{"paper_id":"2308.07329","title":"Variations on the Reinforcement Learning performance of Blackjack","authors":["Avish Buramdoyal","Tim Gebbie"],"paper_date":"2023-08-09","paper_type":"theoretical","paper_url":"https://arxiv.org/pdf/2308.07329v1","page_url":"https://thequant.space/flowcharts/variations-on-the-reinforcement-learning-performance-of-blac/","scores":{"empirical_rigor":5,"math_complexity":3.5,"quadrant":"Street Traders"},"proposes":{"status":"available","statement":"The paper proposes a Q‑learning based agent for optimal blackjack play and investigates how the rate of learning convergence varies with deck size.","quote":"Here we consider a q-learning solution for optimal play and investigate the rate of learning convergence of the algorithm as a function of deck size. The novelty of our work is to place this conceptual understanding of the impact of deck size in the context of learning agent convergence.","summary":"Blackjack or “21” is a popular card-based game of chance and skill. The objective of the game is to win by obtaining a hand total higher than the dealer’s without exceeding 21. The ideal blackjack strategy will maximize financial return in the long run while avoiding gambler’s ruin. The stochastic…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Rewards are not discounted (γ = 1) and terminal rewards represent returns.","type":"model","stated":"explicit","quote":"We note that rewards γ are not discounted, i.e. γ = 1 and so terminal rewards represent returns.","support":"full"},{"assumption":"The state is defined by the player’s hand total and the dealer’s face‑up card.","type":"model","stated":"explicit","quote":"The state depends on the player’s hand total and the dealer’s face up card.","support":"full"},{"assumption":"Episodes begin with a randomly chosen state and action (exploring starts).","type":"model","stated":"explicit","quote":"Episodes are therefore generated with exploring starts. This means that each episode begins with a randomly chosen state and action and then follows the current policy.","support":"full"},{"assumption":"A decaying epsilon greedy policy is used to balance exploration and exploitation.","type":"model","stated":"explicit","quote":"F. Q-Learning agent: Hit and stand only For the base q-learning model, we implement a decaying epsilon q-learning algorithm, using the one step q-learning given by Equation 9 for the blackjack environment assumed by ”blackjack-v0”. To ensure an effective learning process, appropriate values for ϵ. ϵ is a hyperparameter used to control the balance between exploration and exploitation in many reinforcement learning","support":"full"},{"assumption":"The dealer may either stand or hit on a soft 17, as a configurable rule.","type":"model","stated":"explicit","quote":"We allow the dealer the choice to either stand or hit on having a soft 17.","support":"full"},{"assumption":"The card counter uses the Hi‑Low counting method and follows basic strategy perfectly.","type":"model","stated":"explicit","quote":"The card counter uses the Hi-Low method and the other players randomly decide on the % of their bankroll to bet for each round.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Blackjack simulation data (random agents, card counter, dealer)","provider":null,"kind":"synthetic","access":"public","period":null,"universe":null,"quote":"The figure can be recovered using python script random agent.py on GitHub resource [23]. Case 1: 4 Players, 6 Decks, Dealer stands on soft 17 and 10000 simulations Players Win % Draw % Loss % Bankroll Card counter 14.55 % 1.35 % 3.21 % R14, 647, 394 Random agent 1 11.03 % 1.43 % 7.52 % -R6, 835, 650 Random agent 2 11.01 % 1.44 % 7.52 % -R6, 989, 950 Random agent 3 11.02 % 1","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"none","findings":[{"finding":"The random agent achieves an average payoff of –396.29.","kind":"performance","quote":"The average payoff of -396.29 achieved by the random agent is to be compared to the payoff achieved by the q-learning agent for following q-learning policy.","support":"full"},{"finding":"The card counter outperforms random agents across simulation sizes, numbers of players, and deck sizes.","kind":"performance","quote":"1) Performance across simulation size: Figure 8 shows a clear outperformance of the card counter to the random agent against the dealer across both variations in the number of players, deck size and whether the dealer hits or stands on a soft 17 hand. 2) Performance across number of players: Figure 9 also indicates the outperformance of the card counter to the random agent across increased number of players and","support":"full"},{"finding":"The number of decks in play does not influence the win‑percentage discrepancy between the card counter and random agents.","kind":"performance","quote":"The number of decks in play do not seem to influence the discrepancy in win% against the dealer between the card counter and the random agent.","support":"full"},{"finding":"Even with an initial bankroll ratio of 1:200 favoring the house, the dealer goes bankrupt when the card counter uses perfect basic strategy and the Hi‑Lo system.","kind":"performance","quote":"Even for an initial bankroll ratio of 1:200 in favour of the house, the dealer goes bankrupt for having a card counter perfectly using the basic strategy table and the Hi-Lo system.","support":"full"},{"finding":"Learning convergence rate of the Q‑learning algorithm varies as a function of deck size.","kind":"theoretical","quote":"Here we consider a q-learning solution for optimal play and investigate the rate of learning convergence of the algorithm as a function of deck size.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":15,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T20:01:47Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}