Variations on the Reinforcement Learning performance of Blackjack

ArXiv ID: 2308.07329 “View on arXiv”

Authors: Unknown

Abstract

Blackjack or “21” is a popular card-based game of chance and skill. The objective of the game is to win by obtaining a hand total higher than the dealer’s without exceeding 21. The ideal blackjack strategy will maximize financial return in the long run while avoiding gambler’s ruin. The stochastic environment and inherent reward structure of blackjack presents an appealing problem to better understand reinforcement learning agents in the presence of environment variations. Here we consider a q-learning solution for optimal play and investigate the rate of learning convergence of the algorithm as a function of deck size. A blackjack simulator allowing for universal blackjack rules is also implemented to demonstrate the extent to which a card counter perfectly using the basic strategy and hi-lo system can bring the house to bankruptcy and how environment variations impact this outcome. The novelty of our work is to place this conceptual understanding of the impact of deck size in the context of learning agent convergence.

Keywords: Q-learning, Reinforcement Learning (RL), Card Counting, House Edge, Game Theory, General / Gaming

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 5.0/10
• Quadrant: Street Traders
• Why: The paper involves basic game theory and RL (Q-learning) concepts without dense derivations, but implements a simulator and evaluates convergence rates and financial outcomes, making it moderately data and implementation heavy.

  flowchart TD
    subgraph R["Research Goal"]
        G["How does deck size affect Q-learning convergence and card counting effectiveness in Blackjack?"]
    end

    subgraph M["Methodology"]
        S1["Implement Blackjack Simulator"] --> S2["Develop Q-learning Agent"]
        S2 --> S3["Test varying Deck Sizes"]
        S3 --> S4["Simulate Hi-Lo Card Counter"]
    end

    subgraph D["Data & Inputs"]
        D1["Deck Sizes: 1 to 8 Decks"]
        D2["Standard Blackjack Rules"]
        D3["Q-learning Parameters"]
    end

    subgraph P["Computational Process"]
        P1["Train Agent: Update Q-values via rewards"]
        P2["Evaluate Win Rate & Convergence Speed"]
        P3["Simulate Bankroll: Counter vs House Edge"]
    end

    subgraph O["Key Findings/Outcomes"]
        F1["Q-learning converges faster with fewer decks"]
        F2["Card counting significantly reduces house edge"]
        F3["Deck size impacts optimal strategy effectiveness"]
    end

    G --> S1
    D1 & D2 & D3 --> P1
    P1 --> P2 --> P3
    P3 --> F1 & F2 & F3