Paper: arXiv 2212.09192

Authors: Hongda Hu, Arthur Charpentier, Mario Ghossoub, Alexander Schied

Abstract

The classical multi-armed bandit (MAB) problem involves a learner and a collection of K independent arms, each with its own ex ante unknown independent reward distribution. At each one of a finite number of rounds, the learner selects one arm and receives new information. The learner often faces an exploration-exploitation dilemma: exploiting the current information by playing the arm with the highest estimated reward versus exploring all arms to gather more reward information. The design objective aims to maximize the expected cumulative reward over all rounds. However, such an objective does not account for a risk-reward tradeoff, which is often a fundamental precept in many areas of applications, most notably in finance and economics. In this paper, we build upon Sani et al. (2012) and extend the classical MAB problem to a mean-variance setting. Specifically, we relax the assumptions of independent arms and bounded rewards made in Sani et al. (2012) by considering sub-Gaussian arms. We introduce the Risk Aware Lower Confidence Bound (RALCB) algorithm to solve the problem, and study some of its properties. Finally, we perform a number of numerical simulations to demonstrate that, in both independent and dependent scenarios, our suggested approach performs better than the algorithm suggested by Sani et al. (2012).

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 5.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a solid theoretical extension of the Multi-Armed Bandit framework to incorporate a mean-variance trade-off under sub-Gaussian and dependent reward assumptions. While the mathematical formulations and proofs are robust, the empirical component relies primarily on synthetic numerical simulations rather than real-world financial backtesting.

Research Flowchart

  flowchart TD
    A[Research Goal: Extend MAB to Mean-Variance Setting<br>Relaxing Independent & Bounded Assumptions] --> B[Inputs: Sub-Gaussian Reward Distributions<br>Independent & Dependent Arms]
    B --> C[Methodology: Develop Risk Aware Lower<br>Confidence Bound RALCB Algorithm]
    C --> D[Computational Process: Run Numerical Simulations<br>Comparing RALCB vs. Sani et al. 2012]
    D --> E[Key Findings: RALCB Outperforms Baseline<br>in Both Independent & Dependent Scenarios]