Paper: arXiv 2410.04217

Authors: Gustavo de Freitas Fonseca, Lucas Coelho e Silva, Paulo André Lima de Castro

Abstract

In Reinforcement Learning (RL), multi-armed Bandit (MAB) problems have found applications across diverse domains such as recommender systems, healthcare, and finance. Traditional MAB algorithms typically assume stationary reward distributions, which limits their effectiveness in real-world scenarios characterized by non-stationary dynamics. This paper addresses this limitation by introducing and evaluating novel Bandit algorithms designed for non-stationary environments. First, we present the Adaptive Discounted Thompson Sampling (ADTS) algorithm, which enhances adaptability through relaxed discounting and sliding window mechanisms to better respond to changes in reward distributions. We then extend this approach to the Portfolio Optimization problem by introducing the Combinatorial Adaptive Discounted Thompson Sampling (CADTS) algorithm, which addresses computational challenges within Combinatorial Bandits and improves dynamic asset allocation. Additionally, we propose a novel architecture called Bandit Networks, which integrates the outputs of ADTS and CADTS, thereby mitigating computational limitations in stock selection. Through extensive experiments using real financial market data, we demonstrate the potential of these algorithms and architectures in adapting to dynamic environments and optimizing decision-making processes. For instance, the proposed bandit network instances present superior performance when compared to classic portfolio optimization approaches, such as capital asset pricing model, equal weights, risk parity, and Markovitz, with the best network presenting an out-of-sample Sharpe Ratio 20% higher than the best performing classical model.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)

Why this score: The paper introduces novel algorithms (ADTS, CADTS) with mathematical formulations involving discounting, sliding windows, and combinatorial optimization, indicating high math density. It demonstrates strong empirical rigor through extensive experiments on real market data (S&P 500), comparing against baselines and reporting metrics like Sharpe Ratio.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Optimize Portfolio in Non-Stationary Markets"] --> B["Data Input:<br>Real Financial Market Data"]
  B --> C["Core Methodology:<br>Novel Bandit Algorithms"]
  C --> D{"Computational Process:<br>Bandit Networks Architecture"}
  D --> E["Algorithm 1:<br>ADTS<br>Adaptive Discounted TS"]
  D --> F["Algorithm 2:<br>CADTS<br>Combinatorial ADTS"]
  E --> G["Outcome:<br>Dynamic Asset Allocation"]
  F --> G
  G --> H["Key Finding:<br>20% Higher Sharpe Ratio<br>vs. Classical Models"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper introduces Adaptive Discounted Thompson Sampling (ADTS), Combinatorial Adaptive Discounted Thompson Sampling (CADTS), and a novel Bandit Networks architecture that integrates them for portfolio optimization.

Source sentence
We then extend this approach to the Portfolio Optimization problem by introducing the Combinatorial Adaptive Discounted Thompson Sampling (CADTS) algorithm

Assumes 1

  • The experiments use historical daily returns of a fixed set of 44 S&P 500 stocks over a specific period, implying no intraday trading or slippage considerations. data
    Source sentence
    using historical daily returns of 44 S&P 500 stocks from April 2020 to July 2024

Data

Backtest sample: 2020-04 – 2024-07 · 44 S&P 500 stocks · equities · daily

  • Historical daily returns of 44 S&P 500 stocks market-prices unknown
    April 2020 to July 2024 · 44 S&P 500 stocks
    Source sentence
    using historical daily returns of 44 S&P 500 stocks from April 2020 to July 2024

Costs

Not modeled A trading or execution result is reported without costs.

Finds 5

Reported performance (Two Layer ADTS (n=4)): Sharpe 1.59 (benchmark 1.32) · total return 4.9% · max drawdown 0.9%

  • The best Bandit Network instance achieves an out-of-sample Sharpe Ratio 20% higher than the best classical model. performance
    Source sentence
    The best network presenting an out-of-sample Sharpe Ratio 20% higher than the best performing classical model
  • Two-Stage ADTS (n = 4) yields cumulative returns 168% higher than the CAPM, the best classical benchmark. performance
    Source sentence
    cumulative returns of Two-Stage ADTS (n = 4) is 168% higher than the CAPM
  • ADTS consistently achieves the lowest cumulative regret among the tested methods. statistical
    Source sentence
    ADTS algorithm demonstrated superior performance, consistently achieving the lowest cumulative regret
  • Two-Stage ADTS (n = 15) shows the smallest drawdown metric (0.55) among network instances, indicating better risk diversification. performance
    Source sentence
    by selecting fifteen stocks simultaneously, the instance diversifies risks, as suggested by the smallest drawdown metric of the bandit networks instances (0.55)
  • All Bandit Network instances outperform classical portfolio models (CAPM, Equal Weights, Risk Parity, Markowitz) in terms of returns. performance
    Source sentence
    all the bandits network instances present superior performance. In this aspect, the cumulative returns of Two-Stage ADTS (n = 4) is 168% higher than the CAPM, where the last is the best-performing classical model

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 26 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (13)

  1. The index of general nonlinear D A E S (1995)
  2. Clinical implications of dysregulated cytokine production (2000) — link
  3. Quasimonotonicity, regularity and duality for nonlinear systems of partial differential equations (1995)
  4. Algorithms for C omputer A lgebra (1992)
  5. Software engineering—from auxiliary to key technologies (1992)
  6. Conductive P olymers (1981)
  7. Neuromuscular blocking drugs in man (1976)
  8. Isolation and characterization of plasmid deoxyribonucleic acid from Streptomyces fradiae (1978)
and 5 more
  1. Global integrated drought monitoring and prediction system (GIDMaPS) data sets (2014)
  2. Quantum scissors: teleportation of single-mode optical states by means of a nonlocal single photon (2002)
  3. Mixing induced CP asymmetries in inclusive B decays (1997) — link
  4. deep SIP : deep learning of S upernova I a P arameters (2020)
  5. Dark Energy Survey Year 1 Results: Constraints on Extended Cosmological Models from Galaxy Clustering and Weak Lensing (2019) — link

0 of 13 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.