Paper: arXiv 2306.10950

Authors: Marc Velay, Bich-Liên Doan, Arpad Rimmel, Fabrice Popineau, Fabrice Daniel

Abstract

Deep Reinforcement Learning approaches to Online Portfolio Selection have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often been lacking in previous works. We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management. We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)

Why this score: The paper employs advanced Deep Reinforcement Learning mathematics (Policy Gradients, MDPs, entropy regularization) and demonstrates high empirical rigor through a standardized benchmarking process, public data, open-source code, and robust statistical evaluation of DRL agents.

Research Flowchart

  flowchart TD
  A["Research Goal: Benchmark DRL Robustness in Online Portfolio Management"] --> B{"Methodology"}
  B --> C["Market Representation & Data Processing<br>Multi-Asset Time Series"]
  B --> D["Algorithm Selection & Training<br>Classical DRL Agents"]
  C --> E["Backtesting Process<br>Out-of-sample Evaluation"]
  D --> E
  E --> F["Robustness Metrics<br>Generalization & Degradation Analysis"]
  F --> G{"Key Findings / Outcomes"}
  G --> H["Most DRL algorithms lack robustness<br>Strategies generalize poorly<br>Performance degrades quickly"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a training and evaluation process to assess the performance of classical deep reinforcement learning algorithms for portfolio management.

Source sentence
We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management.

Assumes 5

  • The financial market environment is highly uncertain and non‑stationary. market
    Source sentence
    OLPS is unlike many other RL problems due to its high uncertainty
  • Agents can keep a portion of wealth in cash (liquidity) to reduce portfolio volatility. model
    Source sentence
    We add the possibility for the agent to keep liquidity to reduce portfolio volatility.
  • Training uses a fixed starting portfolio value of 100,000 USD. execution
    Source sentence
    The starting value of the portfolio was 100000USD.
  • During training, 20 stocks are randomly subsampled from the S&P500 constituents. data
    Source sentence
    During training, 20 stocks are randomly subsampled.
Show 1 more
  • Backtesting evaluates a manually selected set of 20 assets from diverse industries. execution
    Source sentence
    For backtesting, we manually selected 20 assets from diverse industries with moderate to high returns for the period.

Data

  • S&P500 constituent stocks · public market data (S&P500 constituents as of Jan 2023) market-prices public
    2010-01 to 2022-12 · 500 S&P500 constituent stocks (20‑asset portfolio sampled)
    Source sentence
    We focused on S&P500 constituent stocks at the date of January 2023, for the period from January 2010 to December 2022.

Costs

Not modeled A trading or execution result is reported without costs.

Finds 2

  • Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting. negative
    Source sentence
    We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.
  • Overfitting agents achieved good validation performance but quickly became obsolete when deployed. negative
    Source sentence
    we have found that overfitting agents predicting the same allocation weights regardless of selected assets and market changes had good performance measures in validation but would quickly become obsolete if deployed.

Checked by others

Supported by

  • Myopic Optimality: why reinforcement learning portfolio management strategies lose money (2025)
    On the question: Do reinforcement learning portfolio management strategies provide robust and profitable performance?
    Compare the findings

    This paper: Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.

    That paper: Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.

    Both findings agree that RL methods perform poorly: A reports lack of robustness and quick degradation, while B shows lower/negative returns, higher variance, costs, CVaR, and model risk.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.