Paper: arXiv 2306.10950
Authors: Marc Velay, Bich-Liên Doan, Arpad Rimmel, Fabrice Popineau, Fabrice Daniel
Abstract
Deep Reinforcement Learning approaches to Online Portfolio Selection have grown in popularity in recent years. The sensitive nature of training Reinforcement Learning agents implies a need for extensive efforts in market representation, behavior objectives, and training processes, which have often been lacking in previous works. We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management. We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)
Why this score: The paper employs advanced Deep Reinforcement Learning mathematics (Policy Gradients, MDPs, entropy regularization) and demonstrates high empirical rigor through a standardized benchmarking process, public data, open-source code, and robust statistical evaluation of DRL agents.
Research Flowchart
flowchart TD
A["Research Goal: Benchmark DRL Robustness in Online Portfolio Management"] --> B{"Methodology"}
B --> C["Market Representation & Data Processing<br>Multi-Asset Time Series"]
B --> D["Algorithm Selection & Training<br>Classical DRL Agents"]
C --> E["Backtesting Process<br>Out-of-sample Evaluation"]
D --> E
E --> F["Robustness Metrics<br>Generalization & Degradation Analysis"]
F --> G{"Key Findings / Outcomes"}
G --> H["Most DRL algorithms lack robustness<br>Strategies generalize poorly<br>Performance degrades quickly"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a training and evaluation process to assess the performance of classical deep reinforcement learning algorithms for portfolio management.
Source sentence
We propose a training and evaluation process to assess the performance of classical DRL algorithms for portfolio management.
Assumes 5
- The financial market environment is highly uncertain and non‑stationary. market
Source sentence
OLPS is unlike many other RL problems due to its high uncertainty
- Agents can keep a portion of wealth in cash (liquidity) to reduce portfolio volatility. model
Source sentence
We add the possibility for the agent to keep liquidity to reduce portfolio volatility.
- Training uses a fixed starting portfolio value of 100,000 USD. execution
Source sentence
The starting value of the portfolio was 100000USD.
- During training, 20 stocks are randomly subsampled from the S&P500 constituents. data
Source sentence
During training, 20 stocks are randomly subsampled.
Show 1 more
- Backtesting evaluates a manually selected set of 20 assets from diverse industries. execution
Source sentence
For backtesting, we manually selected 20 assets from diverse industries with moderate to high returns for the period.
Data
- S&P500 constituent stocks market-prices public
Source sentence
We focused on S&P500 constituent stocks at the date of January 2023, for the period from January 2010 to December 2022.
Costs
Not modeled
Finds 2
- Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting. negative
Source sentence
We found that most Deep Reinforcement Learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.
- Overfitting agents achieved good validation performance but quickly became obsolete when deployed. negative
Source sentence
we have found that overfitting agents predicting the same allocation weights regardless of selected assets and market changes had good performance measures in validation but would quickly become obsolete if deployed.
Checked by others
Supported by
- Myopic Optimality: why reinforcement learning portfolio management strategies lose money
Compare the findings
This paper: Most deep reinforcement learning algorithms were not robust, with strategies generalizing poorly and degrading quickly during backtesting.
That paper: Myopic optimization outperforms reinforcement learning: RL yields lower or negative returns, higher variance, larger costs, heavier CVaR, lower profitability, and greater model risk.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.