Paper: arXiv 2309.00626

Abstract

We propose an ensemble method to improve the generalization performance of trading strategies trained by deep reinforcement learning algorithms in a highly stochastic environment of intraday cryptocurrency portfolio trading. We adopt a model selection method that evaluates on multiple validation periods, and propose a novel mixture distribution policy to effectively ensemble the selected models. We provide a distributional view of the out-of-sample performance on granular test periods to demonstrate the robustness of the strategies in evolving market conditions, and retrain the models periodically to address non-stationarity of financial data. Our proposed ensemble method improves the out-of-sample performance compared with the benchmarks of a deep reinforcement learning strategy and a passive investment strategy.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts like MDPs, policy ensembling, and distributional views of performance, but also includes significant empirical elements like a 4-year backtest, portfolio trading, and performance metrics.

Research Flowchart

  flowchart TD
  A["Research Goal: Improve DRL Strategy Generalization for Crypto Trading"] --> B["Data: Intraday Cryptocurrency Prices & Volumes"]
  B --> C["Methodology: Ensemble DRL with Model Selection"]
  C --> D{"Key Component: Mixture Distribution Policy"}
  D --> E["Retraining: Address Data Non-Stationarity"]
  E --> F["Outcome: Out-of-Sample Evaluation"]
  F --> G["Result: Superior Performance vs. DRL & Passive Benchmarks"]