Paper: arXiv 2405.05449
Authors: Gang Hu, Ming Gu
Abstract
Investment portfolios, central to finance, balance potential returns and risks. This paper introduces a hybrid approach combining Markowitz’s portfolio theory with reinforcement learning, utilizing knowledge distillation for training agents. In particular, our proposed method, called KDD (Knowledge Distillation DDPG), consist of two training stages: supervised and reinforcement learning stages. The trained agents optimize portfolio assembly. A comparative analysis against standard financial models and AI frameworks, using metrics like returns, the Sharpe ratio, and nine evaluation indices, reveals our model’s superiority. It notably achieves the highest yield and Sharpe ratio of 2.03, ensuring top profitability with the lowest risk in comparable return scenarios.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 6.0/10 (capped from 6.5: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 6.5)
Why this score: The paper presents substantial mathematical formalisms including Markowitz optimization, MDPs, and continuous RL algorithms (DDPG), while demonstrating empirical rigor through comparative analysis using financial metrics like Sharpe ratio, though it lacks detailed implementation code or raw dataset disclosures.
Research Flowchart
flowchart TD A["Research Goal<br>Optimize portfolio return & risk"] B["Data Inputs<br>Historical market data & asset prices"] C["Methodology<br>Two-stage KDD training"] D["Stage 1<br>Supervised learning (Markowitz reference)"] E["Stage 2<br>Reinforcement learning (DDPG agent)"] F["Computational Process<br>Knowledge distillation & policy optimization"] G["Key Findings<br>Superior performance metrics"] H["Outcome<br>Sharpe Ratio: 2.03<br>Highest yield & lowest risk"] A --> B B --> C C --> D D --> E E --> F F --> G G --> H
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a hybrid portfolio management model called Knowledge Distillation DDPG (KDD) that combines Markowitz theory with reinforcement learning in two training stages.
Source sentence
In particular, our proposed method, called KDD (Knowledge Distillation DDPG), consist of two training stages: supervised and reinforcement learning stages.
Assumes 2
- Investors are risk‑averse, seeking to minimize uncertainty and potential losses while maximizing expected returns for a given level of risk. market
Source sentence
This theory is predicated on the assumption that investors are risk‑averse, seeking to minimize uncertainty and potential losses while maximizing expected returns for a given level of risk.
- The Markowitz model provides optimal portfolio allocations that can be used as target actions for supervised pre‑training of the DDPG actor network. model
Source sentence
Initially, we generate a dataset based on the Markowitz model, simulating optimal investment portfolios under various market conditions.
Data
- Markowitz‑generated synthetic portfolio dataset synthetic unknown
Source sentence
Initially, we generate a dataset based on the Markowitz model, simulating optimal investment portfolios under various market conditions.
Costs
Not modeled
Source sentence
these models frequently overlook critical factors like transaction costs, thus limiting their practicality in real‑world trading scenarios.
Finds 3
Sharpe 2.03
- The KDD model achieves a total return of 138.38%, far surpassing benchmarks such as DJI (51.41%) and the Markowitz portfolio (69.43%). performance
Source sentence
The KDD model demonstrates exceptional performance, with a Total Return of 138.38%, outshining all other strategies, including the DJI and Markowitz portfolio, which yielded returns of 51.41% and 69.43%, respectively.
- The model attains an annualized return of 38.74%, the highest among all compared strategies. performance
Source sentence
Moreover, the model achieves an Annualized Return of 38.74%, surpassing all competitors.
- The KDD model records the highest Sharpe Ratio of 2.03, indicating superior risk‑adjusted performance. performance
Source sentence
In terms of risk‑adjusted returns, the model attains the highest Sharpe Ratio of 2.03, indicating not only higher returns but also a superior risk‑return balance.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (88)
- Markowitz’s" portfolio selection": A fifty-year retrospective (2002)
- Covid-induced economic uncertainty (2020)
- Options, futures, and other derivative securities (1993)
- Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
- Adaptive markets: Financial evolution at the speed of thought (2017)
- Continuous control with deep reinforcement learning (2015) — link
- A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
- Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models — → 2311.05743
and 80 more
- Distilling the knowledge in a neural network (2015) — link
- Stock Market Prediction via Deep Learning Techniques: A Survey — → 2212.12717
- Long short-term memory (1997)
- Deep learning with long short-term memory networks for financial market predictions (2018)
- Human-level control through deep reinforcement learning (2015)
- Markov decision processes: discrete stochastic dynamic programming (1994)
- Mastering the game of Go without human knowledge (2017)
- Sortino: a ‘sharper’ratio (2013)
- Deep Reinforcement Learning for Autonomous Driving (2019)
- Stability tests for alphas and betas over bull and bear market conditions (1977)
- A Comparative Study on the Sharpe Ratio, Sortino Ratio, and Calmar Ratio in Portfolio Optimization
- Market volatility (1992)
- Different measures of win rate for optimal proportional betting (1984)
- The economic value of weather forecasts for decision-making problems in the profit/loss situation (2007)
- Stock returns, expected returns, and real activity (1990)
- Total return strategies for multi-asset portfolios (2007)
- The information ratio (1998)
- Title of the Article (2022)
- Playing Atari with Deep Reinforcement Learning (2013)
- Deep learning for portfolio optimization (2019)
- Benchmarking Robustness of Deep Reinforcement Learning approaches to Online Portfolio Management — → 2306.10950
- Algorithms for portfolio management based on the newton method (2006)
- Proof of activity: Extending bitcoin’s proof of work via proof of stake [extended abstract] y (2014)
- Sok: Research perspectives and challenges for bitcoin and cryptocurrencies (2015)
- On the competitive theory and practice of portfolio selection (2000)
- Can we learn to beat the best stock (2004)
- Technical analysis: The complete resource for financial market technician (2006)
- Universal portfolios (1991)
- Universal data compression and portfolio selection (1996)
- An investigation into the use of reinforcement learning techniques within the algorithmic trading domain (2015) — link
- Meta optimization and its application to portfolio selection (2011) — link
- An automated fx trading system using adaptive reinforcement learning (2006) — link
- Deep direct reinforcement learning for financial signal representation and trading (2017)
- Darkcoin: Peer to peer cryptocurrency with anonymous blockchain transactions and an improved proof of work system (2014)
- Prediction-based portfolio optimization model using neural networks (2009)
- Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position (1980)
- Weighted moving average passive aggressive algorithm for online portfolio selection (2013)
- Bitcoin: An innovative alternative digital currency (2012)
- Nonparametric kernel-based sequential investment strategies (2006)
- Modern investment theory (1986)
- Deep learning for finance: deep portfolios (2016) — link
- On-line portfolio selection using multiplicative updates (1998)
- Forecasting seasonals and trends by exponentially weighted moving averages (2004)
- Robust median reversion strategy for on-line portfolio selection (2013)
- Cryptocurrency portfolio management with deep reinforcement learning (2017)
- A new interpretation of information rate (1956) — link
- Adam: A Method for Stochastic Optimization (2014) — link
- Imagenet classification with deep convolutional neural networks (2012)
- The calculus 7 (1996)
- OLPS: A toolbox for online portfolio selection (2015a)
- Online portfolio selection: A survey (2014)
- CORN: Correlation-driven nonparametric learning approach for portfolio selection (2011)
- PAMR: Passive aggressive mean reversion strategy for portfolio selection (2012) — link
- Confidence weighted mean reversion strategy for online portfolio selection (2013)
- Moving average reversion strategy for on-line portfolio selection (2015b)
- Continuous Control with Deep Reinforcement Learning (2016) — link
- Foundations of technical analysis: Computational algorithms, statistical inference, and empirical implementation (2000)
- Maximum drawdown (2004)
- Portfolio selection: efficient diversification of investments (1968)
- Asynchronous methods for deep reinforcement learning (2016)
- Learning to trade via direct reinforcement (2001) — link
- Performance functions and reinforcement learning for trading systems and portfolios (1998)
- Bitcoin: A peer-to-peer electronic cash system (2008)
- Forecasting S&P 500 index using artificial neural networks and design of experiments (2013) — link
- Performance analysis of log-optimal portfolio strategies with transaction costs (2013)
- Estimating variance from high, low and closing prices (1991)
- Principles of mathematical analysis (1976)
- Convolutional neural networks applied to house numbers digit classification (2012)
- Capital asset prices: A theory of market equilibrium under conditions of risk (1964)
- The Sharpe ratio (1994)
- Deterministic Policy Gradient Algorithms (2014)
- Mastering the game of go with deep neural networks and tree search (2016)
- Universal portfolio selection (1998)
- Generalization of backpropagation with application to a recurrent gas market model (1988)
- Summary of ChatGPT-Related research and perspective towards the future of large language models (2023) — link
- FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement Learning (2022)
- FinRL : Deep reinforcement learning framework to automate trading in quantitative finance (2021)
- FinRL : A deep reinforcement learning library for automated stock trading in quantitative finance (2020)
- Practical deep reinforcement learning approach for stock trading (2018)
- Dynamic Programming and Markov Processes (1960)
3 of 88 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.