Paper: arXiv 2501.17992

Authors: Jinghai He, Cheng Hua, Chunyang Zhou, Zeyu Zheng

Abstract

We develop a portfolio allocation framework that leverages deep learning techniques to address challenges arising from high-dimensional, non-stationary, and low-signal-to-noise market information. Our approach includes a dynamic embedding method that reduces the non-stationary, high-dimensional state space into a lower-dimensional representation. We design a reinforcement learning (RL) framework that integrates generative autoencoders and online meta-learning to dynamically embed market information, enabling the RL agent to focus on the most impactful parts of the state space for portfolio allocation decisions. Empirical analysis based on the top 500 U.S. stocks demonstrates that our framework outperforms common portfolio benchmarks and the predict-then-optimize (PTO) approach using machine learning, particularly during periods of market stress. Traditional factor models do not fully explain this superior performance. The framework’s ability to time volatility reduces its market exposure during turbulent times. Ablation studies confirm the robustness of this performance across various reinforcement learning algorithms. Additionally, the embedding and meta-learning techniques effectively manage the complexities of high-dimensional, noisy, and non-stationary financial data, enhancing both portfolio performance and risk management.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced deep learning concepts including dynamic embedding, generative autoencoders, and online meta-learning, which requires significant mathematical sophistication. Empirically, it is backed by rigorous testing on 500 U.S. stocks over 30 years, with comprehensive ablation studies, transaction cost considerations, and benchmark comparisons.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Portfolio Allocation in High-Dim,<br>Non-Stationary Markets"] -->|"Input"| B["Top 500 U.S. Stocks<br>(High-Dim, Noisy Data)"]
  B --> C["Dynamic Embedding Module<br>Generative Autoencoders"]
  C --> D["Reduced State Representation<br>(Low-Dim, Stationary Features)"]
  D --> E["RL Framework<br>Online Meta-Learning"]
  E --> F["Portfolio Allocation<br>Agent Decision"]
  F --> G["Outcomes:<br>Outperforms Benchmarks & PTO<br>Robust to Market Stress<br>Effective Volatility Timing"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes an end‑to‑end Dynamic Embedding Reinforcement Learning (DERL) framework that combines deep RL, generative autoencoders and online meta‑learning for portfolio allocation.

Source sentence
In this paper, to address the challenges of high-dimensional portfolio allocation in a dynamic non-stationary market, we propose an end-to-end framework named Dynamic Embedding Reinforcement Learning (DERL), which leverages three deep learning methods—deep reinforcement learning, generative encoders, and meta‑learning.

Assumes 6

  • No leverage or short selling is allowed. execution
    Source sentence
    Finally, we assume no leverage or short selling is allowed, aligning our strategy with the constraints typically encountered in mutual fund portfolio management.
  • Investor maximizes Sharpe ratio of net returns after transaction costs. model
    Source sentence
    The investor is assumed to maximize the Sharpe ratio of net portfolio returns after accounting for transaction costs.
  • Transaction cost rate of 0.1% applied to each trade. execution
    Source sentence
    cost rate of 0.1% is applied to the total value of each trade.
  • Portfolio is rebalanced daily using only price‑volume data and technical indicators. data
    Source sentence
    our method relies on daily adjustments solely based on price‑volume data and technical indicators.
Show 2 more
  • Only the top 500 U.S. stocks by market capitalization are considered. data
    Source sentence
    we evaluate out‑of‑sample portfolio performance using top 500 stocks in terms of market capitalization in each subperiod.
  • Sharpe ratio is used as the RL reward function. model
    Source sentence
    we use the Sharpe ratio to measure portfolio performance, which serves as the final reward for the reinforcement learning agent.

Data

Backtest sample: 1993 – 2022 · top 500 US stocks · equities · daily

  • CRSP daily stock file · CRSP (Center for Research in Security Prices) market-prices commercial
    January 1, 1990 to December 31, 2022 · top 500 U.S. stocks by market value
    Source sentence
    We evaluate our model performance using the top 500 stocks by market value, which are actively traded. The trading information for each constituent stock, including daily open (O), high (H), low (L), close (C) prices, trading volumes (V), and returns, is collected from the CRSP (Center for Research in Security Prices)

Recognised sources: Cboe CRSP

Costs

Modeled Costs are part of the reported headline results.

0.1% of the total trade value is charged as transaction cost.

Source sentence
cost rate of 0.1% is applied to the total value of each trade.

Finds 5

Reported performance (DERL (Dynamic Embedding RL) portfolio): Sharpe 1.04 (benchmark 0.48) · annual return 14.8%

  • DERL achieves significantly higher Sharpe and Sortino ratios than the two‑step predict‑then‑optimize method and both value‑ and equal‑weighted portfolios. performance
    Source sentence
    Empirical results show that our DERL framework achieves significantly higher Sharpe and Sortino ratios compared to the two-step predict‑then‑optimize (PTO) method using machine learning models, as well as value- and equal-weighted portfolios.
  • DERL’s outperformance is especially strong during high‑volatility regimes. performance
    Source sentence
    The results demonstrate that DERL’s outperformance is highly significant under high market volatility conditions compared to low‑volatility conditions.
  • The daily risk‑adjusted return alpha of DERL exceeds 0.03%, i.e., about 7.5% per annum, and cannot be fully explained by standard factor models. performance
    Source sentence
    Factor analysis shows that the performance of the DERL framework cannot be fully explained by the Fama and French (1993) three‑factor model or Fama and French (1993)‑Carhart (1997) four‑factor model, with the daily risk‑adjusted return α exceeding 0.03%, or 7.5% per an…
  • Ablation studies confirm that the framework’s performance is robust across different RL algorithms and that embedding and meta‑learning are crucial components. performance
    Source sentence
    All three RL algorithms within the DERL framework outperform versions without dynamic updati…
  • The DERL agent exhibits volatility‑timing capability, reducing market exposure during periods of market stress. performance
    Source sentence
    Further analysis reveals that the DERL agent has volatility timing capability, reducing

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 66 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (57)

  1. Textual Factors: A Scalable, Interpretable, and Data-Driven Approach to Analyzing Unstructured Information (2024) — link
  2. Target PCA: Transfer Learning Large Dimensional Panel Data (2023) — link
  3. Dynamic datasets and market environments for financial reinforcement learning (2023) — link
  4. Deep Reinforcement Learning for Sequential Targeting (2022) — link
  5. Dynamic Coupon Targeting Using Batch Deep Reinforcement Learning: An Application to Livestream Shopping (2022) — link
  6. Machine-Learning-Based Return Predictors and the Spanning Controversy in Macro-Finance (2022) — link
  7. Fully Online Meta-Learning Without Task Boundaries (2022) — link
  8. Recent advances in reinforcement learning in finance (2021) — link
and 49 more
  1. FinRL: deep reinforcement learning framework to automate trading in quantitative finance (2021) — link
  2. Introduction to Reinforcement Learning (2020) — link
  3. Autoencoder asset pricing models (2020) — link
  4. Do Noisy Stock Prices Impede Real Efficiency? (2020) — link
  5. Deep Sequence Modeling: Development and Applications in Asset Pricing (2020) — link
  6. A Transaction-Cost Perspective on the Multitude of Firm Characteristics (2020) — link
  7. Dissecting Characteristics Nonparametrically (2020) — link
  8. An Augmented q-Factor Model with Expected Growth* (2020) — link
  9. Scalable Reinforcement Learning for Multiagent Networked Systems (2019) — link
  10. Dynamics-aware Embeddings (2019) — link
  11. Financial Machine Learning (2019) — link
  12. Adaptive Clinical Trial Designs with Surrogates: When Should We Bother? (2019) — link
  13. Deep Learning in Asset Pricing (2019) — link
  14. Meta Dynamic Pricing: Learning Across Experiments (2019) — link
  15. Empirical Asset Pricing via Machine Learning (2018) — link
  16. Approaching Mean-Variance Efficiency for Large Portfolios (2018) — link
  17. Machine Learning and Portfolio Optimization (2018) — link
  18. Addressing Function Approximation Error in Actor-Critic Methods (2018) — link
  19. Wasserstein Auto-Encoders (2017) — link
  20. Mastering the game of Go without human knowledge (2017) — link
  21. Wasserstein Generative Adversarial Networks (2017)
  22. Adversarial Variational Bayes: Unifying Variational Autoencoders and Generative Adversarial Networks (2017) — link
  23. Asynchronous Methods for Deep Reinforcement Learning (2016) — link
  24. Mastering the game of Go with deep neural networks and tree search (2016) — link
  25. Continuous control with deep reinforcement learning (2015) — link
  26. Deterministic Policy Gradient Algorithms (2014)
  27. Improving Portfolio Selection Using Option-Implied Volatility and Skewness (2013) — link
  28. Reinforcement Learning: A Tutorial Survey and Recent Advances (2009) — link
  29. The Markowitz Optimization Enigma: Is ‘Optimized’ Optimal? (2005) — link
  30. Common risk factors in the returns on stocks and bonds (1993) — link
  31. The Persistence of Mutual Fund Performance (1992) — link
  32. Permanent and Temporary Components of Stock Prices (1988) — link
  33. Stationarity of Market Risk: Random Coefficients Tests for Individual Stocks (1980) — link
  34. Portfolio Selection (1971) — link
  35. Machine Learning vs. Economic Restrictions: Evidence from Stock Return Predictability (2023) — link
  36. Asset-Pricing Factors with Economic Targets (2023) — link
  37. AlphaPortfolio: Direct Construction Through Deep Reinforcement Learning and Interpretable AI (2021)
  38. (Re-)Imag(in)ing Price Trends∗ (2021)
  39. Predicting with proxies: Transfer learning in high dimension (2021)
  40. AUTO-ENCODING VARIATIONAL BAYES (2020)
  41. Time Varying Stock Market Integration and Diversification Opportunities within Emerging and Frontier Markets (2020) — link
  42. NBER WORKING PAPER SERIES CHARACTERISTICS ARE COVARIANCES: A UNIFIED MODEL OF RISK AND RETURN (2018)
  43. GENERATIVE ADVERSARIAL NETS (2018)
  44. A five-factor asset pricing model (2013) — link
  45. Herding and Contrarian Behaviour in Financial Markets (2010)
  46. Illiquidity and Stock Returns: Cross-Section and Time-Series Effects (2002) — link
  47. Smart Money, Noise Trading and Stock Price Behavior (1993) — link
  48. Investigations of Nonstationarity in Prices (1974) — link
  49. org/paper/AlphaPortfolio%3A-Direct-Construction-Through-Deep-AI-Cong-Tang/ 16f7268059915bc194ce54ed379979d6fd2fa631

0 of 57 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.