Paper: arXiv 2311.10801

Authors: Wentao Zhang, Yilei Zhao, Shuo Sun, Jie Ying, Yonggang Xie, Zitao Song, Xinrun Wang, Bo An

Abstract

Portfolio management (PM) is a fundamental financial trading task, which explores the optimal periodical reallocation of capitals into different stocks to pursue long-term profits. Reinforcement learning (RL) has recently shown its potential to train profitable agents for PM through interacting with financial markets. However, existing work mostly focuses on fixed stock pools, which is inconsistent with investors’ practical demand. Specifically, the target stock pool of different investors varies dramatically due to their discrepancy on market states and individual investors may temporally adjust stocks they desire to trade (e.g., adding one popular stocks), which lead to customizable stock pools (CSPs). Existing RL methods require to retrain RL agents even with a tiny change of the stock pool, which leads to high computational cost and unstable performance. To tackle this challenge, we propose EarnMore, a rEinforcement leARNing framework with Maskable stOck REpresentation to handle PM with CSPs through one-shot training in a global stock pool (GSP). Specifically, we first introduce a mechanism to mask out the representation of the stocks outside the target pool. Second, we learn meaningful stock representations through a self-supervised masking and reconstruction process. Third, a re-weighting mechanism is designed to make the portfolio concentrate on favorable stocks and neglect the stocks outside the target pool. Through extensive experiments on 8 subset stock pools of the US stock market, we demonstrate that EarnMore significantly outperforms 14 state-of-the-art baselines in terms of 6 popular financial metrics with over 40% improvement on profit.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced RL theory, deep learning architectures, and representation learning with self-supervised objectives, indicating high mathematical complexity. It also features extensive empirical validation with multiple baselines, specific financial metrics, and a public code repository, demonstrating strong empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Handle Portfolio Management with Customizable Stock Pools<br>using one-shot training"] --> B["Data Input: Global Stock Pool & Financial Markets"]
  B --> C["Methodology: EarnMore Framework"]
  C --> D["Representation Learning<br>Self-supervised Masking & Reconstruction"]
  C --> E["Masking Mechanism<br>Mask representations of stocks outside target pool"]
  C --> F["Re-weighting Mechanism<br>Concentrate on favorable stocks"]
  D & E & F --> G["Computational Process<br>RL Agent Interaction with CSPs"]
  G --> H["Key Findings<br>Outperforms 14 SOTA baselines<br>>40% profit improvement"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes EarnMore, a reinforcement learning framework with maskable stock representation for portfolio management over customizable stock pools.

Source sentence
To tackle this challenge, we propose EarnMore, a rEinforcement leARNing framework with Maskable stOck REpresentation to handle PM with CSPs through one-shot training in a global stock pool (GSP).

Assumes 3

  • Customizable stock pools can be simulated by randomly masking stocks during training. model
    Source sentence
    During training, we utilize the adaptive masking strategy developed by MAGE
  • The maskable stock representation can replace raw market data as input to the actor‑critic networks. model
    Source sentence
    defined in Equation 4 instead of raw market data as input states for Actor and Critic.
  • Daily price and technical indicator data from Yahoo Finance are sufficient for evaluating the RL framework. data
    Source sentence
    In our experiment, we study daily data for 10,273 US stocks from Yahoo Finance, deriving indicators to understand market trends.

Data

  • Yahoo Finance daily US stock data · Yahoo Finance market-prices public
    2007-09-26 to 2022-06-26 · 10,273 US stocks (filtered to 3,094 stocks with 95 technical indicators)
    Source sentence
    In our experiment, we study daily data for 10,273 US stocks from Yahoo Finance, deriving indicators to understand market trends.
  • SP500 subset benchmark public
    2007-09-26 to 2022-06-26 · 420 stocks
    Source sentence
    Global stock pool (GSP) is the full set of stock pools containing 420 and 28 stocks respectively.
  • DJ30 subset benchmark public
    2007-09-26 to 2022-06-26 · 28 stocks
    Source sentence
    Global stock pool (GSP) is the full set of stock pools containing 420 and 28 stocks respectively.

Costs

Unclear No cost treatment was found in the passages the extractor read; that is not evidence that costs are not modeled.

Finds 3

  • EarnMore significantly outperforms 14 baselines on 6 financial metrics, achieving over 40% improvement on profit. performance
    Source sentence
    we demonstrate that EarnMore significantly outperforms 14 state-of-the-art baselines in terms of 6 popular financial metrics with over 40% improvement on profit.
  • On the SP500 global stock pool, EarnMore achieves a profit of 110.15% and ARR of 7.0%, far exceeding other methods. performance
    Source sentence
    EarnMore 110.1507.0 2.28223.7 25.8 -19.7 43.4 95.1
  • Ablation shows that removing maskable representation or re‑weighting reduces profit and increases risk metrics. performance
    Source sentence
    Comparing EarnMore-w/o-M with EarnMore reveals a significant improvement due to the generalized pool-level maskable stock representation.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 5 papers in the archive (5 by independent authors)

Code

https://github.com/DVampire/EarnMore — the authors' own repository

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (60)

  1. Online portfolio selection: A survey (2014)
  2. Market making via reinforcement learning (2018)
  3. Universal trading for order execution with oracle policy distillation (2021)
  4. Mastering the game of go without human knowledge (2017)
  5. An end-to-end optimal trade execution framework based on proximal policy optimization (2020)
  6. Discovering faster matrix multiplication algorithms with reinforcement learning (2022)
  7. Reinforcement learning for quantitative trading (2023)
  8. Time series momentum (2012)
and 52 more
  1. Bollinger on bollinger bands (2002)
  2. Deep direct reinforcement learning for financial signal representation and trading (2016)
  3. Hierarchical multi-scale gaussian transformer for stock movement prediction (2020)
  4. Efficient capital markets: A review of theory and empirical work (1970)
  5. Temporal relational ranking for stock prediction (2019)
  6. Lightgbm: A highly efficient gradient boosting decision tree (2017)
  7. An actor-critic-based portfolio investment method inspired by benefit-risk optimization (2018)
  8. Model-based deep reinforcement learning for dynamic portfolio optimization (2019) — link
  9. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  10. Portfolio management system with reinforcement learning (2020)
  11. Adversarial deep reinforcement learning in portfolio management (2018) — link
  12. Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
  13. Jointly modeling transfer learning of industrial chain information and deep learning for stock prediction (2022)
  14. Transfer Learning for Portfolio Optimization — → 2307.13546
  15. Deep Reinforcement Learning for ESG financial portfolio management — → 2307.09631
  16. Asynchronous methods for deep reinforcement learning (2016)
  17. Trust region policy optimization (2015)
  18. Proximal policy optimization algorithms (2017) — link
  19. Playing atari with deep reinforcement learning (2013) — link
  20. Soft actor-critic algorithms and applications (2018) — link
  21. Continuous control with deep reinforcement learning (2015) — link
  22. Addressing function approximation error in actor-critic methods (2018)
  23. Performance functions and reinforcement learning for trading systems and portfolios (1998)
  24. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  25. Investor-imitator: A framework for trading knowledge extraction (2018)
  26. DeepTrader: A deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
  27. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  28. Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
  29. Masked autoencoders are scalable vision learners (2022)
  30. A time series is worth 64 words: Long-term forecasting with transformers (2022) — link
  31. SimMTM: A simple pre-training framework for masked time-series modeling (2023) — link
  32. Ti-MAE: Self-supervised masked time series autoencoders (2023) — link
  33. Qlib: An AI-oriented quantitative unvestment platform (2020) — link
  34. Mage: Masked generative encoder to unify representation learning and image synthesis (2023)
  35. FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance (2020) — link
  36. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis (2023)
  37. Policy gradient methods for reinforcement learning with function approximation (1999)
  38. Mean reversion in stock prices: Evidence and implications (1988)
  39. Cross-sectional and time-series determinants of momentum returns (2002)
  40. Xgboost: A scalable tree boosting system (2016)
  41. A dual-stage attention-based recurrent neural network for time series prediction (2017) — link
  42. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling (2018) — link
  43. A unified theory of underreaction, momentum trading, and overreaction in asset markets (1999)
  44. Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts (2021)
  45. Deep reinforcement learning for portfolio management of markets with a dynamic number of assets (2021)
  46. Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
  47. 101 formulaic alphas (2016)
  48. SimMIM: A Simple Framework for Masked Image Modeling (2022)
  49. Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
  50. FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2022) — link
  51. Reinforcement learning in financial markets-a survey (2018)
  52. TradeMaster: A Holistic Quantitative Trading Platform Empowered by Reinforcement Learning (2023)

2 of 60 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.