Paper: arXiv 2311.10801
Authors: Wentao Zhang, Yilei Zhao, Shuo Sun, Jie Ying, Yonggang Xie, Zitao Song, Xinrun Wang, Bo An
Abstract
Portfolio management (PM) is a fundamental financial trading task, which explores the optimal periodical reallocation of capitals into different stocks to pursue long-term profits. Reinforcement learning (RL) has recently shown its potential to train profitable agents for PM through interacting with financial markets. However, existing work mostly focuses on fixed stock pools, which is inconsistent with investors’ practical demand. Specifically, the target stock pool of different investors varies dramatically due to their discrepancy on market states and individual investors may temporally adjust stocks they desire to trade (e.g., adding one popular stocks), which lead to customizable stock pools (CSPs). Existing RL methods require to retrain RL agents even with a tiny change of the stock pool, which leads to high computational cost and unstable performance. To tackle this challenge, we propose EarnMore, a rEinforcement leARNing framework with Maskable stOck REpresentation to handle PM with CSPs through one-shot training in a global stock pool (GSP). Specifically, we first introduce a mechanism to mask out the representation of the stocks outside the target pool. Second, we learn meaningful stock representations through a self-supervised masking and reconstruction process. Third, a re-weighting mechanism is designed to make the portfolio concentrate on favorable stocks and neglect the stocks outside the target pool. Through extensive experiments on 8 subset stock pools of the US stock market, we demonstrate that EarnMore significantly outperforms 14 state-of-the-art baselines in terms of 6 popular financial metrics with over 40% improvement on profit.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced RL theory, deep learning architectures, and representation learning with self-supervised objectives, indicating high mathematical complexity. It also features extensive empirical validation with multiple baselines, specific financial metrics, and a public code repository, demonstrating strong empirical rigor.
Research Flowchart
flowchart TD A["Research Goal: Handle Portfolio Management with Customizable Stock Pools<br>using one-shot training"] --> B["Data Input: Global Stock Pool & Financial Markets"] B --> C["Methodology: EarnMore Framework"] C --> D["Representation Learning<br>Self-supervised Masking & Reconstruction"] C --> E["Masking Mechanism<br>Mask representations of stocks outside target pool"] C --> F["Re-weighting Mechanism<br>Concentrate on favorable stocks"] D & E & F --> G["Computational Process<br>RL Agent Interaction with CSPs"] G --> H["Key Findings<br>Outperforms 14 SOTA baselines<br>>40% profit improvement"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes EarnMore, a reinforcement learning framework with maskable stock representation for portfolio management over customizable stock pools.
Source sentence
To tackle this challenge, we propose EarnMore, a rEinforcement leARNing framework with Maskable stOck REpresentation to handle PM with CSPs through one-shot training in a global stock pool (GSP).
Assumes 3
- Customizable stock pools can be simulated by randomly masking stocks during training. model
Source sentence
During training, we utilize the adaptive masking strategy developed by MAGE
- The maskable stock representation can replace raw market data as input to the actor‑critic networks. model
Source sentence
defined in Equation 4 instead of raw market data as input states for Actor and Critic.
- Daily price and technical indicator data from Yahoo Finance are sufficient for evaluating the RL framework. data
Source sentence
In our experiment, we study daily data for 10,273 US stocks from Yahoo Finance, deriving indicators to understand market trends.
Data
- Yahoo Finance daily US stock data market-prices public
Source sentence
In our experiment, we study daily data for 10,273 US stocks from Yahoo Finance, deriving indicators to understand market trends.
- SP500 subset benchmark public
Source sentence
Global stock pool (GSP) is the full set of stock pools containing 420 and 28 stocks respectively.
- DJ30 subset benchmark public
Source sentence
Global stock pool (GSP) is the full set of stock pools containing 420 and 28 stocks respectively.
Costs
Unclear
Finds 3
- EarnMore significantly outperforms 14 baselines on 6 financial metrics, achieving over 40% improvement on profit. performance
Source sentence
we demonstrate that EarnMore significantly outperforms 14 state-of-the-art baselines in terms of 6 popular financial metrics with over 40% improvement on profit.
- On the SP500 global stock pool, EarnMore achieves a profit of 110.15% and ARR of 7.0%, far exceeding other methods. performance
Source sentence
EarnMore 110.1507.0 2.28223.7 25.8 -19.7 43.4 95.1
- Ablation shows that removing maskable representation or re‑weighting reduces profit and increases risk metrics. performance
Source sentence
Comparing EarnMore-w/o-M with EarnMore reveals a significant improvement due to the generalized pool-level maskable stock representation.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
5 papers in the archive
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (60)
- Online portfolio selection: A survey (2014)
- Market making via reinforcement learning (2018)
- Universal trading for order execution with oracle policy distillation (2021)
- Mastering the game of go without human knowledge (2017)
- An end-to-end optimal trade execution framework based on proximal policy optimization (2020)
- Discovering faster matrix multiplication algorithms with reinforcement learning (2022)
- Reinforcement learning for quantitative trading (2023)
- Time series momentum (2012)
and 52 more
- Bollinger on bollinger bands (2002)
- Deep direct reinforcement learning for financial signal representation and trading (2016)
- Hierarchical multi-scale gaussian transformer for stock movement prediction (2020)
- Efficient capital markets: A review of theory and empirical work (1970)
- Temporal relational ranking for stock prediction (2019)
- Lightgbm: A highly efficient gradient boosting decision tree (2017)
- An actor-critic-based portfolio investment method inspired by benefit-risk optimization (2018)
- Model-based deep reinforcement learning for dynamic portfolio optimization (2019) — link
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- Portfolio management system with reinforcement learning (2020)
- Adversarial deep reinforcement learning in portfolio management (2018) — link
- Reinforcement-learning based portfolio management with augmented asset movement prediction states (2020)
- Jointly modeling transfer learning of industrial chain information and deep learning for stock prediction (2022)
- Transfer Learning for Portfolio Optimization — → 2307.13546
- Deep Reinforcement Learning for ESG financial portfolio management — → 2307.09631
- Asynchronous methods for deep reinforcement learning (2016)
- Trust region policy optimization (2015)
- Proximal policy optimization algorithms (2017) — link
- Playing atari with deep reinforcement learning (2013) — link
- Soft actor-critic algorithms and applications (2018) — link
- Continuous control with deep reinforcement learning (2015) — link
- Addressing function approximation error in actor-critic methods (2018)
- Performance functions and reinforcement learning for trading systems and portfolios (1998)
- A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
- Investor-imitator: A framework for trading knowledge extraction (2018)
- DeepTrader: A deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
- DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
- Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
- Masked autoencoders are scalable vision learners (2022)
- A time series is worth 64 words: Long-term forecasting with transformers (2022) — link
- SimMTM: A simple pre-training framework for masked time-series modeling (2023) — link
- Ti-MAE: Self-supervised masked time series autoencoders (2023) — link
- Qlib: An AI-oriented quantitative unvestment platform (2020) — link
- Mage: Masked generative encoder to unify representation learning and image synthesis (2023)
- FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance (2020) — link
- TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis (2023)
- Policy gradient methods for reinforcement learning with function approximation (1999)
- Mean reversion in stock prices: Evidence and implications (1988)
- Cross-sectional and time-series determinants of momentum returns (2002)
- Xgboost: A scalable tree boosting system (2016)
- A dual-stage attention-based recurrent neural network for time series prediction (2017) — link
- An empirical evaluation of generic convolutional and recurrent networks for sequence modeling (2018) — link
- A unified theory of underreaction, momentum trading, and overreaction in asset markets (1999)
- Accurate multivariate stock movement prediction via data-axis transformer with multi-level contexts (2021)
- Deep reinforcement learning for portfolio management of markets with a dynamic number of assets (2021)
- Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
- 101 formulaic alphas (2016)
- SimMIM: A Simple Framework for Masked Image Modeling (2022)
- Bert: Pre-training of deep bidirectional transformers for language understanding (2018) — link
- FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (2022) — link
- Reinforcement learning in financial markets-a survey (2018)
- TradeMaster: A Holistic Quantitative Trading Platform Empowered by Reinforcement Learning (2023)
2 of 60 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.