Paper: arXiv 2402.00515
Authors: Zhenglong Li, Vincent Tam, Kwan L. Yeung
Abstract
Deep or reinforcement learning (RL) approaches have been adapted as reactive agents to quickly learn and respond with new investment strategies for portfolio management under the highly turbulent financial market environments in recent years. In many cases, due to the very complex correlations among various financial sectors, and the fluctuating trends in different financial markets, a deep or reinforcement learning based agent can be biased in maximising the total returns of the newly formulated investment portfolio while neglecting its potential risks under the turmoil of various market conditions in the global or regional sectors. Accordingly, a multi-agent and self-adaptive framework namely the MASA is proposed in which a sophisticated multi-agent reinforcement learning (RL) approach is adopted through two cooperating and reactive agents to carefully and dynamically balance the trade-off between the overall portfolio returns and their potential risks. Besides, a very flexible and proactive agent as the market observer is integrated into the MASA framework to provide some additional information on the estimated market trends as valuable feedbacks for multi-agent RL approach to quickly adapt to the ever-changing market conditions. The obtained empirical results clearly reveal the potential strengths of our proposed MASA framework based on the multi-agent RL approach against many well-known RL-based approaches on the challenging data sets of the CSI 300, Dow Jones Industrial Average and S&P 500 indexes over the past 10 years. More importantly, our proposed MASA framework shed lights on many possible directions for future investigation.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.0)
Why this score: The paper employs advanced concepts from deep reinforcement learning (e.g., TD3, POMDP, multi-agent systems) and operationalizes them into a concrete computational framework, while demonstrating empirical rigor with specific backtest-ready datasets (CSI 300, DJIA, S&P 500), reported hardware (RTX 3090), and comparative performance metrics.
Research Flowchart
flowchart TD A["Research Goal:<br>Develop a Multi-Agent, Self-Adaptive Framework<br>(MASA) for Portfolio Risk Management"] --> B["Methodology:<br>Multi-Agent Reinforcement Learning (MARL)"] B --> C["Data Input:<br>10-Year Historical Market Indices<br>(CSI 300, Dow Jones, S&P 500)"] C --> D["Computational Process:<br>1. Market Observer (Proactive Agent)<br>2. Multi-Agent RL (Cooperating Agents)<br>3. Self-Adaptive Balancing of Risk vs. Return"] D --> E["Key Findings:<br>1. Outperforms Baseline RL Approaches<br>2. Effectively Balances Risk-Return Trade-off<br>3. Dynamic Adaptation to Market Turmoil"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a multi‑agent and self‑adaptive framework (MASA) that combines RL‑based, solver‑based, and market‑observer agents to balance portfolio returns and risks.
Source sentence
Accordingly, a multi-agent and self-adaptive framework namely the MASA is proposed in which a sophisticated multi-agent reinforcement learning (RL) approach is adopted through two cooperating and reactive agents to carefully and dynamically balance the trade-off between the overall portfolio returns and their potential
Assumes 4
- The first five years of data are used for training and the subsequent two years for validation. data
Source sentence
the first five-year data is used to train the model, followed by the subsequent data set of two years to validate the trained model.
- All data sets contain both upward and downward price trends and diverse market conditions to avoid bias. market
Source sentence
all the involved data sets contain both upward and downward trends of stock prices, and also various patterns of fluctuation for different market conditions so as to avoid any possible bias toward a specific approach under the evaluation.
- Portfolio risk is measured by the sampled variance (volatility) of daily returns over the trading period. statistical
Source sentence
The long-term portfolio risk 𝑉𝑜𝑙𝑝 is defined as the strategy volatility that is the sampled variance of the daily return rates 𝑟𝑝,𝑡 of a trading strategy over the whole trading period.
- Performance is evaluated using Sharpe Ratio, defined as (annualized return – risk‑free rate) divided by portfolio volatility. statistical
Source sentence
The Sharpe Ratio (SR) is a performance indicator for evaluating a portfolio in terms of the total annualized returns 𝑅𝑝, risk‑free rate 𝑟𝑓 and annualized long-term portfolio risk 𝑉𝑜𝑙𝑝.
Data
- CSI 300 market-prices unknown
Source sentence
three challenging yet representative data sets of CSI 300, Dow Jones Industrial Average (DJIA) and S&P 500 indexes from September 2013 to August 2023
- Dow Jones Industrial Average (DJIA) market-prices unknown
Source sentence
three challenging yet representative data sets of CSI 300, Dow Jones Industrial Average (DJIA) and S&P 500 indexes from September 2013 to August 2023
- S&P 500 market-prices unknown
Source sentence
three challenging yet representative data sets of CSI 300, Dow Jones Industrial Average (DJIA) and S&P 500 indexes from September 2013 to August 2023
Costs
Not modeled
Finds 3
- MASA without market observer reduces risk (MDD 0.0119) and achieves a higher Sharpe Ratio compared with the single‑agent baseline. performance
Source sentence
The resulting risks and MDD are dropped to 0.0119 and 31.7% respectively, with some further improvement on both the total returns and SR.
- Adding the market observer agent further improves total annual returns and Sharpe Ratio over the dual‑agent MASA‑w/oMktObs. performance
Source sentence
MASA can better estimate the potential risks while pursuing higher returns after considering more latest market information from the market observer agent.
- The MASA framework outperforms ten representative RL‑based and algorithmic methods on the CSI 300, DJIA and S&P 500 data sets over the past 10 years. performance
Source sentence
The obtained empirical results clearly reveal the remarkable performance of our proposed MASA framework based on the multi-agent RL approach when compared against those of other well-known RL-based approaches on the 3 data sets of widely recognised financial indexes
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 17 related papers). Coverage is partial, so this is neither replication nor refutation.
3 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (63)
- The Jensen-Shannon divergence (1997) — link
- On Information and Sufficiency (1951) — link
- Divergence measures based on the Shannon entropy (1991) — link
- A brief review of portfolio optimization techniques (2023)
- A deep learning approach to estimating fill probabilities in a limit order book (2022)
- AI for Finance (2023)
- Predicting stock market trends using machine learning and deep learning algorithms via continuous and binary data; a comparative analysis (2020)
- An optimal least square support vector machine based earnings prediction of blockchain financial products (2020)
and 55 more
- Efficient Capital Markets: A Review of Theory and Empirical Work (1970)
- Multi-agent systems for computational economics and finance (2022) — link
- Agent-Based Models in Finance and Market Simulations (2021) — link
- Fast Agent-Based Simulation Framework with Applications to Reinforcement Learning and the Study of Trading Latency Effects (2022)
- A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment (2020) — link
- Dcn v2: Improved deep & cross network and practical lessons for web-scale learning to rank systems (2021)
- Artificial intelligence and machine learning in finance: Identifying foundations, themes, and research clusters from bibliometric analysis (2021)
- Modern perspectives on reinforcement learning in finance (2020)
- Markov decision processes (1990)
- Mean-variance Analysis in Portfolio Choice and Capital Markets (2000)
- Capital Growth Theory (1995)
- NL-SHADE-RSP algorithm with adaptive archive and selective pressure for CEC 2021 numerical optimization (2021)
- Global Portfolio Optimization (1992)
- Hierarchical Adaptive Temporal-Relational Modeling for Stock Trend Prediction. (2021)
- FactorVAE: A Probabilistic Dynamic Factor Model Based on Variational Autoencoder for Predicting Cross-sectional Stock Returns (2022)
- A Smart Trader for Portfolio Management based on Normalizing Flows (2022)
- An Adaptive News-Driven Method for CVaR-sensitive Online Portfolio Selection in Non-Stationary Financial Markets. (2021)
- Relation-aware Transformer for Portfolio Policy Learning (2021)
- Cost-sensitive Portfolio Selection via Deep Reinforcement Learning (2020)
- Reinforcement-learning Based Portfolio Management with Augmented Asset Movement Prediction States (2020)
- Safe Reinforcement Learning via Shielding (2018)
- End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-critical Continuous Control Tasks (2019)
- Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learning (2022)
- Online Portfolio Selection: A Survey (2014)
- A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
- Control Barrier Functions: Theory and Applications (2019)
- Control Barrier Function based Quadratic Programs with Application to Adaptive Cruise Control (2014)
- On Overfitting and Asymptotic Bias in Batch Reinforcement Learning with Partial Observability (2019)
- Adaptive Quantitative Trading: An Imitative Deep Reinforcement Learning Approach (2020)
- Set Invariance in Control (1999)
- Universal Portfolios (1991)
- On-line Portfolio Selection Using Multiplicative Updates (1998)
- On-Line Portfolio Selection with Moving Average Reversion (2012)
- PAMR: Passive Aggressive Mean Reversion Strategy for Portfolio Selection (2012)
- Corn: Correlation-driven Nonparametric Learning Approach for Portfolio Selection (2011)
- Addressing Function Approximation Error in Actor-critic Methods (2018)
- TD3-Based Ensemble Reinforcement Learning for Financial Portfolio Optimisation (2021)
- Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces (1997)
- Particle swarm optimization (1995)
- Stable-baselines3: Reliable Reinforcement Learning Implementations (2021)
- A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms (2011)
- FinBrain: when finance meets AI 2.0 (2019)
- A bat-neural network multi-agent system (BNNMAS) for stock price prediction: Case study of DAX stock price (2015)
- An agent-based co-evolutionary multi-objective algorithm for portfolio optimization (2017)
- Bi-criteria portfolio optimization models with percentile and symmetric risk measures by mathematical programming (2012)
- Moving averages for financial data smoothing (2013)
- Financial Forecast for the Relative Strength Index (2010)
- Profiling high-frequency equity price movements in directional changes (2017)
- Forecasting directional changes in the fx markets (2016)
- Long short-term memory (1997)
- DeepTrader: a deep reinforcement learning approach for risk-return balanced portfolio management with market conditions Embedding (2021)
- Robust median reversion strategy for online portfolio selection (2016)
- CVXOPT: Convex optimization (2020)
- CVXPY : A P ython-embedded modeling language for convex optimization (2016)
- A rewriting system for convex optimization problems (2018)
0 of 63 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.