Paper: arXiv 2407.14486
Authors: Alejandra de la Rica Escudero, Eduardo C. Garrido-Merchan, Maria Coronado-Vaca
Abstract
Financial portfolio management investment policies computed quantitatively by modern portfolio theory techniques like the Markowitz model rely on a set on assumptions that are not supported by data in high volatility markets. Hence, quantitative researchers are looking for alternative models to tackle this problem. Concretely, portfolio management is a problem that has been successfully addressed recently by Deep Reinforcement Learning (DRL) approaches. In particular, DRL algorithms train an agent by estimating the distribution of the expected reward of every action performed by an agent given any financial state in a simulator. However, these methods rely on Deep Neural Networks model to represent such a distribution, that although they are universal approximator models, they cannot explain its behaviour, given by a set of parameters that are not interpretable. Critically, financial investors policies require predictions to be interpretable, so DRL agents are not suited to follow a particular policy or explain their actions. In this work, we developed a novel Explainable Deep Reinforcement Learning (XDRL) approach for portfolio management, integrating the Proximal Policy Optimization (PPO) with the model agnostic explainable techniques of feature importance, SHAP and LIME to enhance transparency in prediction time. By executing our methodology, we can interpret in prediction time the actions of the agent to assess whether they follow the requisites of an investment policy or to assess the risk of following the agent suggestions. To the best of our knowledge, our proposed approach is the first explainable post hoc portfolio management financial policy of a DRL agent. We empirically illustrate our methodology by successfully identifying key features influencing investment decisions, which demonstrate the ability to explain the agent actions in prediction time.
Complexity vs Empirical Score
- Math Complexity: 6.5/10
- Empirical Rigor: 3.5/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper employs advanced deep reinforcement learning (PPO) and XAI techniques (SHAP, LIME), requiring significant mathematical complexity. However, the empirical validation appears theoretical and lacks concrete backtesting metrics, code, or implementation details, placing it in the theoretical research quadrant.
Research Flowchart
flowchart TD A["Research Goal:<br>Explainable DRL Portfolio Management"] --> B["Data Input:<br>Financial Market Simulator"] B --> C["Core Methodology:<br>Explainable XDRL<br>PPO + SHAP/LIME"] C --> D["Computational Process:<br>Training Agent &<br>Generating Explanations"] D --> E["Key Outcome 1:<br>Identified Key Features<br>for Investment Decisions"] D --> F["Key Outcome 2:<br>Post-hoc Interpretability<br>of DRL Actions"] E --> G["Conclusion:<br>Validated Transparent &<br>Explainable Financial Policy"] F --> G
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a novel Explainable Deep Reinforcement Learning (XDRL) framework for portfolio management that combines PPO with model‑agnostic XAI methods such as feature importance, SHAP and LIME.
Source sentence
we introduce a novel Explainable Deep Reinforcement Learning (XDRL) framework for portfolio management. Concretely, our approach combines the popular Proximal Policy Optimization (PPO) algorithm [20], a state‑of‑the‑art DRL technique, with model‑agnostic XAI methods such as feature importance [21], SHAP (SHapley Additi
Assumes 1
- The market can be perfectly explained by technical data. data
Source sentence
In this work, we assume that the market can be perfectly explained by technical data, focusing hence only in this kind of data.
Data
No dataset named in the text.
Costs
Unclear
Finds 3
- The authors successfully used SHAP, LIME and feature importance to explain the decisions made by their DRL model in portfolio management tasks. other
Source sentence
In this work, we successfully used SHAP, LIME and feature importance to explain the decisions made by our DRL model in portfolio management tasks.
- Key features influencing investment decisions were identified, e.g., the Apple closing price was the most important for the estimated policy. other
Source sentence
We can see how the APPLE close value is the most important for the estimated policy of the DRL agent.
- The three explainability techniques can be implemented independently or jointly, allowing explanation of DRL agent predictions in trading time. other
Source sentence
the three explainability techniques can be implemented independently or jointly, being able to explain the DRL agent predictions in trading time, being able to track throughout the time whether the policy is acting as it is expected or not
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 9 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (59)
- Proximal policy optimization algorithms (2017) — link
- Principles and practice of explainable machine learning (2021)
- The shapley value in machine learning (2022) — link
- Statistical stability indices for LIME: Obtaining reliable explanations for machine learning models (2022)
- Explainable artificial intelligence: an analytical review (2021)
- Intelligible and explainable machine learning: Best practices and practical challenges (2020)
- Deep reinforcement learning: A brief survey (2017)
- Reinforcement learning: An introduction (2018)
and 51 more
- Reinforcement learning: An introduction, 2nd ed. (2020)
- Openai gym (2016) — link
- Deep learning (2015)
- A universal approximation theorem of deep neural networks for expressing probability distributions (2020)
- Can we open the black box of AI ? (2016)
- Financial transparency and corporate governance: you manage what you measure (1996)
- The basics of finance: An introduction to financial markets, business finance, and portfolio management (2010)
- A simplified perspective of the Markowitz portfolio theory (2013)
- True Markowitz or assumptions we break and why it matters (2012)
- Stock market prediction and Portfolio selection models: a survey (2017)
- The different faces of volatility exposure in portfolio management (2013)
- Recent advances in reinforcement learning in finance (2023)
- Human-level control through deep reinforcement learning (2015)
- Policy gradient methods for reinforcement learning with function approximation (1999)
- Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
- Twin-delayed ddpg: A deep reinforcement learning technique to model a continuous movement of an intelligent robot agent (2019)
- A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
- Cryptocurrency portfolio management with deep reinforcement learning (2017)
- Continuous control with deep reinforcement learning (2015) — link
- Deep reinforcement learning for the control of robotic manipulation: a focussed mini-review (2021)
- A survey of deep reinforcement learning in video games (2019) — link
- Explainable deep reinforcement learning: state of the art and challenges (2022)
- Adversarial deep reinforcement learning in portfolio management (2018) — link
- Deep reinforcement learning in cryptocurrency market making (2019) — link
- Recommending cryptocurrency trading points with deep reinforcement learning approach (2020)
- Deep reinforcement learning for trading (2019) — link
- Deep hedging: hedging derivatives under generic market frictions using reinforcement learning (2019)
- Deep hedging of derivatives using reinforcement learning (2021) — link
- Deep hedging of long-term financial derivatives (2021)
- Multi-agent reinforcement learning approach for hedging portfolio problem (2021)
- Modelling stock markets by multi-agent reinforcement learning (2021)
- MSPM: A modularized and scalable multi-agent reinforcement learning-based system for financial portfolio management (2022)
- MAPS: Multi-agent reinforcement learning-based portfolio management system (2020) — link
- Cost-sensitive portfolio selection via deep reinforcement learning (2020)
- GPM: A graph convolutional network based reinforcement learning framework for portfolio management (2022)
- Stock portfolio management by using fuzzy ensemble deep reinforcement learning algorithm (2023)
- Deep reinforcement learning for stock portfolio optimization by connecting with modern portfolio theory (2023)
- Market sentiment-aware deep reinforcement learning approach for stock portfolio allocation (2021)
- Optimistic bull or pessimistic bear: Adaptive deep reinforcement learning for stock portfolio allocation (2019) — link
- Leveraging Deep Learning and Online Source Sentiment for Financial Portfolio Management — → 2309.16679
- Stock trading bot using deep reinforcement learning (2019)
- FinRL: Deep reinforcement learning framework to automate trading in quantitative finance (2021)
- FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance (2020) — link
- Practical deep reinforcement learning approach for stock trading (2018) — link
- Explainable deep reinforcement learning for portfolio management: an empirical approach (2021)
- Towards interpretable reinforcement learning with state abstraction driven by external knowledge (2020)
- X PM : An explainable deep reinforcement learning framework for portfolio management (2021)
- Alphastock: A buying-winners-and-selling-losers investment strategy using interpretable deep reinforcement attention networks (2019)
- Explainable AI: Foundations, methodologies and applications (2023)
- A unified approach to interpreting model predictions (2017)
- " W hy should I trust you?" Explaining the predictions of any classifier (2016)
1 of 59 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.