Paper: arXiv 2511.11481

Authors: Emmanuel Lwele, Sabuni Emmanuel, Sitali Gabriel Sitali

Abstract

This paper presents a deep reinforcement learning (DRL) framework for dynamic portfolio optimization under market uncertainty and risk. The proposed model integrates a Sharpe ratio-based reward function with direct risk control mechanisms, including maximum drawdown and volatility constraints. Proximal Policy Optimization (PPO) is employed to learn adaptive asset allocation strategies over historical financial time series. Model performance is benchmarked against mean-variance and equal-weight portfolio strategies using backtesting on high-performing equities. Results indicate that the DRL agent stabilizes volatility successfully but suffers from degraded risk-adjusted returns due to over-conservative policy convergence, highlighting the challenge of balancing exploration, return maximization, and risk mitigation. The study underscores the need for improved reward shaping and hybrid risk-aware strategies to enhance the practical deployment of DRL-based portfolio allocation models.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematics including gradient ascent derivations for Sharpe ratio optimization and constraints via softmax, while demonstrating empirical rigor with backtesting on historical data against benchmarks and risk metrics like maximum drawdown.

Research Flowchart

  flowchart TD
  A["Research Goal: DRL for Dynamic Portfolio Optimization"] --> B["Methodology: PPO with Sharpe Reward & Risk Constraints"]
  B --> C["Data: Historical High-Performing Equities"]
  C --> D{"Computational Process"}
  D --> E["Training Agent"]
  D --> F["Backtesting"]
  E --> G["Over-conservative Policy Convergence"]
  F --> H["Benchmark vs. Mean-Variance & Equal-Weight"]
  G & H --> I["Outcomes"]
  I --> J["Stable Volatility"]
  I --> K["Degraded Risk-Adjusted Returns"]
  I --> L["Need for Improved Reward Shaping"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a risk‑aware deep reinforcement learning model for dynamic portfolio optimization that uses a Sharpe‑ratio based reward and incorporates maximum drawdown and volatility constraints.

Source sentence
We introduce a risk-aware DRL model that integrates advanced reinforcement learning algorithms, a novel reward function based on the Sharpe ratio, and direct risk control strategies including maximum drawdown and portfolio volatility constraints.

Assumes 6

  • The model is trained and evaluated on historical financial time series data. data
    Source sentence
    Historical financial data was collected, including stock prices, economic indicators, and other relevant financial features.
  • Portfolio rebalancing is performed on a monthly basis. execution
    Source sentence
    Historical data was utilised for model training and backtesting, with monthly portfolio rebalancing.
  • Transaction costs are represented by a linear coefficient µ in the reward function. model
    Source sentence
    The reward at time t is based on the log of the total portfolio value, penalized by transaction costs. It is defined as: R(St−1, At−1) = ln At−1 · Yt−1 −µ
  • The agent’s actions are portfolio weight vectors that sum to one. model
    Source sentence
    The action at time t, denoted as At, is the portfolio weight vector, derived from the weights at the previous time step At−1 = Wt−1.
Show 2 more
  • Proximal Policy Optimization (PPO) can learn a stable policy from the simulated market environment. model
    Source sentence
    This neural network-based policy is optimized using PPO (Proximal Policy Optimization), allowing the agent to gradually improve its decision-making over time by interacting with the market environment simulated through historical data.
  • Regularization (L1/L2) helps mitigate overfitting of the neural network. model
    Source sentence
    We applied regularization techniques such as L1 (Lasso) and L2 (Ridge) to penalize large weights in the neural network and reduce the risk of overfitting.

Data

Backtest sample: high-performing equities · equities · daily

  • Yahoo Finance 15‑asset dataset · Yahoo Finance market-prices public
    2015-01-01 to 2024-01-01 · AAPL, AMZN, GOOG, HD, INTC, JNJ, JPM, MA, META, NFLX, NVDA, PG, TSLA, UNH, V
    Source sentence
    For this experiment, 15 high-performing assets were selected from Yahoo Finance on the basis of their historical performance over a 10-year period spanning January 1, 2015, to January 1, 2024.

Costs

Modeled Costs are part of the reported headline results.

Transaction costs are included in the reward function via a coefficient µ that penalizes changes in portfolio weights.

Source sentence
The reward at time t is based on the log of the total portfolio value, penalized by transaction costs. ... µ

Finds 2

Reported performance (Risk-Aware DRL Portfolio Allocation (PPO)): Sharpe 0.13 · annual return 2.1%

  • After training, the DRL agent’s annualized return fell to 2.10% with volatility reduced to 16.32% and Sharpe ratio dropping to 0.13, indicating a collapse in risk‑adjusted performance. performance
    Source sentence
    After training, the DRL agent’s return declined sharply to 2.10%, with volatility reduced to 16.32%. The Sharpe ratio deteriorated to 0.13, suggesting a collapse in risk‑adjusted performance.
  • The trained model succeeded in stabilizing risk by cutting volatility roughly in half compared to the pre‑training baseline (34.9% → 16.32%). performance
    Source sentence
    shift highlights a critical trade‑off: while the trained model successfully stabilized risk through volatility reduction, it failed to preserve profitability.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (26)

  1. Predicting the direction of stock market prices using tree-based classifiers (2019)
  2. Utilising digital media as a second language (L2) support: A case study on Netflix with translation applications (2020)
  3. Mean–variance portfolio optimization using machine learning-based stock price prediction (2021)
  4. Stock trading with recurrent reinforcement learning (RRL) (2016)
  5. Machine learning algorithms for time series in financial markets (2020)
  6. Deep reinforcement learning for optimal portfolio allocation: A comparative study with mean-variance optimization (2023)
  7. A Bibliometric Analysis of Artificial Intelligence Technique in Financial Market (2021) — link
  8. Multi-agent deep reinforcement learning algorithm with trend consistency regularization for portfolio management (2023)
and 18 more
  1. Performance functions and reinforcement learning for trading systems and portfolios (1998)
  2. Stock market prediction using machine learning classifiers and social media, news (2022)
  3. An efficient word embedding and deep learning based model to forecast the direction of stock exchange market using Twitter and financial news sites: a case of Istanbul stock exchange (BIST 100) (2020)
  4. A Comparative Study of SVM and LSTM Deep Learning Algorithms for Stock Market Prediction. (2019)
  5. URLLC-awared resource allocation for heterogeneous vehicular edge computing (2024)
  6. Deep learning-based feature engineering for stock price movement prediction (2019)
  7. Machine learning-aided modeling of fixed income instruments (2018)
  8. Mobilenet convolutional neural networks and support vector machines for palmprint recognition (2019)
  9. Big data analysis in stock market prediction (2019)
  10. Stock price prediction using news sentiment analysis (2019)
  11. A survey on machine learning for stock price prediction: Algorithms and techniques (2020)
  12. Predicting the stock market index using stochastic time series ARIMA modelling: The sample of BSE and NSE (2019)
  13. Stock market prediction using machine learning (2018)
  14. Forecasting stock market movement direction using sentiment analysis and support vector machine (2018)
  15. Risk-Aware Financial Portfolio Management with Distributional Deep Deterministic Policy Gradient (2021)
  16. Predicting the daily return direction of the stock market using hybrid machine learning algorithms (2019)
  17. Risk-Sensitive Portfolio Management by using Distributional Reinforcement Learning (2019) — link
  18. Distributional reinforcement learning for risk-sensitive policies (2022)

0 of 26 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.