Paper: arXiv 2504.05521

Authors: Andrei Neagu, Frédéric Godin, Leila Kosseim

Abstract

Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing them as sequential decision-making problems. However, most previous work assesses the performance of only one or two DRL algorithms, making an objective comparison across algorithms difficult. In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a novel application to the task of dynamic hedging. In our experiments, we use the Black-Scholes delta hedge as a baseline and simulate the dataset using a GJR-GARCH(1,1) model. Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi-quadratic penalty. Moreover, MCPG is the only algorithm to outperform the Black-Scholes delta hedge baseline with the allotted computational budget, possibly due to the sparsity of rewards in our environment.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematical concepts like GJR-GARCH modeling, risk measures (RSQP), and Bellman equations, while providing rigorous empirical evaluation with code and dataset availability but limited to simulated data without real-world trading implementation.

Research Flowchart

  flowchart TD
  A["Research Goal: Compare performance of eight DRL algorithms in dynamic hedging"] --> B["Data: Simulated dataset using GJR-GARCH(1,1) model"]
  B --> C["Methodology: Apply eight DRL algorithms to hedging problem<br>(MCPG, PPO, 4x DQL, 2x DDPG)"]
  C --> D["Baseline: Black-Scholes delta hedge"]
  C --> E["Computational Process: Train & evaluate algorithms using Root Semi-Quadratic Penalty metric"]
  D --> E
  E --> F["Key Findings:<br>1. MCPG & PPO perform best<br>2. MCPG is the only algorithm to outperform baseline<br>3. Success linked to sparse reward environment"]
  F --> G["Outcome: Validated MCPG effectiveness for dynamic hedging"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper provides a benchmark comparison of eight deep reinforcement learning algorithms for dynamic option hedging, including two novel variants.

Source sentence
In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a

Assumes 6

  • Log‑returns follow a GJR‑GARCH(1,1) process with parameters estimated from monthly S&P 500 data. model
    Source sentence
    are learned through maximum likelihood estimates from monthly S&P 500 prices from 2000/11/15 to 2024/10/15.
  • Risk‑free rate is set to zero. market
    Source sentence
    The risk-free rate is set to rf = 0.
  • The hedging environment provides a single reward only at the final time step (sparse rewards). execution
    Source sentence
    the formulation of the option hedging problem leads to an environment in which rewards are sparse, where only a single reward is obtained from the environment at the final time step.
  • Baseline Black‑Scholes delta hedge assumes no transaction costs and continuous‑time trading under geometric Brownian motion. model
    Source sentence
    In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.
Show 2 more
  • Training is performed on simulated price paths rather than real market execution. data
    Source sentence
    To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7...
  • Hyperparameter search is limited due to computational constraints. other
    Source sentence
    The main limitation common to most DRL algorithms and also present in our work is their heavy computational load coupled with their dependence on hyperparameters, forcing the hyperparameter search to be narrow.

Data

  • Simulated price paths (GJR‑GARCH) synthetic public
    2000/11/15 to 2024/10/15 (parameter esti · S&P 500 (monthly prices for parameter estimation)
    Source sentence
    To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7, where the parameters of the GJR‑GARCH(1,1) model

Costs

Not modeled A trading or execution result is reported without costs.

Source sentence
In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.

Finds 5

  • Monte Carlo Policy Gradient (MCPG) achieves the lowest RSQP (0.8111) and is the only algorithm to outperform the Black‑Scholes delta hedge baseline (0.9038) within the computational budget. performance
    Source sentence
    MCPG is the only algorithm that was capable of significantly outperforming the B‑S DH baseline in the allotted computational budget (0.8111 vs 0.9038).
  • Proximal Policy Optimization (PPO) follows MCPG in performance but does not beat the baseline. performance
    Source sentence
    Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi‑quadratic penalty.
  • MCPG’s training time is more than fourteen times smaller than that of the next best algorithm. performance
    Source sentence
    Additionally, the training time of MCPG is more than fourteen times smaller than the closest competitor.
  • Value‑based algorithms (DQL, DDPG variants) perform poorly due to sparse rewards and do not improve over the baseline. negative
    Source sentence
    the formulation of the option hedging problem leads to an environment in which rewards are sparse.
  • TD3 marginally lowers RSQP compared to vanilla DDPG but still fails to beat the baseline and requires significantly more training time. performance
    Source sentence
    This seems to only marginally lower the RSQP over vanilla DDPG (1.0113 vs 1.0467)

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 51 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (1 by independent authors): Application of Deep Reinforcement Learning to At-the-Money … (discusses its findings)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (79)

  1. Optimal execution of portfolio transactions (2001)
  2. Distributed Distributional Deterministic Policy Gradients (2018) — link
  3. QLBS : Q -learner in the B lack- S choles (- M erton) worlds (2020)
  4. Advanced Option Pricing and Hedging with Q -Learning: Performance Evaluation of the QLBS Algorithm (2025)
  5. Deterministic policy gradient algorithms (2014)
  6. A Distributional Perspective on Reinforcement Learning (2017) — link
  7. Deep Hedging (2019)
  8. The pricing of options and corporate liabilities (1973)
and 71 more
  1. Gamma and vega hedging using deep distributional reinforcement learning (2023)
  2. Deep Hedging of Derivatives Using Reinforcement Learning (2020)
  3. Deep Hedging of Long-Term Financial Derivatives (2021)
  4. Deep equal risk pricing of financial derivatives with non-translation invariant risk measures (2023)
  5. Equal risk pricing of derivatives with deep hedging (2021)
  6. Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments (2024)
  7. Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning (2023)
  8. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  9. DEEP EXECUTION - VALUE AND POLICY BASED REINFORCEMENT LEARNING FOR TRADING AND BEATING MARKET BENCHMARKS (2019) — link
  10. Deep Reinforcement Learning for Option Replication and Hedging (2020)
  11. Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret (2020)
  12. Addressing Function Approximation Error in Actor-Critic Methods (2018)
  13. On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks (1993) — link
  14. Deep Learning (2016)
  15. QLBS: Q-Learner in the B lack- S choles(- M erton) Worlds (2020) — link
  16. Recent Advances in Reinforcement Learning in Finance (2023) — link
  17. A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle (1989)
  18. A reinforcement learning extension to the A lmgren- C hriss framework for optimal trade execution (2014) — link
  19. A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993)
  20. No-Transacion Band Network: A Neural Network Architecture for Efficient Deep Hedging (2023)
  21. Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019) — link
  22. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  23. Cryptocurrency portfolio management with deep reinforcement learning. (2017)
  24. Portfolio Management using Reinforcement Learning (2016)
  25. A new interpretation of information rate (1956)
  26. Online portfolio selection: A survey (2014)
  27. OLPS : A Toolbox for On-Line Portfolio Selection (2016)
  28. Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
  29. Continuous control with deep reinforcement learning (2016)
  30. Optimal Liquidation with Deep Reinforcement Learning (2019)
  31. An End-to-End Optimal Trade Execution Framework based on Proximal Policy Optimization (2020) — link
  32. Practical Deep Reinforcement Learning Approach for Stock Trading (2018)
  33. Computational Finance: A Scientific Perspective (2000) — link
  34. Portfolio Selection (1952)
  35. The Optimization of a Quadratic Function Subject to Linear Constraints (1956)
  36. Portfolio Selection: Efficient Diversification of Investments (1959)
  37. Deep reinforcement learning for option pricing and hedging under dynamic expectile risk measures (2023) — link
  38. Convergence of Q -learning: a simple proof (2001)
  39. On estimating the expected return on the market: An exploratory investigation (1980)
  40. Option pricing when underlying stock returns are discontinuous (1976) — link
  41. Empirical deep hedging (2023) — link
  42. Playing A tari with Deep Reinforcement Learning (2013) — link
  43. Human-level control through deep reinforcement learning (2015)
  44. Deep Hedging with Market Impact — → 2402.13326
  45. Deep Reinforcement Learning Algorithms for Financial Decision-Making (2024)
  46. Kelly C riterion for Multivariate Portfolios: A Model-Free Approach (2014)
  47. Double Deep Q -learning for Optimal Execution (2021)
  48. OpenAI Spinning Up: Deep Deterministic Policy Gradient (2024)
  49. OpenAI Spinning Up: Proximal Policy Optimization (2024)
  50. OpenAI Spinning Up: Trust Region Policy Optimization (2024)
  51. An intelligent financial portfolio trading strategy using deep Q-learning (2020) — link
  52. Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey (2019) — link
  53. Trust Region Policy Optimization (2015) — link
  54. Proximal Policy Optimization Algorithms (2017) — link
  55. Hedging Beyond the Mean: A Distributional Reinforcement Learning Perspective for Hedging Portfolios with Structured Products (2024) — link
  56. Stochastic Calculus for Finance II : Continuous-Time Models (2004)
  57. Deterministic policy gradient algorithms (2014)
  58. Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization (2023) — link
  59. A parsimonious neural network approach to solve portfolio optimization problems without using dynamic programming — → 2303.08968
  60. Reinforcement Learning: An Introduction (2018 )
  61. Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999) — link
  62. Deep hedging under rough volatility (2021)
  63. The K elly C riterion in Blackjack, Sports Betting and the Stock Market (2006)
  64. Deep Reinforcement Learning with Double Q -learning (2015) — link
  65. Option hedging with risk averse reinforcement learning (2021) — link
  66. Dueling network architectures for deep reinforcement learning (2016)
  67. An Asymptotic Analysis of an Optimal Hedging Model for Option Pricing with Transaction Costs (2002) — link
  68. Robust Risk-Aware Option Hedging — → 2303.15216
  69. Optimal Trade Execution Based on Deep Deterministic Policy Gradient (2020)
  70. Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization (2019) — link
  71. Deep Reinforcement Learning for Trading (2020)

3 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.