Paper: arXiv 2504.05521
Authors: Andrei Neagu, Frédéric Godin, Leila Kosseim
Abstract
Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing them as sequential decision-making problems. However, most previous work assesses the performance of only one or two DRL algorithms, making an objective comparison across algorithms difficult. In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a novel application to the task of dynamic hedging. In our experiments, we use the Black-Scholes delta hedge as a baseline and simulate the dataset using a GJR-GARCH(1,1) model. Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi-quadratic penalty. Moreover, MCPG is the only algorithm to outperform the Black-Scholes delta hedge baseline with the allotted computational budget, possibly due to the sparsity of rewards in our environment.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced mathematical concepts like GJR-GARCH modeling, risk measures (RSQP), and Bellman equations, while providing rigorous empirical evaluation with code and dataset availability but limited to simulated data without real-world trading implementation.
Research Flowchart
flowchart TD A["Research Goal: Compare performance of eight DRL algorithms in dynamic hedging"] --> B["Data: Simulated dataset using GJR-GARCH(1,1) model"] B --> C["Methodology: Apply eight DRL algorithms to hedging problem<br>(MCPG, PPO, 4x DQL, 2x DDPG)"] C --> D["Baseline: Black-Scholes delta hedge"] C --> E["Computational Process: Train & evaluate algorithms using Root Semi-Quadratic Penalty metric"] D --> E E --> F["Key Findings:<br>1. MCPG & PPO perform best<br>2. MCPG is the only algorithm to outperform baseline<br>3. Success linked to sparse reward environment"] F --> G["Outcome: Validated MCPG effectiveness for dynamic hedging"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper provides a benchmark comparison of eight deep reinforcement learning algorithms for dynamic option hedging, including two novel variants.
Source sentence
In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a
Assumes 6
- Log‑returns follow a GJR‑GARCH(1,1) process with parameters estimated from monthly S&P 500 data. model
Source sentence
are learned through maximum likelihood estimates from monthly S&P 500 prices from 2000/11/15 to 2024/10/15.
- Risk‑free rate is set to zero. market
Source sentence
The risk-free rate is set to rf = 0.
- The hedging environment provides a single reward only at the final time step (sparse rewards). execution
Source sentence
the formulation of the option hedging problem leads to an environment in which rewards are sparse, where only a single reward is obtained from the environment at the final time step.
- Baseline Black‑Scholes delta hedge assumes no transaction costs and continuous‑time trading under geometric Brownian motion. model
Source sentence
In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.
Show 2 more
- Training is performed on simulated price paths rather than real market execution. data
Source sentence
To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7...
- Hyperparameter search is limited due to computational constraints. other
Source sentence
The main limitation common to most DRL algorithms and also present in our work is their heavy computational load coupled with their dependence on hyperparameters, forcing the hyperparameter search to be narrow.
Data
- Simulated price paths (GJR‑GARCH) synthetic public
Source sentence
To evaluate the performance of the different DRL algorithms, we trained them on simulated price paths following Eq. 7, where the parameters of the GJR‑GARCH(1,1) model
Costs
Not modeled
Source sentence
In the absence of transaction costs, in continuous time, and for market dynamics following a geometric Brownian motion, this procedure is shown to completely eliminate risk.
Finds 5
- Monte Carlo Policy Gradient (MCPG) achieves the lowest RSQP (0.8111) and is the only algorithm to outperform the Black‑Scholes delta hedge baseline (0.9038) within the computational budget. performance
Source sentence
MCPG is the only algorithm that was capable of significantly outperforming the B‑S DH baseline in the allotted computational budget (0.8111 vs 0.9038).
- Proximal Policy Optimization (PPO) follows MCPG in performance but does not beat the baseline. performance
Source sentence
Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi‑quadratic penalty.
- MCPG’s training time is more than fourteen times smaller than that of the next best algorithm. performance
Source sentence
Additionally, the training time of MCPG is more than fourteen times smaller than the closest competitor.
- Value‑based algorithms (DQL, DDPG variants) perform poorly due to sparse rewards and do not improve over the baseline. negative
Source sentence
the formulation of the option hedging problem leads to an environment in which rewards are sparse.
- TD3 marginally lowers RSQP compared to vanilla DDPG but still fails to beat the baseline and requires significantly more training time. performance
Source sentence
This seems to only marginally lower the RSQP over vanilla DDPG (1.0113 vs 1.0467)
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 51 related papers). Coverage is partial, so this is neither replication nor refutation.
2 papers in the archive : Application of Deep Reinforcement Learning to At-the-Money …
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (79)
- Optimal execution of portfolio transactions (2001)
- Distributed Distributional Deterministic Policy Gradients (2018) — link
- QLBS : Q -learner in the B lack- S choles (- M erton) worlds (2020)
- Advanced Option Pricing and Hedging with Q -Learning: Performance Evaluation of the QLBS Algorithm (2025)
- Deterministic policy gradient algorithms (2014)
- A Distributional Perspective on Reinforcement Learning (2017) — link
- Deep Hedging (2019)
- The pricing of options and corporate liabilities (1973)
and 71 more
- Gamma and vega hedging using deep distributional reinforcement learning (2023)
- Deep Hedging of Derivatives Using Reinforcement Learning (2020)
- Deep Hedging of Long-Term Financial Derivatives (2021)
- Deep equal risk pricing of financial derivatives with non-translation invariant risk measures (2023)
- Equal risk pricing of derivatives with deep hedging (2021)
- Deep Equal Risk Pricing of Financial Derivatives with Multiple Hedging Instruments (2024)
- Pseudo-Model-Free Hedging for Variable Annuities via Deep Reinforcement Learning (2023)
- Empirical properties of asset returns: stylized facts and statistical issues (2001)
- DEEP EXECUTION - VALUE AND POLICY BASED REINFORCEMENT LEARNING FOR TRADING AND BEATING MARKET BENCHMARKS (2019) — link
- Deep Reinforcement Learning for Option Replication and Hedging (2020)
- Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regret (2020)
- Addressing Function Approximation Error in Actor-Critic Methods (2018)
- On the Relation between the Expected Value and the Volatility of the Nominal Excess Return on Stocks (1993) — link
- Deep Learning (2016)
- QLBS: Q-Learner in the B lack- S choles(- M erton) Worlds (2020) — link
- Recent Advances in Reinforcement Learning in Finance (2023) — link
- A New Approach to the Economic Analysis of Nonstationary Time Series and the Business Cycle (1989)
- A reinforcement learning extension to the A lmgren- C hriss framework for optimal trade execution (2014) — link
- A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (1993)
- No-Transacion Band Network: A Neural Network Architecture for Efficient Deep Hedging (2023)
- Improving financial trading decisions using deep Q-learning: Predicting the number of shares, action strategies, and transfer learning (2019) — link
- A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
- Cryptocurrency portfolio management with deep reinforcement learning. (2017)
- Portfolio Management using Reinforcement Learning (2016)
- A new interpretation of information rate (1956)
- Online portfolio selection: A survey (2014)
- OLPS : A Toolbox for On-Line Portfolio Selection (2016)
- Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
- Continuous control with deep reinforcement learning (2016)
- Optimal Liquidation with Deep Reinforcement Learning (2019)
- An End-to-End Optimal Trade Execution Framework based on Proximal Policy Optimization (2020) — link
- Practical Deep Reinforcement Learning Approach for Stock Trading (2018)
- Computational Finance: A Scientific Perspective (2000) — link
- Portfolio Selection (1952)
- The Optimization of a Quadratic Function Subject to Linear Constraints (1956)
- Portfolio Selection: Efficient Diversification of Investments (1959)
- Deep reinforcement learning for option pricing and hedging under dynamic expectile risk measures (2023) — link
- Convergence of Q -learning: a simple proof (2001)
- On estimating the expected return on the market: An exploratory investigation (1980)
- Option pricing when underlying stock returns are discontinuous (1976) — link
- Empirical deep hedging (2023) — link
- Playing A tari with Deep Reinforcement Learning (2013) — link
- Human-level control through deep reinforcement learning (2015)
- Deep Hedging with Market Impact — → 2402.13326
- Deep Reinforcement Learning Algorithms for Financial Decision-Making (2024)
- Kelly C riterion for Multivariate Portfolios: A Model-Free Approach (2014)
- Double Deep Q -learning for Optimal Execution (2021)
- OpenAI Spinning Up: Deep Deterministic Policy Gradient (2024)
- OpenAI Spinning Up: Proximal Policy Optimization (2024)
- OpenAI Spinning Up: Trust Region Policy Optimization (2024)
- An intelligent financial portfolio trading strategy using deep Q-learning (2020) — link
- Model-Free Reinforcement Learning for Financial Portfolios: A Brief Survey (2019) — link
- Trust Region Policy Optimization (2015) — link
- Proximal Policy Optimization Algorithms (2017) — link
- Hedging Beyond the Mean: A Distributional Reinforcement Learning Perspective for Hedging Portfolios with Structured Products (2024) — link
- Stochastic Calculus for Finance II : Continuous-Time Models (2004)
- Deterministic policy gradient algorithms (2014)
- Deep Reinforcement Learning for Optimal Portfolio Allocation: A Comparative Study with Mean-Variance Optimization (2023) — link
- A parsimonious neural network approach to solve portfolio optimization problems without using dynamic programming — → 2303.08968
- Reinforcement Learning: An Introduction (2018 )
- Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999) — link
- Deep hedging under rough volatility (2021)
- The K elly C riterion in Blackjack, Sports Betting and the Stock Market (2006)
- Deep Reinforcement Learning with Double Q -learning (2015) — link
- Option hedging with risk averse reinforcement learning (2021) — link
- Dueling network architectures for deep reinforcement learning (2016)
- An Asymptotic Analysis of an Optimal Hedging Model for Option Pricing with Transaction Costs (2002) — link
- Robust Risk-Aware Option Hedging — → 2303.15216
- Optimal Trade Execution Based on Deep Deterministic Policy Gradient (2020)
- Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization (2019) — link
- Deep Reinforcement Learning for Trading (2020)
3 of 79 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.