Deep Reinforcement Learning Algorithms for Option Hedging

ArXiv ID: 2504.05521 “View on arXiv”

Authors: Unknown

Abstract

Dynamic hedging is a financial strategy that consists in periodically transacting one or multiple financial assets to offset the risk associated with a correlated liability. Deep Reinforcement Learning (DRL) algorithms have been used to find optimal solutions to dynamic hedging problems by framing them as sequential decision-making problems. However, most previous work assesses the performance of only one or two DRL algorithms, making an objective comparison across algorithms difficult. In this paper, we compare the performance of eight DRL algorithms in the context of dynamic hedging; Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), along with four variants of Deep Q-Learning (DQL) and two variants of Deep Deterministic Policy Gradient (DDPG). Two of these variants represent a novel application to the task of dynamic hedging. In our experiments, we use the Black-Scholes delta hedge as a baseline and simulate the dataset using a GJR-GARCH(1,1) model. Results show that MCPG, followed by PPO, obtain the best performance in terms of the root semi-quadratic penalty. Moreover, MCPG is the only algorithm to outperform the Black-Scholes delta hedge baseline with the allotted computational budget, possibly due to the sparsity of rewards in our environment.

Keywords: Deep Reinforcement Learning (DRL), Dynamic Hedging, Monte Carlo Policy Gradient (MCPG), Proximal Policy Optimization (PPO), GJR-GARCH, Equity Derivatives

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematical concepts like GJR-GARCH modeling, risk measures (RSQP), and Bellman equations, while providing rigorous empirical evaluation with code and dataset availability but limited to simulated data without real-world trading implementation.

  flowchart TD
    A["Research Goal: Compare performance of eight DRL algorithms in dynamic hedging"] --> B["Data: Simulated dataset using GJR-GARCH(1,1) model"]
    B --> C["Methodology: Apply eight DRL algorithms to hedging problem<br/>(MCPG, PPO, 4x DQL, 2x DDPG)"]
    C --> D["Baseline: Black-Scholes delta hedge"]
    C --> E["Computational Process: Train & evaluate algorithms using Root Semi-Quadratic Penalty metric"]
    D --> E
    E --> F["Key Findings:<br/>1. MCPG & PPO perform best<br/>2. MCPG is the only algorithm to outperform baseline<br/>3. Success linked to sparse reward environment"]
    F --> G["Outcome: Validated MCPG effectiveness for dynamic hedging"]