Hedging American Put Options with Deep Reinforcement Learning

Paper: arXiv 2405.06774

Authors: Reilly Pickard, Finn Wredenhagen, Julio DeJesus, Mario Schlener, Yuri Lawryshyn

Abstract

This article leverages deep reinforcement learning (DRL) to hedge American put options, utilizing the deep deterministic policy gradient (DDPG) method. The agents are first trained and tested with Geometric Brownian Motion (GBM) asset paths and demonstrate superior performance over traditional strategies like the Black-Scholes (BS) Delta, particularly in the presence of transaction costs. To assess the real-world applicability of DRL hedging, a second round of experiments uses a market calibrated stochastic volatility model to train DRL agents. Specifically, 80 put options across 8 symbols are collected, stochastic volatility model coefficients are calibrated for each symbol, and a DRL agent is trained for each of the 80 options by simulating paths of the respective calibrated model. Not only do DRL agents outperform the BS Delta method when testing is conducted using the same calibrated stochastic volatility model data from training, but DRL agents achieves better results when hedging the true asset path that occurred between the option sale date and the maturity. As such, not only does this study present the first DRL agents tailored for American put option hedging, but results on both simulated and empirical market testing data also suggest the optimality of DRL agents over the BS Delta method in real-world scenarios. Finally, note that this study employs a model-agnostic Chebyshev interpolation method to provide DRL agents with option prices at each time step when a stochastic volatility model is used, thereby providing a general framework for an easy extension to more complex underlying asset processes.

Complexity vs Empirical Score

• Math Complexity: 6.5/10
• Empirical Rigor: 7.5/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a strong practical application of Deep Deterministic Policy Gradient (DDPG) reinforcement learning to American put option hedging, incorporating realistic friction such as transaction costs. The methodology is empirically validated using both stochastic volatility models and real market data across multiple underlying options, showing robust real-world performance superior to standard Black-Scholes delta hedging.

Research Flowchart

  flowchart TD
    A["Research Goal: Train DRL Agents (DDPG) to Hedge American Put Options Effectively"] --> B["Data & Inputs: Market Options Data (80 Puts, 8 Symbols) & GBM/Stochastic Volatility Models"]
    B --> C["Price Options via Chebyshev Interpolation Method"]
    C --> D["Train DDPG DRL Agents on Simulated Asset Paths with Transaction Costs"]
    D --> E["Evaluate Hedging Performance against Traditional Black-Scholes Delta Benchmark"]
    E --> F["Key Findings: DRL Outperforms BS Delta on both Simulated & Real-World Empirical Market Data"]

Cited References (32)

1. Option Dynamic Hedging Using Reinforcement Learning (2023) — link
2. A Review on Derivative Hedging Using Reinforcement Learning (2023) — link
3. Empirical deep hedging (2022) — link
4. Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions (2022) — link
5. Gamma and vega hedging using deep distributional reinforcement learning (2022) — link
6. Delta-Gamma-Like Hedging with Transaction Cost under Reinforcement Learning Technique (2022) — link
7. Optimal Option Hedging with Policy Gradient (2021) — link
8. Multi-agent reinforcement learning approach for hedging portfolio problem (2021) — link

0 of 32 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.