Paper: arXiv 2611.10956

Authors: Jacky Chen, Ing-Haw Cheng, John Hull, Landis Li, Zissis Poulos, Justin Shi, Andreas Veneris, Zeyu Wang

Abstract

Options market makers absorb client order flow and must hedge the resulting inventory risk under trading frictions. We study whether an option hedging reinforcement learning (RL) agent trained on historical market data outperforms three alternatives in tail-risk protection: (i) traditional delta hedging, (ii) simple hedge-ratio rules, and (iii) the same agent trained in simulated markets. Most RL hedging agents are trained in model-generated markets that do not capture the crashes and volatility regimes present in real data. We introduce an empirical replay environment using S&P 500 option data from 1996 to 2017 and compare against a matched geometric Brownian motion (GBM) environment, a stochastic-volatility (SABR) simulator, and a hybrid that values historical paths with Black-Scholes, all sharing the same portfolio mechanics, stochastic arrivals, costs, and test period. We train a distributional Twin Delayed Deep Deterministic Policy Gradient (TD3) agent and evaluate it out of sample over 2018-2023. The agent trained on empirical data reduces Conditional Value-at-Risk (CVaR) tail loss by 3.1% compared to delta hedging and outperforms every hedge-ratio rule chosen during the training period and every alternative training environment. The reduction reaches 8.9% in volatile and falling markets, but reverses to a cost of 7-10% in rising markets. Empirical training induces a state-dependent short net delta: the policy under-hedges negative-delta books, most strongly after prices have fallen. This tilt disappears with GBM training and is weaker with SABR or hybrid training. These findings suggest that exposure to historical market dynamics can materially change a tail-risk hedging policy, although the evidence rests on a single test path and the edge is regime-conditional.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a robust empirical study of RL for options market making, leveraging real S&P 500 options data. The methodology is well-defined, and the comparison across different training environments adds significant value, demonstrating strong empirical rigor and a novel approach to training RL agents in finance.

Research Flowchart

  flowchart TD
    A[Research Goal: Outperform traditional hedging in tail-risk protection for options market makers using RL] --> B{Methodology: Compare RL agent to alternatives};
    B --> C[Data/Inputs: Historical S&P 500 Options (1996-2017), Real S&P 500 Test Data (2018-2023), Simulated GBM/SABR/Hybrid Markets];
    C --> D[Computational Process: Train Distributional TD3 RL Agent in Empirical & Simulated Environments];
    D --> E[Computational Process: Evaluate Out-of-Sample Performance (2018-2023) against Delta Hedging & Hedge-Ratio Rules, Measure CVaR];
    E --> F{Key Finding 1: Empirical RL Reduces CVaR Tail Loss by 3.1% overall (8.9% in volatile/falling markets) vs. Delta Hedging};
    F --> G[Key Finding 2: Empirical RL induces state-dependent short net delta (under-hedges negative-delta books after price falls), absent/weaker in simulated training];
    G --> H[Key Finding 3: Edge is regime-conditional (costly in rising markets)];