Reinforcement Learning and Deep Stochastic Optimal Control for Final Quadratic Hedging

ArXiv ID: 2401.08600 “View on arXiv”

Authors: Unknown

Abstract

We consider two data driven approaches, Reinforcement Learning (RL) and Deep Trajectory-based Stochastic Optimal Control (DTSOC) for hedging a European call option without and with transaction cost according to a quadratic hedging P&L objective at maturity (“variance-optimal hedging” or “final quadratic hedging”). We study the performance of the two approaches under various market environments (modeled via the Black-Scholes and/or the log-normal SABR model) to understand their advantages and limitations. Without transaction costs and in the Black-Scholes model, both approaches match the performance of the variance-optimal Delta hedge. In the log-normal SABR model without transaction costs, they match the performance of the variance-optimal Barlett’s Delta hedge. Agents trained on Black-Scholes trajectories with matching initial volatility but used on SABR trajectories match the performance of Bartlett’s Delta hedge in average cost, but show substantially wider variance. To apply RL approaches to these problems, P&L at maturity is written as sum of step-wise contributions and variants of RL algorithms are implemented and used that minimize expectation of second moments of such sums.

Keywords: Reinforcement Learning, Stochastic Optimal Control, Variance-optimal hedging, SABR model, Transaction costs, Derivatives (Options)

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 7.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematical constructs like stochastic optimal control and quadratic hedging P&L objectives, requiring dense derivations and heavy LaTeX formulas. It is empirically rigorous with data-driven methods, simulating hedging under Black-Scholes and SABR models with transaction costs, and includes comparative performance analysis of RL and DTSOC agents, though it lacks real-market backtests.

  flowchart TD
    A["Research Goal<br>Final Quadratic Hedging<br>with/without Transaction Costs"] --> B{"Methodology"}
    
    B --> C["Reinforcement Learning<br>RL Minimizing Expectation<br>of Second Moments"]
    B --> D["Deep Trajectory-based Stochastic<br>Optimal Control DTSOC"]
    
    C --> E["Market Environments"]
    D --> E
    
    E --> F["Black Scholes Model"]
    E --> G["Log Normal SABR Model"]
    
    F --> H["Key Findings"]
    G --> H
    
    H --> I["Without Transaction Costs<br>Match Variance Optimal Delta Hedge<br>in Black Scholes"]
    H --> J["Without Transaction Costs<br>Match Bartlett s Delta Hedge<br>in SABR Model"]
    H --> K["Trained on BS / Tested on SABR<br>Match Average Cost of Bartlett<br>But Wider Variance"]