Paper: arXiv 2406.15612
Authors: Parisa Davar, Frédéric Godin, Jose Garrido
Abstract
This paper tackles the problem of mitigating catastrophic risk (which is risk with very low frequency but very high severity) in the context of a sequential decision making process. This problem is particularly challenging due to the scarcity of observations in the far tail of the distribution of cumulative costs (negative rewards). A policy gradient algorithm is developed, that we call POTPG. It is based on approximations of the tail risk derived from extreme value theory. Numerical experiments highlight the out-performance of our method over common benchmarks, relying on the empirical distribution. An application to financial risk management, more precisely to the dynamic hedging of a financial option, is presented.
Complexity vs Empirical Score
- Math Complexity: 7.0/10
- Empirical Rigor: 6.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper presents an innovative integration of Extreme Value Theory (EVT) into policy gradient reinforcement learning to better handle far-tail risk. It contains rigorous mathematical derivations alongside benchmarked empirical applications, including dynamic option hedging. Additionally, high reproducibility is ensured via a publicly available GitHub repository.
Research Flowchart
flowchart TD
A[Goal: Mitigate Catastrophic Tail Risk in Sequential Decision-Making] --> B[Input: Scarcity of Far-Tail Cost Observations & Financial Market Data]
B --> C[Methodology: Apply Extreme Value Theory EVT to Model Tail Risk]
C --> D[Compute: Peak-Over-Threshold Policy Gradient POTPG Algorithm]
D --> E[Application: Dynamic Hedging of Financial Derivatives]
E --> F[Outcome: POTPG Outperforms Empirical Benchmarks in Tail-Risk Reduction]
Cited References (28)
- Automated threshold selection for extreme value analysis via ordered goodness-of-fit tests with adjustment for false discovery rate (2018)
- Estimation of Generalized Tail Distortion Risk Measures with Applications in Reinsurance (2024)
- Residual life time at great age (1974)
- A sensitivity formula for risk-sensitive cost and the actor–critic algorithm (2001)
- Deep hedging (2019)
- Gamma and vega hedging using deep distributional reinforcement learning (2023)
- Equal risk pricing of derivatives with deep hedging (2021)
- Risk-constrained reinforcement learning with percentile risk criteria (2018)
and 20 more
- Conditionally elicitable dynamic risk measures for deep reinforcement learning (2023)
- An introduction to statistical modeling of extreme values (2001)
- Contingent claim pricing using a normal inverse G aussian probability distortion operator (2012)
- Minimizing CV a R in global dynamic hedging with transaction costs (2016)
- Efficient risk-averse reinforcement learning (2022)
- ADAM : A method for stochastic optimization (2014) — link
- Actor-critic algorithms for risk-sensitive MDP s (2013)
- Deep reinforcement learning for option pricing and hedging under dynamic expectile risk measures (2023)
- Quantitative risk management: concepts, techniques and tools-revised edition (2015)
- Statistical inference using extreme order statistics (1975)
- Risk-sensitive reinforcement learning via policy gradient search (2022)
- Bias-corrected peaks-over-threshold estimation of the cvar (2021)
- Conditional value-at-risk for general loss distributions (2002)
- L 'e vy processes in finance: pricing financial derivatives (2003)
- Reinforcement learning: An introduction (2018)
- Policy gradients with variance related risk criteria (2012)
- Best-Arm Identification Using Extreme Value Theory Estimates of the CV a R (2022)
- A policy gradient approach for optimization of smooth risk measures (2023)
- Minimizing risk models in M arkov decision processes with policies depending on target values (1999)
- Robust risk-aware option hedging — → 2303.15216
1 of 28 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.