Paper: arXiv 2610.27214

Authors: Carlos Pérez Octavio Mendoza, Frédéric Godin

Abstract

Estimating extreme risk in reinforcement learning becomes increasingly difficult as the confidence level increases, since only a small number of observations contribute to empirical estimates of Conditional Value-at-Risk (CVaR). In this paper, we extend an Extreme Value Theory (EVT) approach for CVaR optimization to high-dimensional neural-network policies. Using a peaks-over-threshold representation and method-of-moments estimation, we obtain a differentiable tail-risk objective and show that, conditional on the selected threshold, its finite-sample gradient can be computed directly through automatic differentiation. We first validate the approach in a controlled experiment with a known optimal solution and then investigate its performance in a deep-hedging problem with rare extreme market movements. Our results show similar hedging behavior for EVT- and empirical-CVaR-trained policies under normal conditions, while EVT-trained policies exhibit lower hedging losses under the extreme-scenario evaluation across the training sample sizes considered. These results suggest that EVT can provide a practical alternative for training neural policies when the observations relevant to extreme risk are scarce.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a mathematically sophisticated approach combining EVT with deep reinforcement learning for tail risk. It demonstrates strong empirical validation in controlled and deep-hedging settings, showing practical benefits for scarce extreme observations. The methodology is clearly articulated, suggesting good reproducibility.

Research Flowchart

  flowchart TD
    A[Research Goal: Deep Catastrophic-Risk-Aware RL] --> B{Problem: Estimating Extreme Risk (CVaR) with Scarce Data};
    B --> C{Methodology: Extend EVT for CVaR Optimization to High-Dimensional Neural Policies};
    C --> D[Computational Steps: Peaks-Over-Threshold, Method-of-Moments, Differentiable Tail-Risk Objective, Automatic Differentiation];
    D --> E[Data/Inputs: Controlled Experiment, Deep Hedging (Extreme Market Movements)];
    E --> F[Key Findings: EVT-trained policies show similar hedging under normal conditions, Lower hedging losses under extreme scenarios across sample sizes];
    F --> G[Outcome: EVT is a practical alternative for training neural policies with scarce extreme-risk observations];