Paper: arXiv 2301.12719
Authors: Gero Junike, Solveig Flaig, Ralf Werner
Abstract
Machine learning (ML) methods are becoming increasingly important in the design economic scenario generators for internal models. Validation of data-driven models differs from classical theory-based models. We discuss two novel aspects of such a validation: first, checking dependencies between risk factors and second, detecting unwanted memorization effects. The first task becomes necessary since in ML-based methods dependencies are no longer derived from a financial-mathematical theory but are driven by data. The need for the latter task arises since it cannot be ruled out that ML-based models merely reproduce the empirical data rather than generating new scenarios. To address the first issue, we propose to use an existing test from the literature. For the second issue, we introduce and discuss a novel memorization ratio. Numerical experiments based on real market data are included and an autoencoder-based scenario generator is validated with these two methods.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 6.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper addresses a critical gap in machine learning model risk management by introducing a novel framework to validate dependencies and detect memorization in ML-based Economic Scenario Generators. It balances theoretical formulation of a novel memorization ratio with practical empirical validation using an autoencoder on real market data.
Research Flowchart
flowchart TD
A["Research Goal: Validate ML-Based Economic Scenario Generators (ESGs)"] --> B["Input Data: Real Market Data"]
B --> C["Model: Train Autoencoder-Based Scenario Generator"]
C --> D1["Validation 1: Test Risk Factor Dependencies (Existing Literature Test)"]
C --> D2["Validation 2: Evaluate Memorization (Novel Memorization Ratio)"]
D1 --> E["Key Outcomes: Robust Validation Framework Detecting Data Overfitting & Dependency Errors"]
D2 --> E