Paper: arXiv 2301.10985
Authors: Nassim Nicholas Taleb, Ron Richman, Marcos Carreira, James Sharpe
Abstract
We respond to Tetlock et al. (2022) showing 1) how expert judgment fails to reflect tail risk, 2) the lack of compatibility between forecasting tournaments and tail risk assessment methods (such as extreme value theory). More importantly, we communicate a new result showing a greater gap between the properties of tail expectation and those of the corresponding probability.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 3.5/10
- Quadrant: Lab Rats — theoretically deep, empirically untested
Why this score: The paper provides a strong theoretical and mathematical critique regarding tail expectation versus probability estimation, contrasting mathematical rigor with qualitative forecasting tournaments. However, as a commentary/reply paper, it lacks extensive empirical backtesting or data analysis, relying instead on theoretical proofs and Extreme Value Theory conceptual frameworks.
Research Flowchart
flowchart TD
A["Research Goal: Rebut Tetlock et al. (2022) &<br>Evaluate Expert Judgment in Tail Risk"] --> B["Data & Inputs: Multi-Asset Returns &<br>Forecasting Tournament Data"]
B --> C["Apply Extreme Value Theory EVT &<br>Mathematical Tail Expectation Proofs"]
C --> D["Compare Expert Forecasts vs.<br>EVT Mathematical Properties"]
D --> E1["Outcome 1: Expert judgment fails<br>to reflect true tail risk"]
D --> E2["Outcome 2: Forecasting tournaments are<br>incompatible with EVT methods"]
D --> E3["Outcome 3: Proven wider gap between<br>tail expectation and probability"]