Forking paths in financial economics

Paper: arXiv 2401.08606

Authors: Guillaume Coqueret

Abstract

We argue that spanning large numbers of degrees of freedom in empirical analysis allows better characterizations of effects and thus improves the trustworthiness of conclusions. Our ideas are illustrated in three studies: equity premium prediction, asset pricing anomalies and risk premia estimation. In the first, we find that each additional degree of freedom in the protocol expands the average range of $t$-statistics by at least 30%. In the second, we show that resorting to forking paths instead of bootstrapping in multiple testing raises the bar of significance for anomalies: at the 5% confidence level, the threshold for bootstrapped statistics is 4.5, whereas with paths, it is at least 8.2, a bar much higher than those currently used in the literature. In our third application, we reveal the importance of particular steps in the estimation of premia. In addition, we use paths to corroborate prior findings in the three topics. We document heterogeneity in our ability to replicate prior studies: some conclusions seem robust, others do not align with the paths we were able to generate.

Complexity vs Empirical Score

• Math Complexity: 4.5/10
• Empirical Rigor: 8.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: This paper presents a robust methodological framework for evaluating empirical sensitivity via specification curves (forking paths) across major asset pricing domains. It significantly raises the bar for empirical rigor by demonstrating that traditional bootstrapping severely underestimates multiple testing thresholds.

Research Flowchart

  flowchart TD
    A[Research Goal: Assess impact of analytical degrees of freedom 'forking paths' on empirical finance] --> B[Inputs: 3 Case Studies - Equity Premium, Asset Pricing Anomalies, Risk Premia]
    B --> C[Methodology: Map and execute all possible methodological paths vs traditional bootstrapping]
    C --> D[Compute Path Variations: Evaluate t-statistic distributions across all execution protocols]
    D --> E1[Outcome 1: Equity Premium - Each added degree of freedom expands t-stat range by >=30%]
    D --> E2[Outcome 2: Anomalies - Forking paths raise 5% significance threshold to t >= 8.2 vs 4.5 bootstrapped]
    D --> E3[Outcome 3: Replication - Uncovers sensitivity in risk premia and mixed robustness of prior studies]

Cited References (230)

1. Statistical nonsignificance in empirical economics (2020)
2. Noisy Factors (2023)
3. Intangible Capital and the Value Factor: H as Your Value Definition Just Expired? (2020)
4. Predictive regressions: A reduced-bias estimation method (2004)
5. Scientists rise up against statistical significance (2019)
6. Model selection and multimodel inference (2004)
7. Identification of and correction for publication bias (2019)
8. An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator (1992)

0 of 230 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.