Paper: arXiv 2303.16158

Authors: Murray Z. Frank, Jing Gao, Keer Yang

Abstract

Standard forecast efficiency tests interpret violations as evidence of behavioral bias. We show theoretically and empirically that rational forecasters using optimal regularization systematically violate these tests. Machine learning forecasts show near zero bias at one year horizon, but strong overreaction at two years, consistent with predictions from a model of regularization and measurement noise. We provide three complementary tests: experimental variation in regularization parameters, cross-sectional heterogeneity in firm signal quality, and quasi-experimental evidence from ML adoption around 2013. Technically trained analysts shift sharply toward overreaction post-2013. Our findings suggest reported violations may reflect statistical sophistication rather than cognitive failure.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper provides a novel theoretical framework and robust empirical evidence demonstrating that forecast bias previously attributed to cognitive errors can actually result from rational machine learning regularization. The empirical execution is highly rigorous, utilizing multiple quasi-experimental and cross-sectional tests to validate the model’s predictions.

Research Flowchart

  flowchart TD
    A[Research Goal: Do forecast efficiency violations reflect behavioral bias or optimal ML regularization?] --> B[Data & Inputs: Analyst forecasts, firm signals, and post-2013 ML adoption timeline]
    B --> C[Theoretical Modeling: Model optimal regularization under measurement noise]
    B --> D[Empirical Testing: Cross-sectional signal quality & post-2013 quasi-experiments]
    C --> E[Computational Analysis: Measure forecast bias across 1-year vs 2-year horizons]
    D --> E
    E --> F[Key Findings: Optimal regularization creates apparent overreaction; technically trained analysts shift post-2013]
    F --> G[Outcome: Test violations reflect statistical sophistication rather than cognitive failure]