Paper: arXiv 2604.15531

Authors: Sotirios D. Nikolopoulos

Abstract

Adaptive specification search generates statistically significant backtests even under martingale-difference nulls. We introduce a falsification audit testing complete predictive workflows against synthetic reference classes, including zero-predictability environments and microstructure placebos. Workflows generating significant walk-forward evidence in these environments are falsified. For passing workflows, we quantify selection-induced performance inflation using an absolute magnitude gap linking optimized in-sample evidence to disjoint walk-forward realizations, adjusted for effective multiplicity. Simulations validate extreme-value scaling under correlated searches and demonstrate detection power under genuine structure. Empirical case studies confirm that many apparent findings represent methodological artifacts rather than genuine predictability.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper combines rigorous asymptotic derivations of extreme-value scaling under correlated searches with a sophisticated, backtest-ready falsification framework involving synthetic martingale-difference reference classes. It bridges deep theoretical statistics with practical, high-integrity validation protocols for mitigating data-snooping bias in quantitative trading workflows.

Research Flowchart

  flowchart TD
    A[Goal: Identify Spurious Predictability] --> B[Data: Financial Time Series]
    B --> C[Process: Adaptive Specification Search]
    C --> D[Audit: Falsification via Synthetic Placebos]
    D --> E{Workflow Falsified?}
    E -- Yes --> F[Outcome: Methodological Artifact]
    E -- No --> G[Outcome: Quantified Performance Inflation]

Cited References (58)

  1. Empirical Asset Pricing via Machine Learning (2020)
  2. The Impact of Machine Learning on Economics (2019)
  3. Big Data: New Tricks for Econometrics (2014)
  4. Comparing Predictive Accuracy, Twenty Years Later: A Personal Perspective on the Use and Abuse of Diebold-Mariano Tests (2015) — link
  5. Advances in Financial Machine Learning (2018)
  6. …and the Cross-Section of Expected Returns (2016)
  7. Replicating Anomalies (2020) — link
  8. The Econometrics of Financial Markets (1997)
and 50 more
  1. Autoregressive Conditional Heteroscedasticity with Estimates of the Variance of United Kingdom Inflation (1982)
  2. A Note on the Validity of Cross-Validation for Evaluating Autoregressive Time Series Prediction (2018)
  3. A Reality Check for Data Snooping (2000)
  4. A Test for Superior Predictive Ability (2005)
  5. Causal Inference in Statistics, Social, and Biomedical Sciences (2015)
  6. Inference on Treatment Effects after Selection among High-Dimensional Controls (2014)
  7. Exact Post-Selection Inference, with Application to the Lasso (2016)
  8. A Simple, Positive Semi-Definite, Heteroskedasticity and Autocorrelation Consistent Covariance Matrix (1987)
  9. Automatic Lag Selection in Covariance Matrix Estimation (1994)
  10. A Well-Conditioned Estimator for Large-Dimensional Covariance Matrices (2004)
  11. A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics (2005)
  12. Distribution of Eigenvalues for Some Sets of Random Matrices (1967)
  13. The Sizes of Compact Subsets of Hilbert Space and Continuity of Gaussian Processes (1967)
  14. Oracle Inequalities in Empirical Risk Minimization and Sparse Recovery Problems (2011)
  15. Gaussian Random Processes and Measures of Solid Angles in Hilbert Space (1971)
  16. Gaussian Approximations and Multiplier Bootstrap for Maxima of Sums of High-Dimensional Random Vectors (2013)
  17. Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation (1991)
  18. Martingale Limit Theory and Its Application (1980)
  19. The Brunn-Minkowski Inequality in Gauss Space (1975)
  20. The Probabilistic Sharpe Ratio (2012)
  21. Portfolio Performance Manipulation and Manipulation-Proof Performance Measures (2007)
  22. A Simple Approximate Long-Memory Model of Realized Volatility (2009)
  23. Man versus Machine Learning Revisited (2025)
  24. Expectations Matter: When (not) to Use Machine Learning Earnings Forecasts (2024) — link
  25. Man versus machine learning: The term structure of earnings expectations and conditional biases (2023)
  26. From man vs. machine to man+ machine: The art and AI of stock analyses (2024)
  27. A machine learning approach to volatility forecasting (2023)
  28. Diverging Roads: Theory-Based vs. Machine Learning-Implied Stock Risk Premia (2025)
  29. How does post-earnings announcement sentiment affect firms’ dynamics? New evidence from causal machine learning (2024)
  30. Man vs. machine: Quantitative and discretionary equity management (2023)
  31. Consistent testing for serial correlation of unknown form (1996)
  32. Modelling Dynamic Semiparametric Factor Models (2009) — link
  33. The Herfindahl-Hirschman index (1993)
  34. Horizontal Merger Guidelines (2010) — link
  35. Statistical Analysis of High-Frequency Data (2019)
  36. Dynamic semiparametric factor models in risk neutral density estimation (2009)
  37. A semiparametric factor model for implied volatility surface dynamics (2007)
  38. Granger causality in risk and detection of extreme risk spillover between financial markets (2009)
  39. Common risk factors in the returns on stocks and bonds (1993) — link
  40. High-Dimensional Statistics: A Non-Asymptotic Viewpoint (2019)
  41. Empirical properties of asset returns: stylized facts and statistical issues (2001)
  42. The Variation of Certain Speculative Prices (1963)
  43. …and the Cross-Section of Expected Returns (2016)
  44. The Deflated Sharpe Ratio: Correcting for Selection Bias, Backtest Overfitting, and Non-Normality (2014)
  45. On Persistence in Mutual Fund Performance (1997)
  46. Common risk factors in the returns on stocks and bonds (1993)
  47. Is There a Replication Crisis in Finance? (2023)
  48. Mathematical Foundations of Infinite-Dimensional Statistical Models (2016)
  49. The Stationary Bootstrap (1994)
  50. The Dependent Wild Bootstrap (2010)

0 of 58 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.