Multiple Testing

High-Throughput Asset Pricing ArXiv ID: 2311.10685 “View on arXiv” Authors: Unknown Abstract We apply empirical Bayes (EB) to mine data on 136,000 long-short strategies constructed from accounting ratios, past returns, and ticker symbols. This ``high-throughput asset pricing’’ matches the out-of-sample performance of top journals while eliminating look-ahead bias. Naively mining for the largest Sharpe ratios leads to similar performance, consistent with our theoretical results, though EB uniquely provides unbiased predictions with transparent intuition. Predictability is concentrated in accounting strategies, small stocks, and pre-2004 periods, consistent with limited attention theories. Multiple testing methods popular in finance fail to identify most out-of-sample performers. High-throughput methods provide a rigorous, unbiased framework for understanding asset prices. ...

Is There a Replication Crisis inFinance? ArXiv ID: ssrn-3774514 “View on arXiv” Authors: Unknown Abstract Several papers argue that financial economics faces a replication crisis because the majority of studies cannot be replicated or are the result of multiple test Keywords: replication crisis, multiple testing, publication bias, p-hacking, Financial Economics Complexity vs Empirical Score Math Complexity: 8.0/10 Empirical Rigor: 9.0/10 Quadrant: Holy Grail Why: The paper employs a complex Bayesian statistical model for joint factor estimation, involving advanced priors and shrinkage methods, indicating high mathematical density. It also demonstrates high empirical rigor through extensive global backtesting on a new large dataset (93 countries) and provides open-source data access, making it highly data and implementation-heavy. flowchart TD A["Research Goal<br>Replicability in Finance?"] --> B["Methodology<br>Replicate 200+ Studies"] A --> C["Data Input<br>Prominent Finance Journals"] B --> D["Computational Process<br>Statistical Test & Meta-Analysis"] C --> D D --> E["Key Findings<br>High Failure Rate<br>Significant Publication Bias"]

Is There A Replication Crisis In Finance? ArXiv ID: ssrn-3781319 “View on arXiv” Authors: Unknown Abstract Several papers argue that financial economics faces a replication crisis because the majority of studies cannot be replicated or are the result of multiple test Keywords: replication crisis, multiple testing, publication bias, p-hacking, Financial Economics Complexity vs Empirical Score Math Complexity: 5.0/10 Empirical Rigor: 8.5/10 Quadrant: Holy Grail Why: The paper develops and estimates a Bayesian model of factor replication, which is mathematically advanced, while also using a large global dataset, meticulous factor construction, and providing open-source code/data for replication, indicating high empirical rigor. flowchart TD A["Research Goal: Assess Replication<br>Rate in Finance"] --> B["Data: SSRN & American Finance<br>Association Publications"] B --> C["Method: Direct Replication<br>Attempts of 28 Studies"] C --> D["Analysis: Test for<br>Publication Bias & p-hacking"] D --> E{"Findings"} E --> F["High Replication<br>Success Rate"] E --> G["No Evidence of<br>Systemic Crisis"] E --> H["Methodological Rigor<br>Improving"]