Paper: arXiv 2609.26303
Authors: Masoud Soleimani
Abstract
Forecasters often score the same units per date against one standardized realized outcome. We show that every standardized forecast splits exactly into a component aligned with this common target and a component uncorrelated with it. Three consequences follow: forecast-error correlation largely mirrors forecast correlation and is therefore a poor measure of diversity; an equally weighted combination beats a no-information forecast only when average alignment is large relative to the combination’s dispersion; and the gain from adding a forecaster separates into genuine improvement and mere dilution, which equal-weight admission can mistakenly reward. We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange-traded funds. Selection removes most dilution losses, but no combination beats the no-information forecast.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 7.5/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: This paper presents a highly mathematical decomposition of forecast alignment and its implications for combination, backed by simulations and real-world applications. The theoretical contributions are significant, and the empirical sections provide solid validation, even if the ’null findings’ are somewhat counterintuitive.
Research Flowchart
flowchart TD
A["Research Goal: Understand cross-sectional forecast dynamics"] --> B{"Key Concepts: Target alignment, Dilution, Forecast selection"};
B --> C["Methodology: Decompose standardized forecasts into aligned/uncorrelated components"];
C --> D{"Data/Inputs: Language model forecasts (US equity), Mechanical signals (ETFs)"};
D --> E["Computational Processes: Analyze forecast-error correlation, Simulate combination strategies, Apply cautious selection rule"];
E --> F["Key Findings: Forecast-error correlation mirrors forecast correlation; Equal-weight combination effectiveness depends on alignment/dispersion; Cautious selection removes dilution losses but no combination beats no-information forecast."];
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a cautious three‑way selection rule for admitting forecasts to an equal‑weight pool, based on target alignment and target‑orthogonal dependence.
Source sentence
We develop a cautious selection rule, study it in simulations, and apply it to language-model forecasts of US equity rankings and mechanical signals ranking exchange‑traded funds.
Assumes 6
- Forecasts are standardized and share a common support (common target). model
Source sentence
Exact for standardized forecasts on a common support; no distributional assumptions
- Heteroskedasticity‑ and autocorrelation‑consistent (HAC) variance estimates are valid for the rolling‑origin validation. statistical
Source sentence
Simultaneous per‑step bands conditional on the incumbent pool, asymptotically valid under HAC consistency
- No distributional assumptions are required for the algebraic identity results. statistical
Source sentence
Algebraic identity Exact for standardized forecasts on a common support; no distributional assumptions
- The target alignment and target‑orthogonal dependence fully decompose combination risk. model
Source sentence
their combination risk admits an exact decomposition into target alignment and target‑orthogonal dependence
Show 2 more
- Equal‑weight admission rewards scale dilution when alignment is weak. other
Source sentence
equal‑weight admission is able to reward scale dilution
- Scale‑free admission becomes conservative once incumbent pools are large. other
Source sentence
scale‑free admission leans conservative once incumbent pools are large
Data
- Signal panel (mechanical forecasters ranking exchange‑traded funds) alternative unknown
Source sentence
a cross‑asset fund panel with mechanical forecasters
Costs
Not applicable
Finds 5
- Selection removes most dilution losses, but no combination beats the no‑information forecast; alignment is not detectable in the empirical panels. negative
Source sentence
Selection removes most dilution losses, but no combination beats the no‑information forecast.
- Equal‑weight pooling only outperforms the no‑information forecast when average alignment is large relative to the pool’s dispersion. theoretical
Source sentence
an equally weighted combination beats a no‑information forecast only when average alignment is large relative to the combination’s dispersion
- Out‑of‑sample, the best performer (shrunk ridge weights) improves relative‑score risk by +0.019 [0.006, 0.032] over the no‑information forecast. performance
Source sentence
The best performer, shrunk ridge weights, exhibits a difference of +0.019 [0.006, 0.032].
- In simulations, equal‑weight risk beats the no‑information forecast only in regions where alignment is sufficiently high relative to dependence. theoretical
Source sentence
Only in regions where alignment stands sufficiently high relative to dependence does equal weighting outperform the no‑information forecast.
- The three‑way admission rule reduces risk by 0.20–0.27 relative to equal weighting, but still does not surpass the no‑information benchmark. performance
Source sentence
... three‑way admission and diversity criteria reduce risk by 0.20–0.27. Against the no‑information forecast, none improves.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 32 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (60)
- The combination of forecasts (1969) — link
- Diversity creation methods: A survey and categorisation (2005) — link
- A nonparametric approach to identifying a subset of forecasters that outperforms the simple average (2017) — link
- Econometric evaluation of linear macro-economic models (1986) — link
- The forecast combination puzzle: A simple theoretical explanation (2016) — link
- Combining forecasts: A review and annotated bibliography (1989) — link
- Comparing predictive accuracy (1995) — link
- Machine learning for regularized survey forecast combination: Partially-egalitarian LASSO and its derivatives (2019) — link
and 52 more
- Combining expert forecasts: Can anything beat the simple average? (2013) — link
- Improved methods of combining forecasts (1984) — link
- Tests for forecast encompassing (1998) — link
- Beyond forecast leaderboards: Measuring individual model importance based on contribution to ensemble accuracy (2026) — link
- Neural network ensembles, cross validation, and active learning (1995)
- A well-conditioned estimator for large-dimensional covariance matrices (2004) — link
- Improving the simple average combined forecast via factor-adjusted regularization (2026) — link
- Combining forecasts under structural breaks using graphical LASSO (2026) — link
- On the uncertainty of a combined forecast: The critical role of correlation (2023) — link
- Optimal selection of expert forecasts with integer programming (2018) — link
- The evaluation of economic forecasts (1969)
- A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix (1987) — link
- Automatic lag selection in covariance matrix estimation (1994) — link
- Too similar to combine? O n negative weights in forecast combination (2023) — link
- A simple explanation of the forecast combination puzzle (2009) — link
- Combination forecasts of output growth in a seven-country data set (2004) — link
- Forecast combinations (2006) — link
- Generalization error of ensemble estimators (1996) — link
- Forecast combinations: An over 50-year review (2023) — link
- Persistence in forecasting performance and conditional combination strategies (2006) — link
- The cross-section of volatility and expected returns (2006) — link
- Value and momentum everywhere (2013) — link
- Forecaster diversity and the benefits of combining forecasts (1995) — link
- Valid post-selection inference (2013) — link
- Survivorship bias in performance studies (1992) — link
- Identifying expertise to extract the wisdom of crowds (2015) — link
- Forecast combination with entry and exit of experts (2009) — link
- Limits for the precision and value of information from dependent sources (1985) — link
- The use of confidence or fiducial limits illustrated in the case of the binomial (1934) — link
- Optimal combination of survey forecasts (2015) — link
- Tests of conditional predictive ability (2006) — link
- Assessing look-ahead bias in stock return predictions generated by GPT sentiment analysis — → 2309.17322
- Making and evaluating point forecasts (2011) — link
- The model confidence set (2011) — link
- Tests for multiple forecast encompassing (2000)
- Pooling of forecasts (2004) — link
- Computing the nearest correlation matrix— A problem from finance (2002) — link
- A simple sequentially rejective multiple test procedure (1979) — link
- Simultaneous inference in general parametric models (2008) — link
- Evidence of predictable behavior of security returns (1990) — link
- Returns to buying winners and selling losers: Implications for stock market efficiency (1993) — link
- The jackknife and the bootstrap for general stationary observations (1989) — link
- Honey, I shrunk the sample covariance matrix (2004) — link
- Economic forecast evaluation: Profits versus the conventional error measures (1991)
- Why do some combinations perform better than others? (2020) — link
- Optimal and robust combination of forecasts via constrained optimization and shrinkage (2022) — link
- Stepwise multiple testing as formalized data snooping (2005) — link
- Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy (2024) — link
- A comparison of the two one-sided tests procedure and the power approach for assessing the equivalence of average bioavailability (1987) — link
- Out-of-sample tests of forecasting accuracy: An analysis and review (2000) — link
- Regression shrinkage and selection via the lasso (1996) — link
- A unified theory of diversity in ensemble learning (2023) — link
1 of 60 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.