Paper: arXiv 2506.03780

Authors: Hasan Fallahgoul

Abstract

Recent advances in machine learning have shown promising results for financial prediction using large, over-parameterized models. This paper provides theoretical foundations and empirical validation for understanding when and how these methods achieve predictive success. I examine two key aspects of high-dimensional learning in finance. First, I prove that within-sample standardization in Random Fourier Features implementations fundamentally alters the underlying Gaussian kernel approximation, replacing shift-invariant kernels with training-set dependent alternatives. Second, I establish information-theoretic lower bounds that identify when reliable learning is impossible no matter how sophisticated the estimator. A detailed quantitative calibration of the polynomial lower bound shows that with typical parameter choices, e.g., 12,000 features, 12 monthly observations, and R-square 2-3%, the required sample size to escape the bound exceeds 25-30 years of data–well beyond any rolling-window actually used. Thus, observed out-of-sample success must originate from lower-complexity artefacts rather than from the intended high-dimensional mechanism.

Complexity vs Empirical Score

  • Math Complexity: 9.0/10
  • Empirical Rigor: 4.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper is dense with advanced theoretical constructs including PAC-learning theory, information-theoretic bounds, random matrix theory, and detailed proofs on kernel approximation properties, placing it at the high end of mathematical complexity. While it includes quantitative calibration and numerical validation, the summary and excerpt focus on theoretical proofs and bounds rather than extensive backtesting, code, or dataset implementation, resulting in moderate empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Understanding High-Dim ML Success in Finance"] --> B{"Key Methodology<br>1. RFF Kernel Analysis<br>2. Info-Theoretic Bounds"}
  B --> C["Data/Inputs<br>12k Features, 12 Monthly Observations"]
  C --> D["Computational Process<br>Quantitative Calibration of Bounds"]
  D --> E{"Outcome: Required Sample Size"}
  E -- > 25-30 Years --> F["Key Finding<br>Success from Low-Dim Artefacts"]
  E -- <= 12 Months --> G["False Success Scenario<br>Unachievable Requirements"]
  F --> H["Conclusion<br>Targeted Low-Complexity Models"]
  G --> H

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper provides theoretical foundations and empirical validation for high‑dimensional learning in finance, including analysis of standardization effects on Random Fourier Features and calibrated information‑theoretic sample‑size bounds.

Source sentence
This paper provides theoretical foundations and empirical validation for understanding when and how these methods achieve predictive success.

Assumes 2

  • Affine independence of the augmented training vectors (xt,1), i.e., the (K+1)×T matrix has full column rank T. statistical
    Source sentence
    Assumption 3.4 (Affine Independence of the Sample). Equivalently, the (K + 1) × T matrix formed by these augmented vectors has full column rank T. This geometric condition on the training data is essential for the analysis of the stan-dardized kernel.
  • Sub‑Gaussianity of any linear combination of Random Fourier Features, i.e., for every unit vector u, u⊤z(x) is κ‑sub‑Gaussian. statistical
    Source sentence
    Assumption 3.5 (Sub-Gaussian Random Features). This is a standard concentration assumption in high-dimensional learning theory and is automatically satisfied in all practical RFF implementations.

Data

  • synthetic macro predictor data synthetic unknown
    Source sentence
    19 6.1.1 Data Generation and Model Parameters I generate realistic financial predictor data following the autoregressive structure typical of macroeconomic variables used in return prediction. For each parameter combination (T,K), I construct predictor matrices X ∈RT×K where: Xt = ΦXt−1 + ut, ut ∼N(0,Σu) The persistence parameters Φ = diag(φ1,...,φK) are drawn from the range [0.82,0.98] to match the high persistence

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Within‑sample standardization in Random Fourier Features implementations fundamentally alters the Gaussian kernel approximation, breaking the theoretical kernel assumptions. theoretical
    Source sentence
    First, I prove that within-sample standardization in Random Fourier Features implementations fundamentally alters the underlying Gaussian kernel approximation, replacing shift-invariant kernels with training-set dependent alternatives. First, I prove that within-sample standardization—employed in every practical Random Fourier Features implementation—fundamentally breaks the kernel approximation that underlies
  • Information‑theoretic lower bounds imply that the critical sample size required to overcome weak‑signal limitations exceeds 30 years of monthly data for typical high‑dimensional designs (e.g., P=12,000, R²≈2‑3%). theoretical
    Source sentence
    A detailed quantitative calibration of the polynomial lower bound shows that with typical parameter choices, e.g., 12,000 features, 12 monthly observations, and R-square 2-3%, the required sample size to escape the bound exceeds 25-30 years of data--well beyond any rolling-window actually used. For the high-dimensional design in Kelly et al. (2024), we find: Baseline (P=12,000): Tcrit = 1 1.1 2.2 × 10−3 5 × 10−5 ×
  • Calibrated polynomial minimax lower bound shows that even low‑dimensional models need a sample size on the order of decades to reliably learn weak financial signals. theoretical
    Source sentence
    Using defensible parameters for signal strength (calibrated to an R2 of 1–5%) and noise variance drawn from historical market data, we diagnose the nature of the learning problem faced by financial prediction models. My analysis reveals a profound implication: the critical sample size (Tcrit) required to overcome the limitations imposed by weak signals is on the order 4 of decades, even for low-dimensional models.
  • Empirical calibration using realistic signal strength (R²≈2.3%) and noise variance (σ²≈2.2×10⁻³) yields a critical sample size T₍crit₎≈375 months (≈31 years) for P=12,000 features. theoretical figure from elsewhere in the paper
    Source sentence
    Using our calibrated total variance Var(r) ≈2.2 × 10−3, this implies a range for the signal variance: B2 = R2 × Var(r) ≈(0.01 −0.05) × (2.2 × 10−3) =⇒B2 ≃(2.2 −11) × 10−5. For the high-dimensional design in Kelly et al. (2024), we find: Baseline (P=12,000): Tcrit = 1 1.1 2.2 × 10−3 5 × 10−5 × 9.4 ≈375 months (≈31 years).
  • The performance bound is dictated by signal weakness (B²); increasing sample size improves the bound only up to the complexity‑limited regime, beyond which additional data cannot lower the theoretical performance floor. theoretical
    Source sentence
    The best achievable performance is funda-mentally constrained by the weakness of the signal, B2. In this regime, neither more data nor a simpler model can lower the theoretical performance floor. • If T > Tcrit, the problem is complexity-limited.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 22 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (52)

  1. Seemingly Virtuous Complexity in Return Prediction (2025)
  2. The Virtue of Complexity in Return Prediction (2024)
  3. A Comprehensive Look at the Empirical Performance of Equity Premium Prediction (2008)
  4. Random Features for Large-Scale Kernel Machines (2007)
  5. Introduction to Nonparametric Estimation (2009)
  6. Rademacher and Gaussian Complexities: Risk Bounds and Structural Results (2002)
  7. Stability and Generalization (2002)
  8. Gaussian Processes for Machine Learning (2006)
and 44 more
  1. Reconciling Modern Machine-Learning Practice and the Classical Bias–Variance Trade-off (2019)
  2. Surprises in High-Dimensional Ridgeless Least Squares Interpolation (2022)
  3. Benign Overfitting in Linear Regression (2020)
  4. On the Error of Random Fourier Features (2015)
  5. The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve (2022)
  6. High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections (2024)
  7. Early Stopping, Regularization Paths and Generalization Bounds for Ridge Regression (2014)
  8. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2009)
  9. High-Dimensional Statistics: A Non-Asymptotic Viewpoint (2019)
  10. Foundations of Machine Learning (2018)
  11. Support Vector Machines (2008)
  12. Learning Theory: An Approximation Theory Viewpoint (2007)
  13. Can Machines Learn Weak Signals? (2025)
  14. APT or “AIPT”? The Surprising Dominance of Large Factor Models (2024)
  15. On the equivalence between kernel quadrature rules and random feature expansions (2017)
  16. Neural Network Learning: Theoretical Foundations (1999) — link
  17. On the density of families of sets (1972) — link
  18. A combinatorial problem; stability and order for models and theories in infinitary languages (1972) — link
  19. On the uniform convergence of relative frequencies of events to their probabilities (1971) — link
  20. Statistical Learning Theory (1998)
  21. Learnability and the Vapnik-Chervonenkis dimension (1989) — link
  22. Understanding Machine Learning: From Theory to Algorithms (2014) — link
  23. Forecasting stock returns (2013)
  24. The Virtue of Complexity in Return Prediction (2024) — link
  25. Empirical Asset Pricing via Machine Learning (2020) — link
  26. Deep Learning in Asset Pricing (2024) — link
  27. Taming the Factor Zoo: A Test of New Factors (2020) — link
  28. Bond Risk Premiums with Machine Learning (2021) — link
  29. … and the Cross‐Section of Expected Returns (2016) — link
  30. Does Academic Research Destroy Stock Return Predictability? (2016) — link
  31. Reconciling Modern Machine‐Learning Practice and the Bias–Variance Trade‐off (2019) — link
  32. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2009)
  33. A Theory of the Learnable (1984)
  34. An Introduction to Computational Learning Theory (1994)
  35. Generalization Properties of Learning with Random Features (2017)
  36. An Introduction to Matrix Concentration Inequalities (2015) — link
  37. User-Friendly Tail Bounds for Sums of Random Matrices (2012) — link
  38. The Econometrics of Financial Markets (1997)
  39. High-Dimensional Probability: An Introduction with Applications in Data Science (2018)
  40. Simplified: A Closer Look at the Virtue of Complexity in Return Prediction (2025)
  41. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (2002)
  42. A generalized representer theorem (2001)
  43. Probability inequalities for sums of bounded random variables (1963)
  44. Asset pricing with neural networks: Significance tests (2024)

0 of 52 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.