Paper: arXiv 2604.17166

Authors: Nima Afsharhajari, Jonathan Yu-Meng Li

Abstract

Sparsity or complexity? In modern high-dimensional asset pricing, these are often viewed as competing principles: richer feature spaces appear to favor complexity, while economic intuition has long favored parsimony. We show that this tension is misplaced. We distinguish capacity sparsity-the dimensionality of the candidate feature space-from factor sparsity-the parsimonious structure of priced risks-and argue that the two are complements: expanding capacity enables the discovery of factor sparsity. Revisiting the benchmark empirical design of Didisheim et al. (2025) and pushing it to higher complexity regimes, we show that nonlinear feature expansions combined with basis pursuit yield portfolios whose out-of-sample performance dominates ridgeless benchmarks beyond a critical complexity threshold. The evidence shows that the gains from complexity arise not from retaining more factors, but from enlarging the space from which a sparse structure of priced risks can be identified. The virtue of complexity in asset pricing operates through factor sparsity.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper combines sophisticated statistical learning theory, such as basis pursuit and overparameterized feature expansions, with a rigorous ‘apples-to-apples’ empirical design benchmarked against recent literature. It successfully bridges theoretical implications regarding model capacity and factor sparsity with substantive out-of-sample portfolio performance analysis.

Research Flowchart

  flowchart TD
    A[Research Goal: Reconcile Capacity Sparsity vs Factor Sparsity] --> B[Data: High-Dimensional Asset Pricing Features]
    B --> C[Methodology: Nonlinear Feature Expansion]
    C --> D[Process: Basis Pursuit / Regularization]
    D --> E[Outcome: Discovery of Parsimonious Priced Risks]
    E --> F[Conclusion: Complexity Enables Factor Sparsity]

Cited References (24)

  1. The Virtue of Complexity in Return Prediction (2024) — link
  2. Characteristics are covariances: A unified model of risk and return (2019) — link
  3. Surprises in High-Dimensional Ridgeless Least Squares Interpolation (2022)
  4. Benign Overfitting in Linear Regression (2020)
  5. Deep Learning is Not So Mysterious or Different (2025) — link
  6. The Role of Conditioning Information in Deducing Testable Restrictions Implied by Dynamic Asset Pricing Models (1987)
  7. Random Features for Large-Scale Kernel Machines (2007)
  8. Shrinking the Cross-Section (2020)
and 16 more
  1. Empirical Asset Pricing via Machine Learning (2020)
  2. Deep Learning in Asset Pricing (2024)
  3. \ldots and the Cross-Section of Expected Returns (2016)
  4. Is There a Replication Crisis in Finance? (2023) — link
  5. APT or `` AIPT ’’ ? The Surprising Dominance of Large Factor Models (2025)
  6. Understanding The Virtue of Complexity (2025) — link
  7. Presidential Address: Discount Rates (2011) — link
  8. Seemingly Virtuous Complexity in Return Prediction (2025)
  9. Autoencoder Asset Pricing Models (2021) — link
  10. Taming the Factor Zoo: A Test of New Factors (2020) — link
  11. Open Source Cross-Sectional Asset Pricing (2022)
  12. Does Academic Research Destroy Stock Return Predictability? (2016)
  13. Replicating Anomalies (2020)
  14. Nonparametric Sparsity and Regularization (2013)
  15. Atomic Decomposition by Basis Pursuit (1998) — link
  16. Branch-and-Price: Column Generation for Solving Huge Integer Programs (1998) — link

0 of 24 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.