Paper: arXiv 2411.18830

Authors: Yonghe Lu, Yanrong Yang, Terry Zhang

Abstract

We study the relationship between model complexity and out-of-sample performance in the context of mean-variance portfolio optimization. Representing model complexity by the number of assets, we find that the performance of low-dimensional models initially improves with complexity but then declines due to overfitting. As model complexity becomes sufficiently high, the performance improves with complexity again, resulting in a double ascent Sharpe ratio curve similar to the double descent phenomenon observed in artificial intelligence. The underlying mechanisms involve an intricate interaction between the theoretical Sharpe ratio and estimation accuracy. In high-dimensional models, the theoretical Sharpe ratio approaches its upper limit, and the overfitting problem is reduced because there are more parameters than data restrictions, which allows us to choose well-behaved parameters based on inductive bias.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper employs advanced theoretical frameworks like random matrix theory and derives analytical asymptotic results for Sharpe ratios, indicating high mathematical complexity. However, the excerpt focuses on theoretical derivations and conceptual mechanisms without showing specific backtesting procedures, code, or detailed empirical metrics, resulting in lower empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Investigate relationship between model complexity (number of assets) and out-of-sample Sharpe ratio in mean-variance portfolio optimization"] --> B["Data: Historical equity returns data (N assets, T time periods)"]
  
  B --> C["Methodology: Simulate varying model complexity (N from low to high)"]
  
  C --> D["Computational Process: Construct Mean-Variance portfolios using empirical inputs (sample mean/covariance) across all complexity levels"]
  
  D --> E["Analysis: Evaluate out-of-sample Sharpe ratio performance against model complexity"]
  
  E --> F["Finding 1: Performance improves initially with complexity (low N)"]
  E --> G["Finding 2: Performance declines due to overfitting (moderate N)"]
  E --> H["Finding 3: Performance improves again with high complexity (N > T)"]
  
  F --> I["Key Outcome: Double Descent Sharpe Ratio Curve<br>Mechanism: Interaction between theoretical Sharpe limit & estimation accuracy in high dimensions"]
  G --> I
  H --> I

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes that mean‑variance portfolio optimization exhibits a double‑descent‑like “double ascent” Sharpe ratio curve as model complexity (number of assets) increases.

Source sentence
we find that the performance of low-dimensional models initially improves with complexity but then declines due to overfitting. As model complexity becomes sufficiently high, the performance improves with complexity again, resulting in a “double ascent” Sharpe ratio curve similar to the “double descent” phenomenon obse

Assumes 6

  • High dimensionality: T, N → ∞ with ρ_T = N/T in (0,1)∪(1,∞). statistical
    Source sentence
    A1 (High Dimensionality). We consider asymptotic setup where T, N → ∞
  • Factorless model: excess returns rt = μ + y_t with i.i.d. zero‑mean, unit‑variance, bounded 4th‑moment y_t. model
    Source sentence
    A2 (Factorless Model). The excess returns vector at time t is generated as r_t = μ + y_t, where y_t is a vector of i.i.d. random variables with zero mean, unit variance and bounded 4th moment.
  • Constant scale: the squared theoretical Sharpe ratio θ converges to a bounded limit ˜θ as N → ∞. model
    Source sentence
    A3 (Constant Scale). Assume that the square of theoretical Sharpe ratio, θ (equivalently
  • High dimensionality for factor model: same asymptotic regime with ρ_T in (0,1)∪(1,∞). statistical
    Source sentence
    B1 (High Dimensionality). We consider asymptotic setup where T, N → ∞
Show 2 more
  • Factor model: excess returns rt = b μ_f + b σ_f ζ_t + σ_ε η_t with independent i.i.d. components having bounded 4th moments. model
    Source sentence
    η_t is a vector of i.i.d. random variables with zero mean, unit variance and bounded 4th moment, ζ_t
  • Constant scale for factor model: θ converges to ˜θ, implying ‖b‖^2 converges to a constant ˜b^2. model
    Source sentence
    B3 (Constant Scale). Assume that the square of theoretical Sharpe ratio, θ (equivalently

Data

Backtest sample: US equity returns · equities · monthly

  • CRSP monthly total individual equity returns · Center for Research in Security Prices (CRSP) market-prices public
    January 1967 to June 2019 (630 months) · nearly 30,000 stocks, average >6,200 stocks per month
    Source sentence
    Our study utilizes a comprehensive dataset of monthly total individual equity returns sourced from the Center for Research in Security Prices (CRSP). This dataset includes firms listed on the New York Stock Exchange (NYSE), the American Stock Exchange (AMEX), and NASDAQ. The analysis spans from January 1967 to June 201

Recognised sources: CRSP

Costs

Not modeled A trading or execution result is reported without costs.

Finds 4

Reported performance (mean-variance portfolio optimization with varying asset count): Sharpe 1.40

  • In high‑dimensional regimes (N > T) the out‑of‑sample Sharpe ratio ascends again, showing a double‑ascent pattern. theoretical
    Source sentence
    What is important in this figure is the “ascent” observed in the high-dimensional region, when ρ > 1. From a finance perspective, the observed “ascent” suggests that increasing the number of assets beyond the sample size can enhance the out-of-sample risk-adjusted returns of the estimated portfolio.
  • Empirical results with CRSP data are consistent with the theory, showing the Sharpe ratio first rises with N then plateaus. performance
    Source sentence
    Based on the empirical performance of portfolio models with different complexity, we can also estimate the relationship between the clairvoyant Sharpe ratio and the number of assets.
  • The calibrated clairvoyant Sharpe ratio follows θ(N)=θ_1 e^{−λ(N−1)}+¯θ(1−e^{−λ(N−1)}) and reaches half the upper bound with about 13 assets; at N≈39 the clairvoyant Sharpe is 1.11 (~¾ of the limit). other
    Source sentence
    The speed parameter λ informs us how fast the clairvoyant Sharpe ratio increases with the number of assets. To reach half of the Sharp ratio upper bound, the portfolio needs to have around 13 assets. At the optimal low-dimensional point, when N = 39, the clairvoyant Sharpe ratio is 1.11, which is about three quarters o
  • Theoretical Sharpe ratio upper bound would require about 12 independent asset‑pricing factors, aligning with other literature estimates. theoretical
    Source sentence
    If we assume the asset pricing factors have the same Sharpe ratio as the market, then we need 12 independent asset pricing factors to reach the theoretical Sharpe ratio upper bound.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 26 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (120)

  1. Approaching mean-variance efficiency for large portfolios (2019)
  2. Market efficiency in the age of big data (2022)
  3. Lest we forget: Learn from out-of-sample forecast errors when optimizing portfolios (2022)
  4. A transaction-cost perspective on the multitude of firm characteristics (2020)
  5. Empirical cross-sectional asset pricing: a survey (2012)
  6. Presidential address: Discount rates (2011)
  7. The statistical limit of arbitrage (2024)
  8. Dissecting characteristics nonparametrically (2020)
and 112 more
  1. Parametric portfolio policies: Exploiting characteristics in the cross-section of equity returns (2009)
  2. On asymptotics of eigenvectors of large sample covariance matrix (2007) — link
  3. Asymptotic properties of eigenmatrices of a large sample covariance matrix (2011) — link
  4. Local operator theory, random matrices and Banach spaces (2001)
  5. Matrix analysis (2012)
  6. Statistics on special manifolds (2003)
  7. Multivariate analysis (1979)
  8. Topics in matrix analysis (1994)
  9. The distribution of eigenvalues in certain sets of random matrices (1967)
  10. The strong limits of random matrix spectra for sample matrices of independent elements (1978)
  11. Strong convergence of the empirical distribution of eigenvalues of large dimensional random matrices (1995)
  12. High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation (2010) — link
  13. Random matrix theory and robust covariance matrix estimation for financial data (2005)
  14. Risk and asset allocation (2005)
  15. Quantitative risk management: concepts, techniques and tools-revised edition (2015)
  16. The Student ensemble of correlation matrices: eigenvalue spectrum and Kullback-Leibler entropy (2007) — link
  17. Spectral analysis of large dimensional random matrices (2010)
  18. On the empirical distribution of eigenvalues of a class of large dimensional random matrices (1995)
  19. Spectral convergence for a general class of random matrices (2011)
  20. Overparametrized Regression: Ridgeless Interpolation (2023)
  21. Surprises in high-dimensional ridgeless least squares interpolation (2022)
  22. On asymptotics of eigenvectors of large sample covariance matrix (2007)
  23. Central limit theorem for hotelling’s T 2 statistic under large dimension (2011)
  24. Portfolio Selection (1952) — link
  25. Distribution of eigenvalues for some sets of random matrices (1967)
  26. An introduction to multivariate statistical analysis (1958)
  27. Estimation for Markowitz efficient portfolios (1980)
  28. Economic Forecasts Using Many Noises (2023) — link
  29. Asset allocation under distribution uncertainty (2004)
  30. Portfolio theory: as I still see it (2010)
  31. A brief review of portfolio optimization techniques (2023)
  32. Robust portfolio optimization: a categorized bibliographic review (2020)
  33. Optimal portfolio choice with unknown benchmark efficiency (2024)
  34. Forest through the trees: Building cross-sections of stock returns (2023)
  35. Corporate investment and asset price dynamics: Implications for the cross-section of returns (2004)
  36. The value premium (2005)
  37. A generalized approach to portfolio optimization: Improving performance by constraining portfolio norms (2009)
  38. Estimation Risk and OptimalPortfolio Choice North-Holland (1979)
  39. Risk reduction in large portfolios: Why imposing the wrong constraints helps (2003)
  40. Keynes Meets Markowitz: The Trade of Between Familiarity and Diversification (2010)
  41. Markowitz meets Talmud: A combination of sophisticated and naive diversification strategies (2011)
  42. Anomalies and the expected market return (2022)
  43. Universal Portfolio Shrinkage (2024)
  44. The virtue of complexity everywhere (2022)
  45. APT or “ AIPT ”? The Surprising Dominance of Large Factor Models (2024)
  46. Structural deep learning in conditional asset pricing (2023)
  47. The virtue of complexity in return prediction (2024)
  48. Machine learning and the implementable efficient frontier (2024)
  49. Factor models, machine learning, and asset pricing (2022)
  50. Financial machine learning (2023)
  51. Machine learning vs. economic restrictions: Evidence from stock return predictability (2023)
  52. AlphaPortfolio: Direct construction through deep reinforcement learning and interpretable AI (2022)
  53. Asset pricing with panel tree under global split criteria (2023)
  54. Portfolio choices with many big models (2022)
  55. Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets Goldilocks (2017)
  56. Portfolio selection with parameter and model uncertainty: A multi-prior approach (2007)
  57. Optimal versus naive diversification: How inefficient is the 1/ N portfolio strategy? (2009)
  58. The effect of errors in means, variances, and covariances on optimal portfolio choice (1993)
  59. Complexity in factor pricing models (2023)
  60. Time-varying minimum variance portfolio (2022)
  61. Factor models for portfolio selection in large dimensions: The good, the better and the ugly (2021)
  62. Sparse and stable Markowitz portfolios (2009)
  63. Benign overfitting in linear regression (2020)
  64. Eigenvectors of some large sample covariance matrix ensembles (2011)
  65. A regularized Hotelling’s T 2 test for pathway analysis in proteomic studies (2011)
  66. Empirical asset pricing via machine learning (2020)
  67. Anisotropic local laws for random matrices (2017)
  68. Generalized inversion of modified matrices (1973)
  69. Large sample covariance matrices without independence structures in columns (2008)
  70. Limiting spectral distribution for large sample covariance matrices with m-dependent elements (2010)
  71. The limiting spectral distribution for large sample covariance matrices with unbounded m-dependent entries (2016)
  72. Gaussian fluctuations for sample covariance matrices with dependent data (2013)
  73. Reconciling modern machine-learning practice and the classical bias–variance trade-off (2019)
  74. Harmless interpolation of noisy data in regression (2020)
  75. Two models of double descent for weak features (2020)
  76. Understanding overfitting peaks in generalization error: Analytical risk curves for $ l_2 $ and $ l_1 $ penalized interpolation (2019) — link
  77. JUST INTERPOLATE: KERNEL “RIDGELESS” REGRESSION CAN GENERALIZE (2020)
  78. The generalization error of random features regression: Precise asymptotics and the double descent curve (2022)
  79. Deep learning: a statistical viewpoint (2021)
  80. Exact expressions for double descent and implicit regularization via surrogate random design (2020)
  81. On the role of optimization in double descent: A least squares study (2021)
  82. A farewell to the bias-variance tradeoff? an overview of the theory of overparameterized machine learning (2021) — link
  83. High-dimensional dynamics of generalization error in neural networks (2020)
  84. Linear and nonlinear extension of the pseudo-inverse solution for learning boolean functions (1989)
  85. Benign overfitting and noisy features (2023)
  86. Autoencoder asset pricing models (2021)
  87. Deep learning in asset pricing (2024)
  88. The sampling error in estimates of mean-variance efficient portfolio weights (1999)
  89. Mean–variance portfolio optimization with deep learning based-forecasts for cointegrated stocks (2022)
  90. Optimal portfolio choice with parameter uncertainty (2007)
  91. APT or “AIPT” ? The Surprising Dominance of Large Factor Models (2024)
  92. Growing the efficient frontier on panel trees (2022)
  93. Time-series and cross-sectional stock return forecasting: New machine learning methods (2020)
  94. Bond risk premiums with machine learning (2021)
  95. A well-conditioned estimator for large-dimensional covariance matrices (2004)
  96. Estimation of the global minimum variance portfolio in high dimensions (2018)
  97. Two is better than one: Regularized shrinkage of large minimum variance portfolios (2024)
  98. Optimal portfolio choice with estimation risk: No risk-free asset case (2022)
  99. The risk of expected utility under parameter uncertainty (2024)
  100. Machine learning in asset management—Part 2: Portfolio construction—Weight optimization (2020)
  101. Deep learning for portfolio optimization (2020) — link
  102. Portfolio optimization with 2D relative-attentional gated transformer (2020)
  103. Do Common Factors Really Explain the Cross-Section of Stock Returns? (2020)
  104. Efficient portfolio selection in a large market (2016)
  105. Eigen portfolio selection: A robust approach to sharpe ratio maximization (2018)
  106. Finite sample approximation results for principal component analysis: A matrix perturbation approach (2008)
  107. On the effect of noise on fitting linear regression models (2024) — link
  108. Deep double descent: Where bigger models and more data hurt (2021)
  109. Forecasting using principal components from a large number of predictors (2002)
  110. Inferential theory for factor models of large dimensions (2003)
  111. Large covariance estimation by thresholding principal orthogonal complements (2013)
  112. l2-Relaxation: With Applications to Forecast Combination and Portfolio Analysis (2022)

0 of 120 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.