Paper: arXiv 2603.02620

Authors: Federico Vittorio Cortesi, Giuseppe Iannone, Giulia Crippa, Tomaso Poggio, Pierfrancesco Beneventano

Abstract

Neural networks applied to financial time series operate in a regime of underspecification, where model predictors achieve indistinguishable out-of-sample error. Using large-scale volatility forecasting for S$&$P 500 stocks, we show that different model-training-pipeline pairs with identical test loss learn qualitatively different functions. Across architectures, predictive accuracy remains unchanged, yet optimizer choice reshapes non-linear response profiles and temporal dependence differently. These divergences have material consequences for decisions: volatility-ranked portfolios trace a near-vertical Sharpe-turnover frontier, with nearly $3\times$ turnover dispersion at comparable Sharpe ratios. We conclude that in underspecified settings, optimization acts as a consequential source of inductive bias, thus model evaluation should extend beyond scalar loss to encompass functional and decision-level implications.

Complexity vs Empirical Score

  • Math Complexity: 3.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper provides a highly novel and practical empirical finding: neural network optimizers act as implicit priors in low-signal financial settings, leading to drastically different portfolio turnover despite identical out-of-sample loss. The mathematical derivations are light, but the empirical setup, control grid, diagnostic tools, and open-source code availability make it an exceptionally rigorous paper.

Research Flowchart

  flowchart TD
    A[Research Goal: Evaluate Implicit Optimizer Bias<br/>in Underspecified Financial Forecasting] --> B[Data: S&P 500 Equities<br/>Large-Scale Volatility Time Series]
    B --> C[Train Neural Networks across Architectures<br/>& Optimizers e.g., Adam vs. SGD]
    C --> D[Evaluate Model Outputs at 3 Levels:<br/>Scalar Loss, Function Profile & Portfolio Decision]
    D --> E[Finding 1: Identical Test Loss<br/>across Pipelines]
    D --> F[Finding 2: Distinct Non-Linear<br/>& Temporal Responses]
    D --> G[Finding 3: 3x Turnover Dispersion<br/>at Comparable Sharpe Ratios]

Cited References (138)

  1. Beyond the Edge of Stability via Two-step Gradient Updates (2023) — link
  2. Adaptive Gradient Methods at the Edge of Stability (2022) — link
  3. Self- Stabilization : The Implicit Bias of Gradient Descent at the Edge of Stability (2023) — link
  4. Understanding Optimization in Deep Learning with Central Flows (2024) — link
  5. The Break - Even Point on Optimization Trajectories of Deep Neural Networks (2020) — link
  6. On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length (2019) — link
  7. Underspecification Presents Challenges for Credibility in Modern Machine Learning (2020) — link
  8. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) (2001) — link
and 130 more
  1. Rashomon (1950)
  2. Predictive Multiplicity in Classification (2020) — link
  3. On the Existence of Simpler Machine Learning Models (2022) — link
  4. Amazing Things Come From Having Many Good Models (2024) — link
  5. On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (2017) — link
  6. Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization (2021) — link
  7. Adversarial Examples Are Not Bugs, They Are Features (2019) — link
  8. Robust Models Are More Interpretable Because Attributions Look Normal (2022) — link
  9. Implications of Model Indeterminacy for Explanations of Automated Decisions (2022) — link
  10. The Pitfalls of Simplicity Bias in Neural Networks (2020) — link
  11. Learned feature representations are biased by complexity, learning order, position, and more (2024) — link
  12. N-BEATS: Neural basis expansion analysis for interpretable time series forecasting (2020) — link
  13. One-Embedding-Fits-All: Efficient Zero-Shot Time Series Forecasting by a Model Zoo (2025) — link
  14. Model Multiplicity: Opportunities, Concerns, and Solutions (2022) — link
  15. Muon: An optimizer for hidden layers in neural networks (2024) — link
  16. Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data (2025) — link
  17. On the difficulty of training Recurrent Neural Networks (2013) — link
  18. On the Spectral Bias of Neural Networks (2019) — link
  19. Volatility-Managed Portfolios (2017) — link
  20. FinTSB: A Comprehensive and Practical Benchmark for Financial Time Series Forecasting (2025) — link
  21. A Unified Approach to Interpreting Model Predictions (2017) — link
  22. Highly accurate protein structure prediction with AlphaFold (2021)
  23. Attention Is All You Need (2017)
  24. A Comprehensive 2022 Look at the Empirical Performance of Equity Premium Prediction (2024)
  25. Backtesting on Synthetic Data (2018)
  26. Time-series Generative Adversarial Networks (2019) — link
  27. Empirical asset pricing via machine learning (2020)
  28. Deep learning in asset pricing (2024)
  29. Deconstructing what makes a good optimizer for language models (2024) — link
  30. The marginal value of adaptive gradient methods in machine learning (2017)
  31. Gradient-based learning applied to document recognition (1998) — link
  32. Long short-term memory (1997)
  33. Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability (2022) — link
  34. Edge of Stochastic Stability: Revisiting the Edge of Stability for SGD (2025) — link
  35. Optuna: A Next-generation Hyperparameter Optimization Framework (2019) — link
  36. Adam: A Method for Stochastic Optimization (2017) — link
  37. Interpretable machine learning: Fundamental principles and 10 grand challenges (2022)
  38. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously (2019)
  39. Real-time price discovery in global stock, bond and foreign exchange markets (2007)
  40. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation (1982)
  41. Neural Network Ensembles, Cross Validation, and Active Learning (1994) — link
  42. Time-series forecasting with deep learning: a survey (2021)
  43. Deep learning for finance: deep portfolios (2017)
  44. Generalized autoregressive conditional heteroskedasticity (1986) — link
  45. A Simple Approximate Long-Memory Model of Realized Volatility (2009) — link
  46. Forecasting realized volatility: Does anything beat linear models? (2024) — link
  47. Forecasting realized volatility: HAR against Principal Components Combining, neural networks and GARCH (2017) — link
  48. An artificial neural network-GARCH model for international stock return volatility (1997) — link
  49. Realized Volatility Forecasting with Neural Networks (2020) — link
  50. A Machine Learning Approach to Volatility Forecasting* (2022) — link
  51. Volatility forecasting with machine learning and intraday commonality (2023) — link
  52. The 2024 Foundation Model Transparency Index (2025) — link
  53. A decoder-only foundation model for time-series forecasting (2024) — link
  54. Deep learning in predicting cryptocurrency volatility (2022) — link
  55. Understanding deep learning requires rethinking generalization (2017) — link
  56. The Implicit Bias of Gradient Descent on Separable Data (2024) — link
  57. Theory IIIb: Generalization in Deep Networks (2018) — link
  58. Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (2020) — link
  59. Prevalence of neural collapse during the terminal phase of deep learning training (2020) — link
  60. How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD (2024) — link
  61. Implicit Regularization in Matrix Factorization (2017) — link
  62. One Fits All:Power General Time Series Analysis by Pretrained LM (2023) — link
  63. Foundation Models for Time Series Analysis: A Tutorial and Survey (2024) — link
  64. On the Estimation of Security Price Volatilities from Historical Data (1980)
  65. Forecast Combinations (2006) — link
  66. Deep-learning models for forecasting financial risk premia and their interpretations (2023) — link
  67. Power and Bipower Variation with Stochastic Volatility and Jumps (2004) — link
  68. Deep Learning for Predicting Asset Returns (2018) — link
  69. Deep Learning for Portfolio Optimization (2020) — link
  70. High frequency market making: The role of speed (2024) — link
  71. Advances in forecasting realized volatility: a review of methodologies (2026) — link
  72. The Best of Both Worlds? Predicting Realized Volatility Based on Convolutional Neural Nets and Coloured Intraday Return Images (2023) — link
  73. HARNet: A Convolutional Neural Network for Realized Volatility Forecasting (2022) — link
  74. Realized Volatility Forecasting with Neural Networks (2020)
  75. Forecasting Realized Volatility through Financial Turbulence and Neural Networks (2023) — link
  76. On Stock Volatility Forecasting under Mixed-Frequency Data Based on Hybrid RR-MIDAS and CNN-LSTM Models (2024) — link
  77. Forecasting Stock Market Volatility Using Convolutional Neural Networks (2021) — link
  78. A Combined Structural and Flexible Functional Approach for Modeling Energy Substitution (1989) — link
  79. Generalized Linear Models (1989)
  80. A Path to Simpler Models Starts with Noise (2023) — link
  81. Very Simple Classification Rules Perform Well on Most Commonly Used Datasets (1993) — link
  82. Accurate Intelligible Models with Pairwise Interactions (2013) — link
  83. Learning Certifiably Optimal Rule Lists for Categorical Data (2018) — link
  84. Fast Sparse Decision Tree Optimization via Reference Ensembles (2022) — link
  85. Fast Sparse Classification for Generalized Linear and Additive Models (2022) — link
  86. When Do Neural Nets Outperform Boosted Trees on Tabular Data? (2023) — link
  87. Exploring the Whole Rashomon Set of Sparse Decision Trees (2022) — link
  88. Exploring and Interacting with the Set of Good Sparse Generalized Additive Models (2023) — link
  89. FasterRisk: Fast and Accurate Interpretable Risk Scores (2022) — link
  90. Fast and Interpretable Mortality Risk Scores for Critical Care Patients (2023) — link
  91. On the origin of implicit regularization in stochastic gradient descent (2021) — link
  92. On the trajectories of sgd without replacement (2023) — link
  93. Noise balance and stationary distribution of stochastic gradient descent (2025)
  94. Heavy-tailed class imbalance and why adam outperforms gradient descent on language models (2024)
  95. Why transformers need adam: A hessian perspective (2024)
  96. How Memory in Optimization Algorithms Implicitly Modifies the Loss (2025) — link
  97. The Effect of Mini-Batch Noise on the Implicit Bias of Adam (2026) — link
  98. On the implicit bias of adam (2023) — link
  99. A qualitative study of the dynamic behavior for adaptive gradient algorithms (2022)
  100. Convergence rates for the Adam optimizer (2024) — link
  101. On the convergence of adam and beyond (2019) — link
  102. The Implicit Bias of Gradient Descent on Separable Data (2018) — link
  103. The implicit bias of gradient descent on nonseparable data (2019) — link
  104. Characterizing Implicit Bias in Terms of Optimization Geometry (2018) — link
  105. Gradient Descent Maximizes the Margin of Homogeneous Neural Networks (2020) — link
  106. Implicit Bias of Gradient Descent on Linear Convolutional Networks (2018) — link
  107. Implicit Regularization in Matrix Factorization (2017) — link
  108. Neural Tangent Kernel: Convergence and Generalization in Neural Networks (2018) — link
  109. Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent (2019) — link
  110. On Lazy Training in Differentiable Programming (2019) — link
  111. On the Spectral Bias of Neural Networks (2019) — link
  112. Prevalence of Neural Collapse during the Terminal Phase of Deep Learning Training (2020) — link
  113. Adversarial Examples Are Not Bugs, They Are Features (2019) — link
  114. Shortcut Learning in Deep Neural Networks (2020) — link
  115. The Pitfalls of Simplicity Bias in Neural Networks (2020) — link
  116. An Investigation of Why Overparameterization Exacerbates Spurious Correlations (2020) — link
  117. Invariant Risk Minimization (2019) — link
  118. Underspecification and Solution Multiplicity in Deep Neural Networks (2023)
  119. Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author) (2001) — link
  120. Decoupled Weight Decay Regularization (2019) — link
  121. Flat Minima (1997) — link
  122. Sharp Minima Can Generalize For Deep Nets (2017) — link
  123. Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization (2021) — link
  124. Sharpness-Aware Minimization for Efficiently Improving Generalization (2021) — link
  125. Machine Learning for Realised Volatility Forecasting (2020) — link
  126. On the Opportunities and Risks of Foundation Models (2022) — link
  127. Unified Training of Universal Time Series Forecasting Transformers (2024) — link
  128. MOMENT: A Family of Open Time-series Foundation Models (2024) — link
  129. Recent Advances in Time Series Foundation Models: Have We Reached the “BERT Moment”? (2025) — link
  130. Volatility forecasting using deep recurrent neural networks as GARCH models (2025) — link

0 of 138 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.