Paper: arXiv 2604.00064

Authors: Pierre Andreoletti

Abstract

We study trajectory forecasting under squared loss for time series with weak conditional structure, using highly expressive prediction models. Building on the classical characterization of squared-loss risk minimization, we emphasize regimes in which the conditional expectation of future trajectories is effectively degenerate, leading to trivial Bayes-optimal predictors (flat for prices and zero for returns in standard financial settings). In this regime, increased model expressivity does not improve predictive accuracy but instead introduces spurious trajectory fluctuations around the optimal predictor. These fluctuations arise from the reuse of noise and result in increased prediction variance without any reduction in bias. This provides a process-level explanation for the degradation of Transformerbased forecasts on financial time series. We complement these theoretical results with numerical experiments on high-frequency EUR/USD exchange rate data, analyzing the distribution of trajectory-level forecasting errors. The results show that Transformer-based models yield larger errors than a simple linear benchmark on a large majority of forecasting windows, consistent with the variance-driven mechanism identified by the theory.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a strong theoretical explanation for the failure of deep learning architectures like Transformers in financial time-series forecasting. It effectively links statistical learning theory (variance explosion from noise fitting under squared loss) with empirical backtests against simple benchmarks on high-frequency FX data.

Research Flowchart

  flowchart TD
    A[Research Goal: Explain Transformer Forecast Collapse in Financial Time Series] --> B[Data Input: High-Frequency EUR/USD Exchange Rate Data]
    B --> C[Methodology: Analyze Trajectory Forecasting under Squared-Loss Risk Minimization]
    C --> D[Theory: Prove Conditional Expectation Degenerates to Trivial Bayes-Optimal Predictor]
    D --> E[Mechanism: Expressive Transformers Reuse Noise, Increasing Variance without Reducing Bias]
    E --> F[Empirical Evaluation: Compare Transformer vs. Simple Linear Benchmark Trajectory Errors]
    F --> G[Finding: Transformers Exceed Linear Error on Most Windows via Variance-Driven Collapse]