Paper: arXiv 2304.09947

Authors: Jiaju Miao, Pawel Polak

Abstract

We propose a gradient-free online ensemble learning algorithm that dynamically combines forecasts from a heterogeneous set of machine learning models based on their recent predictive performance, measured by out-of-sample R-squared. The ensemble is model-agnostic, requires no gradient access, and is designed for sequential forecasting under nonstationarity. It adaptively reweights 16 constituent models-three linear benchmarks (OLS, PCR, LASSO) and thirteen nonlinear learners including Random Forests, Gradient-Boosted Trees, and a hierarchy of neural networks (NN1-NN12). We apply the framework to sector rotation, using sector-level features aggregated from firm characteristics. Empirically, sector returns are more predictable and stable than individual asset returns, making them suitable for cross-sectional forecasting. The algorithm constructs sector-specific ensembles that assign adaptive weights in a rolling-window fashion, guided by forecast accuracy. Our key theoretical result bounds the online forecast regret directly in terms of realized out-of-sample R-squared, providing an interpretable guarantee that the ensemble performs nearly as well as the best model in hindsight. Empirically, the ensemble consistently outperforms individual models, equal-weighted averages, and traditional offline ensembles, delivering higher predictive accuracy, stronger risk-adjusted returns, and robustness across macroeconomic regimes, including during the COVID-19 crisis.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a strong combination of theoretical guarantees (bounding online forecast regret) and rigorous empirical evaluation across 16 heterogeneous constituent models. Applying gradient-free online ensemble learning specifically to sector rotation addresses nonstationarity effectively, demonstrating robust out-of-sample and regime-dependent performance.

Research Flowchart

  flowchart TD
    A["<b>Research Goal:</b> Adaptive sector rotation forecasting under nonstationarity"] --> B["<b>Data & Inputs:</b> Firm characteristics aggregated to sector-level features"]
    B --> C["<b>Constituent Models:</b> 16 Learners (OLS, PCR, LASSO, RF, GBDT, NN1-NN12)"]
    C --> D["<b>Online Ensemble Algorithm:</b> Gradient-free reweighting via rolling-window OOS R²"]
    D --> E["<b>Theoretical Outcome:</b> Regret bound tied directly to OOS R²"]
    D --> F["<b>Empirical Findings:</b> Outperforms benchmarks with higher risk-adjusted returns & regime robustness"]