Paper: arXiv 2508.04707

Authors: Alena Chan, Maria Garmonina

Abstract

We evaluate the performance of several optimizers on the task of forecasting S&P 500 Index returns with the MambaStock model. Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors. In contrast, the Lion optimizer offers notably faster training. To combine these advantages, we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper introduces new optimizer variants with a few novel activation functions (e.g., erf), but the core methodology is standard hyperparameter tuning. Empirical rigor is high due to detailed backtesting on S&P 500 data, inclusion of multiple error metrics (MSE, directional accuracy), and a reproducible GitHub link, though it lacks live trading validation.

Research Flowchart

  flowchart TD
  A["Research Goal: Find optimal optimizer for MambaStock on S&P 500"] --> B["Data: S&P 500 Historical Price Data"]
  B --> C["Model: MambaStock for Return Forecasting"]
  C --> D["Training: Parallel Comparison of Optimizers"]
  D --> E{"Evaluation: Test Error & Speed"}
  E --> F["Findings: Adam/RMSProp (Lowest Error)"]
  E --> G["Findings: Lion (Fastest Training)"]
  F & G --> H["Outcome: Roaree Optimizer Family<br>(Balances Speed & Stability)"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a new family of optimizers called Roaree that smooths Lion's oscillatory loss while keeping its training speed.

Source sentence
we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed.

Assumes 3

  • Datasets are kept causal, preventing the model from peeking into the future. data
    Source sentence
    We keep all of our datasets causal – not letting the model peek into the future.
  • The MambaStock architecture is fixed; only the optimizer varies. model
    Source sentence
    No architectural changes were made – the optimizer choice is the sole experimental factor.
  • Financial return targets are very small and noisy, causing large gradient variance across layers. market
    Source sentence
    Because financial-return targets are very small and noisy, gradients can vary by orders of magnitude across the layers of a Mamba block.

Data

  • Weekly S&P 500 Index dataset market-prices unknown
    2000-2019 · S&P 500 Index
    Source sentence
    The dataset contains weekly S&P 500 Index observations from 2000 to 2019 with historical returns, 10 engineered technical-analysis signals, three fundamental valuation ratios, and two crowd-sentiment scores

Costs

Not applicable The paper reports no trading or execution result.

Finds 4

  • Gradient‑smoothing and adaptive‑rate optimizers (e.g., Adam and RMSProp) achieve the lowest test errors among the baselines. predictive-accuracy
    Source sentence
    Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors.
  • The Lion optimizer provides notably faster training than other baselines. performance
    Source sentence
    In contrast, the Lion optimizer offers notably faster training.
  • The Roaree family smooths Lion's convergence and improves prediction accuracy, with the s_κerf surrogate (κ=10) achieving the best trade‑off. predictive-accuracy
    Source sentence
    The best surrogate function choice appears to be s_κerf with κ = 10. It retains extremely low average epoch time (reducing it even further than Lion) while also decreasing Test MSE.
  • Overall, Roaree smooths the convergence behavior of Lion, improving accuracy of predictions. predictive-accuracy
    Source sentence
    In our experiments, the Roaree family of optimizers smooths the convergence behavior of Lion, improving accuracy of predictions.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 44 related papers). Coverage is partial, so this is neither replication nor refutation.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.