Paper: arXiv 2508.04707
Authors: Alena Chan, Maria Garmonina
Abstract
We evaluate the performance of several optimizers on the task of forecasting S&P 500 Index returns with the MambaStock model. Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors. In contrast, the Lion optimizer offers notably faster training. To combine these advantages, we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 7.5/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper introduces new optimizer variants with a few novel activation functions (e.g., erf), but the core methodology is standard hyperparameter tuning. Empirical rigor is high due to detailed backtesting on S&P 500 data, inclusion of multiple error metrics (MSE, directional accuracy), and a reproducible GitHub link, though it lacks live trading validation.
Research Flowchart
flowchart TD
A["Research Goal: Find optimal optimizer for MambaStock on S&P 500"] --> B["Data: S&P 500 Historical Price Data"]
B --> C["Model: MambaStock for Return Forecasting"]
C --> D["Training: Parallel Comparison of Optimizers"]
D --> E{"Evaluation: Test Error & Speed"}
E --> F["Findings: Adam/RMSProp (Lowest Error)"]
E --> G["Findings: Lion (Fastest Training)"]
F & G --> H["Outcome: Roaree Optimizer Family<br>(Balances Speed & Stability)"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a new family of optimizers called Roaree that smooths Lion's oscillatory loss while keeping its training speed.
Source sentence
we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed.
Assumes 3
- Datasets are kept causal, preventing the model from peeking into the future. data
Source sentence
We keep all of our datasets causal – not letting the model peek into the future.
- The MambaStock architecture is fixed; only the optimizer varies. model
Source sentence
No architectural changes were made – the optimizer choice is the sole experimental factor.
- Financial return targets are very small and noisy, causing large gradient variance across layers. market
Source sentence
Because financial-return targets are very small and noisy, gradients can vary by orders of magnitude across the layers of a Mamba block.
Data
- Weekly S&P 500 Index dataset market-prices unknown
Source sentence
The dataset contains weekly S&P 500 Index observations from 2000 to 2019 with historical returns, 10 engineered technical-analysis signals, three fundamental valuation ratios, and two crowd-sentiment scores
Costs
Not applicable
Finds 4
- Gradient‑smoothing and adaptive‑rate optimizers (e.g., Adam and RMSProp) achieve the lowest test errors among the baselines. predictive-accuracy
Source sentence
Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors.
- The Lion optimizer provides notably faster training than other baselines. performance
Source sentence
In contrast, the Lion optimizer offers notably faster training.
- The Roaree family smooths Lion's convergence and improves prediction accuracy, with the s_κerf surrogate (κ=10) achieving the best trade‑off. predictive-accuracy
Source sentence
The best surrogate function choice appears to be s_κerf with κ = 10. It retains extremely low average epoch time (reducing it even further than Lion) while also decreasing Test MSE.
- Overall, Roaree smooths the convergence behavior of Lion, improving accuracy of predictions. predictive-accuracy
Source sentence
In our experiments, the Roaree family of optimizers smooths the convergence behavior of Lion, improving accuracy of predictions.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 44 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.