Paper: arXiv 2603.14453

Authors: Harris Buchanan, Eric Benhamou

Abstract

Trend-following strategies underpin many systematic trading approaches yet struggle under nonstationary and nonlinear market regimes. We propose an LSTM-based framework to forecast next-day trend differences ($Δ_t$) for the top 30 S&P 500 equities, validated across market cycles (2005–2025). Key contributions include: (i) formal proof of bias-variance reduction via differencing, (ii) exhaustive empirical benchmarks against OLS, Ridge, and Lasso, (iii) portfolio simulations confirming economic gains in terms of overall PNL compared to other models like OLS, Ridge, Lasso or LightGBM Regressor

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a solid empirical evaluation over a long historical period (2005-2025) with extensive baseline comparisons against traditional linear models and LightGBM. It balances deep learning application with theoretical backing by providing a formal proof on bias-variance reduction via differencing. While LSTMs for trend forecasting are relatively standard, the multi-decade backtest and theoretical grounding elevate its overall quality.

Research Flowchart

  flowchart TD
    A[Research Goal: Forecast Next-Day Equity Trend Differences Δt] --> B[Data Inputs: Top 30 S&P 500 Equities 2005–2025]
    B --> C[Methodology: Differencing Transformation & LSTM Modeling]
    C --> D[Empirical Benchmarking vs OLS, Ridge, Lasso & LightGBM]
    D --> E[Outcome 1: Formal Proof of Bias-Variance Reduction]
    D --> F[Outcome 2: Portfolio Simulations Show Superior PnL Gains]

Cited References (51)

  1. Two Centuries of Trend Following (2014)
  2. Two Centuries of Trend Following (2014) — link
  3. Dissecting Investment Strategies in the Cross‑Section and Time Series (2015) — link
  4. Dissecting Investment Strategies in the Cross‑Section and Time Series (2015)
  5. Do momentum-based trading strategies work in the commodity futures markets? (2015)
  6. Basis-momentum (2019)
  7. In Pursuit of Trend-Following Beta: The Promise and Pitfalls of Replication (2024)
  8. Momentum Strategies in Futures Markets and Trend‑Following Funds (2013) — link
and 43 more
  1. Beyond Trend Following: Deep Learning for Market Trend Prediction — → 2407.13685
  2. Demystifying time-series momentum strategies: Volatility estimators, trading rules and pairwise correlations (2020)
  3. Autoencoder asset pricing models (2021)
  4. High-order Hidden Markov Model for trend prediction in financial time series (2019)
  5. Machine Learning: A Probabilistic Perspective (2012)
  6. Time Series Momentum (2012) — link
  7. Time Series Momentum (2012)
  8. Futures Markets, Bayesian Forecasting and Risk Modeling (2010)
  9. Implementing Models in Quantitative Finance: Methods and Cases (2008)
  10. XI. The asymmetrical probability-curve (1896)
  11. Following the trend: diversified managed futures trading (2023)
  12. Forecasting, structural time series models and the Kalman filter (1990)
  13. Trend without hiccups: a kalman filter approach (2018) — link
  14. Re-evaluating Short- and Long-Term Trend Factors in CTA Replication: A Bayesian Graphical Approach (2025)
  15. Kalman filter demystified: from intuition to probabilistic graphical model to real case in financial markets (2018) — link
  16. 10 structural time series models (1993)
  17. The horseshoe estimator for sparse signals (2010)
  18. Momentum trading, disposition effects and prediction of future share prices: An experimental study of multiple reference points in responses to short- and long-run return trends (2009) — link
  19. Short-term reversals and longer-term momentum around the world: Theory and evidence (2022)
  20. The risk in hedge fund strategies: Theory and evidence from trend followers (2001)
  21. Path dependent options: The case of lookback options (1991)
  22. Path dependent options:" buy at the low, sell at the high" (1979)
  23. Trend Following Struggles to Return to Vogue (2024) — link
  24. Should Trend Followers Lower Their Horizons? (2024) — link
  25. Should Trend Followers Lower Their Horizons? (2024)
  26. Novel Risk-Off CTA Strategy Passes Tariff Test (2025) — link
  27. Novel Risk-Off CTA Strategy Passes Tariff Test (2025)
  28. Cliff Asness: AI is ‘Still Just Statistics’. The AQR Head on Passive Investing, Private Equity, Tail Insurance, Trend-Following and Much More (2024) — link
  29. Designing robust trend-following system: Behind the scenes of trend-following (2018)
  30. Mutual Fund Performance and Flow–Performance Relationship (2012)
  31. On Persistence in Mutual Fund Performance (1997)
  32. Household Finance (2006)
  33. Inflation Expectations and the Phillips Curve: Evidence from Survey Data (2012)
  34. Time Series Analysis: Forecasting and Control (1976)
  35. Data Analysis Using Regression and Multilevel/Hierarchical Models (2009)
  36. A Forecast Comparison of Volatility Models: Does Anything Beat a GARCH(1,1)? (2005)
  37. Empirical Asset Pricing via Machine Learning (2018) — link
  38. Machine Learning in Finance: From Theory to Practice (2017) — link
  39. Deep learning with long short‐term memory networks for financial market predictions (2018) — link
  40. A Century of Evidence on Trend-\Following Investing (2017)
  41. Does Trend Following Still Work on Stocks? (2025) — link
  42. Trend-Following Strategies: A Practical Guide (2025) — link
  43. Re-evaluating Short- and Long-Term Trend Factors in CTA Replication: A Bayesian Graphical Approach (2025) — link

1 of 51 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.