Less is more: AI Decision-Making using Dynamic Deep Neural Networks for Short-Term Stock Index Prediction

ArXiv ID: 2408.11740 “View on arXiv”

Authors: Unknown

Abstract

In this paper we introduce a multi-agent deep-learning method which trades in the Futures markets based on the US S&P 500 index. The method (referred to as Model A) is an innovation founded on existing well-established machine-learning models which sample market prices and associated derivatives in order to decide whether the investment should be long/short or closed (zero exposure), on a day-to-day decision. We compare the predictions with some conventional machine-learning methods namely, Long Short-Term Memory, Random Forest and Gradient-Boosted-Trees. Results are benchmarked against a passive model in which the Futures contracts are held (long) continuously with the same exposure (level of investment). Historical tests are based on daily daytime trading carried out over a period of 6 calendar years (2018-23). We find that Model A outperforms the passive investment in key performance metrics, placing it within the top quartile performance of US Large Cap active fund managers. Model A also outperforms the three machine-learning classification comparators over this period. We observe that Model A is extremely efficient (doing less and getting more) with an exposure to the market of only 41.95% compared to the 100% market exposure of the passive investment, and thus provides increased profitability with reduced risk.

Keywords: Multi-Agent Systems, Futures Trading, Deep Learning, S&P 500, Algorithmic Trading, Futures

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 7.0/10
• Quadrant: Street Traders
• Why: The paper relies on standard machine learning architectures (LSTM, Random Forest, GBT) with minimal novel mathematical derivations, but demonstrates strong empirical rigor through a 6-year backtest on real futures data with detailed performance metrics and risk adjustments.

  flowchart TD
    A["Research Goal: Develop an efficient<br>AI model for S&P 500 Futures trading"] --> B["Methodology: Model A<br>Multi-Agent Deep Learning"]
    B --> C["Data Input: Historical Prices & Derivatives<br>(2018-2023)"]
    C --> D["Computational Process: Day-to-day<br>Trading Signals (Long/Short/Closed)"]
    D --> E["Comparison: vs. LSTM, RF, GBT<br>and Passive Buy-and-Hold"]
    E --> F["Outcome 1: 41.95% Market Exposure<br>(vs 100% Passive)"]
    E --> G["Outcome 2: Outperformed Passive &<br>ML Comparators in Key Metrics"]
    F & G --> H["Conclusion: High Efficiency<br>(Less Exposure, More Profit)"]