Paper: arXiv 2603.19136

Authors: Mohammad Al Ridhawi, Mahtab Haj Ali, Hussein Al Osman

Abstract

Stock markets exhibit regime-dependent behavior where prediction models optimized for stable conditions often fail during volatile periods. Existing approaches typically treat all market states uniformly or require manual regime labeling, which is expensive and quickly becomes stale as market dynamics evolve. This paper introduces an adaptive prediction framework that adaptively identifies deviations from normal market conditions and routes data through specialized prediction pathways. The architecture consists of three components: (1) an autoencoder trained on normal market conditions that identifies anomalous regimes through reconstruction error, (2) dual node transformer networks specialized for stable and event-driven market conditions respectively, and (3) a Soft Actor-Critic reinforcement learning controller that adaptively tunes the regime detection threshold and pathway blending weights based on prediction performance feedback. The reinforcement learning component enables the system to learn adaptive regime boundaries, defining anomalies as market states where standard prediction approaches fail. Experiments on 20 S&P 500 stocks spanning 1982 to 2025 demonstrate that the proposed framework achieves 0.68% mean absolute percentage error (MAPE) for one-day predictions without the reinforcement controller and 0.59% MAPE with the full adaptive system, compared to 0.80% for the baseline integrated node transformer. Directional accuracy reaches 72% with the complete framework. The system maintains robust performance during high-volatility periods, with MAPE below 0.85% when baseline models exceed 1.5%. Ablation studies confirm that each component contributes meaningfully: autoencoder routing accounts for 36% relative MAPE degradation upon removal, followed by the SAC controller at 15% and the dual-path architecture at 7%.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces an innovative architecture combining Autoencoders, Dual Transformers, and Soft Actor-Critic (SAC) reinforcement learning for market regime routing. It demonstrates strong empirical rigor using a long historical dataset (1982-2025 across 20 S&P 500 stocks) with thorough ablation studies proving the efficacy of each component.

Research Flowchart

  flowchart TD
    A[Research Goal: Predict stock prices across dynamic market regimes without manual labeling] --> B[Data Input: S&P 500 Stocks 1982-2025]
    
    subgraph Methodology & Processing
        B --> C[Autoencoder: Detects anomalous market regimes via reconstruction error]
        C --> D[SAC RL Controller: Dynamically tunes routing thresholds & blending weights]
        D --> E[Dual Node Transformers: Specialized predictions for Stable vs. Event-Driven pathways]
    end

    E --> F[Outcomes: 0.59% MAPE & 72% Directional Accuracy]
    F --> G[Key Insights: High volatility MAPE <0.85%; AE routing contributes 36% relative error reduction]

Cited References (58)

  1. Prospect theory: An analysis of decision under risk (1979) — link
  2. Efficient capital markets: A review of theory and empirical work (1970) — link
  3. Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
  4. Stock market prediction via deep learning techniques: A survey — → 2212.12717
  5. Deep learning with long short-term memory networks for financial market predictions (2018) — link
  6. Attention is all you need (2017)
  7. A comprehensive survey on graph neural networks (2020) — link
  8. NodeFormer: A scalable graph structure learning transformer for node classification (2022)
and 50 more
  1. BERT : Pre-training of deep bidirectional transformers for language understanding (2019) — link
  2. Comparison between autoregressive integrated moving average and long short-term memory models for stock price prediction (2023) — link
  3. The ARIMA model for the Indonesia stock price (2017)
  4. A na "i ve SVM-KNN based stock market trend reversal analysis for I ndian benchmark indices (2015) — link
  5. Prediction of stock index of T ata S teel using hybrid machine learning based optimization techniques (2019) — link
  6. Predicting the direction of stock market prices using tree-based classifiers (2019) — link
  7. Convolutional neural network for stock trading using technical indicators (2022) — link
  8. Stock market trend prediction using high-order information of time series (2019) — link
  9. Recurrent neural networks approach to the financial forecast of G oogle assets (2017)
  10. Stacked deep learning structure with bidirectional long-short term memory for stock market prediction (2020) — link
  11. A new deep network model for stock price prediction (2022) — link
  12. Hybrid deep learning model for stock price prediction (2018) — link
  13. A novel graph convolutional feature based convolutional neural network for stock trend prediction (2021) — link
  14. MG-Conv : A spatiotemporal multi-graph convolutional neural network for stock market index trend prediction (2022) — link
  15. A financial forecasting model based on transformer architecture (2019)
  16. A CNN-BiLSTM-AM method for stock price prediction (2021) — link
  17. Stock price forecasting from sentiment and financial stock data using machine learning (2023) — link
  18. Stock Market Prediction Using Node Transformer Architecture Integrated with BERT Sentiment Analysis — → 2603.05917
  19. A new approach to the economic analysis of nonstationary time series and the business cycle (1989) — link
  20. Regime switching with time-varying transition probabilities (1994)
  21. Non-linear time series: A dynamical system approach (1990)
  22. Regime-based versus static asset allocation: Letting the data speak (2017) — link
  23. A survey of methods for time series change point detection (2017) — link
  24. Reducing the dimensionality of data with neural networks (2006) — link
  25. Auto-encoding variational B ayes (2014)
  26. Credit card fraud detection using deep learning based on auto-encoder and restricted B oltzmann machine (2018) — link
  27. A survey of network anomaly detection techniques (2016) — link
  28. A deep learning framework for financial time series using stacked autoencoders and long-short term memory (2017) — link
  29. Reinforcement learning: An introduction (2018)
  30. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  31. A double deep Q -learning model for energy-efficient edge scheduling (2021) — link
  32. Deep direct reinforcement learning for financial signal representation and trading (2016) — link
  33. Human-level control through deep reinforcement learning (2015) — link
  34. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  35. International asset allocation with regime shifts (2002) — link
  36. Asset allocation under multivariate regime switching (2007) — link
  37. Soft actor-critic algorithms and applications (2019) — link
  38. VIX and stock market volatility predictability: A new approach (2022) — link
  39. SemEval -2017 Task 5: Fine-grained sentiment analysis on financial microblogs and news (2017)
  40. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers (2023)
  41. iTransformer : Inverted Transformers Are Effective for Time Series Forecasting (2024)
  42. TimesNet : Temporal 2D-Variation Modeling for General Time Series Analysis (2023)
  43. Markov-Switching Vector Autoregressions: Modelling, Statistical Inference, and Application to Business Cycle Analysis (1997)
  44. Comparing Predictive Accuracy (1995) — link
  45. Time Series Analysis: Forecasting and Control (1976)
  46. Macroeconomics and Reality (1980) — link
  47. Random Forests (2001) — link
  48. The Nature of Statistical Learning Theory (1995) — link
  49. XGBoost : A Scalable Tree Boosting System (2016) — link
  50. Long Short-Term Memory (1997) — link

2 of 58 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.