Minimal Batch Adaptive Learning Policy Engine for Real-Time Mid-Price Forecasting in High-Frequency Trading

ArXiv ID: 2412.19372 “View on arXiv”

Authors: Unknown

Abstract

High-frequency trading (HFT) has transformed modern financial markets, making reliable short-term price forecasting models essential. In this study, we present a novel approach to mid-price forecasting using Level 1 limit order book (LOB) data from NASDAQ, focusing on 100 U.S. stocks from the S&P 500 index during the period from September to November 2022. Expanding on our previous work with Radial Basis Function Neural Networks (RBFNN), which leveraged automated feature importance techniques based on mean decrease impurity (MDI) and gradient descent (GD), we introduce the Adaptive Learning Policy Engine (ALPE) - a reinforcement learning (RL)-based agent designed for batch-free, immediate mid-price forecasting. ALPE incorporates adaptive epsilon decay to dynamically balance exploration and exploitation, outperforming a diverse range of highly effective machine learning (ML) and deep learning (DL) models in forecasting performance.

Keywords: High-frequency trading, Limit order book, Mid-price forecasting, Reinforcement learning, Adaptive epsilon decay

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail
  • Why: The paper employs advanced reinforcement learning concepts with adaptive epsilon decay and compares against complex deep learning models, demonstrating high mathematical density. It uses a specific, high-quality dataset (100 S&P 500 stocks from NASDAQ over a defined period) and conducts rigorous empirical comparisons against multiple baselines, making it backtest-ready.
  flowchart TD
    A["Research Goal:<br>Real-Time Mid-Price<br>Forecasting in HFT"] --> B["Data Source:<br>NASDAQ LOB Level 1<br>100 S&P 500 Stocks"]
    B --> C["Methodology:<br>Adaptive Learning Policy Engine<br>Reinforcement Learning with<br>Adaptive Epsilon Decay"]
    C --> D["Computational Process:<br>Batch-Free Immediate<br>Forecasting via RL Agent"]
    D --> E{"Outcome: Comparative Analysis"}
    E --> F["Key Finding:<br>ALPE Outperforms<br>ML/DL Baselines"]
    E --> G["Key Finding:<br>Effective Balance of<br>Exploration vs. Exploitation"]