Transformers versus LSTMs for electronic trading

ArXiv ID: 2309.11400 “View on arXiv”

Authors: Unknown

Abstract

With the rapid development of artificial intelligence, long short term memory (LSTM), one kind of recurrent neural network (RNN), has been widely applied in time series prediction. Like RNN, Transformer is designed to handle the sequential data. As Transformer achieved great success in Natural Language Processing (NLP), researchers got interested in Transformer’s performance on time series prediction, and plenty of Transformer-based solutions on long time series forecasting have come out recently. However, when it comes to financial time series prediction, LSTM is still a dominant architecture. Therefore, the question this study wants to answer is: whether the Transformer-based model can be applied in financial time series prediction and beat LSTM. To answer this question, various LSTM-based and Transformer-based models are compared on multiple financial prediction tasks based on high-frequency limit order book data. A new LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction. The experiment result reflects that the Transformer-based model only has the limited advantage in absolute price sequence prediction. The LSTM-based models show better and more robust performance on difference sequence prediction, such as price difference and price movement.

Keywords: LSTM (Long Short-Term Memory), Transformer Models, Time Series Forecasting, Limit Order Book, High-Frequency Data, Equities

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Street Traders
• Why: The paper introduces a novel LSTM-based model (DLSTM) and adapts Transformer architectures for financial prediction, involving moderate algorithmic complexity, while conducting rigorous experiments on real high-frequency limit order book data with profitability simulations and sharing source code.

  flowchart TD
    RQ["Research Question: Can Transformers beat LSTMs in financial time series prediction?"] --> Inputs
    Inputs["Input Data<br>High-Frequency Limit Order Book"] --> Method["Methodology"]
    subgraph Method ["Model Comparison"]
        direction LR
        M1["LSTM Models<br>including new DLSTM"]
        M2["Transformer Models<br>adapted for finance"]
    end
    Method --> Proc["Computational Process<br>Comparison across multiple financial tasks"]
    Proc --> Out["Key Findings"]
    subgraph Out ["Outcomes"]
        F1["Transformers: Limited advantage in<br>absolute price prediction"]
        F2["LSTMs: Better & robust performance<br>in difference/price movement prediction"]
    end