Paper: arXiv 2309.11400
Authors: Paul Bilokon, Yitao Qiu
Abstract
With the rapid development of artificial intelligence, long short term memory (LSTM), one kind of recurrent neural network (RNN), has been widely applied in time series prediction. Like RNN, Transformer is designed to handle the sequential data. As Transformer achieved great success in Natural Language Processing (NLP), researchers got interested in Transformer’s performance on time series prediction, and plenty of Transformer-based solutions on long time series forecasting have come out recently. However, when it comes to financial time series prediction, LSTM is still a dominant architecture. Therefore, the question this study wants to answer is: whether the Transformer-based model can be applied in financial time series prediction and beat LSTM. To answer this question, various LSTM-based and Transformer-based models are compared on multiple financial prediction tasks based on high-frequency limit order book data. A new LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction. The experiment result reflects that the Transformer-based model only has the limited advantage in absolute price sequence prediction. The LSTM-based models show better and more robust performance on difference sequence prediction, such as price difference and price movement.
Complexity vs Empirical Score
- Math Complexity: 5.5/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces a novel LSTM-based model (DLSTM) and adapts Transformer architectures for financial prediction, involving moderate algorithmic complexity, while conducting rigorous experiments on real high-frequency limit order book data with profitability simulations and sharing source code.
Research Flowchart
flowchart TD
RQ["Research Question: Can Transformers beat LSTMs in financial time series prediction?"] --> Inputs
Inputs["Input Data<br>High-Frequency Limit Order Book"] --> Method["Methodology"]
subgraph Method ["Model Comparison"]
direction LR
M1["LSTM Models<br>including new DLSTM"]
M2["Transformer Models<br>adapted for finance"]
end
Method --> Proc["Computational Process<br>Comparison across multiple financial tasks"]
Proc --> Out["Key Findings"]
subgraph Out ["Outcomes"]
F1["Transformers: Limited advantage in<br>absolute price prediction"]
F2["LSTMs: Better & robust performance<br>in difference/price movement prediction"]
end
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper introduces a new LSTM-based model called DLSTM and designs a new Transformer architecture adapted for financial prediction.
Source sentence
A new LSTM-based model called DLSTM is built and new architecture for the Transformer-based model is designed to adapt for financial prediction.
Assumes 5
- Price difference sequences are approximately stationary and easier to predict than absolute prices. statistical
Source sentence
According to Ruppert [5], the absolute price information is generally non-stationary, while the price change information, such as price difference and return and approximately stationary.
- Mid-price movement can be approximated by a classification task using smoothed labels. model
Source sentence
For this reason, an easier classification task for predicting mid-price movement is introduced here.
- Transaction costs are a flat 0.002% per trade in the trading simulation. execution
Source sentence
of 0.002% to further compare the robustness among different models.
- Training, validation and test sets are normalized by the training set mean and standard deviation. data
Source sentence
test set are normalized by the mean and standard deviation of the training set.
Show 1 more
- Models are trained for 10 epochs with early stopping using Adam optimizer. model
Source sentence
All the models are trained for 10 epochs using the Adaptive Momentum Estimation optimizer and L2 loss with early stopping.
Data
- BTC-USDT LOB data (one day) order-book public
Source sentence
one-day LOB data of product BTC-USDT (Bitcoin-U.S. dollar tether) on 2022.07.15 containing 863397 ticks
- BTC-USDT LOB data (four days) order-book public
Source sentence
four days of LOB data for the product BTC-USDT from 2022.07.03 (inclusive) to 2022.07.06 (inclusive) is used, containing 3432211 ticks
- ETH-USDT LOB data (12 days) order-book public
Source sentence
the whole dataset contains 12 days of LOB data of product ETH-USDT (Ethereum-U.S. dollar tether) from 2022.07.03 (inclusive) to 2022.07.14 (inclusive), containing 10255144 ticks
Costs
Partial
A flat 0.002% commission fee is applied in the simple trading simulation.
Source sentence
of 0.002% to further compare the robustness among different models.
Finds 5
- FEDformer and Autoformer achieve lower MSE and MAE than LSTM in mid-price prediction across all horizons. performance
Source sentence
Both FEDformer and Autoformer outperform LSTM and FEDformer has the best performance in all the prediction lengths.
- LSTM outperforms Informer, Reformer, and vanilla Transformer in mid-price prediction, showing robustness to LOB data. performance
Source sentence
LSTM outperforms the other three models: Informer, Reformer, and vanilla Transformer, which indicates that LSTM is robust in handling LOB data
- DLSTM achieves the highest cumulative return (CPR) and Sharpe Ratio (SR) under a 0.002% transaction cost, outperforming all other models. performance
Source sentence
DLSTM’s strong profitability and robustness against the risk brought by the transaction cost. LSTM-based methods’ performance is generally better than Transformer-based methods.
- Transformer-based models’ performance degrades significantly with transaction costs, with Autoformer yielding negative CPRs at longer horizons. negative
Source sentence
Interestingly, Transformer-based models’ performance drops significantly under the transaction cost. The state-of-the-art Autoformer produces even less profit than vanilla Transformer, yielding negative CPRs and SRs in prediction horizon 50 and 100
- In mid-price difference prediction, LSTM-based methods are more robust and achieve higher R² than Transformer-based models, which are limited by their time decomposition architecture. performance
Source sentence
LSTM-based methods show their robustness in time series prediction and perform better than Transformer-based models, which reach the highest 11.5% R2 in around 10 prediction steps.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 37 related papers). Coverage is partial, so this is neither replication nor refutation.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.