{"paper_id":"2308.01915","title":"LOB-Based Deep Learning Models for Stock Price Trend Prediction: A Benchmark Study","authors":["Matteo Prata","Giuseppe Masi","Leonardo Berti","Viviana Arrigoni","Andrea Coletta","Irene Cannistraci","Svitlana Vyetrenko","Paola Velardi","Novella Bartolini"],"paper_date":"2023-07-05","paper_type":"dataset-benchmark","paper_url":"https://arxiv.org/pdf/2308.01915v2","page_url":"https://thequant.space/flowcharts/lob-based-deep-learning-models-for-stock-price-trend-predict/","scores":{"empirical_rigor":7,"math_complexity":6.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper releases LOBCAST, an open‑source modular framework for preprocessing, training, testing and profit analysis of LOB‑based stock price trend models.","quote":"We release a highly modular open-source framework called LOBCAST1, to pre-process data, train, and test stock market models.","summary":"The recent advancements in Deep Learning (DL) research have notably influenced the finance sector. We examine the robustness and generalizability of fifteen state-of-the-art DL models focusing on Stock Price Trend Prediction (SPTP) based on Limit Order Book (LOB) data. To carry out this study, we…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Each trade uses a fixed size of one share, implying negligible market impact.","type":"execution","stated":"explicit","quote":"we set the number of shares per trade to a fixed value of 1, simplifying our analysis and assuming a negligible market impact."},{"assumption":"The simulated trader starts with an initial capital of $10,000.","type":"execution","stated":"explicit","quote":"our simulated trader begins with an initial capital of $10.000"},{"assumption":"No transaction fees are applied in the trading simulation.","type":"execution","stated":"explicit","quote":"we make the assumption of no transaction fees."},{"assumption":"Trading signals are generated every 10 order‑book events.","type":"model","stated":"explicit","quote":"The trading strategy relies on the models and operates by generating signals every 10 events to predict subsequent price movements."},{"assumption":"Orders are filled at the next open price, effectively assuming no slippage.","type":"execution","stated":"implied","quote":"The orders are filled at the next open price."},{"assumption":"Trades are executed at the mid‑price, ignoring market impact and other frictions.","type":"execution","stated":"explicit","quote":"we assume the trades execution at the mid-price, and we ignore transaction costs"}]},"datasets":{"status":"available","items":[{"name":"FI-2010","provider":"Ntakaris et al.","kind":"order-book","access":"public","period":null,"universe":"five Finnish companies (Kesko Oyj, Outokumpu Oyj, Sampo, Rautaruukki, Wärtsilä)","quote":"The most widely spread public LOB dataset is FI-2010 which is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0) and was proposed in 2017 by Ntakaris et al.","known_id":"fi-2010"},{"name":"LOB-2021","provider":"LOBSTER (reconstructed NASDAQ data)","kind":"order-book","access":"commercial","period":null,"universe":null,"quote":"datasets called LOB-2021 and LOB-2022, extrapolated from the LOBSTER dataset","known_id":"lobster"},{"name":"LOB-2022","provider":"LOBSTER (reconstructed NASDAQ data)","kind":"order-book","access":"commercial","period":null,"universe":null,"quote":"datasets called LOB-2021 and LOB-2022, extrapolated from the LOBSTER dataset","known_id":"lobster"},{"name":"LOBSTER","provider":"LOBSTER (online LOB data provider)","kind":"order-book","access":"commercial","period":null,"universe":null,"quote":"To test the generalizability of the models in a more realistic scenario, we used data extracted from LOBSTER [13], an online LOB data provider for order book data, which is not available for free","known_id":"lobster"}],"known":[{"id":"lobster","name":"LOBSTER","provider":"LOBSTER","kind":"order-book","access":"commercial","quote":"To test the generalizability of the models in a more realistic scenario, we used data extracted from LOBSTER [13], an online LOB data provider for order book data, which is not available for free, as is often the case for critical applications such as health and finance [9].","mentions":15,"used_in":4}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"l2-book"},"transaction_costs":{"status":"available","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"conflict","rule":"R4b","candidates":{"front_matter":"n/a","evidence_card":"no","evidence_card_quoted":true},"conflict_kind":"applicability","card_detail":null,"card_quote":"we assume the trades execution at the mid-price, and we ignore transaction costs","meaning":"the platform classifier and the paper-text extraction disagree, so no value is asserted"},"results":{"status":"available","evaluation_type":"forecast-accuracy","findings":[{"finding":"All fifteen DL models suffer a significant performance drop when evaluated on unseen LOBSTER data, questioning real‑world applicability.","kind":"negative","quote":"Our extensive experiments reveal that all models exhibit a significant performance drop when exposed to new data, thereby raising questions about their real-world market applicability."},{"finding":"The BINCTABL model achieves the highest average F1‑score (82.6% ± 7.0) and the best robustness score (99.7) on the FI‑2010 dataset.","kind":"performance","quote":"... the BINCTABL model demonstrates the highest F1-score when averaged over the seeds and prediction horizons, achieving an average of 82.6% ± 7.0."},{"finding":"Generalizability tests show overall F1‑scores drop to 48‑61% on LOB‑2021/2022, far below FI‑2010 results.","kind":"performance","quote":"the overall performance of all models on the LOB-2021/2022 dataset is still significantly lower than on the FI-2010 dataset, ranging 48-61% in F1-score."},{"finding":"Trading simulation profits are positive only for stocks with high daily returns (e.g., LSTR, NFLX) and negative for stocks with high negative returns (e.g., SOFI, SHLS).","kind":"performance","quote":"the two stocks with the highest positive daily returns (namely LSTR and NFLX) are the only ones for which the strategy is profitable. On the other hand, the two stocks with the highest negative daily returns (SOFI and SHLS) are the ones for which most models show a negative return."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":18,"citations":{"cited_by_in_corpus":8,"by_independent_authors":6,"how_cited":{"used_as_baseline":0,"uses_method_or_data":1,"extends":0,"discusses_findings":2,"mentions":2,"unclassified":3},"independent_papers":[{"paper":{"paper_id":"2602.00776","title":"Explainable Patterns in Cryptocurrency Microstructure","paper_date":"2026-01-31","authors":["Bartosz Bieganowski","Robert Ślepaczuk"],"empirical_rigor":8.5,"page_url":"https://thequant.space/flowcharts/explainable-patterns-in-cryptocurrency-microstructure/"},"how":"discusses_findings","quote":"conduct a benchmark study of LOB-based deep learning models, confirming their robustness in trend prediction."},{"paper":{"paper_id":"2505.22678","title":"An Efficient deep learning model to Predict Stock Price Movement Based on Limit Order Book","paper_date":"2025-05-14","authors":["Jiahao Yang","Ran Fang","Ming Zhang","Jun Zhou"],"empirical_rigor":6.5,"page_url":"https://thequant.space/flowcharts/an-efficient-deep-learning-model-to-predict-stock-price-move/"},"how":"discusses_findings","quote":"Extensive experiments reveal significant performance drops in all models when exposed to new data [CITED] ."},{"paper":{"paper_id":"2405.18938","title":"HLOB -- Information Persistence and Structure in Limit Order Books","paper_date":"2024-05-29","authors":["Antonio Briola","Silvia Bartolucci","Tomaso Aste"],"empirical_rigor":7.5,"page_url":"https://thequant.space/flowcharts/hlob----information-persistence-and-structure-in-limit-order/"},"how":"uses_method_or_data","quote":null}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/matteoprata/LOBCAST","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:25:15Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}