{"paper_id":"2411.16585","title":"MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series","authors":["Aaron Wheeler","Jeffrey D. Varner"],"paper_date":"2024-11-25","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2411.16585v1","page_url":"https://thequant.space/flowcharts/marketgpt-developing-a-pre-trained-transformer-gpt-for-mo/","scores":{"empirical_rigor":7,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a generative pre‑trained transformer (GPT) that serves as an order‑generation engine within a discrete‑event simulator to model financial time‑series and limit‑order‑book dynamics.","quote":"This work presents a generative pre-trained transformer (GPT) designed for modeling financial time series.","summary":"This work presents a generative pre-trained transformer (GPT) designed for modeling financial time series. The GPT functions as an order generation engine within a discrete event simulator, enabling realistic replication of limit order book dynamics. Our model leverages recent advancements in large…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The model is trained and evaluated on Nasdaq TotalView‑ITCH 5.0 message data, excluding pre‑market (before 9:30 AM ET) and after‑hours (after 4:00 PM ET) messages.","type":"data","stated":"implied","quote":"We excluded all pre-market (before 9:30 AM ET) and after-hours messages (after 4:00 PM ET) and included messages from all price levels of the order book.","support":"partial"},{"assumption":"A full level‑3 representation of the order book is maintained, assuming that deeper book messages contain valuable information for the simulation.","type":"model","stated":"implied","quote":"Maintaining a full level-3 representation of orders across the book was also vital for evaluation purposes, such as estimating functional forms and computing distribution tails.","support":"full"},{"assumption":"The transformer architecture relies on attention sinks and a dedicated sink token to enable streaming generation beyond the original context window.","type":"model","stated":"implied","quote":"The intuition behind this approach is that models learn to use the first few tokens in a sequence for offloading attention scores (since softmax must be summed to one regardless of how relevant the previous tokens are) and attention sinks serve as a mechanism for the model to do so once the initial context length is surpassed. We found that the use of a single dedicated sink token during training and inference time","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Nasdaq TotalView‑ITCH 5.0 message data","provider":"Nasdaq","kind":"order-book","access":"unknown","period":null,"universe":null,"quote":"2 Materials and Methods Data We utilized historical Nasdaq TotalView-ITCH 5.0 message data to train our model.","support":"full","known_id":"itch"}],"known":[{"id":"itch","name":"Nasdaq TotalView-ITCH","provider":"Nasdaq","kind":"order-book","access":"commercial","quote":"2 Materials and Methods Data We utilized historical Nasdaq TotalView-ITCH 5.0 message data to train our model.","mentions":5,"used_in":1}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"l2-book"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"The model successfully reproduces key features of order‑flow data, including the distribution of message types, order sizes, and inter‑arrival times.","kind":"statistical","quote":"Our results demonstrate that the model successfully reproduces key features of order flow data, even when the initial order flow prompt is no longer present within the model's context window. We found that our model was able to reproduce several key features of the order flow data, including the distribution of message types, order sizes, and interarrival times.","support":"full"},{"finding":"Evaluations show that the model captures several stylized facts of real financial markets, such as heavy‑tailed return distributions and volatility clustering.","kind":"statistical","quote":"Moreover, evaluations reveal that the model captures several statistical properties, or 'stylized facts', characteristic of real financial markets and broader macro-scale data distributions. We also found that our model was able to reproduce the heavy-tailed property of the returns distribution, the volatility clustering of returns, and the power law behavior of absolute returns.","support":"full"},{"finding":"The model can generate realistic order flow well after the initial prompt, enabling long‑sequence generation of plausible artificial market data.","kind":"statistical","quote":"Our model demonstrates the ability to produce realistic order flow data well after the initial prompt, which is a key feature for generating long sequences of plausible artifical market data.","support":"full"},{"finding":"Generated price series exhibit a Brownian‑like roughness and remain within a close region of each other, despite the model not being trained to forecast prices.","kind":"statistical","quote":"These price series appear qualitatively similar (they share a distinct roughness and overall ”Brownian” appearance) and stay within close region of one other. We found this result impressive considering that this model was not trained to forecast price series whatsoever, but rather predict the next token (message sub-element) in the sequence.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"available","supports":[],"contradicts":[],"qualifies":[],"compared_with":28,"citations":{"cited_by_in_corpus":7,"by_independent_authors":7,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":1,"discusses_findings":0,"mentions":4,"unclassified":2},"independent_papers":[{"paper":{"paper_id":"2509.00982","title":"Prospects of Imitating Trading Agents in the Stock Market","paper_date":"2025-08-31","authors":["Mateusz Wilinski","Juho Kanniainen"],"empirical_rigor":2.5,"page_url":"https://thequant.space/flowcharts/prospects-of-imitating-trading-agents-in-the-stock-market/"},"how":"extends","quote":null}],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/aaron-wheeler/MarketGPT","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:53:48Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}