Paper: arXiv 2508.02247
Authors: Yang Li, Zhi Chen
Abstract
Generative modeling of high-frequency limit order book (LOB) dynamics is a critical yet unsolved challenge in quantitative finance, essential for robust market simulation and strategy backtesting. Existing approaches are often constrained by simplifying stochastic assumptions or, in the case of modern deep learning models like Transformers, rely on tokenization schemes that affect the high-precision, numerical nature of financial data through discretization and binning. To address these limitations, we introduce ByteGen, a novel generative model that operates directly on the raw byte streams of LOB events. Our approach treats the problem as an autoregressive next-byte prediction task, for which we design a compact and efficient 32-byte packed binary format to represent market messages without information loss. The core novelty of our work is the complete elimination of feature engineering and tokenization, enabling the model to learn market dynamics from its most fundamental representation. We achieve this by adapting the H-Net architecture, a hybrid Mamba-Transformer model that uses a dynamic chunking mechanism to discover the inherent structure of market messages without predefined rules. Our primary contributions are: 1) the first end-to-end, byte-level framework for LOB modeling; 2) an efficient packed data representation; and 3) a comprehensive evaluation on high-frequency data. Trained on over 34 million events from CME Bitcoin futures, ByteGen successfully reproduces key stylized facts of financial markets, generating realistic price distributions, heavy-tailed returns, and bursty event timing. Our findings demonstrate that learning directly from byte space is a promising and highly flexible paradigm for modeling complex financial systems, achieving competitive performance on standard market quality metrics without the biases of tokenization.
Complexity vs Empirical Score
- Math Complexity: 6.0/10
- Empirical Rigor: 8.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces a novel, mathematically dense hybrid Mamba-Transformer architecture with dynamic chunking and operates on raw byte streams, but it is backed by strong empirical validation on 34 million real-world events with comprehensive backtest-ready metrics.
Research Flowchart
flowchart TD A["Research Goal: Generative modeling of high-frequency LOB dynamics without tokenization or feature engineering."] --> B["Data Input: Raw byte streams of LOB events<br>(34M events from CME Bitcoin futures)"] B --> C["Methodology: ByteGen Framework<br>• Packed binary format (32 bytes)<br>• Autoregressive next-byte prediction<br>• H-Net architecture (Mamba-Transformer hybrid)"] C --> D["Computational Process: Dynamic chunking<br>Discovers inherent market structure without predefined rules."] D --> E["Key Findings: Competitive performance<br>• Realistic price distributions<br>• Heavy-tailed returns<br>• Bursty event timing<br>(Without tokenization biases)"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes ByteGen, a generative model that learns directly from raw byte streams of limit order book events, eliminating feature engineering and tokenization.
Source sentence
we introduce ByteGen, a novel generative model that operates directly on the raw byte streams of LOB events.
Assumes 4
- The 32‑byte packed binary format represents market messages without information loss. data
Source sentence
design a compact and efficient 32-byte packed binary format to represent market messages without information loss.
- The H‑Net architecture can discover the inherent structure of market messages without predefined rules. model
Source sentence
the H‑Net architecture, a hybrid Mamba‑Transformer model that uses a dynamic chunking mechanism to discover the inherent structure of market messages without predefined rules.
- Financial markets are non‑stationary, so a fixed binning strategy may become inappropriate under regime changes. market
Source sentence
Financial markets are non‑stationary.
- A fixed binning strategy that is appropriate for a low‑volatility regime will be entirely inappropriate during a high‑volatility period. model
Source sentence
A fixed binning strategy that is appropriate for a low‑volatility regime will be entirely inappropriate during a high‑volatility period.
Data
- CME Bitcoin futures (BTCX4) Level 3 (Market‑By‑Order) data order-book commercial
Source sentence
We evaluate ByteGen on CME Bitcoin futures (BTCX4) Level 3 (Market‑By‑Order) data from CME Group via Databento.
- Training dataset order-book unknown
Source sentence
Our training dataset spans November 11‑15, 2024 (5 trading days) and contains 34.2 million order‑book messages totaling 1.02 GB in our packed 32-byte format.
Costs
Not applicable
Finds 2
- The model captures heavy‑tailed price returns but with slightly lower volatility. statistical
Source sentence
The model captures the heavy‑tailed nature of price movements, though with slightly lower volatility (12.6 bps vs 16.9 bps in real data).
- Event type distribution is biased: more cancel orders (47% vs 31%) and fewer trades than observed. performance
Source sentence
event type frequencies show systematic biases, with the model generating more cancel orders (47% vs 31%) and fewer trades than observed in real data.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (29)
- LOB-Bench: Benchmarking Generative AI for Finance – an Application to Limit Order Book Data — → 2502.09172
- MarketGPT: Developing a Pre-trained transformer (GPT) for Modeling Financial Time Series — → 2411.16585
- Generative AI for End-to-End Limit Order Book Modelling: A Token-Level Autoregressive Generative Model of Message Flow Using a Deep State Space Network — → 2309.00638
- Dynamic Chunking for End-to-End Hierarchical Sequence Modeling (2025) — link
- Volatility clustering in financial markets: empirical facts and agent-based models (2007)
- A stochastic model for order book dynamics (2010)
- Algorithmic and high-frequency trading (2015)
- Universal features of price formation in financial markets: perspectives from deep learning (2019)
and 21 more
- DeepLOB: Deep convolutional neural networks for limit order books (2019)
- Conditional generators for limit order book environments: Explainability, challenges, and robustness — → 2306.12806
- Generative AI for End-to-End Limit Order Book Modelling: A Conditional Diffusion Model Approach — → 2309.00638
- Order flow dynamics around extreme price movements on limit order markets (2021)
- Spectra of some self-exciting and mutually exciting point processes (1971)
- Hawkes processes in finance (2015)
- Simulating and analyzing order book data: The queue-reactive model (2015)
- Multi-agent-based order book model of financial markets (2006)
- An agent based model of the E-Mini S&P 500 applied to flash crash analysis (2012)
- Byt5: Towards a token-free future with pre-trained byte-to-byte models (2022)
- Neural machine translation with byte-level subwords (2020)
- Language Modeling Is Compression (2023) — link
- Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inference (2024)
- Byte Latent Transformer: Patches Scale Better Than Tokens (2024) — link
- Efficiently modeling long sequences with structured state spaces (2021) — link
- Transformers are ssms: Generalized models and efficient algorithms through structured state space duality (2024) — link
- Simplified state space layers for sequence modeling (2022) — link
- Mamba: Linear-time sequence modeling with selective state spaces (2023) — link
- Ml-mamba: Efficient multi-modal large language model utilizing mamba-2 (2024) — link
- FlowHFT: Imitation Learning via Flow Matching Policy for Optimal High-Frequency Trading under Diverse Market Conditions — → 2505.05784
- FlowOE: Imitation Learning with Flow Policy from Ensemble RL Experts for Optimal Execution under Heston Volatility and Concave Market Impacts — → 2506.05755
7 of 29 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.