ByteGen: A Tokenizer-Free Generative Model for Orderbook Events in Byte Space

ArXiv ID: 2508.02247 “View on arXiv”

Authors: Yang Li, Zhi Chen

Abstract

Generative modeling of high-frequency limit order book (LOB) dynamics is a critical yet unsolved challenge in quantitative finance, essential for robust market simulation and strategy backtesting. Existing approaches are often constrained by simplifying stochastic assumptions or, in the case of modern deep learning models like Transformers, rely on tokenization schemes that affect the high-precision, numerical nature of financial data through discretization and binning. To address these limitations, we introduce ByteGen, a novel generative model that operates directly on the raw byte streams of LOB events. Our approach treats the problem as an autoregressive next-byte prediction task, for which we design a compact and efficient 32-byte packed binary format to represent market messages without information loss. The core novelty of our work is the complete elimination of feature engineering and tokenization, enabling the model to learn market dynamics from its most fundamental representation. We achieve this by adapting the H-Net architecture, a hybrid Mamba-Transformer model that uses a dynamic chunking mechanism to discover the inherent structure of market messages without predefined rules. Our primary contributions are: 1) the first end-to-end, byte-level framework for LOB modeling; 2) an efficient packed data representation; and 3) a comprehensive evaluation on high-frequency data. Trained on over 34 million events from CME Bitcoin futures, ByteGen successfully reproduces key stylized facts of financial markets, generating realistic price distributions, heavy-tailed returns, and bursty event timing. Our findings demonstrate that learning directly from byte space is a promising and highly flexible paradigm for modeling complex financial systems, achieving competitive performance on standard market quality metrics without the biases of tokenization.

Keywords: Limit Order Book (LOB), Generative Modeling, High-Frequency Trading, Market Simulation, Byte-Level Modeling

Complexity vs Empirical Score

• Math Complexity: 6.0/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail
• Why: The paper introduces a novel, mathematically dense hybrid Mamba-Transformer architecture with dynamic chunking and operates on raw byte streams, but it is backed by strong empirical validation on 34 million real-world events with comprehensive backtest-ready metrics.

  flowchart TD
    A["Research Goal: Generative modeling of high-frequency LOB dynamics without tokenization or feature engineering."] --> B["Data Input: Raw byte streams of LOB events<br>(34M events from CME Bitcoin futures)"]
    B --> C["Methodology: ByteGen Framework<br>• Packed binary format (32 bytes)<br>• Autoregressive next-byte prediction<br>• H-Net architecture (Mamba-Transformer hybrid)"]
    C --> D["Computational Process: Dynamic chunking<br>Discovers inherent market structure without predefined rules."]
    D --> E["Key Findings: Competitive performance<br>• Realistic price distributions<br>• Heavy-tailed returns<br>• Bursty event timing<br>(Without tokenization biases)"]