Paper: arXiv 2610.28753

Authors: Valentin Mohl, Alexandre Bismuth, Aramis Marti-Shahandeh, George Nigmatulin, Kang Li, Stefan Zohren, Jakob Foerster, Mihai Cucuringu, Anisoara Calinescu

Abstract

Autoregressive foundation models offer state-of-the-art capabilities to simulate the market by generating the message flow of limit order books (LOBs). Yet, without understanding their inner workings, research directions are hard to narrow and practical use in real-world trading environments remains uncertain. This study therefore asks what these models actually learn about market microstructure. We find that, similarly to games where sequence models build an internal representation of the board, an 8.2M-parameter Mamba-3 state-space model trained autoregressively on the tokenized message stream of eight large-cap US equities learns an internal representation of the market and its dynamics. In particular, order-flow imbalance, a central driver of short-horizon price moves, is decodable at $R^2 \approx 0.59$, far above controls and baselines. This representation extends to unseen stocks, suggesting that these models learn general representations rather than ticker-specific patterns. The model also generalizes across metrics beyond order-flow imbalance: results hold across traded volume and volatility. Additionally, the representation is causally steerable: pushing the state along a feature’s axis shifts the model’s next-message beliefs as that feature implies, monotonically in dose, consistent with the emergence of a world model.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a highly novel approach to understanding LOB foundation models using state-space models, demonstrating strong empirical rigor through extensive data training and robust probing/steering methodologies. The combination of advanced mathematical models with rigorous empirical validation places it firmly in the ‘Holy Grail’ quadrant, offering significant contributions to both theory and practice.

Research Flowchart

  flowchart TD
    A[Research Goal: What do state-space foundation models learn about market microstructure?] --> B{Methodology: Train and analyze 8.2M-parameter Mamba-3 model};
    B --> C[Data Input: Tokenized message stream of 8 large-cap US equities];
    C --> D[Computational Process: Autoregressive training & internal representation decoding];
    D --> E[Finding 1: Internal market representation learned];
    E --> F[Finding 2: Order-flow imbalance decodable (R^2 ~ 0.59), generalizes across stocks & metrics (volume, volatility)];
    F --> G[Finding 3: Representation causally steerable, consistent with emergence of a world model].