Paper: arXiv 2611.14117

Authors: Fedor Shabashev

Abstract

Generative models have become the standard approach in language and vision. These models are typically pretrained first and then fine-tuned. Recent work demonstrates how to pretrain an autoregressive generative model on Level-3 order-flow data. However, to our knowledge, no one has fine-tuned such a model yet for downstream tasks. Here we first pretrain a 300M-parameter generative model on Level-3 order-flow data collected from two crypto exchanges, Bitstamp and Bitfinex. Then we fine-tune the pretrained model to predict the microprice return and its realized volatility. We show that our fine-tuned model outperforms classical baselines, such as gradient boosting on handcrafted features, in predicting microprice returns over sub-second horizons. Furthermore, the predictive ability of the model transfers zero-shot to assets it was never fine-tuned on. Due to the short-term nature of these forecasts, the predicted microprice moves are smaller than the bid-ask spread plus fees, so the natural applications are in execution and market making. These results indicate that a pretrained generative model can be turned into a competitive short-horizon price forecaster via fine-tuning.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a novel application of foundation models to Level-3 order-flow data for sub-second forecasting, demonstrating strong empirical results against baselines. The methodology is clearly outlined, and the zero-shot transfer capability is a significant finding. While the mathematical underpinnings of the generative model are complex, the paper focuses more on the empirical application and results.

Research Flowchart

  flowchart TD
    A[Research Goal: Fine-tune order-flow foundation model for sub-second forecasting] --> B{Key Methodology: Pretraining & Fine-tuning Generative Model};
    B --> C[Data/Inputs: Level-3 Order-flow from Bitstamp & Bitfinex];
    C --> D[Computational Process: Pretrain 300M-parameter autoregressive generative model];
    D --> E[Computational Process: Fine-tune pretrained model for microprice return & volatility prediction];
    E --> F[Key Findings: Outperforms classical baselines in microprice return prediction];
    F --> G[Key Findings: Zero-shot transferability to unseen assets];
    G --> H[Outcomes: Applications in execution & market making];