Paper: arXiv 2511.02136
Authors: Valentin Mohl, Sascha Frey, Reuben Leyland, Kang Li, George Nigmatulin, Mihai Cucuringu, Stefan Zohren, Jakob Foerster, Anisoara Calinescu
Abstract
Agent-based modelling (ABM) approaches for high-frequency financial markets are difficult to calibrate and validate, partly due to the large parameter space created by defining fixed agent policies. Multi-agent reinforcement learning (MARL) enables more realistic agent behaviour and reduces the number of free parameters, but the heavy computational cost has so far limited research efforts. To address this, we introduce JaxMARL-HFT (JAX-based Multi-Agent Reinforcement Learning for High-Frequency Trading), the first GPU-accelerated open-source multi-agent reinforcement learning environment for high-frequency trading (HFT) on market-by-order (MBO) data. Extending the JaxMARL framework and building on the JAX-LOB implementation, JaxMARL-HFT is designed to handle a heterogeneous set of agents, enabling diverse observation/action spaces and reward functions. It is designed flexibly, so it can also be used for single-agent RL, or extended to act as an ABM with fixed-policy agents. Leveraging JAX enables up to a 240x reduction in end-to-end training time, compared with state-of-the-art reference implementations on the same hardware. This significant speed-up makes it feasible to exploit the large, granular datasets available in high-frequency trading, and to perform the extensive hyperparameter sweeps required for robust and efficient MARL research in trading. We demonstrate the use of JaxMARL-HFT with independent Proximal Policy Optimization (IPPO) for a two-player environment, with an order execution and a market making agent, using one year of LOB data (400 million orders), and show that these agents learn to outperform standard benchmarks. The code for the JaxMARL-HFT framework is available on GitHub.
Complexity vs Empirical Score
- Math Complexity: 4.0/10
- Empirical Rigor: 8.0/10
- Quadrant: Street Traders — empirical and applied, lower math complexity
Why this score: The paper focuses heavily on implementing a practical GPU-accelerated framework (JAX) and scaling experiments with real-world data (400M orders), but lacks advanced mathematical derivations or novel theory.
Research Flowchart
flowchart TD
A["Research Goal: Develop GPU-accelerated<br>Multi-Agent RL for HFT"] --> B["JaxMARL-HFT Framework Development"]
B --> C["Input: 1 Year MBO Data<br>400M Orders"]
C --> D{"Computational Process:<br>GPU-Accelerated Training"}
D --> E["240x Speed-Up vs SOTA<br>Enables Large-Scale Sweeps"]
E --> F["Agents: Market Making<br>& Order Execution"]
F --> G["Method: IPPO Training"]
G --> H["Key Findings:<br>Agents Outperform Benchmarks<br>Open Source GitHub Code"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes JaxMARL-HFT, a GPU‑accelerated open‑source multi‑agent reinforcement learning environment for high‑frequency trading on market‑by‑order data.
Source sentence
we introduce JaxMARL-HFT (JAX-based Multi-Agent Reinforcement Learning for High-Frequency Trading), the first GPU-accelerated open-source multi-agent reinforcement learning environment for high-frequency trading (HFT) on market-by-order (MBO) data.
Assumes 5
- Order quantities are fixed a priori via an environment configuration variable. execution
Source sentence
The quantity associated with each order is fixed a priori through an environment configuration variable: the agents following these strategies will quote constant volumes associated with each bid and ask message.
- Agents receive constant volume orders equal to the minimum lot size, assumed realistic for market participants. market
Source sentence
In practice, this quantity can be set to the minimum lot size, which, depending on the asset price, is often the most commonly used trade size by market participants, making this assumption reasonably realistic.
- Reward functions are parameterised by adjustable hyper‑parameters, allowing diverse risk tolerances. model
Source sentence
All functions are parameterised by adjustable hyperparameters, which presents the user with a highly diverse set of possible reward functions.
- Posted orders from previous steps are cancelled at each RL step, which disadvantages market making in a price‑time priority LOB. execution
Source sentence
It is assumed that one of the hindering factors which makes profitable market making difficult is the fact that posted orders from previous steps are cancelled at each RL step.
Show 1 more
- Episodes can be defined to overlap and are initialised with the LOB state at that time. model
Source sentence
At the beginning of each episode, the LOB is initialised as described in Frey et al. [12], using the current state of the book at that time.
Data
- LOBSTER AMZN order data order-book public
Source sentence
... a year’s worth of pre-processed AMZN order data to occupy just 4GB of GPU memory
ABIDES simulator LOBSTER
Costs
Modeled
Execution cost reduced by up to 20% of a tick on average compared to TWAP.
Source sentence
execution cost can be decreased by up to 20% of a tick on average as compared to TWAP
Finds 4
- Training throughput is increased by up to 240× compared with state‑of‑the‑art CPU implementations. performance
Source sentence
Leveraging JAX enables up to a 240x reduction in end-to-end training time, compared with state-of-the-art reference implementations on the same hardware.
- Learned policies outperform baseline TWAP execution and Avellaneda‑Stoikov market‑making models in their respective quality metrics. performance
Source sentence
Both baselines are outperformed by the learned agent in terms of their respective quality metrics (Portfolio Value and Slippage).
- Market‑making agents lose about 0.2 ticks on average but this loss is considered a surprisingly good result given the environment constraints. negative
Source sentence
none of the learned strategies for market making are able to make net profits, but instead lose approximately 0.2 ticks on average.
- Execution agents achieve up to a 20% tick reduction in execution cost relative to TWAP, likely by trading more passively while guaranteeing execution. performance
Source sentence
execution cost can be decreased by up to 20% of a tick on average as compared to TWAP, likely by trading more passively, whilst still guaranteeing execution.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 30 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (41)
- GPUD rive: Data-driven, multi-agent driving simulation at 1 million FPS (2025) — link
- An Extensible, Data-Oriented Architecture for High-Performance, Many-World Simulation (2023)
- Trades, quotes and prices: financial markets under the microscope (2018)
- Robust Autonomy Emerges from Self-Play (2025) — link
- Limit order books (2013)
- Mbt-gym: Reinforcement learning for model-based limit order book trading (2023)
- Deep Q-learning market makers in a multi-agent simulated stock market (2022) — link
- LOBSTER : Limit order book reconstruction system (2011)
and 33 more
- Modelling crypto markets by multi-agent reinforcement learning — → 2402.10803
- Generative AI for end-to-end limit order book modelling: A token-level autoregressive generative model of message flow using a deep state space network — → 2309.00638
- Double deep q-learning for optimal execution (2021)
- A reinforcement learning approach to optimal execution (2022)
- Reinforcement learning approaches to optimal market making (2021)
- Market making via reinforcement learning (2018)
- Deep reinforcement learning for active high frequency trading (2021) — link
- An application of deep reinforcement learning to algorithmic trading (2021)
- Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
- Allocative efficiency of markets with zero-intelligence traders: Market as a partial substitute for individual rationality (1993)
- Get real: realism metrics for robust limit order book market simulations (2021) — link
- An agent-based modeling approach to study price impact (2012) — link
- The predictive power of zero intelligence in financial markets (2005)
- Artificial economic life: a simple model of a stockmarket (1994)
- JaxMARL : Multi-agent RL Environments and Algorithms in JAX (2024)
- JaxMARL: Multi-agent RL environments and algorithms in JAX (2024)
- ABIDES -gym: gym environments for multi-agent discrete event simulation and application to financial markets (2022)
- A stochastic partial differential equation model for limit order book dynamics (2021)
- Reinforcement learning in agent-based market simulation: Unveiling realistic stylized facts and behavior — → 2403.19781
- Many learning agents interacting with an agent-based market model — → 2303.07393
- Modelling stock markets by multi -agent reinforcement learning (2021) — link
- An agent - based market simulator for back - testing deep reinforcement learning based trade execution strategies (2021)
- JAX-LOB: A GPU-Accelerated limit order book simulator to unlock large scale reinforcement learning for trading — → 2308.13289
- ABIDES : Towards high - fidelity multi - agent market simulation (2020)
- A financial market simulation environment for trading agents using deep reinforcement learning (2024)
- Towards multi-agent reinforcement learning-driven over-the-counter market simulations (2024) — link
- Robust market making via adversarial reinforcement learning (2021)
- Multi-agent reinforcement learning in a realistic limit order book market simulation (2020) — link
- Towards a fully RL -based market simulator (2022)
- Tianshou: A highly modularized deep reinforcement learning library (2022)
- JAX : composable transformations of P ython+ N um P y programs (2018) — link
- Fast agent-based simulation framework with applications to reinforcement learning and the study of trading latency effects (2022)
- High-frequency trading in a limit order book (2008)
5 of 41 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.