Paper: arXiv 2511.02016
Authors: Patrick Cheridito, Jean-Loup Dupret, Zhexin Wu
Abstract
We present ABIDES-MARL, a framework that combines a new multi-agent reinforcement learning (MARL) methodology with a new realistic limit-order-book (LOB) simulation system to study equilibrium behavior in complex financial market games. The system extends ABIDES-Gym by decoupling state collection from kernel interruption, enabling synchronized learning and decision-making for multiple adaptive agents while maintaining compatibility with standard RL libraries. It preserves key market features such as price-time priority and discrete tick sizes. Methodologically, we use MARL to approximate equilibrium-like behavior in multi-period trading games with a finite number of heterogeneous agents-an informed trader, a liquidity trader, noise traders, and competing market makers-all with individual price impacts. This setting bridges optimal execution and market microstructure by embedding the liquidity trader’s optimization problem within a strategic trading environment. We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon. We then extend the analysis to a liquidity trader’s problem where market liquidity arises endogenously and show that, at equilibrium, execution strategies shape market-maker behavior and price dynamics. ABIDES-MARL provides a reproducible foundation for analyzing equilibrium and strategic adaptation in realistic markets and contributes toward building economically interpretable agentic AI systems for finance.
Complexity vs Empirical Score
- Math Complexity: 7.0/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper introduces a complex multi-agent reinforcement learning framework and connects it to theoretical models like Kyle’s model, which involves substantial mathematical modeling and equilibrium concepts. However, while it presents a simulation system (ABIDES-MARL) with backtest-ready features like LOB simulation and agents, the empirical validation is primarily theoretical (recovering known models) rather than data-heavy backtesting on real market data.
Research Flowchart
flowchart TD A["Research Goal<br>Model equilibrium behavior in<br>complex financial market games"] --> B["Methodology: MARL Framework"] B --> C["Core Components<br>Limit Order Book Simulation &<br>Synchronized Multi-Agent RL"] C --> D["Key Processes"] D --> E["Solve Extended Kyle Model"] D --> F["Simulate Endogenous Liquidity"] E --> G["Findings & Outcomes"] F --> G G --> H["Recovered Gradual Price Discovery"] G --> I["Equilibrium Strategies Shape<br>Market-Maker Behavior & Price Dynamics"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes ABIDES-MARL, a new multi‑agent reinforcement learning framework combined with a realistic limit‑order‑book simulation to study equilibrium behavior in financial market games.
Source sentence
We present ABIDES-MARL, a framework that combines a new multi-agent reinforcement learning (MARL) methodology with a new realistic limit-order-book (LOB) simulation system to study equilibrium behavior in complex financial market games.
Assumes 6
- Perfect competition among market makers in the classical Kyle model. model
Source sentence
A central assumption of the Kyle model is perfect competition among market makers.
- Finite set of competing market makers replaces the infinite‑maker assumption. model
Source sentence
First, we replace the assumption of infinitely many market makers with a finite set of competing market makers.
- The market clears at each step, meaning all orders are absorbed by market makers. market
Source sentence
We assume that the market clears at each step, meaning all orders are absorbed by market makers.
- Market impact λ(n) can vary over time, reflecting dynamic microstructure. execution
Source sentence
Our formulation allows λ(n) to vary over time, consistent with the
Show 2 more
- Risk aversion is modeled through a running penalty on inventory. model
Source sentence
Risk aversion is modeled through a running penalty on inventory.
- Agents observe either the full LOB (exchange) or a hidden LOB with VWAP inference (OTC). market
Source sentence
State representations. We distinguish between two scenarios. The first, denoted as exchange, assumes that all market participants can observe the full limit order book (LOB). The second, denoted as OTC, assumes the LOB is hidden; participants must instead infer its structure through the volume-weighted average price (V
Data
- ABIDES-MARL simulation synthetic unknown
Source sentence
We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon.
ABIDES simulator
Costs
Modeled
Market impact λ(n) is used as the cost model, varying over time and depth‑weighted across market makers.
Source sentence
Our formulation allows λ(n) to vary over time, consistent with the
Finds 5
- The ABIDES-MARL framework recovers the gradual price discovery phenomenon when solving the extended Kyle model. theoretical
Source sentence
We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon.
- Execution strategies shape market‑maker behavior and price dynamics at equilibrium. theoretical
Source sentence
show that, at equilibrium, execution strategies shape market-maker behavior and price dynamics.
- The multi‑agent PPO policy underperformed because counterparties adapted adversarially during training. performance
Source sentence
The multi-agent PPO policy underperformed because counterparties had adapted adversarially during training, highlighting that execution costs in adaptive markets are inherently endogenous to the learning dynamics of competitors.
- Across strategies, the time‑weighted average price (TWAP) rule remained the most robust under linear market dynamics. performance
Source sentence
Across strategies, the time-weighted average price (TWAP) rule remained the most robust under linear market dynamics
- A retrained single‑agent PPO policy outperformed others under non‑linear settings. performance
Source sentence
a retrained single-agent PPO policy outperformed others under non‑linear settings
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 42 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (78)
- Algorithmic and High-Frequency Trading (2015)
- Optimal execution strategies in limit order books with general shape functions (2010)
- Decentralized policy optimization (2022) — link
- On stationary point convergence of PPO-Clip (2023)
- Ppo-clip attains global optimality: Towards deeper understandings of clipping (2024)
- Continuous auctions and insider trading (1985)
- Bid, ask and transaction prices in a specialist market with heterogeneously informed traders (1985) — link
- Liquidity and market structure (1988)
and 70 more
- Optimal dealer pricing under transactions and return uncertainty (1981) — link
- Long-lived private information and imperfect competition (1992)
- The effect of public information and competition on trading volume and price volatility (1993)
- Deep reinforcement learning with double Q-learning (2016)
- Proximal policy optimization algorithms (2017) — link
- Multi-agent actor-critic for mixed cooperative-competitive environments (2017) — link
- ABIDES: Towards high-fidelity multi-agent market simulation (2020) — link
- ABIDES-Gym: Gym environments for multi-agent discrete event simulation and application to financial markets (2021) — link
- Sample selection bias as a specification error (1979) — link
- Explanatory Model Analysis: Interpretability for Machine Learning Models (2021)
- Equilibrium fast trading (2015) — link
- Optimal execution of portfolio transactions (2001) — link
- Optimal control of execution costs (1998) — link
- Continuous control with deep reinforcement learning (2019) — link
- The surprising effectiveness of PPO in cooperative, multi-agent games (2022)
- Trust region policy optimisation in multi-agent reinforcement learning (2022) — link
- Multi-agent reinforcement learning in a realistic limit order book market simulation (2020) — link
- RLlib: Abstractions for distributed reinforcement learning (2018) — link
- MARLlib: A scalable and efficient multi-agent reinforcement learning library (2023)
- Q-learning (1992) — link
- Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992) — link
- Deterministic policy gradient algorithms (2014) — link
- Reinforcement learning for optimized trade execution (2006)
- A reinforcement learning extension to the Almgren–Chriss framework for optimal trade execution (2014) — link
- Superhuman AI for multiplayer poker (2019) — link
- Learning to communicate with deep multi-agent reinforcement learning (2016) — link
- Deep investing in Kyle’s single period model (2020) — link
- High-frequency trading in a limit order book (2008) — link
- Reinforcement learning for market making in a multi-agent dealer market (2019) — link
- Gymnasium: A standard interface for reinforcement learning environments (2024) — link
- PettingZoo: Gym for multi-agent reinforcement learning (2021) — link
- Optimal execution with reinforcement learning — → 2411.06389
- Asynchronous deep double dueling Q-learning for trading-signal execution in limit order book markets (2023) — link
- Spoofing the limit order book: A strategic agent-based analysis (2021) — link
- Strategic trading when agents forecast the forecasts of others (1996)
- Optimal liquidity trading (2005) — link
- Episodic liquidity crises: Cooperative and predatory trading (2007) — link
- Market making via reinforcement learning (2018) — link
- Robust market making via adversarial reinforcement learning (2020) — link
- Price manipulation and quasi-arbitrage (2004)
- Market architecture: Limit-order books versus dealership markets (2002) — link
- Floors, dealer markets and limit order markets (1998) — link
- Market Liquidity: Theory, Evidence, and Policy (2023) — link
- Double deep Q-learning for optimal execution (2021) — link
- An end-to-end optimal trade execution framework based on proximal policy optimization (2020) — link
- Informed speculation with imperfect competition (1989) — link
- Stable-Baselines3: Reliable reinforcement learning implementations (2021)
- An agent-based market simulator for back-testing deep reinforcement learning based trade execution strategies (2021)
- Robustness of equilibrium in the Kyle model of informed speculation (2015) — link
- Robust strategies for optimal order execution in the Almgren–Chriss framework (2013) — link
- From deep learning to LLMs: A survey of AI in quantitative investment — → 2503.21422
- FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
- A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist — → 2402.18485
- FinRobot: An open-source AI agent platform for financial applications using large language models — → 2405.14767
- Chain-of-thought is not explainability (2025) — link
- Deep differentiable reinforcement learning and optimal trading (2022) — link
- Do price trajectory data increase the efficiency of market impact estimation? (2024) — link
- Optimal liquidation under indirect price impact with propagator (2025) — link
- A reinforcement learning approach to optimal execution (2022) — link
- No-dynamic-arbitrage and market impact (2010) — link
- Optimal trading strategy and supply/demand dynamics (2013) — link
- Optimal execution with non-linear transient market impact (2017) — link
- Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation (1982) — link
- statsmodels: Econometric and statistical modeling with Python (2010)
- Mean field games (2007) — link
- Large population stochastic dynamic games: Closed-loop McKean–Vlasov systems and the Nash certainty equivalence principle (2006) — link
- Reinforcement learning for trade execution with market impact — → 2507.06345
- Reinforcement learning in queue-reactive models: application to optimal execution — → 2511.15262
- Agentic Artificial Intelligence in Finance: A Comprehensive Survey — → 2604.21672
- Optimal trade execution under stochastic volatility and liquidity (2014) — link
8 of 78 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.