Paper: arXiv 2511.02016

Authors: Patrick Cheridito, Jean-Loup Dupret, Zhexin Wu

Abstract

We present ABIDES-MARL, a framework that combines a new multi-agent reinforcement learning (MARL) methodology with a new realistic limit-order-book (LOB) simulation system to study equilibrium behavior in complex financial market games. The system extends ABIDES-Gym by decoupling state collection from kernel interruption, enabling synchronized learning and decision-making for multiple adaptive agents while maintaining compatibility with standard RL libraries. It preserves key market features such as price-time priority and discrete tick sizes. Methodologically, we use MARL to approximate equilibrium-like behavior in multi-period trading games with a finite number of heterogeneous agents-an informed trader, a liquidity trader, noise traders, and competing market makers-all with individual price impacts. This setting bridges optimal execution and market microstructure by embedding the liquidity trader’s optimization problem within a strategic trading environment. We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon. We then extend the analysis to a liquidity trader’s problem where market liquidity arises endogenously and show that, at equilibrium, execution strategies shape market-maker behavior and price dynamics. ABIDES-MARL provides a reproducible foundation for analyzing equilibrium and strategic adaptation in realistic markets and contributes toward building economically interpretable agentic AI systems for finance.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a complex multi-agent reinforcement learning framework and connects it to theoretical models like Kyle’s model, which involves substantial mathematical modeling and equilibrium concepts. However, while it presents a simulation system (ABIDES-MARL) with backtest-ready features like LOB simulation and agents, the empirical validation is primarily theoretical (recovering known models) rather than data-heavy backtesting on real market data.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Model equilibrium behavior in<br>complex financial market games"] --> B["Methodology: MARL Framework"]
  B --> C["Core Components<br>Limit Order Book Simulation &<br>Synchronized Multi-Agent RL"]
  C --> D["Key Processes"]
  D --> E["Solve Extended Kyle Model"]
  D --> F["Simulate Endogenous Liquidity"]
  E --> G["Findings & Outcomes"]
  F --> G
  G --> H["Recovered Gradual Price Discovery"]
  G --> I["Equilibrium Strategies Shape<br>Market-Maker Behavior & Price Dynamics"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes ABIDES-MARL, a new multi‑agent reinforcement learning framework combined with a realistic limit‑order‑book simulation to study equilibrium behavior in financial market games.

Source sentence
We present ABIDES-MARL, a framework that combines a new multi-agent reinforcement learning (MARL) methodology with a new realistic limit-order-book (LOB) simulation system to study equilibrium behavior in complex financial market games.

Assumes 6

  • Perfect competition among market makers in the classical Kyle model. model
    Source sentence
    A central assumption of the Kyle model is perfect competition among market makers.
  • Finite set of competing market makers replaces the infinite‑maker assumption. model
    Source sentence
    First, we replace the assumption of infinitely many market makers with a finite set of competing market makers.
  • The market clears at each step, meaning all orders are absorbed by market makers. market
    Source sentence
    We assume that the market clears at each step, meaning all orders are absorbed by market makers.
  • Market impact λ(n) can vary over time, reflecting dynamic microstructure. execution
    Source sentence
    Our formulation allows λ(n) to vary over time, consistent with the
Show 2 more
  • Risk aversion is modeled through a running penalty on inventory. model
    Source sentence
    Risk aversion is modeled through a running penalty on inventory.
  • Agents observe either the full LOB (exchange) or a hidden LOB with VWAP inference (OTC). market
    Source sentence
    State representations. We distinguish between two scenarios. The first, denoted as exchange, assumes that all market participants can observe the full limit order book (LOB). The second, denoted as OTC, assumes the LOB is hidden; participants must instead infer its structure through the volume-weighted average price (V

Data

  • ABIDES-MARL simulation synthetic unknown
    Source sentence
    We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon.

Recognised sources: ABIDES simulator

Costs

Modeled Costs are part of the reported headline results.

Market impact λ(n) is used as the cost model, varying over time and depth‑weighted across market makers.

Source sentence
Our formulation allows λ(n) to vary over time, consistent with the

Finds 5

  • The ABIDES-MARL framework recovers the gradual price discovery phenomenon when solving the extended Kyle model. theoretical
    Source sentence
    We validate the approach by solving an extended Kyle model within the simulation system, recovering the gradual price discovery phenomenon.
  • Execution strategies shape market‑maker behavior and price dynamics at equilibrium. theoretical
    Source sentence
    show that, at equilibrium, execution strategies shape market-maker behavior and price dynamics.
  • The multi‑agent PPO policy underperformed because counterparties adapted adversarially during training. performance
    Source sentence
    The multi-agent PPO policy underperformed because counterparties had adapted adversarially during training, highlighting that execution costs in adaptive markets are inherently endogenous to the learning dynamics of competitors.
  • Across strategies, the time‑weighted average price (TWAP) rule remained the most robust under linear market dynamics. performance
    Source sentence
    Across strategies, the time-weighted average price (TWAP) rule remained the most robust under linear market dynamics
  • A retrained single‑agent PPO policy outperformed others under non‑linear settings. performance
    Source sentence
    a retrained single-agent PPO policy outperformed others under non‑linear settings

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 42 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (78)

  1. Algorithmic and High-Frequency Trading (2015)
  2. Optimal execution strategies in limit order books with general shape functions (2010)
  3. Decentralized policy optimization (2022) — link
  4. On stationary point convergence of PPO-Clip (2023)
  5. Ppo-clip attains global optimality: Towards deeper understandings of clipping (2024)
  6. Continuous auctions and insider trading (1985)
  7. Bid, ask and transaction prices in a specialist market with heterogeneously informed traders (1985) — link
  8. Liquidity and market structure (1988)
and 70 more
  1. Optimal dealer pricing under transactions and return uncertainty (1981) — link
  2. Long-lived private information and imperfect competition (1992)
  3. The effect of public information and competition on trading volume and price volatility (1993)
  4. Deep reinforcement learning with double Q-learning (2016)
  5. Proximal policy optimization algorithms (2017) — link
  6. Multi-agent actor-critic for mixed cooperative-competitive environments (2017) — link
  7. ABIDES: Towards high-fidelity multi-agent market simulation (2020) — link
  8. ABIDES-Gym: Gym environments for multi-agent discrete event simulation and application to financial markets (2021) — link
  9. Sample selection bias as a specification error (1979) — link
  10. Explanatory Model Analysis: Interpretability for Machine Learning Models (2021)
  11. Equilibrium fast trading (2015) — link
  12. Optimal execution of portfolio transactions (2001) — link
  13. Optimal control of execution costs (1998) — link
  14. Continuous control with deep reinforcement learning (2019) — link
  15. The surprising effectiveness of PPO in cooperative, multi-agent games (2022)
  16. Trust region policy optimisation in multi-agent reinforcement learning (2022) — link
  17. Multi-agent reinforcement learning in a realistic limit order book market simulation (2020) — link
  18. RLlib: Abstractions for distributed reinforcement learning (2018) — link
  19. MARLlib: A scalable and efficient multi-agent reinforcement learning library (2023)
  20. Q-learning (1992) — link
  21. Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992) — link
  22. Deterministic policy gradient algorithms (2014) — link
  23. Reinforcement learning for optimized trade execution (2006)
  24. A reinforcement learning extension to the Almgren–Chriss framework for optimal trade execution (2014) — link
  25. Superhuman AI for multiplayer poker (2019) — link
  26. Learning to communicate with deep multi-agent reinforcement learning (2016) — link
  27. Deep investing in Kyle’s single period model (2020) — link
  28. High-frequency trading in a limit order book (2008) — link
  29. Reinforcement learning for market making in a multi-agent dealer market (2019) — link
  30. Gymnasium: A standard interface for reinforcement learning environments (2024) — link
  31. PettingZoo: Gym for multi-agent reinforcement learning (2021) — link
  32. Optimal execution with reinforcement learning — → 2411.06389
  33. Asynchronous deep double dueling Q-learning for trading-signal execution in limit order book markets (2023) — link
  34. Spoofing the limit order book: A strategic agent-based analysis (2021) — link
  35. Strategic trading when agents forecast the forecasts of others (1996)
  36. Optimal liquidity trading (2005) — link
  37. Episodic liquidity crises: Cooperative and predatory trading (2007) — link
  38. Market making via reinforcement learning (2018) — link
  39. Robust market making via adversarial reinforcement learning (2020) — link
  40. Price manipulation and quasi-arbitrage (2004)
  41. Market architecture: Limit-order books versus dealership markets (2002) — link
  42. Floors, dealer markets and limit order markets (1998) — link
  43. Market Liquidity: Theory, Evidence, and Policy (2023) — link
  44. Double deep Q-learning for optimal execution (2021) — link
  45. An end-to-end optimal trade execution framework based on proximal policy optimization (2020) — link
  46. Informed speculation with imperfect competition (1989) — link
  47. Stable-Baselines3: Reliable reinforcement learning implementations (2021)
  48. An agent-based market simulator for back-testing deep reinforcement learning based trade execution strategies (2021)
  49. Robustness of equilibrium in the Kyle model of informed speculation (2015) — link
  50. Robust strategies for optimal order execution in the Almgren–Chriss framework (2013) — link
  51. From deep learning to LLMs: A survey of AI in quantitative investment — → 2503.21422
  52. FinMem: A performance-enhanced LLM trading agent with layered memory and character design — → 2311.13743
  53. A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist — → 2402.18485
  54. FinRobot: An open-source AI agent platform for financial applications using large language models — → 2405.14767
  55. Chain-of-thought is not explainability (2025) — link
  56. Deep differentiable reinforcement learning and optimal trading (2022) — link
  57. Do price trajectory data increase the efficiency of market impact estimation? (2024) — link
  58. Optimal liquidation under indirect price impact with propagator (2025) — link
  59. A reinforcement learning approach to optimal execution (2022) — link
  60. No-dynamic-arbitrage and market impact (2010) — link
  61. Optimal trading strategy and supply/demand dynamics (2013) — link
  62. Optimal execution with non-linear transient market impact (2017) — link
  63. Autoregressive conditional heteroscedasticity with estimates of the variance of United Kingdom inflation (1982) — link
  64. statsmodels: Econometric and statistical modeling with Python (2010)
  65. Mean field games (2007) — link
  66. Large population stochastic dynamic games: Closed-loop McKean–Vlasov systems and the Nash certainty equivalence principle (2006) — link
  67. Reinforcement learning for trade execution with market impact — → 2507.06345
  68. Reinforcement learning in queue-reactive models: application to optimal execution — → 2511.15262
  69. Agentic Artificial Intelligence in Finance: A Comprehensive Survey — → 2604.21672
  70. Optimal trade execution under stochastic volatility and liquidity (2014) — link

8 of 78 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.