Paper: arXiv 2510.22206

Authors: Ollie Olby, Andreea Bacalum, Rory Baggott, Namid Stillman

Abstract

Execution algorithms are vital to modern trading, they enable market participants to execute large orders while minimising market impact and transaction costs. As these algorithms grow more sophisticated, optimising them becomes increasingly challenging. In this work, we present a reinforcement learning (RL) framework for discovering optimal execution strategies, evaluated within a reactive agent-based market simulator. This simulator creates reactive order flow and allows us to decompose slippage into its constituent components: market impact and execution risk. We assess the RL agent’s performance using the efficient frontier based on work by Almgren and Chriss, measuring its ability to balance risk and cost. Results show that the RL-derived strategies consistently outperform baselines and operate near the efficient frontier, demonstrating a strong ability to optimise for risk and impact. These findings highlight the potential of reinforcement learning as a powerful tool in the trader’s toolkit.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a reinforcement learning framework with defined mathematical formulations like the Almgren-Chriss efficient frontier and slippage decomposition, indicating moderate math complexity. It employs an agent-based market simulator for backtesting, a concrete implementation detail that demonstrates high empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Optimise<br>Execution Algorithms"] --> B["Methodology: Reinforcement Learning<br>in Agent-Based Market Simulator"]
  B --> C["Inputs: Reactive Order Flow<br>Order Book Data"]
  C --> D["Process: RL Agent Learning<br>Slippage Decomposition"]
  D --> E["Evaluation: Efficient Frontier<br>Almgren-Chriss Metrics"]
  E --> F["Outcome: RL Strategies Outperform<br>Baselines Near Efficient Frontier"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a reinforcement learning framework for discovering optimal execution strategies, evaluated within a reactive agent‑based market simulator.

Source sentence
In this work, we present a reinforcement learning (RL) framework for discovering optimal execution strategies, evaluated within a reactive agent-based market simulator.

Assumes 6

  • Price dynamics follow an arithmetic Brownian motion, implying normally distributed trading costs with known mean and variance. market
    Source sentence
    They assumed that price dynamics followed an arithmetic Brownian motion. This means trading costs are normally distributed with a known mean and variance.
  • Slippage is the sum of market risk and market impact. execution
    Source sentence
    Slippage, 𝜁, can be defined as the sum of market risk, 𝜁𝑀𝑅, and market impact, 𝜁𝑀𝐼, 𝜁= 𝜁𝑀𝑅+ 𝜁𝑀𝐼
  • Order arrivals for both limit and market orders follow a Poisson process. model
    Source sentence
    The rate of order submission (both market and limit) can be estimated by assuming a Poisson model.
  • The agent‑based market simulator can generate realistic reactive order flow that decomposes market impact and risk from slippage. other
    Source sentence
    ABMs provide the functionality to model the diverse strategies, reactions, and mechanisms of various market participants, generating synthetic data that reflects plausible market conditions.
Show 2 more
  • The RL policy network receives a constant dummy state because no contextual information is provided. model
    Source sentence
    where 𝑥 is the input state vector (which we set as a constant dummy state to account for the lack of context in the model)
  • EU AI Act of 2024 requires clear explanations for high‑risk AI systems used in trading decisions. other
    Source sentence
    the EU AI Act of 2024 dictates that, if using a high-risk AI system (such as is typically used in trading decisions), any person subject to the output of the AI has the right to clear and meaningful explanations

Data

  • Simudyne Pulse intraday market simulator synthetic unknown
    Source sentence
    Figure 1: Overview of the proposed framework using the Simudyne Pulse intraday market simulator.

Costs

Modeled Costs are part of the reported headline results.

Slippage is modeled as the sum of market impact and market risk; the RL loss is computed from market slippage, impact and risk measured in the simulator.

Source sentence
The loss is determined by executing the order schedule through our market simulator and computing the market slippage, impact and risk.

Finds 5

  • RL‑derived strategies consistently outperform TWAP and VWAP baselines in slippage and lie close to the Almgren‑Chriss efficient frontier. performance
    Source sentence
    Our RL agent, trained in a realistic agent-based simulation, produced execution strategies that outperform traditional baselines such as TWAP and VWAP in terms of slippage and proximity to the efficient frontier.
  • The bi‑modal unbounded distribution trained under the slippage objective yields the most efficient execution strategy by optimizing impact and risk separately. performance
    Source sentence
    The bi-modal unbounded distribution trained under the slippage objective produced the most efficient execution strategy by using its two modes to optimise impact and risk separately.
  • When optimizing directly for market impact, the RL agent reduces impact values compared to baselines but incurs higher market risk. performance
    Source sentence
    ... the RL agent consistently reduced impact values compared to both baselines and the slippage‑optimised strategies. However, this came at the cost of increased market risk.
  • Under the slippage loss function, RL strategies achieve similar market impact to baselines but improve overall transaction costs by reducing risk. performance
    Source sentence
    strategies... we see improvement for all distributions over the baselines and a considerable improvement for the unbounded distribution.
  • The uni‑modal distribution trained with the slippage function can achieve optimal VaR at a 95% confidence level. performance
    Source sentence
    The uni-modal distribution trained with the slippage function produces an execution strategy which can produce optimal VaR at a 95% confidence level.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (32)

  1. A reinforcement learning extension to the Almgren-Chriss framework for optimal trade execution (2014)
  2. Hedging using reinforcement learning: Contextual k-armed bandit versus Q-learning (2023)
  3. Survey on applications of multi-armed and contextual bandits (2020)
  4. Optimal execution of portfolio transactions (2000)
  5. Reinforcement Learning for Optimized Trade Execution (2006)
  6. Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior — → 2403.19781
  7. Playing Atari with Deep Reinforcement Learning (2013) — link
  8. OpenAI Gym (2016) — link
and 24 more
  1. An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
  2. Optimal Execution with Reinforcement Learning — → 2411.06389
  3. Deep Reinforcement Learning in Non-Markov Market-Making (2025)
  4. VWAP execution as an optimal strategy (2015)
  5. A Modular Framework for Reinforcement Learning Optimal Execution (2022) — link
  6. Static vs adapted optimal execution strategies in two benchmark trading models (2016) — link
  7. Intraday periodicity in algorithmic trading (2014) — link
  8. The intraday patterns of the spread and depth in a market without market makers: The Stock Exchange of Hong Kong (1999)
  9. Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992)
  10. Co-existence of Trend and Value in Financial Markets: Estimating an Extended Chiarella Model (2018) — link
  11. A simulation analysis of the microstructure of double auction markets (2002) — link
  12. The predictive power of zero intelligence in financial markets (2005) — link
  13. Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality (1993) — link
  14. Multi-agent–based Order Book Model of financial markets (2006) — link
  15. ABIDES: Towards High-Fidelity Market Simulation for AI Research (2019) — link
  16. Learning to simulate realistic limit order book markets from data as a World Agent (2022) — link
  17. Get Real: Realism Metrics for Robust Limit Order Book Market Simulations (2019) — link
  18. Fast Agent-Based Simulation Framework with Applications to Reinforcement Learning and the Study of Trading Latency Effects (2022) — link
  19. How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation? (2019) — link
  20. Agent-based Liquidity Risk Modelling for Financial Markets — → 2505.15296
  21. Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks — → 2311.11913
  22. Trades, Quotes and Prices: Financial Markets Under the Microscope (2018)
  23. Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) (2024) — link
  24. A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges (2025) — link

4 of 32 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.