Paper: arXiv 2510.22206
Authors: Ollie Olby, Andreea Bacalum, Rory Baggott, Namid Stillman
Abstract
Execution algorithms are vital to modern trading, they enable market participants to execute large orders while minimising market impact and transaction costs. As these algorithms grow more sophisticated, optimising them becomes increasingly challenging. In this work, we present a reinforcement learning (RL) framework for discovering optimal execution strategies, evaluated within a reactive agent-based market simulator. This simulator creates reactive order flow and allows us to decompose slippage into its constituent components: market impact and execution risk. We assess the RL agent’s performance using the efficient frontier based on work by Almgren and Chriss, measuring its ability to balance risk and cost. Results show that the RL-derived strategies consistently outperform baselines and operate near the efficient frontier, demonstrating a strong ability to optimise for risk and impact. These findings highlight the potential of reinforcement learning as a powerful tool in the trader’s toolkit.
Complexity vs Empirical Score
- Math Complexity: 5.5/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper presents a reinforcement learning framework with defined mathematical formulations like the Almgren-Chriss efficient frontier and slippage decomposition, indicating moderate math complexity. It employs an agent-based market simulator for backtesting, a concrete implementation detail that demonstrates high empirical rigor.
Research Flowchart
flowchart TD A["Research Goal: Optimise<br>Execution Algorithms"] --> B["Methodology: Reinforcement Learning<br>in Agent-Based Market Simulator"] B --> C["Inputs: Reactive Order Flow<br>Order Book Data"] C --> D["Process: RL Agent Learning<br>Slippage Decomposition"] D --> E["Evaluation: Efficient Frontier<br>Almgren-Chriss Metrics"] E --> F["Outcome: RL Strategies Outperform<br>Baselines Near Efficient Frontier"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a reinforcement learning framework for discovering optimal execution strategies, evaluated within a reactive agent‑based market simulator.
Source sentence
In this work, we present a reinforcement learning (RL) framework for discovering optimal execution strategies, evaluated within a reactive agent-based market simulator.
Assumes 6
- Price dynamics follow an arithmetic Brownian motion, implying normally distributed trading costs with known mean and variance. market
Source sentence
They assumed that price dynamics followed an arithmetic Brownian motion. This means trading costs are normally distributed with a known mean and variance.
- Slippage is the sum of market risk and market impact. execution
Source sentence
Slippage, 𝜁, can be defined as the sum of market risk, 𝜁𝑀𝑅, and market impact, 𝜁𝑀𝐼, 𝜁= 𝜁𝑀𝑅+ 𝜁𝑀𝐼
- Order arrivals for both limit and market orders follow a Poisson process. model
Source sentence
The rate of order submission (both market and limit) can be estimated by assuming a Poisson model.
- The agent‑based market simulator can generate realistic reactive order flow that decomposes market impact and risk from slippage. other
Source sentence
ABMs provide the functionality to model the diverse strategies, reactions, and mechanisms of various market participants, generating synthetic data that reflects plausible market conditions.
Show 2 more
- The RL policy network receives a constant dummy state because no contextual information is provided. model
Source sentence
where 𝑥 is the input state vector (which we set as a constant dummy state to account for the lack of context in the model)
- EU AI Act of 2024 requires clear explanations for high‑risk AI systems used in trading decisions. other
Source sentence
the EU AI Act of 2024 dictates that, if using a high-risk AI system (such as is typically used in trading decisions), any person subject to the output of the AI has the right to clear and meaningful explanations
Data
- Simudyne Pulse intraday market simulator synthetic unknown
Source sentence
Figure 1: Overview of the proposed framework using the Simudyne Pulse intraday market simulator.
Costs
Modeled
Slippage is modeled as the sum of market impact and market risk; the RL loss is computed from market slippage, impact and risk measured in the simulator.
Source sentence
The loss is determined by executing the order schedule through our market simulator and computing the market slippage, impact and risk.
Finds 5
- RL‑derived strategies consistently outperform TWAP and VWAP baselines in slippage and lie close to the Almgren‑Chriss efficient frontier. performance
Source sentence
Our RL agent, trained in a realistic agent-based simulation, produced execution strategies that outperform traditional baselines such as TWAP and VWAP in terms of slippage and proximity to the efficient frontier.
- The bi‑modal unbounded distribution trained under the slippage objective yields the most efficient execution strategy by optimizing impact and risk separately. performance
Source sentence
The bi-modal unbounded distribution trained under the slippage objective produced the most efficient execution strategy by using its two modes to optimise impact and risk separately.
- When optimizing directly for market impact, the RL agent reduces impact values compared to baselines but incurs higher market risk. performance
Source sentence
... the RL agent consistently reduced impact values compared to both baselines and the slippage‑optimised strategies. However, this came at the cost of increased market risk.
- Under the slippage loss function, RL strategies achieve similar market impact to baselines but improve overall transaction costs by reducing risk. performance
Source sentence
strategies... we see improvement for all distributions over the baselines and a considerable improvement for the unbounded distribution.
- The uni‑modal distribution trained with the slippage function can achieve optimal VaR at a 95% confidence level. performance
Source sentence
The uni-modal distribution trained with the slippage function produces an execution strategy which can produce optimal VaR at a 95% confidence level.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 27 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (32)
- A reinforcement learning extension to the Almgren-Chriss framework for optimal trade execution (2014)
- Hedging using reinforcement learning: Contextual k-armed bandit versus Q-learning (2023)
- Survey on applications of multi-armed and contextual bandits (2020)
- Optimal execution of portfolio transactions (2000)
- Reinforcement Learning for Optimized Trade Execution (2006)
- Reinforcement Learning in Agent-Based Market Simulation: Unveiling Realistic Stylized Facts and Behavior — → 2403.19781
- Playing Atari with Deep Reinforcement Learning (2013) — link
- OpenAI Gym (2016) — link
and 24 more
- An end-to-end optimal trade execution framework based on proximal policy optimization (2021)
- Optimal Execution with Reinforcement Learning — → 2411.06389
- Deep Reinforcement Learning in Non-Markov Market-Making (2025)
- VWAP execution as an optimal strategy (2015)
- A Modular Framework for Reinforcement Learning Optimal Execution (2022) — link
- Static vs adapted optimal execution strategies in two benchmark trading models (2016) — link
- Intraday periodicity in algorithmic trading (2014) — link
- The intraday patterns of the spread and depth in a market without market makers: The Stock Exchange of Hong Kong (1999)
- Simple statistical gradient-following algorithms for connectionist reinforcement learning (1992)
- Co-existence of Trend and Value in Financial Markets: Estimating an Extended Chiarella Model (2018) — link
- A simulation analysis of the microstructure of double auction markets (2002) — link
- The predictive power of zero intelligence in financial markets (2005) — link
- Allocative Efficiency of Markets with Zero-Intelligence Traders: Market as a Partial Substitute for Individual Rationality (1993) — link
- Multi-agent–based Order Book Model of financial markets (2006) — link
- ABIDES: Towards High-Fidelity Market Simulation for AI Research (2019) — link
- Learning to simulate realistic limit order book markets from data as a World Agent (2022) — link
- Get Real: Realism Metrics for Robust Limit Order Book Market Simulations (2019) — link
- Fast Agent-Based Simulation Framework with Applications to Reinforcement Learning and the Study of Trading Latency Effects (2022) — link
- How to Evaluate Trading Strategies: Single Agent Market Replay or Multiple Agent Interactive Simulation? (2019) — link
- Agent-based Liquidity Risk Modelling for Financial Markets — → 2505.15296
- Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks — → 2311.11913
- Trades, Quotes and Prices: Financial Markets Under the Microscope (2018)
- Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) (2024) — link
- A Comprehensive Survey of Reinforcement Learning: From Algorithms to Practical Challenges (2025) — link
4 of 32 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.