{"paper_id":"2510.27334","title":"When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making","authors":["Ali Raza Jafree","Konark Jain","Nick Firoozye"],"paper_date":"2025-10-31","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2510.27334v1","page_url":"https://thequant.space/flowcharts/when-ai-trading-agents-compete-adverse-selection-of-meta-or/","scores":{"empirical_rigor":3,"math_complexity":8,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper proposes an impulse‑control reinforcement‑learning market‑making agent trained with Proximal Policy Optimisation and self‑imitation learning to operate in a Hawkes limit‑order‑book simulation and exploit meta‑order execution.","quote":"Given the real-world impracticalities of the market maker updating strategies for every event in the LOB, we formulate the high-frequency market making agent via an impulse control reinforcement learning framework (Jain et al. 2025). The RL used in the simulation utilises Proximal Policy Optimisation (PPO) and self-imitation learning.","summary":"We investigate the mechanisms by which medium-frequency trading agents are adversely selected by opportunistic high-frequency traders. We use reinforcement learning (RL) within a Hawkes Limit Order Book (LOB) model in order to replicate the behaviours of high-frequency market makers. In contrast to…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The market maker can only react to a subset of LOB events due to technological and practical constraints.","type":"model","stated":"explicit","quote":"Given the real-world impracticalities of the market maker updating strategies for every event in the LOB, we formulate the high-frequency market making agent via an impulse control reinforcement learning framework (Jain et al. 2025). The RL framework approximates the solution to the high dimensional Hamilton-Jacobian-Bellman Quasi-Variational Inequality (HJB-QVI), brought about by the fact that a market maker will","support":"full"},{"assumption":"The Hawkes LOB model captures endogenous price impact and key market properties.","type":"model","stated":"explicit","quote":"In contrast to the classical models with exogenous price impact assumptions, the Hawkes model accounts for endogenous price impact and other key properties of the market (Jain et al. 2024a).","support":"full"},{"assumption":"TWAP is an acceptable representation of a meta‑order execution strategy for testing adverse selection.","type":"execution","stated":"explicit","quote":"While TWAP is not the most sophisticated meta-order execution method, it is still actively used in less devel-oped markets and as a part of quantitative trading strategies (Velu, Hardy, and Nehren 2020).","support":"full"},{"assumption":"The TWAP start time is randomised (normal distribution with µ=150) during training to expose the RL agent to varied market conditions.","type":"execution","stated":"explicit","quote":"The first was to randomise the TWAP starting time using a normal distribution with µ = 150, for 300 seconds of trading time.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Hawkes LOB simulation","provider":null,"kind":"synthetic","access":"unknown","period":null,"universe":null,"quote":"Code — https://github.com/konqr/lobSimulations/tree/ master/HawkesRLTrading Introduction In this study, we use the LOB simulation by Jain and Firoozye (Jain et al. 2024a,b,c) to construct a simulated trading environment in which a medium-frequency trader faces strategic adversarial selection from HFTs, replicating the empirically observed phenomenon of medium-frequency traders (MFTs) being adversely selected by","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"available","modeled":"yes","headline_net_of_costs":null,"detail":"Slippage is calculated using the target arrival method: compare final execution cost to the arrival mid‑price recorded when the TWAP agent starts trading.","quote":"Specifically, we calcu-late slippage using the target arrival method. When TWAP starts trading, the midprice is recorded (arrival price), and the final execution cost is compared with the cost if the en-tire order was executed at the arrival price.","source":"front_matter","rule":"R3","candidates":{"front_matter":"yes","evidence_card":"yes","evidence_card_quoted":true},"card_detail":"Slippage is calculated using the target arrival method: compare final execution cost to the arrival mid‑price recorded when the TWAP agent starts trading.","card_quote":"Specifically, we calcu-late slippage using the target arrival method. When TWAP starts trading, the midprice is recorded (arrival price), and the final execution cost is compared with the cost if the en-tire order was executed at the arrival price.","meaning":"costs are part of the reported headline results"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"The RL market‑making agent learns to capitalise on the price drift induced by a meta‑order, reproducing the adverse‑selection phenomenon.","kind":"performance","quote":"To replicate the adverse selection phenomenon, we test the RL agent trading against a medium frequency trader (MFT) executing a meta-order and demonstrate that, with training against the MFT meta-order execution agent, the RL market making agent learns to capitalise on the price drift induced by the meta-order.","support":"full"},{"finding":"The fRL agent’s performance improvement is asymmetric: it benefits strongly from buy‑side meta‑orders but struggles under sell‑side pressure.","kind":"performance","quote":"The results reveal a notable asymmetry in the fRL agent’s per-formance when facing the different TWAP agent sides. In sum, the fRL agent has learnt asymmetrically: it thrives when the market is characterised by upward price momen-tum driven by the buy TWAP agent but struggles to profit under sustained sell-side pressure.","support":"full"},{"finding":"TWAP agent slippage does not increase significantly when trading against the fRL market maker; in some cases execution costs even decrease.","kind":"performance","quote":"Buy Slippage Sell Slippage 0.24 2.31 Table 6: TWAP Slippage vs fRL, in bps As seen in Table 6, though the MM has clearly learnt to profit from trading against the buy side TWAP agent, the results from the same testing episodes showed that the TWAP agent’s slippage did not increase significantly. In fact, when comparing with Table 1, it appears that trading against fRL decreased execution costs for TWAP.","support":"full"},{"finding":"The higher profitability of the fRL market maker does not stem from worsening execution costs for the medium‑frequency trader, suggesting more efficient liquidity provision or inventory management.","kind":"theoretical","quote":"Interestingly, despite the fRL agent’s higher profitability during TWAP buy executions, the slippage experienced by the TWAP agent did not increase significantly. This suggests that while the RL market maker learned to capitalise on pre-dictable order flow, the profits did not necessarily stem from worsening execution costs for the medium-frequency trader.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":28,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/konqr/lobSimulations","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:46:49Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}