Paper: arXiv 2510.15995

Authors: Luigi Foscari, Emanuele Guidotti, Nicolò Cesa-Bianchi, Tatjana Chavdarova, Alfio Ferrara

Abstract

We study the emergence of tacit collusion between adaptive trading agents in a stochastic market with endogenous price formation. Using a two-player repeated game between a market maker and a market taker, we characterize feasible and collusive strategy profiles that raise prices beyond competitive levels. We show that, when agents follow simple learning algorithms (e.g., gradient ascent) to maximize their own wealth, the resulting dynamics converge to collusive strategy profiles, even in highly liquid markets with small trade sizes. By highlighting how simple learning strategies naturally lead to tacit collusion, our results offer new insights into the dynamics of AI-driven markets.

Complexity vs Empirical Score

  • Math Complexity: 7.0/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper employs advanced mathematical modeling including stochastic processes, game theory, and formal proofs of convergence for gradient ascent dynamics, indicating high math complexity. However, it relies on theoretical simulations and stylized models rather than real-world historical data, backtesting, or implementation-heavy empirical validation.

Research Flowchart

  flowchart TD
  A["Research Goal<br>How do adaptive agents<br>achieve tacit collusion?"] --> B["Methodology<br>Stochastic Game Framework"]
  B --> C["Setup<br>Two-Player Repeated Game<br>Market Maker vs Taker"]
  C --> D["Learning Process<br>Gradient Ascent Optimization"]
  D --> E["Simulation<br>Adaptive Agents in<br>Stochastic Market"]
  E --> F{"Convergence?"}
  F -->|"Yes"| G["Outcome 1<br>Collusive Equilibrium<br>Price > Competitive Level"]
  F -->|"No"| H["Outcome 2<br>Competitive Dynamics<br>Price ≈ Competitive Level"]
  G --> I["Key Insight<br>Simple learning strategies<br>naturally lead to tacit collusion"]
  H --> I

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

We introduce a repeated stochastic game between a market maker and a market taker with endogenous price impact and cash‑inventory constraints to study persistent overpricing.

Source sentence
To do so, we introduce a repeated stochastic game between two representative agents: a market maker and a market taker. Both agents hold cash and inventory and trade to maximize the value of their own portfolios, subject to constraints that rule out short and leveraged positions.

Assumes 3

  • Exogenous shocks are i.i.d. with finite variance and mean one. model
    Source sentence
    We make the common assumption that the shocks are independent and identically distributed with finite variance, and normalize their mean to one. Assumption 1. (εt)t is an i.i.d. stochastic process such that εt > 0 has finite variance and E[εt] = 1.
  • Price impact follows a square‑root functional form in trade size. model
    Source sentence
    We adopt a square-root specification, following a large empirical literature on market impact that finds a square-root dependence on trade size (Lillo et al., 2003; Tóth et al., 2011; Mastromatteo et al., 2014; Tóth et al., 2016; Bouchaud et al., 2018). Assumption 2. Let Qt be the quantity traded at time t.
  • Agents hold non‑negative aggregate cash and inventory (no short or leveraged positions). market
    Source sentence
    Hence the representative maker and taker hold nonnegative aggregate cash and inventory positions.

Data

Synthetic data only.

Costs

Not applicable The paper reports no trading or execution result.

Finds 5

  • Feasible overpricing profiles are characterized via a low‑dimensional parameterization of strategies, identifying the region that generates persistent overpricing. theoretical
    Source sentence
    Our contributions are threefold. • We characterize feasible overpricing profiles: we introduce a repeated game with endogenous price impact and cash-inventory constraints, derive a low-dimensional parameterization of strategies, and identify the region of the parameter space that generates persistent overpricing (section 2). • Through a strategic decomposition of the stage game,
  • Decentralized learning reaches the overpricing region in finite time for adaptive updates satisfying a stochastic progress condition, verified for projected stochastic gradient ascent. theoretical
    Source sentence
    We provide a sufficient condition for decentralized learning to reach the overpricing region in finite time, and we show that this condition is satisfied, in particular, by projected stochastic gradient ascent. We then prove that decentralized learning reaches the overpricing region in finite time for adaptive updates satisfying a stochastic progress condition, and verify this condition for projected stochastic
  • The same overpricing incentives arise for agents optimizing infinite‑horizon wealth, showing the phenomenon is not purely myopic. theoretical
    Source sentence
    We then prove that decentralized learning reaches the overpricing region in finite time for adaptive updates satisfying a stochastic progress condition, and verify this condition for projected stochastic gradient ascent (section 3). • We show that this phenomenon is not purely myopic: agents optimizing infinite-horizon wealth face the same overpricing incentives as greedy myopic agents and reach the same overpricing
  • A strategic decomposition reveals a competitive component favoring zero price impact and a collaborative component that makes overpricing jointly profitable when aggregate inventory is positive. theoretical
    Source sentence
    A key step in the analysis is a decomposition of the game into a competitive component, which favors zero price impact, and a collaborative component, which makes overpricing jointly profitable when aggregate inventory is positive.
  • Simulations of projected stochastic gradient ascent illustrate learning trajectories that converge to the overpricing region. performance
    Source sentence
    훼(푣훼) (a) Ask parameters 0.0 0.5 1.0 푘훽 0.00 0.25 0.50 0.75 1.00 푣훽 Feasible region 푓훽(푣훽) (b) Bid parameters 0.0 0.1 0.2 0.3 푘훼푣훼 0.0 0.1 0.2 0.3 푘훽푣훽 Overpricing region 휇휋= 0 (c) Joint parameter space Figure 1: Plots a and b show a simulation of the learning trajectories of PSGA across the feasible space of parameters with φ = 1/2. The dark line shows the learning trajectory obtained by noisy gradients, in cyan

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 14 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (107)

  1. Potential games (1996)
  2. Markov Games as a Framework for Multi-Agent Reinforcement Learning (1994)
  3. On Multiagent Q-Learning in a Semi-Competitive Domain (1995)
  4. The extragradient method for finding saddle points and other problems (1976)
  5. Competing mechanisms in a common value environment (2000)
  6. Heterogeneous beliefs and routes to chaos in a simple asset pricing model (1998) — link
  7. A modification of the Arrow–Hurwicz method for search of saddle points (1980)
  8. Multiagent Reinforcement Learning: Theoretical Framework and an Algorithm (1998)
and 99 more
  1. Foresight-based pricing algorithms in an economy of software agents (1998) — link
  2. Foresight-based pricing algorithms in agent economies (2000) — link
  3. Pricing in agent economies using multi-agent Q-learning (2002) — link
  4. Price dynamics and quality in information markets (2000) — link
  5. Developing Competition Law for Collusion by Autonomous Artificial Agents (2018) — link
  6. Q-learning agents in a Cournot oligopoly model (2008)
  7. Stochastic Games* (1953) — link
  8. Q-learning (1992) — link
  9. Deep Q-Learning in Airline Dynamic Pricing and Tacit Collusion: Experimental deep RL in airline revenue management in a duopoly setting (2023)
  10. The complexity of computing a Nash equilibrium (2009) — link
  11. Human-level control through deep reinforcement learning (2015)
  12. A Model of Price Formation in Frictionless Markets
  13. Universal portfolios with side information (1996) — link
  14. On-line portfolio selection (1996) — link
  15. A Modern Introduction to Online Learning (2023) — link
  16. Online portfolio selection: A survey (2014) — link
  17. Growth Optimal Investment with Transaction Costs (2008) — link
  18. Universal portfolios with and without transaction costs (1997) — link
  19. Weak aggregating specialist algorithm for online portfolio selection (2023) — link
  20. Game theory, alive (2017)
  21. Competing Against Nash Equilibria in Adversarially Changing Zero - Sum Games (2019) — link
  22. Computing the optimal strategy to commit to — link
  23. Optimal Machine Strategies to Commit to in Two-Person Repeated Games (2015) — link
  24. Reinforcement learning for optimized trade execution (2006)
  25. Market making and mean reversion (2011)
  26. Adaptive market making via online learning (2013)
  27. On the Stackelberg strategy in nonzero-sum games (1973) — link
  28. On the dominance of capitalists leadership in a ` Feedback-Stackelberg ’ solution of a differential game model of capitalism (1985) — link
  29. An anticipative feedback solution for the infinite-horizon, linear-quadratic, dynamic, Stackelberg game (2002) — link
  30. On Coincidence of Feedback Nash Equilibria and Stackelberg Equilibria in Economic Applications of Differential Games (2006) — link
  31. Model-free Resilient Controller Design based on Incentive Feedback Stackelberg Game and Q-learning (2024) — link
  32. Equilibrium computation for extensive games
  33. Computing equilibria for two-person games — link
  34. Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Games — link
  35. Equilibrium computation for two-player games in strategic and extensive form
  36. Meta- Learning in Games (2023) — link
  37. Global Convergence of Multi - Agent Policy Gradient in Markov Potential Games (2021) — link
  38. On a new class of nonzero-sum discounted stochastic games having stationary Nash equilibrium points (2003) — link
  39. Existence of stationary equilibrium strategies in non-zero sum discounted stochastic games with uncountable state space and state-independent transitions (1989) — link
  40. A Simple Adaptive Procedure Leading to Correlated Equilibrium (2000) — link
  41. An introduction to the theory of asymptotic martingales (1983) — link
  42. Artificial Intelligence , Algorithmic Pricing , and Collusion (2020) — link
  43. Artificial Intelligence and Auction Design (2022) — link
  44. Algorithmic collusion: A critical review (2021) — link
  45. Market Making without Regret — → 2411.13993
  46. Uniswap Liquidity Provision : An Online Learning Approach (2023) — link
  47. Finding the Right Curve : Optimal Design of Constant Function Market Makers (2023) — link
  48. Bandits with Knapsacks (2018) — link
  49. Bandits with Replenishable Knapsacks : the Best of both Worlds (2023) — link
  50. Non-monotonic Resource Utilization in the Bandits with Knapsacks Problem (2022) — link
  51. A Unifying Framework for Online Optimization with Long - Term Constraints (2022) — link
  52. Algorithmic Pricing and Liquidity in Securities Markets (2022) — link
  53. On Stochastic and Worst-case Models for Investing (2009)
  54. Universal portfolios with short sales and margin (1998) — link
  55. Minimax regret portfolios for restricted stock sequences (2004) — link
  56. Model-based Reinforcement Learning for Predictions and Control for Limit Order Books (2019) — link
  57. Learning to simulate realistic limit order book markets from data as a World Agent (2022) — link
  58. Reinforcement Learning for Market Making in a Multi -agent Dealer Market (2019) — link
  59. Deep reinforcement learning for high-frequency market making (2023) — link
  60. Market making via reinforcement learning (2018)
  61. Robust market making via adversarial reinforcement learning (2020)
  62. Robust market making via adversarial reinforcement learning (2021)
  63. Efficient market making via convex optimization, and a connection to online learning (2013) — link
  64. Adapting to a Market Shock: Optimal Sequential Market-Making (2008)
  65. Efficient algorithms for universal portfolios (2000) — link
  66. Cooperation between independent market makers (2022) — link
  67. Trading and exchanges: Market microstructure for practitioners (2002)
  68. The pricing of options and corporate liabilities (1973)
  69. Continuous Auctions and Insider Trading (1985) — link
  70. AI -powered trading, algorithmic collusion, and price efficiency (2025)
  71. Efficient capital markets: A review of theory and empirical work (1970)
  72. Proof that properly anticipated prices fluctuate randomly (2016)
  73. Proof That Properly Anticipated Prices Fluctuate Randomly (1965)
  74. Master curve for price-impact function (2003)
  75. Anomalous price impact and the critical nature of liquidity in financial markets (2011)
  76. Agent-based models for latent liquidity and concave price impact (2014)
  77. Trades, quotes and prices: financial markets under the microscope (2018)
  78. Toward a fully automated stock exchange, part I (1971)
  79. The Square-Root Impace Law Also Holds for Option Markets (2016)
  80. How efficiency shapes market impact (2013)
  81. Q-learning agents in a Cournot oligopoly model (2008) — link
  82. Markets with a continuum of traders (1964)
  83. On the impossibility of informationally efficient markets (1980)
  84. Adaptive Algorithms and Collusion via Coupling (2023) — link
  85. Strategic decompositions of normal form games: Zero -sum games and potential games (2020) — link
  86. Best response equivalence (2004)
  87. Game Theory (2013) — link
  88. Collusion by mistake: Does algorithmic sophistication drive supra-competitive profits? (2024)
  89. Risk aversion in the small and in the large (1978) — link
  90. An online portfolio selection algorithm with regret logarithmic in price variation (2015) — link
  91. Pushing the Efficiency - Regret Pareto Frontier for Online Learning of Portfolios and Quantum States (2022) — link
  92. Efficient and Near - Optimal Online Portfolio Selection (2025) — link
  93. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches (2024) — link
  94. Algorithmic collusion and a folk theorem from learning with bounded rationality (2026) — link
  95. Data dependent regret bounds for online portfolio selection with predicted returns (2025) — link
  96. The Real Effects of Financial Markets (2012)
  97. Learning Unfair Trading : a Market Manipulation Analysis From the Reinforcement Learning Perspective (2015) — link
  98. Mispricing and Algorithm Trading (2025) — link
  99. Recent Advances in Reinforcement Learning in Finance (2021) — link

1 of 107 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.