Paper: arXiv 2601.17247

Authors: Julius Graf, Thibaut Mastrolia

Abstract

In this work, we investigate the market-making problem on a trading session in which a continuous phase on a limit order book is followed by a closing auction. Whereas standard optimal market-making models typically rely on terminal inventory penalties to manage end-of-day risk, ignoring the significant liquidity events available in closing auctions, we propose a Deep Q-Learning framework that explicitly incorporates this mechanism. We introduce a market-making framework designed to explicitly anticipate the closing auction, continuously refining the projected clearing price as the trading session evolves. We develop a generative stochastic market model to simulate the trading session and to emulate the market. Our theoretical model and Deep Q-Learning method is applied on the generator in two settings: (1) when the mid price follows a rough Heston model with generative data from this stochastic model; and (2) when the mid price corresponds to historical data of assets from the S&P 500 index and the performance of our algorithm is compared with classical benchmarks from optimal market making.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper tackles a novel and practically relevant problem by incorporating closing auction dynamics into reinforcement learning-based optimal market making. It bridges rigorous stochastic modeling (including rough Heston dynamics and formal proofs like existence of clearing prices) with solid empirical evaluation on S&P 500 historical data.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimize Market Making with Closing Auction Mechanics] --> B[Data Inputs: Rough Heston Model & Historical S&P 500 Data]
    B --> C[Develop Generative Stochastic LOB & Auction Simulator]
    C --> D[Deep Q-Learning Framework with Projected Clearing Price Estimation]
    D --> E[Benchmark Comparison against Classical Market-Making Models]
    E --> F[Outcomes: Improved Inventory Risk Management & PnL via Closing Auction Exploitation]

Cited References (77)

  1. Is Q-learning Provably Efficient? (2018)
  2. Neural fitted Q iteration–first experiences with a data efficient neural reinforcement learning method (2005)
  3. Statistical properties of share volume traded in financial markets (2000)
  4. Human-level control through deep reinforcement learning (2015)
  5. Reinforcement learning: An introduction (1998)
  6. Market making via reinforcement learning (2019)
  7. Optimal execution of portfolio transactions (2001)
  8. Twin-delayed ddpg: A deep reinforcement learning technique to model a continuous movement of an intelligent robot agent (2019)
and 69 more
  1. Soft actor-critic algorithms and applications (2018) — link
  2. Deep deterministic policy gradient algorithm: A systematic review (2024)
  3. Volatility is rough (2022)
  4. Multifactor approximation of rough volatility models (2019)
  5. Portfolio optimization in fractional and rough H eston models (2020)
  6. Markov decision processes: discrete stochastic dynamic programming (2014)
  7. High frequency traders: Taking advantage of speed (2013)
  8. Special issue on machine learning in finance (2024)
  9. Machine Learning and Data Sciences for Financial Markets: A Guide to Contemporary Practices (2023)
  10. Market liquidity and competition among designated market makers (2025)
  11. Putting auction theory to work (2004)
  12. Playing Atari with deep reinforcement learning (2013) — link
  13. Double deep Q -learning for optimal execution (2021)
  14. Market making and incentives design in the presence of a dark pool: a Stackelberg actor–critic approach (2023)
  15. A theoretical analysis of deep Q -learning (2020)
  16. Machine learning for market microstructure and high-frequency trading (2013)
  17. Reinforcement learning for market making in a multi-agent dealer market (2019) — link
  18. Enhancing Q -learning for optimal asset allocation (1997)
  19. Q -learning (1992)
  20. Transaction Cost (In) Transparency: Coasian Dynamics in Frequent Batch Auctions (2024)
  21. Equity auction dynamics: latent liquidity models with activity acceleration (2024)
  22. Optimal dynamic auctions for revenue management (2002)
  23. Auctions and bidding: A primer (1989)
  24. Deep reinforcement learning for market making in corporate bonds: beating the curse of dimensionality (2019)
  25. Algorithmic market making for options (2021)
  26. Algorithmic and high-frequency trading (2015)
  27. Price discovery and learning during the preopening period in the Paris Bourse (1999)
  28. IPO auctions: English , Dutch , \ldots French , and Internet (2002) — link
  29. Price discovery in auction markets: A look inside the black box (2000)
  30. The effect of a closing call auction on market quality and trading strategies (2012)
  31. Continuous auctions and insider trading (1985)
  32. Auction market design: Recent innovations (2019)
  33. AHEAD : Ad hoc electronic auction design (2024)
  34. Welfare and optimal trading frequency in dynamic double auctions (2014)
  35. The influence of economic research on financial mathematics: Evidence from the last 25 years (2022)
  36. Size discovery (2017)
  37. Equilibria and incentives for illiquid auction markets — → 2307.15805
  38. Market making with scaled beta policies (2022)
  39. Market Making with Learned Beta Policies (2024)
  40. Mbt-gym: Reinforcement learning for model-based limit order book trading (2023)
  41. Power laws in economics and finance: some ideas from physics (2001)
  42. Power laws in economics and finance (2009)
  43. Clearing time randomization and transaction fees for auction market design — → 2405.09764
  44. Optimal Rebate Design: Incentives, Competition and Efficiency in Auction Markets — → 2501.12591
  45. Implementation details for frequent batch auctions: Slowing down markets to the blink of an eye (2014)
  46. The growing importance of the closing auction in share trading volumes (2020)
  47. Logarithmic regret in the ergodic A vellaneda- S toikov market making model — → 2409.02025
  48. Market Making without Regret — → 2411.13993
  49. Deep learning for limit order books (2019)
  50. Recent advances in reinforcement learning in finance (2023)
  51. Reinforcement learning for optimized trade execution (2006)
  52. Market making with signals through deep reinforcement learning (2021)
  53. Breaking the Sample Complexity Barrier to Regret-Optimal Model-Free Reinforcement Learning (2021)
  54. Dealing with the inventory risk: a solution to the market making problem (2013)
  55. Optimal auction duration: A price formation viewpoint (2021)
  56. The economics of competitive bidding: a selective survey (1985)
  57. Trading mechanisms in securities markets (1992)
  58. On the discrete-time simulation of the rough Heston model (2023)
  59. High-frequency trading in a limit order book (2008)
  60. Learning a functional control for high-frequency finance (2022)
  61. Lifting the H eston model (2019)
  62. Deep reinforcement learning with double Q -learning (2016)
  63. Continuous control with deep reinforcement learning (2015)
  64. Addressing function approximation error in actor-critic methods (2018)
  65. Spinning Up in Deep Reinforcement Learning (2018)
  66. Deep reinforcement learning at the edge of the statistical precipice (2021)
  67. Deep reinforcement learning that matters (2018)
  68. Learning Optimal Liquidation with Closing Auctions : Replication bundle (2026)
  69. Stable-Baselines3: Reliable reinforcement learning implementations (2021)

5 of 77 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.