Paper: arXiv 2308.08918

Authors: Hui Niu, Siyuan Li, Jiahao Zheng, Zhouchi Lin, Jian Li, Jian Guo, Bo An

Abstract

Market making (MM) has attracted significant attention in financial trading owing to its essential function in ensuring market liquidity. With strong capabilities in sequential decision-making, Reinforcement Learning (RL) technology has achieved remarkable success in quantitative trading. Nonetheless, most existing RL-based MM methods focus on optimizing single-price level strategies which fail at frequent order cancellations and loss of queue priority. Strategies involving multiple price levels align better with actual trading scenarios. However, given the complexity that multi-price level strategies involves a comprehensive trading action space, the challenge of effectively training profitable RL agents for MM persists. Inspired by the efficient workflow of professional human market makers, we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently. The framework start with introducing effective state and action representations adept at encoding information about multi-price level orders. Furthermore, IMM integrates a representation learning unit capable of capturing both short- and long-term market trends to mitigate adverse selection risk. Subsequently, IMM formulates an expert strategy based on signals and trains the agent through the integration of RL and imitation learning techniques, leading to efficient learning. Extensive experimental results on four real-world market datasets demonstrate that IMM outperforms current RL-based market making strategies in terms of several financial criteria. The findings of the ablation study substantiate the effectiveness of the model components.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper involves advanced reinforcement learning and representation learning with complex mathematical formulations (e.g., TCSA network, reward shaping), while also presenting extensive experiments on four real-world datasets with ablation studies, indicating strong empirical backing.

Research Flowchart

  flowchart TD
  A["Research Goal: Develop efficient RL agent for multi-price level market making"] --> B{"Data & Setup"};
  B --> C["State & Action<br>Representations"];
  B --> D["Signal-Based<br>Expert Strategies"];
  C & D --> E{"Computational Model<br>IMM Framework"};
  E --> F["Predictive Representation<br>Learning"];
  E --> G["Imitation Learning<br>Integration"];
  F & G --> H["Outcome: Trained Multi-Price<br>Market Making Agent"];
  H --> I{"Evaluation"};
  I --> J["Key Findings:<br>Outperforms baselines &<br>Validates model components"];

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes Imitative Market Maker (IMM), a novel reinforcement learning framework that combines expert signals and imitation learning to efficiently learn multi-price level market making strategies.

Source sentence
we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently.

Assumes 2

  • Buy limit orders are placed on the bid side and sell orders on the ask side. market
    Source sentence
    It is assumed that buy limit orders are placed on the bid side, and sell ones on the ask side.
  • A stable reference price (pref) can be defined to encode the LOB and multi‑price level orders. model
    Source sentence
    enabling effective encoding of LOB and multi-price level orders.

Data

  • four real‑world market datasets order-book unknown
    Source sentence
    Extensive experimental results on four real‑world market datasets demonstrate that IMM outperforms current RL‑based market making strategies in terms of several financial criteria.
  • four futures markets order-book unknown
    Source sentence
    Experiments on four futures markets demonstrate that IMM outperforms the benchmarks, and further ablation studies verify the effectiveness of the components in the proposed method.

Costs

Unclear No cost treatment was found in the passages the extractor read; that is not evidence that costs are not modeled.

Finds 4

  • IMM significantly outperforms benchmark methods in both profitability and risk management across all test datasets. performance
    Source sentence
    These comparison results indicate that the proposed approach significantly outperforms the benchmarks in terms of both profitability and risk management.
  • On the RB dataset, IMM achieves the highest terminal wealth and risk‑adjusted return, with only a slightly higher MAP compared to the expert and other RL‑based methods. performance
    Source sentence
    As demonstrated in Table 1, on the RB dataset, the proposed method attains the highest terminal wealth as well as risk‑ajusted return, albeit with a slightly elevated MAP in comparison to the expert and the two RL‑based methods.
  • On the FU dataset, IMM obtains the highest terminal wealth, the best return‑to‑risk performance, and superior spread‑capturing ability while maintaining the second‑lowest inventory level. performance
    Source sentence
    On the FU dataset, IMM not only achieves the highest terminal wealth but also demonstrates the most favorable return‑to‑risk performance and spread‑capturing ability, while maintaining the second‑lowest inventory level.
  • Ablation studies confirm that the state/action representations, the SL‑based prediction model, and the TCSA network each contribute positively to IMM’s performance. performance
    Source sentence
    The findings of the ablation study substantiate the effectiveness of the model components.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 19 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.