Paper: arXiv 2308.08918
Authors: Hui Niu, Siyuan Li, Jiahao Zheng, Zhouchi Lin, Jian Li, Jian Guo, Bo An
Abstract
Market making (MM) has attracted significant attention in financial trading owing to its essential function in ensuring market liquidity. With strong capabilities in sequential decision-making, Reinforcement Learning (RL) technology has achieved remarkable success in quantitative trading. Nonetheless, most existing RL-based MM methods focus on optimizing single-price level strategies which fail at frequent order cancellations and loss of queue priority. Strategies involving multiple price levels align better with actual trading scenarios. However, given the complexity that multi-price level strategies involves a comprehensive trading action space, the challenge of effectively training profitable RL agents for MM persists. Inspired by the efficient workflow of professional human market makers, we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently. The framework start with introducing effective state and action representations adept at encoding information about multi-price level orders. Furthermore, IMM integrates a representation learning unit capable of capturing both short- and long-term market trends to mitigate adverse selection risk. Subsequently, IMM formulates an expert strategy based on signals and trains the agent through the integration of RL and imitation learning techniques, leading to efficient learning. Extensive experimental results on four real-world market datasets demonstrate that IMM outperforms current RL-based market making strategies in terms of several financial criteria. The findings of the ablation study substantiate the effectiveness of the model components.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper involves advanced reinforcement learning and representation learning with complex mathematical formulations (e.g., TCSA network, reward shaping), while also presenting extensive experiments on four real-world datasets with ablation studies, indicating strong empirical backing.
Research Flowchart
flowchart TD
A["Research Goal: Develop efficient RL agent for multi-price level market making"] --> B{"Data & Setup"};
B --> C["State & Action<br>Representations"];
B --> D["Signal-Based<br>Expert Strategies"];
C & D --> E{"Computational Model<br>IMM Framework"};
E --> F["Predictive Representation<br>Learning"];
E --> G["Imitation Learning<br>Integration"];
F & G --> H["Outcome: Trained Multi-Price<br>Market Making Agent"];
H --> I{"Evaluation"};
I --> J["Key Findings:<br>Outperforms baselines &<br>Validates model components"];
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes Imitative Market Maker (IMM), a novel reinforcement learning framework that combines expert signals and imitation learning to efficiently learn multi-price level market making strategies.
Source sentence
we propose Imitative Market Maker (IMM), a novel RL framework leveraging both knowledge from suboptimal signal-based experts and direct policy interactions to develop multi-price level MM strategies efficiently.
Assumes 2
- Buy limit orders are placed on the bid side and sell orders on the ask side. market
Source sentence
It is assumed that buy limit orders are placed on the bid side, and sell ones on the ask side.
- A stable reference price (pref) can be defined to encode the LOB and multi‑price level orders. model
Source sentence
enabling effective encoding of LOB and multi-price level orders.
Data
- four real‑world market datasets order-book unknown
Source sentence
Extensive experimental results on four real‑world market datasets demonstrate that IMM outperforms current RL‑based market making strategies in terms of several financial criteria.
- four futures markets order-book unknown
Source sentence
Experiments on four futures markets demonstrate that IMM outperforms the benchmarks, and further ablation studies verify the effectiveness of the components in the proposed method.
Costs
Unclear
Finds 4
- IMM significantly outperforms benchmark methods in both profitability and risk management across all test datasets. performance
Source sentence
These comparison results indicate that the proposed approach significantly outperforms the benchmarks in terms of both profitability and risk management.
- On the RB dataset, IMM achieves the highest terminal wealth and risk‑adjusted return, with only a slightly higher MAP compared to the expert and other RL‑based methods. performance
Source sentence
As demonstrated in Table 1, on the RB dataset, the proposed method attains the highest terminal wealth as well as risk‑ajusted return, albeit with a slightly elevated MAP in comparison to the expert and the two RL‑based methods.
- On the FU dataset, IMM obtains the highest terminal wealth, the best return‑to‑risk performance, and superior spread‑capturing ability while maintaining the second‑lowest inventory level. performance
Source sentence
On the FU dataset, IMM not only achieves the highest terminal wealth but also demonstrates the most favorable return‑to‑risk performance and spread‑capturing ability, while maintaining the second‑lowest inventory level.
- Ablation studies confirm that the state/action representations, the SL‑based prediction model, and the TCSA network each contribute positively to IMM’s performance. performance
Source sentence
The findings of the ablation study substantiate the effectiveness of the model components.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 19 related papers). Coverage is partial, so this is neither replication nor refutation.
2 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.