Paper: arXiv 2305.15821
Authors: Hong Guo, Jianwu Lin, Fanlin Huang
Abstract
Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.
Complexity vs Empirical Score
- Math Complexity: 8.0/10
- Empirical Rigor: 6.0/10 (capped from 6.5: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 6.5)
Why this score: The paper employs advanced deep learning architectures (CNNs, attention mechanisms) and reinforcement learning concepts with a custom continuous action space, indicating high mathematical density, while it includes comprehensive experiments on latency, interpretability, and a simulated backtesting environment, showing strong empirical implementation focus.
Research Flowchart
flowchart TD
A["Research Goal: Market Making with<br>Deep RL using Limit Order Books"] --> B{"Data & Inputs"}
B --> B1["Limit Order Book Data"]
B --> B2["Hybrid Reward Function"]
B --> B3["Continuous Action Space"]
B --> C["Key Methodology:<br>Attn-LOB Neural Network"]
C --> C1["Convolutional Filters<br>extract local features"]
C --> C2["Attention Mechanism<br>captures dynamic relevance"]
C1 & C2 --> D["Computational Process:<br>Reinforcement Learning Agent"]
D --> E["Optimized Bid/Ask Quotes"]
E --> F["Key Findings & Outcomes"]
F --> F1["Good Applicability<br>(Latency & Interpretability)"]
F --> F2["Superior Feature Extraction<br>vs Handcrafted Features"]
F --> F3["Effective MM Strategy<br>without Strong Assumptions"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a deep reinforcement learning agent for market making that uses a CNN‑Attention network (Attn‑LOB) for automatic feature extraction from limit order book data, a novel continuous action space, and a hybrid reward function.
Source sentence
We propose an effective framework to train an RL agent with automatic feature extraction and pre-training from LOB data for market making problems, which is a novel challenge in the HFT domain.
Assumes 4
- The environment is model‑free and does not require explicit modeling of price or order arrival processes. model
Source sentence
In contrast to previous MM models [9], [11] which rely on strong assumptions, we use a model‑free approach which need not to explicitly model the environment.
- Transaction costs are zero in the simulated trading environment. execution
Source sentence
The transaction cost is set to 0, and the agent can hold negative cash or inventory.
- The agent cannot quote if its absolute inventory exceeds a maximum inventory limit defined as ω times the minimum trade unit. execution
Source sentence
If the absolute inventory exceeds max inventory = ω ∗minimum trade unit, the agent will be prohibited from quoting in that direction.
- The simulator only executes the agent’s order when the real historical order arrives, ensuring realistic transaction timing. execution
Source sentence
To ensure the reality of simulated transactions, the simulator executes the agent’s order only when the real historical order arrives.
Data
- Shenzhen Stock Exchange historical orders and trades (Nov 2019) order-book commercial
Source sentence
... We select three stocks in different industries to study the applicability of proposed agent, which are Ping An Bank Co.,Ltd. (SZ.000001) in banking, Wuliangye Yibin Co.,Ltd. (SZ.000858) in liquor‑making, and Hikvision Technology Co.,Ltd. (SZ.002415) in technology.
Costs
Not modeled
Transaction cost is set to 0 in the simulator.
Source sentence
The transaction cost is set to 0, and the agent can hold negative cash or inventory.
Finds 2
- In pre‑training, Attn‑LOB outperforms other models on recall and F1 score while using a shorter input sequence. predictive-accuracy
Source sentence
Attn‑LOB outperforms other methods on recall and F1 score, and we only use half the time length as input compared with DeepLOB.
- The agent’s attention visualizations show it focuses on the nearest events while still considering early market changes, indicating learned market‑making skills similar to humans. other
Source sentence
We explain our results by visualizing attention weights. The agent paid the most attention to the nearest events while still looking at very early changes. And we find that the agent can learn some market‑making skills like humans.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.
3 papers in the archive
Code
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.