Paper: arXiv 2305.15821

Authors: Hong Guo, Jianwu Lin, Fanlin Huang

Abstract

Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 6.0/10 (capped from 6.5: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 6.5)

Why this score: The paper employs advanced deep learning architectures (CNNs, attention mechanisms) and reinforcement learning concepts with a custom continuous action space, indicating high mathematical density, while it includes comprehensive experiments on latency, interpretability, and a simulated backtesting environment, showing strong empirical implementation focus.

Research Flowchart

  flowchart TD
  A["Research Goal: Market Making with<br>Deep RL using Limit Order Books"] --> B{"Data & Inputs"}
  B --> B1["Limit Order Book Data"]
  B --> B2["Hybrid Reward Function"]
  B --> B3["Continuous Action Space"]
  
  B --> C["Key Methodology:<br>Attn-LOB Neural Network"]
  C --> C1["Convolutional Filters<br>extract local features"]
  C --> C2["Attention Mechanism<br>captures dynamic relevance"]
  
  C1 & C2 --> D["Computational Process:<br>Reinforcement Learning Agent"]
  D --> E["Optimized Bid/Ask Quotes"]
  
  E --> F["Key Findings & Outcomes"]
  F --> F1["Good Applicability<br>(Latency & Interpretability)"]
  F --> F2["Superior Feature Extraction<br>vs Handcrafted Features"]
  F --> F3["Effective MM Strategy<br>without Strong Assumptions"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a deep reinforcement learning agent for market making that uses a CNN‑Attention network (Attn‑LOB) for automatic feature extraction from limit order book data, a novel continuous action space, and a hybrid reward function.

Source sentence
We propose an effective framework to train an RL agent with automatic feature extraction and pre-training from LOB data for market making problems, which is a novel challenge in the HFT domain.

Assumes 4

  • The environment is model‑free and does not require explicit modeling of price or order arrival processes. model
    Source sentence
    In contrast to previous MM models [9], [11] which rely on strong assumptions, we use a model‑free approach which need not to explicitly model the environment.
  • Transaction costs are zero in the simulated trading environment. execution
    Source sentence
    The transaction cost is set to 0, and the agent can hold negative cash or inventory.
  • The agent cannot quote if its absolute inventory exceeds a maximum inventory limit defined as ω times the minimum trade unit. execution
    Source sentence
    If the absolute inventory exceeds max inventory = ω ∗minimum trade unit, the agent will be prohibited from quoting in that direction.
  • The simulator only executes the agent’s order when the real historical order arrives, ensuring realistic transaction timing. execution
    Source sentence
    To ensure the reality of simulated transactions, the simulator executes the agent’s order only when the real historical order arrives.

Data

  • Shenzhen Stock Exchange historical orders and trades (Nov 2019) · Shenzhen Stock Exchange order-book commercial
    November 2019 (21 trading days) · Ping An Bank Co.,Ltd. (SZ.000001), Wuliangye Yibin Co.,Ltd. (SZ.000858), Hikvision Technol
    Source sentence
    ... We select three stocks in different industries to study the applicability of proposed agent, which are Ping An Bank Co.,Ltd. (SZ.000001) in banking, Wuliangye Yibin Co.,Ltd. (SZ.000858) in liquor‑making, and Hikvision Technology Co.,Ltd. (SZ.002415) in technology.

Costs

Not modeled A trading or execution result is reported without costs.

Transaction cost is set to 0 in the simulator.

Source sentence
The transaction cost is set to 0, and the agent can hold negative cash or inventory.

Finds 2

  • In pre‑training, Attn‑LOB outperforms other models on recall and F1 score while using a shorter input sequence. predictive-accuracy
    Source sentence
    Attn‑LOB outperforms other methods on recall and F1 score, and we only use half the time length as input compared with DeepLOB.
  • The agent’s attention visualizations show it focuses on the nearest events while still considering early market changes, indicating learned market‑making skills similar to humans. other
    Source sentence
    We explain our results by visualizing attention weights. The agent paid the most attention to the nearest events while still looking at very early changes. And we find that the agent can learn some market‑making skills like humans.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 11 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 3 papers in the archive (3 by independent authors)

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.