Market Making with Deep Reinforcement Learning from Limit Order Books

ArXiv ID: 2305.15821 “View on arXiv”

Authors: Unknown

Abstract

Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making. The modeling of evolving, high-dimensional and low signal-to-noise ratio LOB data is a critical challenge. Traditional MM strategy relied on strong assumptions such as price process, order arrival process, etc. Previous reinforcement learning (RL) works handcrafted market features, which is insufficient to represent the market. This paper proposes a RL agent for market making with LOB data. We leverage a neural network with convolutional filters and attention mechanism (Attn-LOB) for feature extraction from LOB. We design a new continuous action space and a hybrid reward function for the MM task. Finally, we conduct comprehensive experiments on latency and interpretability, showing that our agent has good applicability.

Keywords: market making, reinforcement learning, limit order book (LOB), convolutional neural network, Equities

Complexity vs Empirical Score

• Math Complexity: 8.0/10
• Empirical Rigor: 6.5/10
• Quadrant: Holy Grail
• Why: The paper employs advanced deep learning architectures (CNNs, attention mechanisms) and reinforcement learning concepts with a custom continuous action space, indicating high mathematical density, while it includes comprehensive experiments on latency, interpretability, and a simulated backtesting environment, showing strong empirical implementation focus.

  flowchart TD
    A["Research Goal: Market Making with<br>Deep RL using Limit Order Books"] --> B{"Data & Inputs"}
    B --> B1["Limit Order Book Data"]
    B --> B2["Hybrid Reward Function"]
    B --> B3["Continuous Action Space"]
    
    B --> C["Key Methodology:<br>Attn-LOB Neural Network"]
    C --> C1["Convolutional Filters<br>extract local features"]
    C --> C2["Attention Mechanism<br>captures dynamic relevance"]
    
    C1 & C2 --> D["Computational Process:<br>Reinforcement Learning Agent"]
    D --> E["Optimized Bid/Ask Quotes"]
    
    E --> F["Key Findings & Outcomes"]
    F --> F1["Good Applicability<br>(Latency & Interpretability)"]
    F --> F2["Superior Feature Extraction<br>vs Handcrafted Features"]
    F --> F3["Effective MM Strategy<br>without Strong Assumptions"]