Paper: arXiv 2307.01816

Authors: Zhou Fang, Haiqing Xu

Abstract

The over-the-counter (OTC) market is characterized by a unique feature that allows market makers to adjust bid-ask spreads based on order size. However, this flexibility introduces complexity, transforming the market-making problem into a high-dimensional stochastic control problem that presents significant challenges. To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper uses advanced mathematical concepts like high-dimensional stochastic control, HJB equations, and Poisson processes, indicating high mathematical complexity. However, the empirical analysis relies solely on simulated data without real-world backtesting or implementation details, resulting in low empirical rigor.

Research Flowchart

  flowchart TD
  Start["Research Goal<br>Optimize Bid-Ask Spreads in OTC Market<br>via Reinforcement Learning"] --> Inputs["Data & Assumptions<br>Linear Inverse Relationship:<br>Spread vs. Arrival Intensity"]
  
  Inputs --> Method["Methodology<br>High-Dimensional Stochastic Control<br>Reinforcement Learning Algorithms"]
  
  Method --> Process["Computational Process<br>1. Model Stochastic Market Dynamics<br>2. Train RL Agents (PPO/SAC)<br>3. Simulate Order Book & Inventory"]
  
  Process --> Outcome["Key Findings & Outcomes<br>- Optimal Policy: Gaussian Distribution<br>- Return Distribution Analysis<br>- Bid-Ask Spreads vs. Time/Inventory"]
  
  Outcome --> End["Conclusion<br>RL Effectively Solves<br>Complex OTC Market-Making"]
  
  style Start fill:#e1f5fe,stroke:#01579b
  style Inputs fill:#fff3e0,stroke:#e65100
  style Method fill:#e8f5e9,stroke:#1b5e20
  style Process fill:#f3e5f5,stroke:#4a148c
  style Outcome fill:#fce4ec,stroke:#880e4f
  style End fill:#e0e0e0,stroke:#424242

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a reinforcement learning framework with a stochastic policy for OTC market making, showing that the optimal bid‑ask spread policy follows a Gaussian distribution.

Source sentence
To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. In this research paper, we present a novel framework based on reinforcement learning to address the in-tricate challenges of the multi-dimensional market-making problem encountered by over-the-counter (OTC) market makers.

Assumes 6

  • Market order arrival intensity is linearly inverse to bid‑ask spreads. model
    Source sentence
    By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. 2 As a result, the dynamics of the inventory can be described as follows: dqt = K X k=1 zk �dN + t (k) −dN − t (k)  (2) in this paper, we assume a simple setting, which is the market orders arrival intensity is linearly inverse
  • The volatility of the underlying asset is constant over the trading period. model
    Source sentence
    To streamline the problem and facilitate analysis, we adopt the assumption that the volatility of the underlying asset remains constant throughout the entire trading period.
  • The market maker does not externalize its inventory. model
    Source sentence
    In this paper, we simplify the setting by assuming that the market maker does not externalize its inventory.
  • Buy and sell order executions follow Poisson processes. execution
    Source sentence
    Specifically, we denote the number of buy order executions as N + t (k) and the number of sell order executions as N − t (k). The intensities of these Poisson processes are denoted as λ+(k) and λ−(k), respectively.
Show 2 more
  • The optimal bid‑ask spread policy can be modeled as a Gaussian distribution. statistical
    Source sentence
    Under this assumption, we demonstrate that the optimal policy for bid-ask spread determination can be modeled as a Gaussian distribution.
  • A stochastic policy is used to determine bid‑ask spreads. model
    Source sentence
    Our approach centers on the utilization of a stochastic policy, which enables the market maker to determine bid-ask spreads.

Data

  • Simulated data synthetic unknown
    Source sentence
    To validate the effectiveness of our proposed framework, we conduct extensive numerical experiments using simulated data, showcasing various performance metrics and bid-ask spread outcomes for specific inventory levels.

Costs

Unclear The classifier was not confident.

Finds 5

  • Under the linear inverse intensity assumption, the optimal bid‑ask spread policy follows a Gaussian distribution. theoretical
    Source sentence
    By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. Under this assumption, we demonstrate that the optimal policy for bid-ask spread determination can be modeled as a Gaussian distribution.
  • Numerical experiments reveal the return distribution and bid‑ask spreads under different time and inventory levels. performance
    Source sentence
    We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels.
  • The policy iteration algorithm effectively refines the policy and improves return performance across iterations. performance
    Source sentence
    This suggests that the 12 Figure 7: return distribution of after third policy iteration Figure 8: return distribution of after fourth policy iteration Figure 9: return distribution of after fifth policy iteration policy iteration algorithm is effective in refining the policy and enhancing the performance in terms of returns.
  • The actor‑critic algorithm provides better control of bid‑ask spreads and faster training compared to policy iteration. performance
    Source sentence
    In contrast, the actor-critic algorithm addresses this challenge by employing a neural network to model the mean output of the bid-ask policy, allowing for better control over bid-ask spreads. This advantage is attributed to the use of suitable activation functions, which constrain the proposed spreads within a reasonable range.
  • Inflated bid‑ask spreads in policy iteration stem from the linear intensity assumption becoming unrealistic for large spreads. negative
    Source sentence
    One of the primary reasons for the policy iteration algorithm yielding inflated bid-ask spreads lies in the assumption that the arrival intensity of market orders is linearly dependent on the bid-ask spreads. This modeling assumption becomes unrealistic when confronted with excessively large bid-ask spreads.

Checked by others

No other paper in the archive has been found to support or contradict these findings. Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.