Over-the-Counter Market Making via Reinforcement Learning

ArXiv ID: 2307.01816 “View on arXiv”

Authors: Unknown

Abstract

The over-the-counter (OTC) market is characterized by a unique feature that allows market makers to adjust bid-ask spreads based on order size. However, this flexibility introduces complexity, transforming the market-making problem into a high-dimensional stochastic control problem that presents significant challenges. To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels.

Keywords: Over-the-counter (OTC) market, Reinforcement learning, Stochastic control, Bid-ask spread optimization, OTC Securities

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 3.0/10
• Quadrant: Lab Rats
• Why: The paper uses advanced mathematical concepts like high-dimensional stochastic control, HJB equations, and Poisson processes, indicating high mathematical complexity. However, the empirical analysis relies solely on simulated data without real-world backtesting or implementation details, resulting in low empirical rigor.

  flowchart TD
    Start["Research Goal<br>Optimize Bid-Ask Spreads in OTC Market<br>via Reinforcement Learning"] --> Inputs["Data & Assumptions<br>Linear Inverse Relationship:<br>Spread vs. Arrival Intensity"]
    
    Inputs --> Method["Methodology<br>High-Dimensional Stochastic Control<br>Reinforcement Learning Algorithms"]
    
    Method --> Process["Computational Process<br>1. Model Stochastic Market Dynamics<br>2. Train RL Agents (PPO/SAC)<br>3. Simulate Order Book & Inventory"]
    
    Process --> Outcome["Key Findings & Outcomes<br>- Optimal Policy: Gaussian Distribution<br>- Return Distribution Analysis<br>- Bid-Ask Spreads vs. Time/Inventory"]
    
    Outcome --> End["Conclusion<br>RL Effectively Solves<br>Complex OTC Market-Making"]
    
    style Start fill:#e1f5fe,stroke:#01579b
    style Inputs fill:#fff3e0,stroke:#e65100
    style Method fill:#e8f5e9,stroke:#1b5e20
    style Process fill:#f3e5f5,stroke:#4a148c
    style Outcome fill:#fce4ec,stroke:#880e4f
    style End fill:#e0e0e0,stroke:#424242