Reinforcement Learning with Hawkes-Structured Networks for Market Making

Paper: arXiv 2610.16524

Authors: Fernando Acero, Philipp Plank, Yufei Zhang

Abstract

Market making under realistic order-flow dynamics is challenging due to the clustered, self-exciting nature of order arrivals, which are naturally modeled by non-Markovian Hawkes processes. We propose a model-free reinforcement learning (RL) framework for market making with general Hawkes order-flow dynamics. We first establish non-asymptotic performance bounds that quantify the loss incurred by approximating the infinite-dimensional Hawkes state using a finite collection of exponentially weighted features. Guided by this analysis, we introduce the Hawkes-Structured Network (HSN), a novel neural architecture that encodes historical order flow through a learnable mixture-of-exponentials filter. HSN is trained end-to-end using model-free RL, jointly optimizing the feature representation and quoting policy directly from reward feedback. Experiments on single- and multi-asset market-making problems with mixture-of-exponentials, power-law, and regime-switching kernels demonstrate that HSN outperforms general-purpose neural architectures, achieving higher trading rewards while implicitly recovering the underlying order-flow intensities.

Complexity vs Empirical Score

• Math Complexity: 8.0/10
• Empirical Rigor: 7.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a novel neural architecture (HSN) for market making under complex Hawkes processes, backed by theoretical performance bounds. It combines advanced mathematical modeling of order flow with a robust, model-free RL framework. The empirical results demonstrate strong performance against baselines across various kernel types.

Research Flowchart

  flowchart TD
    A[Research Goal: Market Making under Realistic Order-Flow] --> B{Key Methodology: Model-Free RL with Hawkes Processes};
    B --> C[Data/Inputs: Historical Order Flow Dynamics];
    C --> D{Computational Process: Hawkes-Structured Network (HSN) Training};
    D --> E[Key Findings: HSN Outperforms Baselines, Recovers Intensities];