Paper: arXiv 2406.12983

Authors: Samuel Atkins, Ali Fathi, Sammy Assefa

Abstract

A corporate bond trader in a typical sell side institution such as a bank provides liquidity to the market participants by buying/selling securities and maintaining an inventory. Upon receiving a request for a buy/sell price quote (RFQ), the trader provides a quote by adding a spread over a \textit{“prevalent market price”}. For illiquid bonds, the market price is harder to observe, and traders often resort to available benchmark bond prices (such as MarketAxess, Bloomberg, etc.). In \cite{“Bergault2023ModelingLI”}, the concept of \textit{“Fair Transfer Price”} for an illiquid corporate bond was introduced which is derived from an infinite horizon stochastic optimal control problem (for maximizing the trader’s expected P&L, regularized by the quadratic variation). In this paper, we consider the same optimization objective, however, we approach the estimation of an optimal bid-ask spread quoting strategy in a data driven manner and show that it can be learned using Reinforcement Learning. Furthermore, we perform extensive outcome analysis to examine the reasonableness of the trained agent’s behavior.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper relies on advanced stochastic optimal control and Hamilton-Jacobi-Bellman equations (high math), while also detailing a data-driven RL approach with outcome analysis, though specifics on backtest metrics and implementation details are limited in the provided excerpt.

Research Flowchart

  flowchart TD
  A["Research Goal: Learn optimal bid-ask<br>spread quoting for illiquid bonds"] --> B["Methodology: Reinforcement Learning<br>vs Stochastic Optimal Control"]
  
  B --> C{"Data/Inputs"}
  C --> C1["Corporate Bond RFQs"]
  C --> C2["Trade Execution Records"]
  C --> C3["Market Reference Prices"]
  
  C --> D["Computational Process"]
  D --> D1["RL Agent Training<br>Deep Q-Network"]
  D --> D2["Compute Fair Transfer Price<br>Stochastic Control Solution"]
  D --> D3["Compare Quote Strategies"]
  
  D --> E["Key Findings/Outcomes"]
  E --> E1["RL learns spread <br>driven by inventory risk"]
  E --> E2["Optimal spread inversely <br>correlated with liquidity"]
  E --> E3["RL agents adapt to market <br>regimes without manual rules"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes to estimate the optimal bid‑ask spread quoting strategy for corporate bond RFQs in a data‑driven manner by training a reinforcement‑learning agent, focusing on a finite‑time‑horizon version of the Fair Transfer Price objective.

Source sentence
In this paper, we consider the same optimization objective, however, we approach the estimation of an optimal bid-ask spread quoting strategy in a data driven manner and show that it can be learned using Reinforcement Learning. More precisely, we take the stochastic optimal control objective at the heart of the FTP and try to estimate the optimal control policy (response to bid-ask RFQs) via Reinforcement learning

Assumes 6

  • Bid and ask RFQ arrival intensities follow a two‑dimensional Markov‑modulated Poisson process. model
    Source sentence
    For modelling the variability in liquidity, they assume that the market maker receives RFQs which are governed by a Poisson process where the intensities are also stochastic. More concretely, the bid-ask RFQ ￿ow is modeled by a two dimensional Markov-modulated Poisson process (MMPP) which we describe below.
  • The bond price dynamics are modeled as an arithmetic Brownian motion. model
    Source sentence
    In this framework, the dynamics of the bond price are given by arithmetic Brownian motion: 3(C = ^(_0 C −_1 C )3C + f3,C. (￿￿) The trader answers to bid-ask RFQs by providing the quotes (C −X1 C and (C + X0 C respectively.
  • The trader can observe the current intensity states (_1, _0) of the RFQ flow. market
    Source sentence
    Assuming that the trader is able to identify the state of the RFQ intensities (_ 91,1, _ 90,0) is at any point in time, the authors write down and solve (by approximation of the value functions) the resulting system of HJB equations for the family of value functions {\ 90,91}￿91<1,￿90<0.
  • The RFQ success probability is given by calibrated sigmoid‑type functions of the spread. model
    Source sentence
    We assume that the probability of success in RFQ response is calibrated in sigmoid-type functions 5 1(X1 C ) ￿ and 5 0(X0 C ) (see ￿gure ￿).
Show 2 more
  • Training episodes are simulated over a fixed finite horizon of several days. execution
    Source sentence
    With the above points in mind, we propose a simpli￿cation of the FTP optimization objective and consider the trader responding to the RFQs trying to optimize the objective (￿￿) within a pre-determined time horizon. We simulate episodes of the length ￿￿-days.
  • The agent’s action space (spreads) is continuous and bounded after min‑max scaling. other
    Source sentence
    Forsymmetry, wede￿netheactionspaceforeach X ascontinuousvaluesrangingfrom−￿and￿. Then, we transform these values using min-max scaling to range from −￿.￿￿to ￿.￿.

Data

  • Simulated bond price and RFQ intensity data synthetic proprietary
    Source sentence
    Data Simulation We skip the parameter estimation and replicate the exact simulation environment, based on the speci￿ed model and parameters calibrated in [BG￿￿]￿. We set the generator matrix for the MMPP as: & = ©≠≠≠ ´ −￿￿.￿￿ ￿.￿￿ ￿.￿￿ ￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ™ÆÆÆ ¨ (￿￿) The probability of bid, ask-RFQ quotes are represented by a single (-curve: 5 1(X) = 5 0(X)

Costs

Unclear No cost treatment was found in the passages the extractor read; that is not evidence that costs are not modeled.

Finds 2

  • PPO training converges stably without extensive parameter tuning, yielding a reliable RL agent. performance
    Source sentence
    We demonstrated that the PPO algorithm for training the RL agent results in a stable training convergence.
  • When the implied price movement is positive, the agent reduces the bid spread to purchase more bonds, and vice‑versa. other
    Source sentence
    Further, when the ^-implied price movement is positive, meaning the price will move upwards in the near future, the agent reduces X1 to purchase more of the bond and vice-versa. ￿￿ Figure ￿: The top plot delineates the ^-implied price movement relative to the current price.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 13 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (6)

  1. $\epsilon$-Policy Gradient for Online Pricing — → 2405.03624
  2. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems (2020) — link
  3. Proximal Policy Optimization Algorithms (2017) — link
  4. Modeling liquidity in corporate bond markets: applications to price adjustments (2023)
  5. Stable-Baselines3: Reliable Reinforcement Learning Implementations (2021)
  6. Continuous-time stochastic control and optimization with � nancial applications , volume ��

1 of 6 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.