{"paper_id":"2406.12983","title":"Reinforcement Learning for Corporate Bond Trading: A Sell Side Perspective","authors":["Samuel Atkins","Ali Fathi","Sammy Assefa"],"paper_date":"2024-06-18","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2406.12983v1","page_url":"https://thequant.space/flowcharts/reinforcement-learning-for-corporate-bond-trading-a-sell-si/","scores":{"empirical_rigor":6,"math_complexity":7.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes to estimate the optimal bid‑ask spread quoting strategy for corporate bond RFQs in a data‑driven manner by training a reinforcement‑learning agent, focusing on a finite‑time‑horizon version of the Fair Transfer Price objective.","quote":"In this paper, we consider the same optimization objective, however, we approach the estimation of an optimal bid-ask spread quoting strategy in a data driven manner and show that it can be learned using Reinforcement Learning. More precisely, we take the stochastic optimal control objective at the heart of the FTP and try to estimate the optimal control policy (response to bid-ask RFQs) via Reinforcement learning","summary":"A corporate bond trader in a typical sell side institution such as a bank provides liquidity to the market participants by buying/selling securities and maintaining an inventory. Upon receiving a request for a buy/sell price quote (RFQ), the trader provides a quote by adding a spread over a…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Bid and ask RFQ arrival intensities follow a two‑dimensional Markov‑modulated Poisson process.","type":"model","stated":"explicit","quote":"For modelling the variability in liquidity, they assume that the market maker receives RFQs which are governed by a Poisson process where the intensities are also stochastic. More concretely, the bid-ask RFQ ￿ow is modeled by a two dimensional Markov-modulated Poisson process (MMPP) which we describe below.","support":"full"},{"assumption":"The bond price dynamics are modeled as an arithmetic Brownian motion.","type":"model","stated":"explicit","quote":"In this framework, the dynamics of the bond price are given by arithmetic Brownian motion: 3(C = ^(_0 C −_1 C )3C + f3,C. (￿￿) The trader answers to bid-ask RFQs by providing the quotes (C −X1 C and (C + X0 C respectively.","support":"full"},{"assumption":"The trader can observe the current intensity states (_1, _0) of the RFQ flow.","type":"market","stated":"explicit","quote":"Assuming that the trader is able to identify the state of the RFQ intensities (_ 91,1, _ 90,0) is at any point in time, the authors write down and solve (by approximation of the value functions) the resulting system of HJB equations for the family of value functions {\\ 90,91}￿91<1,￿90<0.","support":"full"},{"assumption":"The RFQ success probability is given by calibrated sigmoid‑type functions of the spread.","type":"model","stated":"explicit","quote":"We assume that the probability of success in RFQ response is calibrated in sigmoid-type functions 5 1(X1 C ) ￿ and 5 0(X0 C ) (see ￿gure ￿).","support":"full"},{"assumption":"Training episodes are simulated over a fixed finite horizon of several days.","type":"execution","stated":"explicit","quote":"With the above points in mind, we propose a simpli￿cation of the FTP optimization objective and consider the trader responding to the RFQs trying to optimize the objective (￿￿) within a pre-determined time horizon. We simulate episodes of the length ￿￿-days.","support":"full"},{"assumption":"The agent’s action space (spreads) is continuous and bounded after min‑max scaling.","type":"other","stated":"explicit","quote":"Forsymmetry, wede￿netheactionspaceforeach X ascontinuousvaluesrangingfrom−￿and￿. Then, we transform these values using min-max scaling to range from −￿.￿￿to ￿.￿.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Simulated bond price and RFQ intensity data","provider":null,"kind":"synthetic","access":"proprietary","period":null,"universe":null,"quote":"Data Simulation We skip the parameter estimation and replicate the exact simulation environment, based on the speci￿ed model and parameters calibrated in [BG￿￿]￿. We set the generator matrix for the MMPP as: & = ©≠≠≠ ´ −￿￿.￿￿ ￿.￿￿ ￿.￿￿ ￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ ￿￿.￿￿ −￿￿.￿￿ ™ÆÆÆ ¨ (￿￿) The probability of bid, ask-RFQ quotes are represented by a single (-curve: 5 1(X) = 5 0(X)","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"none_found","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"none","rule":"R0b","candidates":{"front_matter":"unclear","evidence_card":"no","evidence_card_quoted":false},"meaning":"no cost treatment was found in the passages the extractor read; that is not evidence that costs are not modeled"},"results":{"status":"available","evaluation_type":"none","findings":[{"finding":"PPO training converges stably without extensive parameter tuning, yielding a reliable RL agent.","kind":"performance","quote":"We demonstrated that the PPO algorithm for training the RL agent results in a stable training convergence.","support":"full"},{"finding":"When the implied price movement is positive, the agent reduces the bid spread to purchase more bonds, and vice‑versa.","kind":"other","quote":"Further, when the ^-implied price movement is positive, meaning the price will move upwards in the near future, the agent reduces X1 to purchase more of the bond and vice-versa. ￿￿ Figure ￿: The top plot delineates the ^-implied price movement relative to the current price.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":13,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":1,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:56:45Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}