{"paper_id":"2307.01816","title":"Over-the-Counter Market Making via Reinforcement Learning","authors":["Zhou Fang","Haiqing Xu"],"paper_date":"2023-07-04","paper_type":"perspective-policy","paper_url":"https://arxiv.org/pdf/2307.01816v1","page_url":"https://thequant.space/flowcharts/over-the-counter-market-making-via-reinforcement-learning/","scores":{"empirical_rigor":3,"math_complexity":8.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper proposes a reinforcement learning framework with a stochastic policy for OTC market making, showing that the optimal bid‑ask spread policy follows a Gaussian distribution.","quote":"To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. In this research paper, we present a novel framework based on reinforcement learning to address the in-tricate challenges of the multi-dimensional market-making problem encountered by over-the-counter (OTC) market makers.","summary":"The over-the-counter (OTC) market is characterized by a unique feature that allows market makers to adjust bid-ask spreads based on order size. However, this flexibility introduces complexity, transforming the market-making problem into a high-dimensional stochastic control problem that presents…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Market order arrival intensity is linearly inverse to bid‑ask spreads.","type":"model","stated":"explicit","quote":"By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. 2 As a result, the dynamics of the inventory can be described as follows: dqt = K X k=1 zk \u0000dN + t (k) −dN − t (k) \u0001 (2) in this paper, we assume a simple setting, which is the market orders arrival intensity is linearly inverse","support":"full"},{"assumption":"The volatility of the underlying asset is constant over the trading period.","type":"model","stated":"explicit","quote":"To streamline the problem and facilitate analysis, we adopt the assumption that the volatility of the underlying asset remains constant throughout the entire trading period.","support":"full"},{"assumption":"The market maker does not externalize its inventory.","type":"model","stated":"explicit","quote":"In this paper, we simplify the setting by assuming that the market maker does not externalize its inventory.","support":"full"},{"assumption":"Buy and sell order executions follow Poisson processes.","type":"execution","stated":"explicit","quote":"Specifically, we denote the number of buy order executions as N + t (k) and the number of sell order executions as N − t (k). The intensities of these Poisson processes are denoted as λ+(k) and λ−(k), respectively.","support":"full"},{"assumption":"The optimal bid‑ask spread policy can be modeled as a Gaussian distribution.","type":"statistical","stated":"explicit","quote":"Under this assumption, we demonstrate that the optimal policy for bid-ask spread determination can be modeled as a Gaussian distribution.","support":"full"},{"assumption":"A stochastic policy is used to determine bid‑ask spreads.","type":"model","stated":"explicit","quote":"Our approach centers on the utilization of a stochastic policy, which enables the market maker to determine bid-ask spreads.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"Simulated data","provider":null,"kind":"synthetic","access":"unknown","period":null,"universe":null,"quote":"To validate the effectiveness of our proposed framework, we conduct extensive numerical experiments using simulated data, showcasing various performance metrics and bid-ask spread outcomes for specific inventory levels.","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"not_extracted","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"none","rule":"R0","candidates":{"front_matter":"unclear","evidence_card":"unclear","evidence_card_quoted":false},"meaning":"the classifier was not confident"},"results":{"status":"available","evaluation_type":"none","findings":[{"finding":"Under the linear inverse intensity assumption, the optimal bid‑ask spread policy follows a Gaussian distribution.","kind":"theoretical","quote":"By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. Under this assumption, we demonstrate that the optimal policy for bid-ask spread determination can be modeled as a Gaussian distribution.","support":"full"},{"finding":"Numerical experiments reveal the return distribution and bid‑ask spreads under different time and inventory levels.","kind":"performance","quote":"We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels.","support":"full"},{"finding":"The policy iteration algorithm effectively refines the policy and improves return performance across iterations.","kind":"performance","quote":"This suggests that the 12 Figure 7: return distribution of after third policy iteration Figure 8: return distribution of after fourth policy iteration Figure 9: return distribution of after fifth policy iteration policy iteration algorithm is effective in refining the policy and enhancing the performance in terms of returns.","support":"full"},{"finding":"The actor‑critic algorithm provides better control of bid‑ask spreads and faster training compared to policy iteration.","kind":"performance","quote":"In contrast, the actor-critic algorithm addresses this challenge by employing a neural network to model the mean output of the bid-ask policy, allowing for better control over bid-ask spreads. This advantage is attributed to the use of suitable activation functions, which constrain the proposed spreads within a reasonable range.","support":"full"},{"finding":"Inflated bid‑ask spreads in policy iteration stem from the linear intensity assumption becoming unrealistic for large spreads.","kind":"negative","quote":"One of the primary reasons for the policy iteration algorithm yielding inflated bid-ask spreads lies in the assumption that the arrival intensity of market orders is linearly dependent on the bid-ask spreads. This modeling assumption becomes unrealistic when confronted with excessively large bid-ask spreads.","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"not_extracted","supports":[],"contradicts":[],"qualifies":[],"compared_with":0,"citations":{"cited_by_in_corpus":1,"by_independent_authors":1,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":1},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T20:02:12Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}