{"paper_id":"2305.15821","title":"Market Making with Deep Reinforcement Learning from Limit Order Books","authors":["Hong Guo","Jianwu Lin","Fanlin Huang"],"paper_date":"2023-05-25","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2305.15821v1","page_url":"https://thequant.space/flowcharts/market-making-with-deep-reinforcement-learning-from-limit-or/","scores":{"empirical_rigor":6,"math_complexity":8,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a deep reinforcement learning agent for market making that uses a CNN‑Attention network (Attn‑LOB) for automatic feature extraction from limit order book data, a novel continuous action space, and a hybrid reward function.","quote":"We propose an effective framework to train an RL agent with automatic feature extraction and pre-training from LOB data for market making problems, which is a novel challenge in the HFT domain.","summary":"Market making (MM) is an important research topic in quantitative finance, the agent needs to continuously optimize ask and bid quotes to provide liquidity and make profits. The limit order book (LOB) contains information on all active limit orders, which is an essential basis for decision-making.…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The environment is model‑free and does not require explicit modeling of price or order arrival processes.","type":"model","stated":"explicit","quote":"In contrast to previous MM models [9], [11] which rely on strong assumptions, we use a model‑free approach which need not to explicitly model the environment."},{"assumption":"Transaction costs are zero in the simulated trading environment.","type":"execution","stated":"explicit","quote":"The transaction cost is set to 0, and the agent can hold negative cash or inventory."},{"assumption":"The agent cannot quote if its absolute inventory exceeds a maximum inventory limit defined as ω times the minimum trade unit.","type":"execution","stated":"explicit","quote":"If the absolute inventory exceeds max inventory = ω ∗minimum trade unit, the agent will be prohibited from quoting in that direction."},{"assumption":"The simulator only executes the agent’s order when the real historical order arrives, ensuring realistic transaction timing.","type":"execution","stated":"explicit","quote":"To ensure the reality of simulated transactions, the simulator executes the agent’s order only when the real historical order arrives."}]},"datasets":{"status":"available","items":[{"name":"Shenzhen Stock Exchange historical orders and trades (Nov 2019)","provider":"Shenzhen Stock Exchange","kind":"order-book","access":"commercial","period":"November 2019 (21 trading days)","universe":"Ping An Bank Co.,Ltd. (SZ.000001), Wuliangye Yibin Co.,Ltd. (SZ.000858), Hikvision Technol","quote":"... We select three stocks in different industries to study the applicability of proposed agent, which are Ping An Bank Co.,Ltd. (SZ.000001) in banking, Wuliangye Yibin Co.,Ltd. (SZ.000858) in liquor‑making, and Hikvision Technology Co.,Ltd. (SZ.002415) in technology."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"l2-book"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":"Transaction cost is set to 0 in the simulator.","quote":"The transaction cost is set to 0, and the agent can hold negative cash or inventory.","source":"front_matter","rule":"R3","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":true},"card_detail":"Transaction cost is set to 0 in the simulator.","card_quote":"The transaction cost is set to 0, and the agent can hold negative cash or inventory.","meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"trading-pnl","findings":[{"finding":"In pre‑training, Attn‑LOB outperforms other models on recall and F1 score while using a shorter input sequence.","kind":"predictive-accuracy","quote":"Attn‑LOB outperforms other methods on recall and F1 score, and we only use half the time length as input compared with DeepLOB."},{"finding":"The agent’s attention visualizations show it focuses on the nearest events while still considering early market changes, indicating learned market‑making skills similar to humans.","kind":"other","quote":"We explain our results by visualizing attention weights. The agent paid the most attention to the nearest events while still looking at very early changes. And we find that the agent can learn some market‑making skills like humans."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":11,"citations":{"cited_by_in_corpus":3,"by_independent_authors":3,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":3},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":true,"code_url":"https://github.com/imTurkey/Market-Making-with-Deep-Reinforcement-Learning-from-Limit-Order-Books","note":null},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:26:31Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}