{"paper_id":"2306.17178","title":"Optimal Execution Using Reinforcement Learning","authors":["Cong Zheng","Jiafa He","Can Yang"],"paper_date":"2023-06-19","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2306.17178v1","page_url":"https://thequant.space/flowcharts/optimal-execution-using-reinforcement-learning/","scores":{"empirical_rigor":4,"math_complexity":7.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper proposes a reinforcement‑learning based optimal execution framework that incorporates cross‑exchange signals to predict cryptocurrency price changes.","quote":"we attempted for the first time to use cross-exchange signals to predict asset price changes.","summary":"This work is about optimal order execution, where a large order is split into several small orders to maximize the implementation shortfall. Based on the diversity of cryptocurrency exchanges, we attempt to extract cross-exchange signals by aligning data from multiple exchanges for the first time.…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The agent’s own orders have no impact on market price in the baseline model.","type":"execution","stated":"explicit","quote":"So far, we have always assumed that our own orders have no impact on the market price."},{"assumption":"A flat execution fee of 3 basis points is incurred per trade.","type":"market","stated":"explicit","quote":"It can be seen that considering the execution fee of 3 bps, the IS of the TWAP execution algorithm is about -3.1 bps"},{"assumption":"Cross‑exchange signals may be delayed due to network latency between exchange servers.","type":"data","stated":"implied","quote":"due to the different locations of servers of different exchanges, there may be network delay issues in the process of obtaining cross-exchange signals"}]},"datasets":{"status":"available","items":[{"name":"Binance perpetual data","provider":"Binance","kind":"crypto-onchain","access":"public","period":null,"universe":null,"quote":"Our work focuses on analyzing perpetual data from several top-ranked cryptocurrency exchanges, including Binance, OKEx, and Bybit.","known_id":"binance"},{"name":"OKEx perpetual data","provider":"OKEx","kind":"crypto-onchain","access":"public","period":null,"universe":null,"quote":"Our work focuses on analyzing perpetual data from several top-ranked cryptocurrency exchanges, including Binance, OKEx, and Bybit.","known_id":"okx"},{"name":"Bybit perpetual data","provider":"Bybit","kind":"crypto-onchain","access":"public","period":null,"universe":null,"quote":"Our work focuses on analyzing perpetual data from several top-ranked cryptocurrency exchanges, including Binance, OKEx, and Bybit.","known_id":"bybit"}],"known":[{"id":"bybit","name":"Bybit","provider":"Bybit","kind":"market-prices","access":"public","quote":"Taking Bybit as an example, we use regression models for both a single exchange and cross-exchange signals to predict short-term price movements for BTC on Bybit.","mentions":32,"used_in":4},{"id":"okx","name":"OKX","provider":"OKX","kind":"market-prices","access":"public","quote":"2 High-Frequency Data And Preprocess Our work focuses on analyzing perpetual data from several top-ranked cryptocurrency exchanges, including Binance, OKEx, and Bybit.","mentions":19,"used_in":2},{"id":"binance","name":"Binance","provider":"Binance","kind":"market-prices","access":"public","quote":"2 High-Frequency Data And Preprocess Our work focuses on analyzing perpetual data from several top-ranked cryptocurrency exchanges, including Binance, OKEx, and Bybit.","mentions":12,"used_in":2}],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"available","modeled":"unclear","headline_net_of_costs":null,"detail":null,"quote":null,"source":"conflict","rule":"R4b","candidates":{"front_matter":"yes","evidence_card":"partial","evidence_card_quoted":true},"conflict_kind":"graded","card_detail":"A flat execution fee of 3 bps is applied; price impact is later modeled with a parameter β=1e-5.","card_quote":"It can be seen that considering the execution fee of 3 bps, the IS of the TWAP execution algorithm is about -3.1 bps","meaning":"the platform classifier and the paper-text extraction disagree, so no value is asserted"},"results":{"status":"available","evaluation_type":"execution-cost","findings":[{"finding":"Cross‑exchange order‑flow signals consistently outperform single‑exchange signals in predictive power across all time horizons.","kind":"predictive-accuracy","quote":"In Figure A and Figure B, we observe that the cross-exchange signal consistently outperforms the single-exchange signal in terms of predictive power, regardless of the prediction time scale."},{"finding":"The RL agent using cross‑exchange signals (PPOcross) achieves lower implementation shortfall than TWAP and single‑exchange RL (PPOsingle) on both Bybit and OKEx.","kind":"performance","quote":"Compared to the TWAP execution algorithm, the agent further gains 0.68 bps of improvement. This result further proves that cross-exchange signals can provide additional improvement compared to using only single-exchange signals."},{"finding":"Using cross‑exchange signals increases variance of execution outcomes, likely due to network‑delay induced signal inaccuracies.","kind":"statistical","quote":"We can understand this as follows: due to the different locations of servers of different exchanges, there may be network delay issues"}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":13,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:25:30Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}