{"paper_id":"2307.10649","title":"An Adaptive Dual-level Reinforcement Learning Approach for Optimal Trade Execution","authors":["Soohan Kim","Jimyeong Kim","Hong Kee Sul","Youngjoon Hong"],"paper_date":"2023-07-20","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2307.10649v1","page_url":"https://thequant.space/flowcharts/an-adaptive-dual-level-reinforcement-learning-approach-for-o/","scores":{"empirical_rigor":6,"math_complexity":8.5,"quadrant":"Holy Grail"},"proposes":{"status":"available","statement":"The paper proposes a dual‑level reinforcement‑learning strategy that uses a Transformer to allocate daily volume based on the U‑shaped intraday pattern and an LSTM with PPO to execute orders within each interval.","quote":"We propose a novel approach for optimal trade execution that utilizes a dual‑level strategy. Orders are allocated through two stages: in the first stage we utilize the U‑shaped pattern of intraday volumes and in the second stage we implement deep reinforcement learning.","summary":"The purpose of this research is to devise a tactic that can closely track the daily cumulative volume-weighted average price (VWAP) using reinforcement learning. Previous studies often choose a relatively short trading horizon to implement their models, making it difficult to accurately track the…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"The actions taken by the model only affect a temporary market and the market recovers to equilibrium at the next time step.","type":"market","stated":"explicit","quote":"We assume that the actions taken by our model only affect a temporary market, and that the market will recover to the equilibrium level at the next time step."},{"assumption":"Commissions and exchange fees are ignored.","type":"execution","stated":"explicit","quote":"Commissions and exchange fees are ignored."},{"assumption":"Orders are traded immediately without order arrival delays.","type":"execution","stated":"explicit","quote":"Our model’s orders are traded immediately without order arrival delays."}]},"datasets":{"status":"available","items":[{"name":"KRX millisecond trade and limit order book (LOB) tick data","provider":"Korea Exchange (KRX)","kind":"order-book","access":"proprietary","period":"2021-01-01 to 2021-12-31","universe":"Samsung Electronics, SK Hynix, Kia Corporation, POSCO Holdings","quote":"Our millisecond trade and limit order book (LOB) tick data is from the Korea Exchange (KRX). We use the daily trade and LOB data of Samsung Electronics (SE), SK Hynix (SH), Kia Corporation (KC) and POSCO Holdings Inc (PH) from January 1st, 2021 to December 31st, 2021."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"unclear"},"transaction_costs":{"status":"available","modeled":"no","headline_net_of_costs":null,"detail":null,"quote":"Commissions and exchange fees are ignored.","source":"front_matter","rule":"R3","candidates":{"front_matter":"no","evidence_card":"no","evidence_card_quoted":true},"card_detail":null,"card_quote":"Commissions and exchange fees are ignored.","meaning":"a trading or execution result is reported without costs"},"results":{"status":"available","evaluation_type":"execution-cost","findings":[{"finding":"The dual‑level architecture improves the accuracy of approximating the cumulative VWAP compared with previous reinforcement‑learning‑based models.","kind":"performance","quote":"The results from our experiments suggest that this dual‑level architecture improves the accuracy of approximating the cumulative VWAP, when compared to previous reinforcement learning‑based models."},{"finding":"The U‑shape Transformer outperforms the statistical U‑shape method in tracking daily cumulative VWAP.","kind":"performance","quote":"Our experiments demonstrate that the U‑shape Transformer performs even better."}],"reported_performance":null,"out_of_sample":{"method":"holdout","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":9,"citations":{"cited_by_in_corpus":4,"by_independent_authors":4,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":2,"unclassified":2},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T16:29:15Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}