Paper: arXiv 2307.10649
Authors: Soohan Kim, Jimyeong Kim, Hong Kee Sul, Youngjoon Hong
Abstract
The purpose of this research is to devise a tactic that can closely track the daily cumulative volume-weighted average price (VWAP) using reinforcement learning. Previous studies often choose a relatively short trading horizon to implement their models, making it difficult to accurately track the daily cumulative VWAP since the variations of financial data are often insignificant within the short trading horizon. In this paper, we aim to develop a strategy that can accurately track the daily cumulative VWAP while minimizing the deviation from the VWAP. We propose a method that leverages the U-shaped pattern of intraday stock trade volumes and use Proximal Policy Optimization (PPO) as the learning algorithm. Our method follows a dual-level approach: a Transformer model that captures the overall(global) distribution of daily volumes in a U-shape, and a LSTM model that handles the distribution of orders within smaller(local) time intervals. The results from our experiments suggest that this dual-level architecture improves the accuracy of approximating the cumulative VWAP, when compared to previous reinforcement learning-based models.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.0)
Why this score: The paper employs advanced mathematics including Markov Decision Processes, Transformer models, and LSTM networks, earning a high math score. Its empirical rigor is strong due to a clearly defined experimental setup with comparisons to previous RL models and use of real financial data, though it lacks explicit backtesting metrics like code or specific datasets.
Research Flowchart
flowchart TD A["Research Goal<br>Devise RL tactic to track daily VWAP<br>minimizing deviation over long horizon"] --> B["Methodology: Dual-Level Reinforcement Learning"] B --> C1["Global Level: Transformer<br>Models U-shaped daily volume distribution"] B --> C2["Local Level: LSTM<br>Models order distribution in small time intervals"] C1 & C2 --> D["Core Algorithm: Proximal Policy Optimization PPO"] D --> E["Inputs: Historical Equities Data"] E --> F["Simulation Environment<br>Calculates PnL & VWAP Deviation"] F -- Feedback --> D D --> G["Outcome: Adaptive Trade Execution Strategy"] G --> H["Key Finding: Dual-level architecture<br>outperforms baseline RL models<br>in tracking daily cumulative VWAP"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a dual‑level reinforcement‑learning strategy that uses a Transformer to allocate daily volume based on the U‑shaped intraday pattern and an LSTM with PPO to execute orders within each interval.
Source sentence
We propose a novel approach for optimal trade execution that utilizes a dual‑level strategy. Orders are allocated through two stages: in the first stage we utilize the U‑shaped pattern of intraday volumes and in the second stage we implement deep reinforcement learning.
Assumes 3
- The actions taken by the model only affect a temporary market and the market recovers to equilibrium at the next time step. market
Source sentence
We assume that the actions taken by our model only affect a temporary market, and that the market will recover to the equilibrium level at the next time step.
- Commissions and exchange fees are ignored. execution
Source sentence
Commissions and exchange fees are ignored.
- Orders are traded immediately without order arrival delays. execution
Source sentence
Our model’s orders are traded immediately without order arrival delays.
Data
- KRX millisecond trade and limit order book (LOB) tick data order-book proprietary
Source sentence
Our millisecond trade and limit order book (LOB) tick data is from the Korea Exchange (KRX). We use the daily trade and LOB data of Samsung Electronics (SE), SK Hynix (SH), Kia Corporation (KC) and POSCO Holdings Inc (PH) from January 1st, 2021 to December 31st, 2021.
Costs
Not modeled
Source sentence
Commissions and exchange fees are ignored.
Finds 2
- The dual‑level architecture improves the accuracy of approximating the cumulative VWAP compared with previous reinforcement‑learning‑based models. performance
Source sentence
The results from our experiments suggest that this dual‑level architecture improves the accuracy of approximating the cumulative VWAP, when compared to previous reinforcement learning‑based models.
- The U‑shape Transformer outperforms the statistical U‑shape method in tracking daily cumulative VWAP. performance
Source sentence
Our experiments demonstrate that the U‑shape Transformer performs even better.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 9 related papers). Coverage is partial, so this is neither replication nor refutation.
4 papers in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.