Paper: arXiv 2212.14670

Authors: Xiaodong Li, Pangjing Wu, Chenxin Zou, Qing Li

Abstract

Designing an intelligent volume-weighted average price (VWAP) strategy is a critical concern for brokers, since traditional rule-based strategies are relatively static that cannot achieve a lower transaction cost in a dynamic market. Many studies have tried to minimize the cost via reinforcement learning, but there are bottlenecks in improvement, especially for long-duration strategies such as the VWAP strategy. To address this issue, we propose a deep learning and hierarchical reinforcement learning jointed architecture termed Macro-Meta-Micro Trader (M3T) to capture market patterns and execute orders from different temporal scales. The Macro Trader first allocates a parent order into tranches based on volume profiles as the traditional VWAP strategy does, but a long short-term memory neural network is used to improve the forecasting accuracy. Then the Meta Trader selects a short-term subgoal appropriate to instant liquidity within each tranche to form a mini-tranche. The Micro Trader consequently extracts the instant market state and fulfils the subgoal with the lowest transaction cost. Our experiments over stocks listed on the Shanghai stock exchange demonstrate that our approach outperforms baselines in terms of VWAP slippage, with an average cost saving of 1.16 base points compared to the optimal baseline.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces a novel 3-tier hierarchical deep reinforcement learning architecture (M3T) specifically tailored for multi-scale temporal order execution. It validates the approach empirically on Shanghai Stock Exchange data, demonstrating a statistically significant reduction in VWAP slippage over traditional baselines.

Research Flowchart

  flowchart TD
    A[Goal: Minimize VWAP Order Execution Slippage] --> B[Data: Shanghai Stock Exchange Market Data]
    B --> C[Macro Trader: LSTM-based Volume Profile & Parent Order Allocation]
    C --> D[Meta Trader: Dynamic Subgoal Selection via HRL]
    D --> E[Micro Trader: Instant Market State Analysis & Order Execution]
    E --> F[Outcome: Reduced Cost with 1.16 bps Average Slippage Improvement]