Paper: arXiv 2406.14537

Authors: Chuqiao Zong, Chaojie Wang, Molei Qin, Lei Feng, Xinrun Wang, Bo An

Abstract

High-frequency trading (HFT) that executes algorithmic trading in short time scales, has recently occupied the majority of cryptocurrency market. Besides traditional quantitative trading methods, reinforcement learning (RL) has become another appealing approach for HFT due to its terrific ability of handling high-dimensional financial data and solving sophisticated sequential decision-making problems, \emph{“e.g.,”} hierarchical reinforcement learning (HRL) has shown its promising performance on second-level HFT by training a router to select only one sub-agent from the agent pool to execute the current transaction. However, existing RL methods for HFT still have some defects: 1) standard RL-based trading agents suffer from the overfitting issue, preventing them from making effective policy adjustments based on financial context; 2) due to the rapid changes in market conditions, investment decisions made by an individual agent are usually one-sided and highly biased, which might lead to significant loss in extreme markets. To tackle these problems, we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, \emph{“a.k.a.”} MacroHFT, which consists of two training phases: 1) we first train multiple types of sub-agents with the market data decomposed according to various financial indicators, specifically market trend and volatility, where each agent owns a conditional adapter to adjust its trading policy according to market conditions; 2) then we train a hyper-agent to mix the decisions from these sub-agents and output a consistently profitable meta-policy to handle rapid market fluctuations, equipped with a memory mechanism to enhance the capability of decision-making. Extensive experiments on various cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.

Complexity vs Empirical Score

  • Math Complexity: 6.0/10
  • Empirical Rigor: 6.0/10 (capped from 7.0: transaction costs only partially modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 7.0)

Why this score: The paper employs advanced reinforcement learning concepts (HRL, conditional adapters, memory mechanisms) and involves complex training phases, indicating high mathematical density. It provides extensive experiments on real cryptocurrency markets with minute-level data, includes code release, and compares against state-of-the-art baselines, demonstrating strong empirical rigor.

Research Flowchart

  flowchart TD
  Goal["Research Goal<br>Create context-aware RL for HFT<br>addressing overfitting & bias"] --> Method["MacroHFT Methodology<br>Two-Phase Training"]
  
  Data["Input: Cryptocurrency Market Data<br>Decomposed by Trends & Volatility"] --> Phase1["Phase 1: Sub-Agents Training<br>Multiple agents with<br>Conditional Adapters"]
  
  Phase1 --> Phase2["Phase 2: Hyper-Agent Training<br>Memory-Augmented Mixing<br>Meta-Policy Optimization"]
  
  Phase2 --> Findings["Key Findings<br>State-of-the-art performance on<br>minute-level HFT tasks"]
  
  style Goal fill:#e1f5e1
  style Findings fill:#fff3cd

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes MacroHFT, a novel memory-augmented context-aware reinforcement learning method for high-frequency cryptocurrency trading.

Source sentence
we propose a novel Memory Augmented Context-aware Reinforcement learning method On HFT, a.k.a. MacroHFT

Assumes 2

  • A commission fee of 0.02% is applied to all trades. execution
    Source sentence
    the commission fee rate is 0.02% for all four cryptocurrencies following the policy of Binance.
  • The cryptocurrency market can be decomposed into categories based on trend and volatility indicators. market
    Source sentence
    MacroHFT decomposes the cryptocurrency market into different categories based on trend and volatility indicators.

Data

  • four popular cryptocurrency markets (e.g., BTCUSDT, DOTUSDT, etc.) market-prices unknown
    Source sentence
    cryptocurrency markets demonstrate that MacroHFT can achieve state-of-the-art performance on minute-level trading tasks.

Costs

Unclear - sources disagree The platform classifier and the paper-text extraction give different answers, so no value is asserted.

Platform classifier: Partial

Paper text extraction: Modeled - commission fee rate is 0.02% per trade

Source sentence (the authors' words)
the commission fee rate is 0.02% for all four cryptocurrencies following the policy of Binance.

Finds 2

  • MacroHFT achieves the highest profit and highest risk-adjusted profit across all four cryptocurrency markets. performance
    Source sentence
    our method achieves the highest profit and the highest risk-adjusted profit in all 4 cryptocurrency markets for most of the evaluation metrics.
  • MacroHFT significantly surpasses multiple state-of-the-art trading methods in profit-making while maintaining competitive risk management ability on minute-level tasks. performance
    Source sentence
    MacroHFT significantly surpasses multiple state-of-the-art trading methods in profit-making while maintaining competitive risk managing ability, and achieves superior performance on minute-level trading tasks.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 17 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 11 papers in the archive (10 by independent authors): AllocFlow: End-to-End Portfolio Trading Action Generation … (used as a baseline); Mixture-of-Experts for Cryptocurrency Order Execution: … (uses its method or data)

Code

https://github.com/ZONG0004/MacroHFT — the authors' own repository

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (31)

  1. Earnhft: Efficient hierarchical reinforcement learning for high frequency trading — → 2309.12891
  2. Quantitative trading through random perturbation Q-network with nonlinear transaction costs (2022)
  3. Towards generalizable reinforcement learning for trade execution — → 2307.11685
  4. Dueling network architectures for deep reinforcement learning (2016)
  5. Scalable diffusion models with transformers (2023)
  6. A dynamic model of the limit order book (2009)
  7. Market microstructure: A survey (2000)
  8. Neural episodic control (2017)
and 23 more
  1. Episodic memory deep Q-networks (2018) — link
  2. Order imbalance, liquidity, and market returns (2002)
  3. Technical Analysis of the Futures Markets: A Comprehensive Guide to Trading Methods and Applications, New York Institute of Finance (1999)
  4. 101 formulaic alphas (2016)
  5. Enforcing Network Safety-Margins in Industrial Process Control Using MACD Indicators (2022)
  6. Various moving average convergence divergence trading strategies: A comparison (2016)
  7. Human-level control through deep reinforcement learning (2015)
  8. Proximal policy optimization algorithms (2017) — link
  9. Deep reinforcement learning for active high frequency trading (2021) — link
  10. DeepScalper: A risk-aware reinforcement learning framework to capture fleeting intraday trading opportunities (2022)
  11. Commission fee is not enough: A hierarchical reinforced framework for portfolio management (2021)
  12. MetaTrader: An reinforcement learning approach integrating diverse policies for portfolio optimization (2022)
  13. A systematic literature review of investor behavior in the cryptocurrency markets (2023)
  14. An application of deep reinforcement learning to algorithmic trading (2021)
  15. Cryptocurrency: A new investment opportunity? (2017)
  16. Cryptocurrency trading: a Comprehensive Survey (2022)
  17. Deep direct reinforcement learning for financial signal representation and trading (2016)
  18. Deep reinforcement learning for trading (2020)
  19. FinRL: A deep reinforcement learning library for automated stock trading in quantitative finance (2020) — link
  20. Quantitative trading on stock market based on deep reinforcement learning (2019)
  21. Adaptive quantitative trading: An imitative deep reinforcement learning approach (2020)
  22. Deep robust reinforcement learning for practical algorithmic trading (2019)
  23. A novel deep reinforcement learning based automated stock trading system using cascaded lstm networks — → 2212.02721

3 of 31 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.