Reinforcement Learning Pair Trading: A Dynamic Scaling approach

ArXiv ID: 2407.16103 “View on arXiv”

Authors: Unknown

Abstract

Cryptocurrency is a cryptography-based digital asset with extremely volatile prices. Around USD 70 billion worth of cryptocurrency is traded daily on exchanges. Trading cryptocurrency is difficult due to the inherent volatility of the crypto market. This study investigates whether Reinforcement Learning (RL) can enhance decision-making in cryptocurrency algorithmic trading compared to traditional methods. In order to address this question, we combined reinforcement learning with a statistical arbitrage trading technique, pair trading, which exploits the price difference between statistically correlated assets. We constructed RL environments and trained RL agents to determine when and how to trade pairs of cryptocurrencies. We developed new reward shaping and observation/action spaces for reinforcement learning. We performed experiments with the developed reinforcement learner on pairs of BTC-GBP and BTC-EUR data separated by 1 min intervals (n=263,520). The traditional non-RL pair trading technique achieved an annualized profit of 8.33%, while the proposed RL-based pair trading technique achieved annualized profits from 9.94% to 31.53%, depending upon the RL learner. Our results show that RL can significantly outperform manual and traditional pair trading techniques when applied to volatile markets such as~cryptocurrencies.

Keywords: Reinforcement Learning (RL), Pair trading, Statistical arbitrage, Cryptocurrency trading, Reward shaping, Cryptocurrency

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail
  • Why: The paper employs advanced mathematics including MDP formulation, reward shaping, and statistical arbitrage concepts, while demonstrating empirical rigor through backtesting on large-scale cryptocurrency datasets (263,520 data points) and reporting annualized profit metrics.
  flowchart TD
    A["Research Goal:<br>RL vs Traditional Methods<br>for Crypto Pair Trading"] --> B["Data Collection:<br>BTC-GBP & BTC-EUR Data<br>1-min intervals, n=263,520"]
    B --> C["Methodology:<br>Construct RL Environment<br>Define Reward/Action Spaces"]
    C --> D["Training:<br>Train RL Agents<br>(RL Learners)"]
    D --> E["Comparison:<br>RL vs Traditional<br>Pair Trading"]
    E --> F["Outcome:<br>RL Outperformance:<br>9.94% - 31.53% vs 8.33%"]
    style F fill:#d4edda,stroke:#155724