MOT: A Mixture of Actors Reinforcement Learning Method by Optimal Transport for Algorithmic Trading

ArXiv ID: 2407.01577 “View on arXiv”

Authors: Unknown

Abstract

Algorithmic trading refers to executing buy and sell orders for specific assets based on automatically identified trading opportunities. Strategies based on reinforcement learning (RL) have demonstrated remarkable capabilities in addressing algorithmic trading problems. However, the trading patterns differ among market conditions due to shifted distribution data. Ignoring multiple patterns in the data will undermine the performance of RL. In this paper, we propose MOT,which designs multiple actors with disentangled representation learning to model the different patterns of the market. Furthermore, we incorporate the Optimal Transport (OT) algorithm to allocate samples to the appropriate actor by introducing a regularization loss term. Additionally, we propose Pretrain Module to facilitate imitation learning by aligning the outputs of actors with expert strategy and better balance the exploration and exploitation of RL. Experimental results on real futures market data demonstrate that MOT exhibits excellent profit capabilities while balancing risks. Ablation studies validate the effectiveness of the components of MOT.

Keywords: Reinforcement Learning, Optimal Transport, Disentangled Representation Learning, Algorithmic Trading, Imitation Learning, Futures

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Holy Grail
• Why: The paper employs advanced mathematical concepts including optimal transport (OT) theory, Markov Decision Processes (MDP), and differential Sharpe ratio derivations, indicating high mathematical complexity. It demonstrates strong empirical rigor by using real futures market data, conducting ablation studies, and detailing specific implementations for backtesting, though it lacks the extreme reproducibility cues like GitHub links or exhaustive statistical metrics.

  flowchart TD
    A["Research Goal<br/>'How to address market condition shifts<br/>in RL for algorithmic trading?'"] --> B["Data Input<br/>Real futures market data"]
    B --> C["Core Methodology<br/>MOT: Mixture of Actors with Optimal Transport"]
    C --> D["Computational Process<br/>1. Disentangled RL Actors<br/>2. OT-based sample allocation<br/>3. Pretrain Module for imitation"]
    D --> E["Key Findings<br/>1. Superior profit & risk balance<br/>2. Validated components via ablation<br/>3. Robustness to market shifts"]