Advancing Algorithmic Trading: A Multi-Technique Enhancement of Deep Q-Network Models

ArXiv ID: 2311.05743 “View on arXiv”

Authors: Unknown

Abstract

This study enhances a Deep Q-Network (DQN) trading model by incorporating advanced techniques like Prioritized Experience Replay, Regularized Q-Learning, Noisy Networks, Dueling, and Double DQN. Extensive tests on assets like BTC/USD and AAPL demonstrate superior performance compared to the original model, with marked increases in returns and Sharpe Ratio, indicating improved risk-adjusted rewards. Notably, convolutional neural network (CNN) architectures, both 1D and 2D, significantly boost returns, suggesting their effectiveness in market trend analysis. Across instruments, these enhancements have yielded stable and high gains, eclipsing the baseline and highlighting the potential of CNNs in trading systems. The study suggests that applying sophisticated deep learning within reinforcement learning can greatly enhance automated trading, urging further exploration into advanced methods for broader financial applicability. The findings advocate for the continued evolution of AI in finance.

Keywords: Deep Q-Network (DQN), Reinforcement learning, Convolutional neural networks (CNN), Prioritized Experience Replay, Automated trading

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 9.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced deep reinforcement learning techniques (Prioritized Experience Replay, Dueling/Double DQN, Noisy Networks, CNN architectures) with explicit mathematical formulations, earning a high complexity score. It demonstrates high empirical rigor through extensive backtesting on real financial data (BTC/USD and AAPL), reporting specific metrics like returns and Sharpe Ratios, and comparing performance against baselines.

  flowchart TD
    A["Research Goal:<br>Enhance DQN Trading Model"] --> B["Methodology:<br>Advanced Techniques"]
    B --> C["Data & Inputs:<br>Historical Market Data<br>BTC/USD & AAPL"]
    C --> D["Computational Process:<br>CNN Architectures<br>1D & 2D Integration"]
    D --> E["Key Findings & Outcomes:<br>Superior Performance<br>Higher Returns & Sharpe Ratio"]
    
    subgraph B
        B1["Prioritized Experience Replay"]
        B2["Regularized Q-Learning"]
        B3["Noisy Networks & Dueling"]
        B4["Double DQN"]
    end