Paper: arXiv 2311.05743
Authors: Gang Hu
Abstract
This study enhances a Deep Q-Network (DQN) trading model by incorporating advanced techniques like Prioritized Experience Replay, Regularized Q-Learning, Noisy Networks, Dueling, and Double DQN. Extensive tests on assets like BTC/USD and AAPL demonstrate superior performance compared to the original model, with marked increases in returns and Sharpe Ratio, indicating improved risk-adjusted rewards. Notably, convolutional neural network (CNN) architectures, both 1D and 2D, significantly boost returns, suggesting their effectiveness in market trend analysis. Across instruments, these enhancements have yielded stable and high gains, eclipsing the baseline and highlighting the potential of CNNs in trading systems. The study suggests that applying sophisticated deep learning within reinforcement learning can greatly enhance automated trading, urging further exploration into advanced methods for broader financial applicability. The findings advocate for the continued evolution of AI in finance.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 9.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced deep reinforcement learning techniques (Prioritized Experience Replay, Dueling/Double DQN, Noisy Networks, CNN architectures) with explicit mathematical formulations, earning a high complexity score. It demonstrates high empirical rigor through extensive backtesting on real financial data (BTC/USD and AAPL), reporting specific metrics like returns and Sharpe Ratios, and comparing performance against baselines.
Research Flowchart
flowchart TD
A["Research Goal:<br>Enhance DQN Trading Model"] --> B["Methodology:<br>Advanced Techniques"]
B --> C["Data & Inputs:<br>Historical Market Data<br>BTC/USD & AAPL"]
C --> D["Computational Process:<br>CNN Architectures<br>1D & 2D Integration"]
D --> E["Key Findings & Outcomes:<br>Superior Performance<br>Higher Returns & Sharpe Ratio"]
subgraph B
B1["Prioritized Experience Replay"]
B2["Regularized Q-Learning"]
B3["Noisy Networks & Dueling"]
B4["Double DQN"]
end
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes an enhanced Deep Q‑Network trading model that incorporates Prioritized Experience Replay, Regularized Q‑Learning, Noisy Networks, Dueling DQN, and Double DQN.
Source sentence
This study enhances a Deep Q-Network (DQN) trading model by incorporating advanced techniques like Prioritized Experience Replay, Regularized Q-Learning, Noisy Networks, Dueling, and Double DQN.
Assumes 2
- The market data are represented as daily candlestick charts. data
Source sentence
All datasets were formatted into daily candlestick charts.
- Benchmarking uses the same datasets as prior work [8]. data
Source sentence
For benchmarking the model’s effectiveness, the same datasets as those employed by [8] were utilized
Data
- GOOGL market-prices public
Source sentence
GOOGL 2010/01/01 2018/01/01 2020/08/24
- AAPL market-prices public
Source sentence
AAPL 2010/01/01 2018/01/01 2020/08/24
- KSS market-prices public
Source sentence
KSS 1999/01/01 2018/01/01 2020/08/24
- BTC-USD crypto-onchain public
Source sentence
BTC-USD 2014/09/17 2018/01/01 2020/08/26
Costs
Unclear
Finds 4
- The DQN‑windowed variant achieved a Total Return of 737% and Sharpe Ratio of 0.195, the highest among all models on AAPL. performance
Source sentence
Our DQN-windowed variant achieved an outstanding Total Return of 737%, surpassing all other models, accompanied by the highest Sharpe Ratio of 0.195 in this cohort.
- The improved DQN‑vanilla model yielded a Total Return of 517% and Sharpe Ratio of 0.181, surpassing the prior DQN‑vanilla implementation. performance
Source sentence
The improved DQN-vanilla model’s performance was particularly remarkable, yielding a Total Return of 517% and a Sharpe Ratio of 0.181
- On BTC/USD, CNN2D achieved returns of 3897% and Sharpe Ratio improvement to 0.085, far outstripping prior benchmarks. performance
Source sentence
the CNN1D and CNN2D models, through the introduction of convolutional layers
- The DQN‑vanilla variant on BTC/USD improved arithmetic return from 261% to 287% and Sharpe Ratio from 0.076 to 0.085. performance
Source sentence
the DQN-vanilla variant exhibited an outstanding arithmetic return of 287% compared to the original’s 261%, and a Sharpe Ratio improvement from 0.076 to 0.085
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 33 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (25)
- Quantitative Trading: How to Build Your Own Algorithmic Trading Business (2021)
- High Frequency Trading (2015)
- Deep Learning for Portfolio Optimization (2020) — link
- Learning Financial Asset-Specific Trading Rules via Deep Reinforcement Learning (2022)
- Using Genetic Algorithms to Find Technical Trading Rules (1999)
- Generating Trading Rules on the Stock Markets with Genetic Programming (2004)
- Mining Associative Classification Rules with Stock Trading Data–A GA-Based Method (2010)
- A Reinforcement Learning Based Encoder-Decoder Framework for Learning Stock Trading Rules (2021)
and 17 more
- High-dimensional stock portfolio trading with deep reinforcement learning (2022)
- Deep reinforcement learning with double q-learning (2016)
- Dueling network architectures for deep reinforcement learning (2016)
- Noisy networks for exploration (2017) — link
- Prioritized experience replay (2015) — link
- Human-level control through deep reinforcement learning (2015)
- Modeling purposeful adaptive behavior with the principle of maximum causal entropy (2010)
- first_page settings Order Article Reprints Open AccessArticle Pore-Level Multiphase Simulations of Realistic Distillation Membranes for Water Desalination (2022)
- Perspectives on predictive power of multimodal deep learning: Surprises and future directions (2018)
- The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care (2018)
- Financial trading as a game: A deep reinforcement learning approach (2018) — link
- An application of deep reinforcement learning to algorithmic trading (2021)
- Application of deep reinforcement learning in stock trading strategies and stock forecasting (2020)
- Regularization in reinforcement learning (2011)
- Practical deep reinforcement learning approach for stock trading (2018) — link
- Adaptive stock trading strategies with deep reinforcement learning methods (2020)
- Deep Q-trading. cslt. riit. tsinghua. edu. cn (2017)
0 of 25 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.