Paper: arXiv 2506.04658
Authors: Jędrzej Maskiewicz, Paweł Sakowski
Abstract
The paper explores the use of Deep Reinforcement Learning (DRL) in stock market trading, focusing on two algorithms: Double Deep Q-Network (DDQN) and Proximal Policy Optimization (PPO) and compares them with Buy and Hold benchmark. It evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023. The results demonstrate DRL’s effectiveness in trading and its ability to manage risk by strategically avoiding trades in unfavorable conditions, providing a substantial edge over classical approaches, based on supervised learning in terms of risk-adjusted returns.
Complexity vs Empirical Score
- Math Complexity: 8.0/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs advanced mathematics and deep learning architectures (Transformers, PPO), while being grounded in practical trading evaluations with real data (2019-2023) across multiple assets, demonstrating high empirical rigor.
Research Flowchart
flowchart TD A["Research Question: Can AI Trade the Stock Market?"] --> B["Methodology: DDQN & PPO vs Buy & Hold"] B --> C["Data: Daily Price Data (2019-2023)"] C --> D["Assets: S&P 500, Forex, Bitcoin"] D --> E["Simulation & Training"] E --> F["Risk Management: Avoiding Unfavorable Conditions"] F --> G["Findings: DRL Outperforms Buy & Hold on Risk-Adjusted Returns"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes using Deep Reinforcement Learning, specifically DDQN and PPO algorithms, for stock market trading and compares them to a buy‑and‑hold benchmark.
Source sentence
The paper explores the use of Deep Reinforcement Learning (DRL) in stock market trading, focusing on two algorithms: Double Deep Q-Network (DDQN) and Proximal Policy Optimization (PPO) and compares them with Buy and Hold benchmark.
Assumes 3
- The RL model assumes that the agent's decision changes the future state of the market. model
Source sentence
the RL model assumes that its decision changes the future state 𝑠!"# that he would end up after the action a, which is inherently not true in our case.
- TD learning does not introduce a look‑ahead bias. model
Source sentence
TD learning does not introduce a look-ahead bias, as might be suggested by the leading subscripts of formula [1].
- Reinforcement learning is applied only to the training dataset, with no information from validation or test samples used. data
Source sentence
reinforcement learning is applied exclusively to the training dataset, ensuring that no information from validation or test samples is used.
Data
2019 – 2023 · USD/JPY · fx · daily
- EUR/USD, EUR/JPY, USD/JPY currency pairs market-prices unknown
Source sentence
... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.
- S&P 500 index market-prices unknown
Source sentence
... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.
- Bitcoin crypto-onchain unknown
Source sentence
... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.
Costs
Partial
Source sentence
Starting with an initial capital of $10,000, it is crucial to decide whether our strategy can achieve a net positive return, taking into account factors such as commissions, bid-ask spread and provisions.
Finds 4
Sharpe 2.18 · annual return 17.3% · total return 122.1% · max drawdown 5.3%
- The PPO_T strategy achieved a final balance of 18,120.64, an annualised return of 12.62% and a Sharpe ratio of 1.752, outperforming the perfect annual strategy and buy‑and‑hold benchmarks. performance
Source sentence
The PPO_T strategy stands out as the top performer, achieving a final balance of 18,120.64 and an impressive annualised return of 12.62%. It has a Sharpe ratio of 1.752, indicating excellent risk-adjusted returns
- The DDQN_T strategy posted a final balance of $16,105.72, an annualised return of 10%, a Sharpe ratio of 1.252 and a maximum drawdown of -8.37%, with 257 trades and an average position duration of 5.45 days. performance
Source sentence
The DDQN_T strategy also demonstrated strong performance with a final balance of $16,105.72 and an annualised return of 10%. It achieved a Sharpe ratio of 1.252 and experienced a maximum drawdown of -8.37%.
- The PPO_NN strategy achieved a final balance of $13,980.63, an annualised return of 6.93% and a Sharpe ratio of 1.122, with 273 trades and an average position duration of 3.08 days. performance
Source sentence
The PPO_NN strategy achieved a final balance of $13,980.63 with an annualised return of 6.93% and a Sharpe ratio of 1.122, suggesting good risk-adjusted returns.
- The DDQN_NN strategy yielded a final balance of 11,960.99, an annualised return of 3.64% and a Sharpe ratio of 0.554, indicating moderate performance. performance
Source sentence
The DDQN_NN strategy exhibited moderate performance with a final balance of 11,960.99 and an annualised return of 3.64% with Sharpe ratio of 0.554.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 20 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.