Paper: arXiv 2506.04658

Authors: Jędrzej Maskiewicz, Paweł Sakowski

Abstract

The paper explores the use of Deep Reinforcement Learning (DRL) in stock market trading, focusing on two algorithms: Double Deep Q-Network (DDQN) and Proximal Policy Optimization (PPO) and compares them with Buy and Hold benchmark. It evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023. The results demonstrate DRL’s effectiveness in trading and its ability to manage risk by strategically avoiding trades in unfavorable conditions, providing a substantial edge over classical approaches, based on supervised learning in terms of risk-adjusted returns.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced mathematics and deep learning architectures (Transformers, PPO), while being grounded in practical trading evaluations with real data (2019-2023) across multiple assets, demonstrating high empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Question: Can AI Trade the Stock Market?"] --> B["Methodology: DDQN & PPO vs Buy & Hold"]
  B --> C["Data: Daily Price Data (2019-2023)"]
  C --> D["Assets: S&P 500, Forex, Bitcoin"]
  D --> E["Simulation & Training"]
  E --> F["Risk Management: Avoiding Unfavorable Conditions"]
  F --> G["Findings: DRL Outperforms Buy & Hold on Risk-Adjusted Returns"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes using Deep Reinforcement Learning, specifically DDQN and PPO algorithms, for stock market trading and compares them to a buy‑and‑hold benchmark.

Source sentence
The paper explores the use of Deep Reinforcement Learning (DRL) in stock market trading, focusing on two algorithms: Double Deep Q-Network (DDQN) and Proximal Policy Optimization (PPO) and compares them with Buy and Hold benchmark.

Assumes 3

  • The RL model assumes that the agent's decision changes the future state of the market. model
    Source sentence
    the RL model assumes that its decision changes the future state 𝑠!"# that he would end up after the action a, which is inherently not true in our case.
  • TD learning does not introduce a look‑ahead bias. model
    Source sentence
    TD learning does not introduce a look-ahead bias, as might be suggested by the leading subscripts of formula [1].
  • Reinforcement learning is applied only to the training dataset, with no information from validation or test samples used. data
    Source sentence
    reinforcement learning is applied exclusively to the training dataset, ensuring that no information from validation or test samples is used.

Data

Backtest sample: 2019 – 2023 · USD/JPY · fx · daily

  • EUR/USD, EUR/JPY, USD/JPY currency pairs market-prices unknown
    2019-2023 · three currency pairs
    Source sentence
    ... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.
  • S&P 500 index market-prices unknown
    2019-2023
    Source sentence
    ... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.
  • Bitcoin crypto-onchain unknown
    2019-2023
    Source sentence
    ... evaluates these algorithms across three currency pairs, the S&P 500 index and Bitcoin, on the daily data in the period of 2019-2023.

Costs

Partial Costs appear only as a robustness check, a flat fee without spread or impact, or outside the headline numbers.

Source sentence
Starting with an initial capital of $10,000, it is crucial to decide whether our strategy can achieve a net positive return, taking into account factors such as commissions, bid-ask spread and provisions.

Finds 4

Reported performance (PPO with Transformer Network): Sharpe 2.18 · annual return 17.3% · total return 122.1% · max drawdown 5.3%

  • The PPO_T strategy achieved a final balance of 18,120.64, an annualised return of 12.62% and a Sharpe ratio of 1.752, outperforming the perfect annual strategy and buy‑and‑hold benchmarks. performance
    Source sentence
    The PPO_T strategy stands out as the top performer, achieving a final balance of 18,120.64 and an impressive annualised return of 12.62%. It has a Sharpe ratio of 1.752, indicating excellent risk-adjusted returns
  • The DDQN_T strategy posted a final balance of $16,105.72, an annualised return of 10%, a Sharpe ratio of 1.252 and a maximum drawdown of -8.37%, with 257 trades and an average position duration of 5.45 days. performance
    Source sentence
    The DDQN_T strategy also demonstrated strong performance with a final balance of $16,105.72 and an annualised return of 10%. It achieved a Sharpe ratio of 1.252 and experienced a maximum drawdown of -8.37%.
  • The PPO_NN strategy achieved a final balance of $13,980.63, an annualised return of 6.93% and a Sharpe ratio of 1.122, with 273 trades and an average position duration of 3.08 days. performance
    Source sentence
    The PPO_NN strategy achieved a final balance of $13,980.63 with an annualised return of 6.93% and a Sharpe ratio of 1.122, suggesting good risk-adjusted returns.
  • The DDQN_NN strategy yielded a final balance of 11,960.99, an annualised return of 3.64% and a Sharpe ratio of 0.554, indicating moderate performance. performance
    Source sentence
    The DDQN_NN strategy exhibited moderate performance with a final balance of 11,960.99 and an annualised return of 3.64% with Sharpe ratio of 0.554.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 20 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.