Paper: arXiv 2407.09557

Authors: Alireza Mohammadshafie, Akram Mirzaeinia, Haseebullah Jumakhan, Amir Mirzaeinia

Abstract

Recent deep reinforcement learning (DRL) methods in finance show promising outcomes. However, there is limited research examining the behavior of these DRL algorithms. This paper aims to investigate their tendencies towards holding or trading financial assets as well as purchase diversity. By analyzing their trading behaviors, we provide insights into the decision-making processes of DRL models in finance applications. Our findings reveal that each DRL algorithm exhibits unique trading patterns and strategies, with A2C emerging as the top performer in terms of cumulative rewards. While PPO and SAC engage in significant trades with a limited number of stocks, DDPG and TD3 adopt a more balanced approach. Furthermore, SAC and PPO tend to hold positions for shorter durations, whereas DDPG, A2C, and TD3 display a propensity to remain stationary for extended periods.

Complexity vs Empirical Score

  • Math Complexity: 5.0/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs deep reinforcement learning algorithms like A2C, PPO, SAC, DDPG, and TD3, which involve stochastic optimization and neural network architectures, adding mathematical complexity, while its empirical rigor is high due to the use of a specific dataset (Dow Jones 30 companies from Yahoo Finance), defined train/test splits, and analysis of trading behaviors.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Investigate DRL Trading Behavior<br>& Purchase Diversity"] --> B{"Methodology"}
  B --> C["Data: Financial Asset Time Series"]
  B --> D["Compute: Train DRL Models<br>DDPG, TD3, A2C, PPO, SAC"]
  C --> D
  D --> E{"Analyze Trading Behavior"}
  E --> F["Findings & Outcomes"]
  F --> G["Performance<br>A2C: Highest Cumulative Reward"]
  F --> H["Trade Volume<br>PPO/SAC: High Trades, Few Stocks<br>DDPG/TD3: Balanced Approach"]
  F --> I["Holding Time<br>SAC/PPO: Short Duration<br>DDPG/A2C/TD3: Long Duration"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper investigates the tendencies of deep reinforcement learning algorithms to hold or trade assets and their purchase diversity in finance.

Source sentence
This paper aims to investigate their tendencies towards holding or trading financial assets as well as purchase diversity.

Assumes 1

  • The analysis uses hourly price data from Yahoo Finance without accounting for market impact or slippage. data
    Source sentence
    The financial data used in this analysis was obtained from Yahoo Finance. The data covers hourly intervals

Data

  • Yahoo Finance · Yahoo market-prices public
    Source sentence
    The financial data used in this analysis was obtained from Yahoo Finance. The data covers hourly intervals

Recognised sources: Yahoo Finance

Costs

Unclear No cost treatment was found in the passages the extractor read; that is not evidence that costs are not modeled.

Finds 5

  • A2C emerges as the top performer in terms of cumulative rewards among the DRL algorithms studied. performance
    Source sentence
    Our findings reveal that each DRL algorithm exhibits unique trading patterns and strategies, with A2C emerging as the top performer in terms of cumulative rewards.
  • PPO and SAC engage in significant trades with a limited number of stocks. performance
    Source sentence
    While PPO and SAC engage in significant trades with a limited number of stocks
  • DDPG and TD3 adopt a more balanced approach, trading smaller amounts across a wider range of securities. performance
    Source sentence
    DDPG and TD3 adopt a more balanced approach, trading smaller amounts across a wider range of securities.
  • SAC and PPO tend to hold positions for shorter durations, frequently buying and selling stocks. performance
    Source sentence
    SAC and PPO tend to hold positions for shorter durations, frequently buying and selling stocks
  • DDPG, A2C, and TD3 display a propensity to remain stationary for extended periods. performance
    Source sentence
    D DPG, A2C, and TD3 display a propensity to remain stationary for extended periods

Checked by others

Supported by

  • Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation (2023)
    On the question: Which deep reinforcement learning algorithm achieves the highest cumulative reward for portfolio optimisation?
    Compare the findings

    This paper: A2C emerges as the top performer in terms of cumulative rewards among the DRL algorithms studied.

    That paper: On‑policy algorithms (PPO, A2C) with generalized advantage estimation can handle reward noise and learn a near‑optimal policy.

    Both findings highlight A2C as the top performer; A notes on‑policy methods (PPO, A2C) handle reward noise, B confirms A2C leads in cumulative rewards.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (9)

  1. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  2. Proximal Policy Optimization Algorithms (2017) — link
  3. FinRL-Podracer: High performance and scalable deep reinforcement learning for quantitative finance (2021)
  4. Scalable Actions in CNN-based Deep Reinforcement Learning in Finance (2023)
  5. Cnn-drl with shuffled features in finance — → 2402.03338
  6. Continuous control with deep reinforcement learning (2015) — link
  7. Addressing function approximation error in actor-critic methods (2018)
  8. Asynchronous methods for deep reinforcement learning (2016)
and 1 more
  1. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)

1 of 9 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.