Paper: arXiv 2407.09557
Authors: Alireza Mohammadshafie, Akram Mirzaeinia, Haseebullah Jumakhan, Amir Mirzaeinia
Abstract
Recent deep reinforcement learning (DRL) methods in finance show promising outcomes. However, there is limited research examining the behavior of these DRL algorithms. This paper aims to investigate their tendencies towards holding or trading financial assets as well as purchase diversity. By analyzing their trading behaviors, we provide insights into the decision-making processes of DRL models in finance applications. Our findings reveal that each DRL algorithm exhibits unique trading patterns and strategies, with A2C emerging as the top performer in terms of cumulative rewards. While PPO and SAC engage in significant trades with a limited number of stocks, DDPG and TD3 adopt a more balanced approach. Furthermore, SAC and PPO tend to hold positions for shorter durations, whereas DDPG, A2C, and TD3 display a propensity to remain stationary for extended periods.
Complexity vs Empirical Score
- Math Complexity: 5.0/10
- Empirical Rigor: 7.0/10
- Quadrant: Holy Grail — high math complexity, high empirical rigor
Why this score: The paper employs deep reinforcement learning algorithms like A2C, PPO, SAC, DDPG, and TD3, which involve stochastic optimization and neural network architectures, adding mathematical complexity, while its empirical rigor is high due to the use of a specific dataset (Dow Jones 30 companies from Yahoo Finance), defined train/test splits, and analysis of trading behaviors.
Research Flowchart
flowchart TD
A["Research Goal<br>Investigate DRL Trading Behavior<br>& Purchase Diversity"] --> B{"Methodology"}
B --> C["Data: Financial Asset Time Series"]
B --> D["Compute: Train DRL Models<br>DDPG, TD3, A2C, PPO, SAC"]
C --> D
D --> E{"Analyze Trading Behavior"}
E --> F["Findings & Outcomes"]
F --> G["Performance<br>A2C: Highest Cumulative Reward"]
F --> H["Trade Volume<br>PPO/SAC: High Trades, Few Stocks<br>DDPG/TD3: Balanced Approach"]
F --> I["Holding Time<br>SAC/PPO: Short Duration<br>DDPG/A2C/TD3: Long Duration"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper investigates the tendencies of deep reinforcement learning algorithms to hold or trade assets and their purchase diversity in finance.
Source sentence
This paper aims to investigate their tendencies towards holding or trading financial assets as well as purchase diversity.
Assumes 1
- The analysis uses hourly price data from Yahoo Finance without accounting for market impact or slippage. data
Source sentence
The financial data used in this analysis was obtained from Yahoo Finance. The data covers hourly intervals
Data
- Yahoo Finance market-prices public
Source sentence
The financial data used in this analysis was obtained from Yahoo Finance. The data covers hourly intervals
Costs
Unclear
Finds 5
- A2C emerges as the top performer in terms of cumulative rewards among the DRL algorithms studied. performance
Source sentence
Our findings reveal that each DRL algorithm exhibits unique trading patterns and strategies, with A2C emerging as the top performer in terms of cumulative rewards.
- PPO and SAC engage in significant trades with a limited number of stocks. performance
Source sentence
While PPO and SAC engage in significant trades with a limited number of stocks
- DDPG and TD3 adopt a more balanced approach, trading smaller amounts across a wider range of securities. performance
Source sentence
DDPG and TD3 adopt a more balanced approach, trading smaller amounts across a wider range of securities.
- SAC and PPO tend to hold positions for shorter durations, frequently buying and selling stocks. performance
Source sentence
SAC and PPO tend to hold positions for shorter durations, frequently buying and selling stocks
- DDPG, A2C, and TD3 display a propensity to remain stationary for extended periods. performance
Source sentence
D DPG, A2C, and TD3 display a propensity to remain stationary for extended periods
Checked by others
Supported by
- Evaluation of Deep Reinforcement Learning Algorithms for Portfolio Optimisation
Compare the findings
This paper: A2C emerges as the top performer in terms of cumulative rewards among the DRL algorithms studied.
That paper: On‑policy algorithms (PPO, A2C) with generalized advantage estimation can handle reward noise and learn a near‑optimal policy.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (9)
- FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
- Proximal Policy Optimization Algorithms (2017) — link
- FinRL-Podracer: High performance and scalable deep reinforcement learning for quantitative finance (2021)
- Scalable Actions in CNN-based Deep Reinforcement Learning in Finance (2023)
- Cnn-drl with shuffled features in finance — → 2402.03338
- Continuous control with deep reinforcement learning (2015) — link
- Addressing function approximation error in actor-critic methods (2018)
- Asynchronous methods for deep reinforcement learning (2016)
and 1 more
- Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
1 of 9 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.