Paper: arXiv 2411.07585

Authors: Alhassan S. Yasin, Prabdeep S. Gill

Abstract

The inherent volatility and dynamic fluctuations within the financial stock market underscore the necessity for investors to employ a comprehensive and reliable approach that integrates risk management strategies, market trends, and the movement trends of individual securities. By evaluating specific data, investors can make more informed decisions. However, the current body of literature lacks substantial evidence supporting the practical efficacy of reinforcement learning (RL) agents, as many models have only demonstrated success in back testing using historical data. This highlights the urgent need for a more advanced methodology capable of addressing these challenges. There is a significant disconnect in the effective utilization of financial indicators to better understand the potential market trends of individual securities. The disclosure of successful trading strategies is often restricted within financial markets, resulting in a scarcity of widely documented and published strategies leveraging RL. Furthermore, current research frequently overlooks the identification of financial indicators correlated with various market trends and their potential advantages. This research endeavors to address these complexities by enhancing the ability of RL agents to effectively differentiate between positive and negative buy/sell actions using financial indicators. While we do not address all concerns, this paper provides deeper insights and commentary on the utilization of technical indicators and their benefits within reinforcement learning. This work establishes a foundational framework for further exploration and investigation of more complex scenarios.

Complexity vs Empirical Score

  • Math Complexity: 4.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Philosophers — conceptual discussion, lower math and empirical scores

Why this score: The paper introduces RL concepts like MDP and uses basic financial indicators (e.g., SMA) without deep mathematical derivations or novel algorithms, keeping math complexity moderate. However, it lacks concrete backtesting results, statistical metrics, or implementation details (e.g., specific code or datasets), focusing instead on conceptual framework and challenges, resulting in low empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal<br>Enhance RL agents' ability to differentiate<br>buy/sell actions using financial indicators"] --> B["Data & Inputs<br>Stock Market Data + Financial Indicators"]
  B --> C["Methodology<br>RL Framework Integration<br>Technical Indicators + Risk Management"]
  C --> D["Computational Process<br>Agent Training & Market Simulation"]
  D --> E{"Outcomes & Findings"}
  E --> F1["Foundation for Complex Scenarios"]
  E --> F2["Deeper Insights on Indicator Utilization"]
  E --> F3["Improved Action Differentiation"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a framework that enhances reinforcement learning agents' ability to differentiate positive and negative buy/sell actions by incorporating financial technical indicators.

Source sentence
This research endeavors to address these complexities by enhancing the ability of RL agents to effectively differentiate between positive and negative buy/sell actions using financial indicators.

Assumes 4

  • The Markov Decision Process assumes that all necessary information for decision‑making is captured in the current state. model
    Source sentence
    The Markov Decision Process states that the future state will depend on the current state and action, it assumes all necessary information is captured in the current state.
  • Time‑series data is treated as a sequence of observations without explicit modeling of longer historical dependencies. data
    Source sentence
    However, this can be problematic in the sense that time-series data is never static. That is to say the Markov Decision Process does not capture all relevant information from a historical perspective.
  • Immediate reward at each timestep drives the agent’s behavior. model
    Source sentence
    In this work we primarily focused on immediate rewards as this provided the RL agent to behave an a manner that would maximize the financial gain on each trade.
  • Trading costs are incurred proportionally to the number of trades (higher commission fees for more trades). execution
    Source sentence
    On an important note, the greater number of trades executed will results in higher commission fees when tying this concept back to a dynamically shifting market.

Data

Backtest sample: APPL · equities

  • APPL daily price data market-prices unknown
    2020-01-01 to 2022-01-01 · APPL
    Source sentence
    The initial training dataset was compiled from two years of day data for APPL from 2020-01-01 to 2022-01-01.

Costs

Partial Costs appear only as a robustness check, a flat fee without spread or impact, or outside the headline numbers.

Higher commission fees are mentioned for a larger number of trades, but costs are not quantified in the reported returns.

Source sentence
On an important note, the greater number of trades executed will results in higher commission fees when tying this concept back to a dynamically shifting market.

Finds 2

Reported performance (Deep Q-Network (DQN)): Sharpe 1.07 · total return 92.3%

  • Using a learning rate of 1e-4 the DQN model achieved a 13.56% total return over the two‑year period with a Sharpe ratio of 0.24 and a win rate of 45.9%. performance
    Source sentence
    Table 2 reflects this as there was a 13.5 percent return over the same 2-year period
  • Increasing the learning rate to 1e-2 dramatically improved performance, yielding a 92.32% total return, Sharpe ratio of 1.07 and a 100% win rate, though the model only traded once. performance
    Source sentence
    ... Win Rate [%] 100.0 ... Interestingly, the model only traded a single time compared to the 61 trades while utilizing a larger learning rate.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 6 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (20)

  1. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  2. Deep reinforcement learning approach for trading automation in the stock market (2022)
  3. Deep reinforcement learning in quantitative algorithmic trading: A review (2021) — link
  4. A deep reinforcement learning-based decision support system for automated stock market trading (2022)
  5. Deep direct reinforcement learning for financial signal representation and trading (2016)
  6. Reinforcement learning in financial markets-a survey (2018)
  7. Recent advances in reinforcement learning in finance (2023)
  8. Application of Deep Reinforcement Learning in Financial Quantitative Trading (2022)
and 12 more
  1. Deep Reinforcement Learning for Quantitative Trading — → 2312.15730
  2. Stock trading bot using deep reinforcement learning (2019)
  3. Stock Trading Optimization through Model-based Reinforcement Learning with Resistance Support Relative Strength (2022) — link
  4. Proximal policy optimization algorithms (2017) — link
  5. Deep Reinforcement Learning for Quantitative Finance: Time Series Forecasting using Proximal Policy Optimization (2022)
  6. Asynchronous methods for deep reinforcement learning (2016)
  7. Playing atari with deep reinforcement learning (2013) — link
  8. Explainable deep reinforcement learning for portfolio management: an empirical approach (2021)
  9. Trust region policy optimization (2015)
  10. Stock trading strategies based on deep reinforcement learning (2022)
  11. Technical analysis of the financial markets: A comprehensive guide to trading methods and applications (1999)
  12. Risk-sensitive compact decision trees for autonomous execution in presence of simulated market response (2019) — link

1 of 20 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.