Paper: arXiv 2402.03338

Authors: Sina Montazeri, Akram Mirzaeinia, Amir Mirzaeinia

Abstract

In prior methods, it was observed that the application of Convolutional Neural Networks agent in Deep Reinforcement Learning to financial data resulted in an enhanced reward. In this study, a specific permutation was applied to the feature vector, thereby generating a CNN matrix that strategically positions more pertinent features in close proximity. Our comprehensive experimental evaluations unequivocally demonstrate a substantial enhancement in reward attainment.

Complexity vs Empirical Score

  • Math Complexity: 4.0/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Street Traders — empirical and applied, lower math complexity

Why this score: The paper uses standard DRL and CNN concepts without heavy derivations or novel math, keeping complexity moderate, while it employs a concrete financial environment, specific features, and backtest-ready metrics like Sharpe ratio and reward tracking, indicating solid empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Enhance DRL Reward in Financial Markets"] --> B["Data Input: Financial Time Series Data"]
  B --> C["Feature Engineering: Apply Strategic Permutation to Features"]
  C --> D["CNN-DRL Processing: Convolutional Neural Network on Permuted Data"]
  D --> E["Computational Process: Deep Reinforcement Learning Agent"]
  E --> F["Outcome: Substantial Enhancement in Reward Attainment"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a CNN-based deep reinforcement learning agent that uses a shuffled feature vector to improve reward performance.

Source sentence
a specific permutation was applied to the feature vector, thereby generating a CNN matrix that strategically positions more pertinent features in close proximity.

Assumes 3

  • The turbulence exclusion period is not activated, and the agent trades during turbulent market periods. market
    Source sentence
    Machine learning models typically exclude the period of uncertainty in the stock market, often characterized by a turbulence factor, in order to stabilize their learning process. In contrast, we chose not to activate the turbulence exclusion and instead closely monitored and took action during this period as well as ot
  • Trading costs are represented by fixed percentage fees as defined in the FinRL framework. execution
    Source sentence
    We employ identical fixed cost percentages as those developed in the FinRL framework.
  • The initial reward is set to one million dollars and later scaled down to one for learning stability. model
    Source sentence
    The environment employs an initial reward of one million dollars, in line with established research practices. This reward is later scaled down to one when it is returned from the environment to the agent, aiding in the learning process.

Data

  • Dow Jones 30 price data · Yahoo Finance and Wharton dataset market-prices public
    Jan 2015 - Sep 2023 · 30 Dow Jones companies
    Source sentence
    We obtained the stock prices of thirty companies listed in the Dow Jones index, spanning from January 2015 to September 2023. This timeframe encompasses the pandemic, a particularly challenging period in the field of finance. This data is sourced from both Yahoo Finance and the Wharton dataset [10].

Recognised sources: Yahoo Finance

Costs

Partial Costs appear only as a robustness check, a flat fee without spread or impact, or outside the headline numbers.

Source sentence
We employ identical fixed cost percentages as those developed in the FinRL framework.

Finds 2

  • The shuffled-feature CNN agent shows a clear improvement over the original CNN and decisively outperforms the MLP agent, especially from January 2020 to January 2023, outpacing the MLP by almost 100% during its peak around October 2023. performance
    Source sentence
    By even a mere glance, the shuffled-featured CNN demonstrates a clear improvement over the original CNN and decisively outperforms the MLP agent. What is most interesting is that both CNN policies, particularly the shuffled-featured one performed vastly better from January 2020 to Jan 2023, outpacing the MLP by almost
  • The proposed CNN architecture with shuffled features yields substantially higher cumulative rewards compared to conventional MLP models. performance
    Source sentence
    This study showcases the capability of convolutional neural networks as a proficient deep reinforcement learning framework for stock trading in settings characterized by shuffled feature vectors. Through empirical assessments using historical market data and the FinRL platform, our specialized CNN agent in addition to

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 8 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 3 papers in the archive

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (11)

  1. FinRL-Meta: Market environments and benchmarks for data-driven financial reinforcement learning (2022)
  2. ABIDES-gym: gym environments for multi-agent discrete event simulation and application to financial markets (2021)
  3. Towards a fully RL-based Market Simulator (2021)
  4. Towards Realistic Market Simulations: a Generative Adversarial Networks Approach. arXiv e-prints, page (2021) — link
  5. Practical deep reinforcement learning approach for stock trading (2018) — link
  6. Proximal Policy Optimization Algorithms (2017) — link
  7. Deep reinforcement learning for automated stock trading: An ensemble strategy (2020)
  8. Deep reinforcement learning for trading (2019) — link
and 3 more
  1. FinRL-Podracer: High performance and scalable deep reinforcement learning for quantitative finance (2021)
  2. Wharton — link
  3. Scalable Actions in CNN-based Deep Reinforcement Learning in Finance (2023)

0 of 11 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.