Paper: arXiv 2409.08426

Authors: Jinyang Li

Abstract

In this research paper, we investigate into a paper named “A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem” [“arXiv:1706.10059”]. It is a portfolio management problem which is solved by deep learning techniques. The original paper proposes a financial-model-free reinforcement learning framework, which consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. Three different instants are used to realize this framework, namely a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). The performance is then examined by comparing to a number of recently reviewed or published portfolio-selection strategies. We have successfully replicated their implementations and evaluations. Besides, we further apply this framework in the stock market, instead of the cryptocurrency market that the original paper uses. The experiment in the cryptocurrency market is consistent with the original paper, which achieve superior returns. But it doesn’t perform as well when applied in the stock market.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced machine learning theory (deep reinforcement learning, PVM, OSBL) with significant mathematical formalism, while also demonstrating high empirical rigor through full implementation, code/GitHub availability, and comparative backtests on cryptocurrency and stock data with explicit performance metrics.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Deep RL for Portfolio Management"] --> B["Data & Inputs:<br>Multi-Asset Time Series"]
  B --> C["Core Methodology:<br>Ensemble of Identical Independent Evaluators EIIE"]
  C --> D{"Key Components"}
  D --> E["Portfolio-Vector Memory PVM"]
  D --> F["Online Stochastic Batch Learning OSBL"]
  D --> G["Explicit Reward Function"]
  E & F & G --> H["Computational Models:<br>CNN / RNN / LSTM"]
  H --> I["Outcomes"]
  I --> J["Crypto Market:<br>Superior Returns"]
  I --> K["Stock Market:<br>Underperformed"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a financial‑model‑free reinforcement learning framework comprising an Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio‑Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and an explicit reward function.

Source sentence
The original paper proposes a financial-model-free reinforcement learning framework, which consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function.

Assumes 2

  • Zero slippage: market volume is high enough that each order can be filled at the market price. execution
    Source sentence
    Zero slippage: we assume that the volume of the market is high enough that each order can be filled with the market price at placement.
  • Zero money impact: the portfolio capital is small enough to have no influence on the market. execution
    Source sentence
    Zero money impact: the capital of the portfolio is small enough that it has no influence on the market.

Data

Backtest sample: 2015-07 – 2017-07 · cryptocurrency market · crypto · daily

  • cryptocurrency market dataset crypto-onchain unknown
    Source sentence
    The original paper tests its validity and profitability in the cryptocurrency exchange market.
  • stock market dataset market-prices unknown
    Source sentence
    we further evaluate this framework in the stock market.

Costs

Modeled Costs are part of the reported headline results.

Costs are assumed to come from a constant commission rate; transaction remainder factor µt is derived from commission rates for buying and selling.

Source sentence
In our framework, we assume that the costs mainly come from commission fee and we assume a constant commission rate using the recursive formula by Ormos and Urbán (2013).

Finds 4

Reported performance (EIIE deep reinforcement learning framework): total return 793.8%

  • The three EIIE models (CNN, RNN, LSTM) significantly beat all other portfolio‑selection strategies in the cryptocurrency market. performance
    Source sentence
    The EIIEs significantly beat all the other strategies when it is in the cryptocurrency market, but the performance is average in stock market.
  • In the stock market the framework’s performance is only average, comparable to a Uniform Constant Rebalanced Portfolio. performance
    Source sentence
    Surprisingly, when it comes to the stock market, the magic of this framework disappeared and fell back to be equivalent to the Uniformly Constant Rebalanced Portfolios algorithm or the equally weighted strategy.
  • Among the three EIIE networks, LSTM obtains lower scores than CNN and RNN. performance
    Source sentence
    Among the three EIIE networks, LSTM has lower scores than that of CNN and RNN.
  • A vanilla RNN outperforms LSTM in exploiting repetitive price patterns, likely due to its inability to forget history. performance
    Source sentence
    Not being designed to forget its input history, a vanilla RNN is more able than a LSTM to exploit

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 15 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors): Comparing Normalization Methods for Portfolio Optimization … (discusses its findings)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (25)

  1. A deep reinforcement learning framework for the financial portfolio management problem (2017) — link
  2. Portfolio selection (1952)
  3. Portfolio selection: Efficient diversification of investments (1959)
  4. A new interpretation of information rate (1956)
  5. Capital growth theory (1995)
  6. Deep learning for finance: Deep portfolios (2017)
  7. Forecasting \uppercase S&P 500 index using artificial neural networks and design of experiments (2013)
  8. Prediction-based portfolio optimization model using neural networks (2009)
and 17 more
  1. Online portfolio selection: A survey (2014)
  2. OLPS: a toolbox for on-line portfolio selection (2016)
  3. Bitcoin and cryptocurrency technologies: A comprehensive introduction (2016)
  4. Cryptocurrencies: a brief thematic review (2017)
  5. The concise fintech compendium (2017)
  6. If banks want benefits of blockchains they must go permissionless (2015)
  7. Bitcoin 101 white paper (2015)
  8. Learning to trade via direct reinforcement (2001)
  9. An automated \uppercase FX trading system using adaptive reinforcement learning (2006)
  10. An investigation into the use of reinforcement learning techniques within the algorithmic trading domain (2015)
  11. Deep direct reinforcement learning for financial signal representation and trading (2016)
  12. Reinforcement learning: A survey (1996)
  13. Performance analysis of log-optimal portfolio strategies with transaction costs (2013)
  14. Technical analysis: The complete resource for financial market technicians (2010)
  15. Foundations of technical analysis: Computational algorithms, statistical inference, and empirical implementation (2000)
  16. Asynchronous methods for deep reinforcement learning (2016)
  17. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? (2007)

0 of 25 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.