A Deep Reinforcement Learning Framework For Financial Portfolio Management

ArXiv ID: 2409.08426 “View on arXiv”

Authors: Unknown

Abstract

In this research paper, we investigate into a paper named “A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem” [“arXiv:1706.10059”]. It is a portfolio management problem which is solved by deep learning techniques. The original paper proposes a financial-model-free reinforcement learning framework, which consists of the Ensemble of Identical Independent Evaluators (EIIE) topology, a Portfolio-Vector Memory (PVM), an Online Stochastic Batch Learning (OSBL) scheme, and a fully exploiting and explicit reward function. Three different instants are used to realize this framework, namely a Convolutional Neural Network (CNN), a basic Recurrent Neural Network (RNN), and a Long Short-Term Memory (LSTM). The performance is then examined by comparing to a number of recently reviewed or published portfolio-selection strategies. We have successfully replicated their implementations and evaluations. Besides, we further apply this framework in the stock market, instead of the cryptocurrency market that the original paper uses. The experiment in the cryptocurrency market is consistent with the original paper, which achieve superior returns. But it doesn’t perform as well when applied in the stock market.

Keywords: Deep Reinforcement Learning, portfolio management, Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), Portfolio-Vector Memory (PVM), Multi-Asset

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 8.0/10
• Quadrant: Holy Grail
• Why: The paper employs advanced machine learning theory (deep reinforcement learning, PVM, OSBL) with significant mathematical formalism, while also demonstrating high empirical rigor through full implementation, code/GitHub availability, and comparative backtests on cryptocurrency and stock data with explicit performance metrics.

  flowchart TD
    A["Research Goal:<br>Deep RL for Portfolio Management"] --> B["Data & Inputs:<br>Multi-Asset Time Series"]
    B --> C["Core Methodology:<br>Ensemble of Identical Independent Evaluators EIIE"]
    C --> D{"Key Components"}
    D --> E["Portfolio-Vector Memory PVM"]
    D --> F["Online Stochastic Batch Learning OSBL"]
    D --> G["Explicit Reward Function"]
    E & F & G --> H["Computational Models:<br>CNN / RNN / LSTM"]
    H --> I["Outcomes"]
    I --> J["Crypto Market:<br>Superior Returns"]
    I --> K["Stock Market:<br>Underperformed"]