A Novel Experts Advice Aggregation Framework Using Deep Reinforcement Learning for Portfolio Management

Paper: arXiv 2212.14477

Authors: MohammadAmin Fazli, Mahdi Lashkari, Hamed Taherkhani, Jafar Habibi

Abstract

Solving portfolio management problems using deep reinforcement learning has been getting much attention in finance for a few years. We have proposed a new method using experts signals and historical price data to feed into our reinforcement learning framework. Although experts signals have been used in previous works in the field of finance, as far as we know, it is the first time this method, in tandem with deep RL, is used to solve the financial portfolio management problem. Our proposed framework consists of a convolutional network for aggregating signals, another convolutional network for historical price data, and a vanilla network. We used the Proximal Policy Optimization algorithm as the agent to process the reward and take action in the environment. The results suggested that, on average, our framework could gain 90 percent of the profit earned by the best expert.

Complexity vs Empirical Score

• Math Complexity: 5.5/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces an original approach combining expert signal aggregation with deep reinforcement learning (specifically PPO) for portfolio management. While the empirical results demonstrate strong relative performance against benchmark experts, the paper would benefit from deeper statistical validation and published code for full reproducibility.

Research Flowchart

  flowchart TD
    A[Research Goal: Improve Portfolio Management using Deep RL & Expert Signals] --> B[Data Inputs: Historical Price Data & Expert Signals]
    B --> C[Neural Architecture: CNNs for Aggregation & Price Data + Vanilla Net]
    C --> D[Agent Training: Proximal Policy Optimization Algorithm]
    D --> E[Outcome: Framework achieves ~90% of the best expert's profit]