Paper: arXiv 2302.00586

Authors: Shuo Sun, Molei Qin, Xinrun Wang, Bo An

Abstract

The financial markets, which involve more than $90 trillion market capitals, attract the attention of innumerable investors around the world. Recently, reinforcement learning in financial markets (FinRL) has emerged as a promising direction to train agents for making profitable investment decisions. However, the evaluation of most FinRL methods only focuses on profit-related measures and ignores many critical axes, which are far from satisfactory for financial practitioners to deploy these methods into real-world financial markets. Therefore, we introduce PRUDEX-Compass, which has 6 axes, i.e., Profitability, Risk-control, Universality, Diversity, rEliability, and eXplainability, with a total of 17 measures for a systematic evaluation. Specifically, i) we propose AlphaMix+ as a strong FinRL baseline, which leverages mixture-of-experts (MoE) and risk-sensitive approaches to make diversified risk-aware investment decisions, ii) we evaluate 8 FinRL methods in 4 long-term real-world datasets of influential financial markets to demonstrate the usage of our PRUDEX-Compass, iii) PRUDEX-Compass together with 4 real-world datasets, standard implementation of 8 FinRL methods and a portfolio management environment is released as public resources to facilitate the design and comparison of new FinRL methods. We hope that PRUDEX-Compass can not only shed light on future FinRL research to prevent untrustworthy results from stagnating FinRL into successful industry deployment but also provide a new challenging algorithm evaluation scenario for the reinforcement learning (RL) community.

Complexity vs Empirical Score

  • Math Complexity: 5.5/10
  • Empirical Rigor: 8.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper addresses a critical gap in Financial Reinforcement Learning (FinRL) by introducing a comprehensive, multi-dimensional evaluation framework (PRUDEX-Compass) alongside a robust MoE-based baseline (AlphaMix+). The empirical rigor is exceptional, featuring extensive testing across multiple real-world market datasets and open-source public resources for reproducibility.

Research Flowchart

  flowchart TD
    A[Research Goal: Address narrow profit-only evaluation in Financial RL] --> B[Introduce PRUDEX-Compass Evaluation Framework<br/>6 Axes & 17 Measures]
    B --> C[Inputs: 4 Long-term Real-World Market Datasets<br/>+ Portfolio Management Environment]
    C --> D[Propose AlphaMix+ Baseline<br/>Combines MoE & Risk-Sensitive RL]
    D --> E[Benchmark 8 FinRL Methods<br/>Across PRUDEX-Compass Axes]
    E --> F[Outcomes: Released Open-Source FinRL Benchmark<br/>& Demonstrated AlphaMix+ Superiority]