Paper: arXiv 2301.05300

Authors: Jiwon Kim, Moon-Ju Kang, KangHun Lee, HyungJun Moon, Bo-Kwan Jeon

Abstract

Recently, there are many trials to apply reinforcement learning in asset allocation for earning more stable profits. In this paper, we compare performance between several reinforcement learning algorithms - actor-only, actor-critic and PPO models. Furthermore, we analyze each models’ character and then introduce the advanced algorithm, so called Reward clipping model. It seems that the Reward Clipping model is better than other existing models in finance domain, especially portfolio optimization - it has strength both in bull and bear markets. Finally, we compare the performance for these models with traditional investment strategies during decreasing and increasing markets.

Complexity vs Empirical Score

  • Math Complexity: 5.0/10
  • Empirical Rigor: 5.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a standard application of Deep Reinforcement Learning (PPO, Actor-Critic) adapted with a custom reward clipping heuristic for portfolio optimization. While it offers practical comparison across market regimes, the theoretical mathematical novelty is limited and empirical details remain relatively basic.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimize Multi-Asset Allocation using DRL] --> B[Data Inputs: Historical Financial Market Data]
    B --> C[Evaluate Baseline DRL Models: Actor-Only, Actor-Critic, PPO]
    C --> D[Develop Advanced Method: PPO with Reward Clipping]
    D --> E[Backtest Models vs. Traditional Strategies in Bull/Bear Markets]
    E --> F[Findings: Reward Clipping outperforms baselines with stable returns in both market phases]