Paper: arXiv 2306.07013

Authors: Zhenglong Li, Hejun Huang, Vincent Tam

Abstract

Reinforcement learning (RL) based investment strategies have been widely adopted in portfolio management (PM) in recent years. Nevertheless, most RL-based approaches may often emphasize on pursuing returns while ignoring the risks of the underlying trading strategies that may potentially lead to great losses especially under high market volatility. Therefore, a risk-manageable PM investment framework integrating both RL and barrier functions (BF) is proposed to carefully balance the needs for high returns and acceptable risk exposure in PM applications. Up to our understanding, this work represents the first attempt to combine BF and RL for financial applications. While the involved RL approach may aggressively search for more profitable trading strategies, the BF-based risk controller will continuously monitor the market states to dynamically adjust the investment portfolio as a controllable measure for avoiding potential losses particularly in downtrend markets. Additionally, two adaptive mechanisms are provided to dynamically adjust the impact of risk controllers such that the proposed framework can be flexibly adapted to uptrend and downtrend markets. The empirical results of our proposed framework clearly reveal such advantages against most well-known RL-based approaches on real-world data sets. More importantly, our proposed framework shed lights on many possible directions for future investigation.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 7.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper features advanced mathematics including POMDP formulations, second-order cone programming, and barrier functions, while providing empirical results on real-world datasets, demonstrating a balanced combination of theoretical depth and practical validation.

Research Flowchart

  flowchart TD
  A["Research Goal: Combine RL & Barrier Functions for Risk-Managed Portfolio Optimization"] --> B["Methodology: RL Agent + Barrier Function Risk Controller"]
  B --> C["Input: Real-world Financial Data"]
  C --> D["Computational Process: Dynamic Programming & Adaptive Market Mechanisms"]
  D --> E["Outcome: High Returns with Controlled Risk Exposure"]
  E --> F["Key Findings: Outperforms Standard RL & Offers Downtrend Protection"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes the RiPO framework, the first integration of reinforcement learning and barrier function based constraint programming for portfolio optimization.

Source sentence
The RiPO framework is the first attempt to integrate RL and BF-based constraint programming for financial applications.

Assumes 2

  • The portfolio will be only considered from long positions. model
    Source sentence
    Assumption 1. The portfolio will be only considered from long positions in this work.
  • The turnover rate of assets in a portfolio satisfies the requirements of each order execution. execution
    Source sentence
    Assumption 2. The turnover rate of assets in a portfolio satisfies the requirements of each order execution.

Data

  • S&P500 constituent stocks daily data · Yahoo Finance market-prices public
    2015-2022 (approx.) · Top 10 S&P500 stocks by market cap
    Source sentence
    the daily OLHCV data of constitute stocks of S&P500 index in the U.S. market is collected from Yahoo Finance. The top 10 stocks are selected to construct a portfolio

Recognised sources: Yahoo Finance

Costs

Modeled Costs are part of the reported headline results.

Transaction cost ς and slippage ξ (uniformly sampled) are included in the reward function.

Source sentence
To close the realistic trading environment, two practical factors including transaction cost ς and slippage ξ are considered in each transaction. The reward at timestamp t can be defined as rt = [−ς +

Finds 4

  • RiPO achieves returns at least 5% higher than other methods, with a maximum drawdown around 22% and the best Sharpe Ratio of 0.72, demonstrating a balance of profit and risk. performance
    Source sentence
    the returns of the RiPO framework are at least 5% higher than other methods while performing relatively low MDD at around 22%
  • Increasing the impact of the risk controller (higher m) improves AR from –9.43% to –6.58% and reduces MDD from 29.47% to 25.56% in downtrend markets, though overly high m may miss profitable opportunities. performance
    Source sentence
    the higher m performs better AR (from −9.43% to −6.58%) and lower MDD (from 29.47% to 25.56%) by enhancing the impact of risk controller to the RL-based agents in downtrend markets.
  • Adaptive Risk Strategy (ARS) reduces losses from 23.32% to 6.69% (without DCM) and from 18.76% to 6.58% (with DCM), while also halving risk exposure. performance
    Source sentence
    the ARS significantly reduces the losses from 23.32% to 6.69% when without using the DCM and from 18.76% to 6.58% when integrating the DCM. Meanwhile, the risk exposure decreases to half of the setting without the ARS.
  • Using a scaling factor µ=3 in MS-2 helps RiPO balance expected returns and short-term risks by dynamically adjusting risk constraint strength. performance
    Source sentence
    using the appropriate scaling factor µ at 3 in the MS-2 encourages the RiPO framework to balance the expected returns and short-term risks by dynamically adjusting the strength of risk constraints.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 38 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 2 papers in the archive (2 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.