Paper: arXiv 2311.04946

Authors: Yasuhiro Nakayama, Tomochika Sawaki

Abstract

In this study, we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques. To begin with, we have addressed the crucial issue of incorporating non-stationarity of financial time series data into reinforcement learning algorithms, which is a significant implementation in the application of reinforcement learning in investment strategies. Our findings highlight the significance of introducing certain variables such as regime change in the environment setting to enhance the prediction accuracy. Furthermore, the application of reinforcement learning in investment strategies provides a remarkable advantage of setting the optimization problem flexibly. This enables the integration of practical constraints faced by investors into the algorithm, resulting in efficient optimization. Our study has categorized the investment strategy formulation conditions into three main categories, including performance measurement indicators, portfolio management rules, and other constraints. We have evaluated the impact of incorporating these conditions into the environment and rewards in a reinforcement learning framework and examined how they influence investment behavior.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
  • Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)

Why this score: The paper employs advanced reinforcement learning algorithms (SARSA, Q-learning) with non-trivial state spaces and reward engineering, scoring high on mathematical complexity. It features a rigorous out-of-sample backtest over 22 years, specific data handling, and statistical significance testing, indicating high empirical rigor.

Research Flowchart

  flowchart TD
  A["Research Goal: Develop RL-based Dynamic Asset Allocation Strategy"] --> B{"Methodology & Data"}
  
  B --> C["Data: Financial Time Series<br>Non-stationary Trends"]
  B --> D["Core Methodology: Reinforcement Learning"]
  
  C & D --> E["Computational Process: Environment Setup"]
  E --> F["Process: Incorporate Investment Constraints<br>Rules & Regime Changes"]
  F --> G["Process: Optimize Policy via<br>RL Algorithm"]
  
  G --> H["Key Findings / Outcomes"]
  
  H --> I["Strategy: Flexible Optimization<br>Integrates Practical Constraints"]
  H --> J["Performance: Enhanced Accuracy<br>via Non-stationarity Handling"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a dynamic asset allocation investment strategy using reinforcement learning that incorporates non‑stationarity and practical investment constraints.

Source sentence
we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques.

Assumes 5

  • Funding costs, transaction costs, and cash ratios are ignored. execution
    Source sentence
    We don’t take into account funding costs, transaction costs, and cash ratios.
  • Rebalancing is performed daily. execution
    Source sentence
    the rebalancing frequency is daily, and transaction costs are not considered.
  • Financial time‑series mean, variance, and covariance are not constant over time. market
    Source sentence
    mean, variance, and covariance are not constant over time and that several events have been historically observed
  • Correlation state variable is discretized into three bins using a ±0.2 threshold. model
    Source sentence
    The correlation coefficient state variable is divided into three states of positive correlation/no correlation/negative correlation with a threshold of ±0.2
Show 1 more
  • Table‑based reinforcement learning (SARSA and Q‑learning) is sufficient for the problem. model
    Source sentence
    we utilize table-based reinforcement learning methods such as SARSA and Q‑learning.

Data

  • S&P500 total return index (USD) · Bloomberg market-prices commercial
    April 2000 to March 2023 · single risky asset
    Source sentence
    We choose the total return index of the S & P500, denominated in USD, as the risky asset
  • US Treasury Index total return (USD) · Bloomberg market-prices commercial
    April 2000 to March 2023 · single non‑risky asset
    Source sentence
    the total return index of US Treasury Index, also denominated in USD, as the non‑risky asset

Recognised sources: Bloomberg

Costs

Not modeled A trading or execution result is reported without costs.

Source sentence
We don’t take into account funding costs, transaction costs, and cash ratios.

Finds 5

  • The non‑stationary model’s average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model and the difference is statistically significant. performance
    Source sentence
    The non‑stationary model's average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model, and the difference is statistically significant.
  • The non‑stationary model also outperforms the median Sharpe ratio of the random model on average. performance
    Source sentence
    The non‑stationary model also outperforms the median of the random model on average Sharpe ratios.
  • Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly). performance
    Source sentence
    As expected, the unconstrained condition exhibits the highest average Sharpe ratio, and performance gradually decreases as the interval is extended.
  • Higher signal accuracy increases the selection ratio of non‑risky assets, with a weaker but positive effect for risky assets. performance
    Source sentence
    The rate of selection for non‑risky assets increases as signal accuracy improved, whereas for risky assets, the trend is discerned in the order of each Accuracy, albeit it is weaker than that for non‑risky assets.
  • In periods where signal accuracy varies, the proportion of assets selected according to the signal rises with higher accuracy for both risky and non‑risky assets. performance
    Source sentence
    As the horizontal axis shifts rightward and the accuracy in one aspect becomes higher, the proportion of selecting the asset indicated by the signals increases. This outcome is confirmed for both risky and non‑risky assets.

Checked by others

Contradicted by

  • Investment Portfolio Optimization Based on Modern Portfolio Theory and Deep Learning Models (2025)
    On the question: What is the impact of rebalancing frequency on portfolio performance (e.g., Sharpe ratio)?
    Compare the findings

    This paper: Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly).

    That paper: Strategies with less frequent rebalancing (e.g., 30‑, 60‑, 90‑day) outperform those with very short rebalancing periods.

    A finds performance declines as rebalancing moves from no limit to weekly/bi‑weekly/monthly, while B finds 30‑, 60‑, 90‑day intervals outperform very short periods.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.