Paper: arXiv 2311.04946
Authors: Yasuhiro Nakayama, Tomochika Sawaki
Abstract
In this study, we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques. To begin with, we have addressed the crucial issue of incorporating non-stationarity of financial time series data into reinforcement learning algorithms, which is a significant implementation in the application of reinforcement learning in investment strategies. Our findings highlight the significance of introducing certain variables such as regime change in the environment setting to enhance the prediction accuracy. Furthermore, the application of reinforcement learning in investment strategies provides a remarkable advantage of setting the optimization problem flexibly. This enables the integration of practical constraints faced by investors into the algorithm, resulting in efficient optimization. Our study has categorized the investment strategy formulation conditions into three main categories, including performance measurement indicators, portfolio management rules, and other constraints. We have evaluated the impact of incorporating these conditions into the environment and rewards in a reinforcement learning framework and examined how they influence investment behavior.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 6.0/10 (capped from 8.0: transaction costs not modelled; see penalty flags)
- Quadrant: Holy Grail — high math complexity, high empirical rigor (effective empirical rigor capped from 8.0)
Why this score: The paper employs advanced reinforcement learning algorithms (SARSA, Q-learning) with non-trivial state spaces and reward engineering, scoring high on mathematical complexity. It features a rigorous out-of-sample backtest over 22 years, specific data handling, and statistical significance testing, indicating high empirical rigor.
Research Flowchart
flowchart TD
A["Research Goal: Develop RL-based Dynamic Asset Allocation Strategy"] --> B{"Methodology & Data"}
B --> C["Data: Financial Time Series<br>Non-stationary Trends"]
B --> D["Core Methodology: Reinforcement Learning"]
C & D --> E["Computational Process: Environment Setup"]
E --> F["Process: Incorporate Investment Constraints<br>Rules & Regime Changes"]
F --> G["Process: Optimize Policy via<br>RL Algorithm"]
G --> H["Key Findings / Outcomes"]
H --> I["Strategy: Flexible Optimization<br>Integrates Practical Constraints"]
H --> J["Performance: Enhanced Accuracy<br>via Non-stationarity Handling"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes a dynamic asset allocation investment strategy using reinforcement learning that incorporates non‑stationarity and practical investment constraints.
Source sentence
we have developed a dynamic asset allocation investment strategy using reinforcement learning techniques.
Assumes 5
- Funding costs, transaction costs, and cash ratios are ignored. execution
Source sentence
We don’t take into account funding costs, transaction costs, and cash ratios.
- Rebalancing is performed daily. execution
Source sentence
the rebalancing frequency is daily, and transaction costs are not considered.
- Financial time‑series mean, variance, and covariance are not constant over time. market
Source sentence
mean, variance, and covariance are not constant over time and that several events have been historically observed
- Correlation state variable is discretized into three bins using a ±0.2 threshold. model
Source sentence
The correlation coefficient state variable is divided into three states of positive correlation/no correlation/negative correlation with a threshold of ±0.2
Show 1 more
- Table‑based reinforcement learning (SARSA and Q‑learning) is sufficient for the problem. model
Source sentence
we utilize table-based reinforcement learning methods such as SARSA and Q‑learning.
Data
- S&P500 total return index (USD) market-prices commercial
Source sentence
We choose the total return index of the S & P500, denominated in USD, as the risky asset
- US Treasury Index total return (USD) market-prices commercial
Source sentence
the total return index of US Treasury Index, also denominated in USD, as the non‑risky asset
Costs
Not modeled
Source sentence
We don’t take into account funding costs, transaction costs, and cash ratios.
Finds 5
- The non‑stationary model’s average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model and the difference is statistically significant. performance
Source sentence
The non‑stationary model's average Sharpe ratio from FY2001 to FY2022 is higher than that of the base model, and the difference is statistically significant.
- The non‑stationary model also outperforms the median Sharpe ratio of the random model on average. performance
Source sentence
The non‑stationary model also outperforms the median of the random model on average Sharpe ratios.
- Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly). performance
Source sentence
As expected, the unconstrained condition exhibits the highest average Sharpe ratio, and performance gradually decreases as the interval is extended.
- Higher signal accuracy increases the selection ratio of non‑risky assets, with a weaker but positive effect for risky assets. performance
Source sentence
The rate of selection for non‑risky assets increases as signal accuracy improved, whereas for risky assets, the trend is discerned in the order of each Accuracy, albeit it is weaker than that for non‑risky assets.
- In periods where signal accuracy varies, the proportion of assets selected according to the signal rises with higher accuracy for both risky and non‑risky assets. performance
Source sentence
As the horizontal axis shifts rightward and the accuracy in one aspect becomes higher, the proportion of selecting the asset indicated by the signals increases. This outcome is confirmed for both risky and non‑risky assets.
Checked by others
Contradicted by
- Investment Portfolio Optimization Based on Modern Portfolio Theory and Deep Learning Models
Compare the findings
This paper: Unconstrained (no rebalancing frequency limit) yields the highest average Sharpe ratio; performance declines as the rebalancing interval is lengthened (weekly, bi‑weekly, monthly).
That paper: Strategies with less frequent rebalancing (e.g., 30‑, 60‑, 90‑day) outperform those with very short rebalancing periods.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.