Paper: arXiv 2508.20103
Authors: Rongwei Liu, Jin Zheng, John Cartlidge
Abstract
The optimal asset allocation between risky and risk-free assets is a persistent challenge due to the inherent volatility in financial markets. Conventional methods rely on strict distributional assumptions or non-additive reward ratios, which limit their robustness and applicability to investment goals. To overcome these constraints, this study formulates the optimal two-asset allocation problem as a sequential decision-making task within a Markov Decision Process (MDP). This framework enables the application of reinforcement learning (RL) mechanisms to develop dynamic policies based on simulated financial scenarios, regardless of prerequisites. We use the Kelly criterion to balance immediate reward signals against long-term investment objectives, and we take the novel step of integrating the Time-series Dense Encoder (TiDE) into the Deep Deterministic Policy Gradient (DDPG) RL framework for continuous decision-making. We compare DDPG-TiDE with a simple discrete-action Q-learning RL framework and a passive buy-and-hold investment strategy. Empirical results show that DDPG-TiDE outperforms Q-learning and generates higher risk adjusted returns than buy-and-hold. These findings suggest that tackling the optimal asset allocation problem by integrating TiDE within a DDPG reinforcement learning framework is a fruitful avenue for further exploration.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 3.0/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper employs advanced mathematical concepts such as Markov Decision Processes, the Kelly criterion, and the Deep Deterministic Policy Gradient (DDPG) algorithm, requiring a dense understanding of reinforcement learning and stochastic optimization. However, its empirical rigor is limited as it relies on simulated financial scenarios without clear evidence of real-world data, backtesting protocols, or implementation details, making it more theoretical than immediately deployable.
Research Flowchart
flowchart TD A["Research Goal: Optimal Asset Allocation<br>between Risky & Risk-Free Assets"] --> B["Formulate as MDP & Kelly Criterion"] B --> C["Modeling DDPG-TiDE Framework<br>Continuous Action RL with Time-series Encoder"] B --> D["Baselines: Q-Learning &<br>Passive Buy-and-Hold"] C & D --> E["Simulation & Training on<br>Financial Market Data"] E --> F["Performance Evaluation<br>Risk-Adjusted Returns"] F --> G["Outcome: DDPG-TiDE Outperforms<br>Baselines (Higher Risk-Adjusted Returns)"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes integrating the Time-series Dense Encoder (TiDE) into a Deep Deterministic Policy Gradient (DDPG) reinforcement learning framework for optimal two‑asset allocation.
Source sentence
into the Deep Deterministic Policy Gradient (DDPG) [14] mechanism to enhance the handling of temporal relationships in multivariate time series within the state space.
Assumes 5
- The market portfolio (CRSP index) is observable and broadly representative of the investable universe. market
Source sentence
This study seeks to fill the above research gaps by examining portfolio allocation between a market index—the CRSP index, serving as a representative proxy for the aggregate U.S. equity market—and a risk-free asset.
- Investor utility follows a constant relative risk aversion (CRRA) utility function consistent with the Kelly criterion. model
Source sentence
To ensure theoretical rigor, the study adopts the Kelly criterion through the use of the constant relative risk aversion (CRRA) utility function and the accumulated reward, providing mathematical consistency between log-utility maximization and long-term capital growth.
- The reinforcement learning problem is formulated as a Markov Decision Process (MDP). model
Source sentence
the optimal two-asset allocation problem as a sequential decision-making task within a Markov Decision Process (MDP).
- Trading occurs without explicit transaction costs or market impact in the reported results. execution
Source sentence
We have deliberately used a simplistic simulation framework.
Show 1 more
- Leverage can be applied up to 50% (w > 1) to allow borrowing for investment. execution
Source sentence
We also introduce the opportunity for Q-learning and DDPG-TiDE strategies to use leverage (i.e., to select values w > 1) to see if they can learn strategies that profit from borrowing to invest.
Data
1989 – 2019 · market index · equities · monthly
- Monthly market return and risk‑free rate series with macro predictors macro public
Source sentence
We used a dataset consisting of monthly time series on the market return rate rt (Mkt) and risk‑free asset return rate r f (RF) from 1927 to 2019.
Costs
Not modeled
Source sentence
We have deliberately used a simplistic simulation framework.
Finds 3
Sharpe 1.13 (benchmark 0.95)
- DDPG‑TiDE without leverage achieves a final portfolio value of 14.85, slightly below buy‑and‑hold (17.77), while with 50% leverage it reaches 21.06, slightly above buy‑and‑hold. performance
Source sentence
The final portfolio value for DDPG‑TiDE without leverage (14.85) is slightly below buy‑and‑hold, and the final portfolio value for DDPG‑TiDE with 50% leverage available (21.06) is slightly above buy‑and‑hold.
- DDPG‑TiDE attains an average Sharpe ratio of 1.13 (no leverage) and 0.99 (with leverage), both modestly higher than the buy‑and‑hold average Sharpe ratio of 0.95. performance
Source sentence
... average Sharpe ratio of 1.13 when no leverage is available, and average Sharpe ratio of 0.99 when leverage is available. Both of these values are slightly higher than the average Sharpe ratio for buy‑and‑hold (0.95).
- DDPG‑TiDE outperforms the discrete‑action Q‑learning strategy on all evaluated metrics, including portfolio value and risk‑adjusted returns. performance
Source sentence
We showed that a continuous‑action DDPG‑TiDE strategy outperforms a simple discrete‑action Q‑learning strategy in all metrics; it is also able to generate more profits than buy‑and‑hold by applying leverage during bull periods; and it generates the highest average Sharpe ratio.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 15 related papers). Coverage is partial, so this is neither replication nor refutation.
1 paper in the archive
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (36)
- What is the value of the cross-sectional approach to deep reinforcement learning? (2021) — link
- A Comprehensive 2022 Look at the Empirical Performance of Equity Premium Prediction (2024) — link
- Maximizing Returns with Reinforcement Learning: A Paradigm Shift in Stock Market Portfolio Management (2023) — link
- The Sampling Error in Estimates of Mean‐Variance Efficient Portfolio Weights (2002) — link
- AlphaPortfolio : Direct construction through deep reinforcement learning and interpretable AI (2020) — link
- Long-term Forecasting with TiDE: Time-series Dense Encoder (2023) — link
- Recent advances in reinforcement learning in finance (2023) — link
- The reinforcement learning Kelly strategy (2022) — link
and 28 more
- A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
- Optimal Portfolio Choice with Parameter Uncertainty (2007) — link
- Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
- Continuous control with deep reinforcement learning (2015) — link
- Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy (2023) — link
- Medium Term Simulations of The Full Kelly and Fractional Kelly Investment Strategies (2010) — link
- Long-term capital growth: the good and bad properties of the Kelly and fractional Kelly capital growth criteria (2011) — link
- Deep Reinforcement Learning in Agent-Based Financial Market Simulation (2020) — link
- Portfolio Selection (1952) — link
- Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case (1969) — link
- Optimum consumption and portfolio rules in a continuous-time model (1971) — link
- An intelligent financial portfolio trading strategy using deep Q -learning (2020) — link
- Trading financial indices with reinforcement learning agents (2018) — link
- Multi-agent reinforcement learning approach for hedging portfolio problem (2021) — link
- Machine learning portfolio allocation (2022) — link
- Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999) — link
- Deep Parametric Portfolio Policies (2022) — link
- Attention is all you need (2017) — link
- Research on Portfolio Optimization Based on Deep Reinforcement Learning (2022) — link
- Q-learning (1992) — link
- Model-based Reinforcement Learning for Predictions and Control for Limit Order Books (2019) — link
- Practical Deep Reinforcement Learning Approach for Stock Trading (2018) — link
- Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization (2019) — link
- QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE — → 2409.05144
- A Universal End-to-End Approach to Portfolio Optimization via Deep Learning (2021) — link
- Artificial intelligence for collective intelligence: A national-scale research strategy (2024) — link
- Artificial intelligence for collective intelligence: A national-scale research strategy (2024) — link
- Agent-based model exploration of latency arbitrage in fragmented financial markets (2018) — link
1 of 36 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.