Paper: arXiv 2508.20103

Authors: Rongwei Liu, Jin Zheng, John Cartlidge

Abstract

The optimal asset allocation between risky and risk-free assets is a persistent challenge due to the inherent volatility in financial markets. Conventional methods rely on strict distributional assumptions or non-additive reward ratios, which limit their robustness and applicability to investment goals. To overcome these constraints, this study formulates the optimal two-asset allocation problem as a sequential decision-making task within a Markov Decision Process (MDP). This framework enables the application of reinforcement learning (RL) mechanisms to develop dynamic policies based on simulated financial scenarios, regardless of prerequisites. We use the Kelly criterion to balance immediate reward signals against long-term investment objectives, and we take the novel step of integrating the Time-series Dense Encoder (TiDE) into the Deep Deterministic Policy Gradient (DDPG) RL framework for continuous decision-making. We compare DDPG-TiDE with a simple discrete-action Q-learning RL framework and a passive buy-and-hold investment strategy. Empirical results show that DDPG-TiDE outperforms Q-learning and generates higher risk adjusted returns than buy-and-hold. These findings suggest that tackling the optimal asset allocation problem by integrating TiDE within a DDPG reinforcement learning framework is a fruitful avenue for further exploration.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper employs advanced mathematical concepts such as Markov Decision Processes, the Kelly criterion, and the Deep Deterministic Policy Gradient (DDPG) algorithm, requiring a dense understanding of reinforcement learning and stochastic optimization. However, its empirical rigor is limited as it relies on simulated financial scenarios without clear evidence of real-world data, backtesting protocols, or implementation details, making it more theoretical than immediately deployable.

Research Flowchart

  flowchart TD
  A["Research Goal: Optimal Asset Allocation<br>between Risky & Risk-Free Assets"] --> B["Formulate as MDP & Kelly Criterion"]
  B --> C["Modeling DDPG-TiDE Framework<br>Continuous Action RL with Time-series Encoder"]
  B --> D["Baselines: Q-Learning &<br>Passive Buy-and-Hold"]
  C & D --> E["Simulation & Training on<br>Financial Market Data"]
  E --> F["Performance Evaluation<br>Risk-Adjusted Returns"]
  F --> G["Outcome: DDPG-TiDE Outperforms<br>Baselines (Higher Risk-Adjusted Returns)"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes integrating the Time-series Dense Encoder (TiDE) into a Deep Deterministic Policy Gradient (DDPG) reinforcement learning framework for optimal two‑asset allocation.

Source sentence
into the Deep Deterministic Policy Gradient (DDPG) [14] mechanism to enhance the handling of temporal relationships in multivariate time series within the state space.

Assumes 5

  • The market portfolio (CRSP index) is observable and broadly representative of the investable universe. market
    Source sentence
    This study seeks to fill the above research gaps by examining portfolio allocation between a market index—the CRSP index, serving as a representative proxy for the aggregate U.S. equity market—and a risk-free asset.
  • Investor utility follows a constant relative risk aversion (CRRA) utility function consistent with the Kelly criterion. model
    Source sentence
    To ensure theoretical rigor, the study adopts the Kelly criterion through the use of the constant relative risk aversion (CRRA) utility function and the accumulated reward, providing mathematical consistency between log-utility maximization and long-term capital growth.
  • The reinforcement learning problem is formulated as a Markov Decision Process (MDP). model
    Source sentence
    the optimal two-asset allocation problem as a sequential decision-making task within a Markov Decision Process (MDP).
  • Trading occurs without explicit transaction costs or market impact in the reported results. execution
    Source sentence
    We have deliberately used a simplistic simulation framework.
Show 1 more
  • Leverage can be applied up to 50% (w > 1) to allow borrowing for investment. execution
    Source sentence
    We also introduce the opportunity for Q-learning and DDPG-TiDE strategies to use leverage (i.e., to select values w > 1) to see if they can learn strategies that profit from borrowing to invest.

Data

Backtest sample: 1989 – 2019 · market index · equities · monthly

  • Monthly market return and risk‑free rate series with macro predictors · Fama/French website macro public
    1927-2019 · CRSP market index and risk‑free asset
    Source sentence
    We used a dataset consisting of monthly time series on the market return rate rt (Mkt) and risk‑free asset return rate r f (RF) from 1927 to 2019.

Costs

Not modeled A trading or execution result is reported without costs.

Source sentence
We have deliberately used a simplistic simulation framework.

Finds 3

Reported performance (DDPG with TiDE (continuous-action) asset allocation): Sharpe 1.13 (benchmark 0.95)

  • DDPG‑TiDE without leverage achieves a final portfolio value of 14.85, slightly below buy‑and‑hold (17.77), while with 50% leverage it reaches 21.06, slightly above buy‑and‑hold. performance
    Source sentence
    The final portfolio value for DDPG‑TiDE without leverage (14.85) is slightly below buy‑and‑hold, and the final portfolio value for DDPG‑TiDE with 50% leverage available (21.06) is slightly above buy‑and‑hold.
  • DDPG‑TiDE attains an average Sharpe ratio of 1.13 (no leverage) and 0.99 (with leverage), both modestly higher than the buy‑and‑hold average Sharpe ratio of 0.95. performance
    Source sentence
    ... average Sharpe ratio of 1.13 when no leverage is available, and average Sharpe ratio of 0.99 when leverage is available. Both of these values are slightly higher than the average Sharpe ratio for buy‑and‑hold (0.95).
  • DDPG‑TiDE outperforms the discrete‑action Q‑learning strategy on all evaluated metrics, including portfolio value and risk‑adjusted returns. performance
    Source sentence
    We showed that a continuous‑action DDPG‑TiDE strategy outperforms a simple discrete‑action Q‑learning strategy in all metrics; it is also able to generate more profits than buy‑and‑hold by applying leverage during bull periods; and it generates the highest average Sharpe ratio.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 15 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 1 paper in the archive (1 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (36)

  1. What is the value of the cross-sectional approach to deep reinforcement learning? (2021) — link
  2. A Comprehensive 2022 Look at the Empirical Performance of Equity Premium Prediction (2024) — link
  3. Maximizing Returns with Reinforcement Learning: A Paradigm Shift in Stock Market Portfolio Management (2023) — link
  4. The Sampling Error in Estimates of Mean‐Variance Efficient Portfolio Weights (2002) — link
  5. AlphaPortfolio : Direct construction through deep reinforcement learning and interpretable AI (2020) — link
  6. Long-term Forecasting with TiDE: Time-series Dense Encoder (2023) — link
  7. Recent advances in reinforcement learning in finance (2023) — link
  8. The reinforcement learning Kelly strategy (2022) — link
and 28 more
  1. A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (2017) — link
  2. Optimal Portfolio Choice with Parameter Uncertainty (2007) — link
  3. Adversarial Deep Reinforcement Learning in Portfolio Management (2018) — link
  4. Continuous control with deep reinforcement learning (2015) — link
  5. Dynamic Portfolio Optimization via Augmented DDPG with Quantum Price Levels-Based Trading Strategy (2023) — link
  6. Medium Term Simulations of The Full Kelly and Fractional Kelly Investment Strategies (2010) — link
  7. Long-term capital growth: the good and bad properties of the Kelly and fractional Kelly capital growth criteria (2011) — link
  8. Deep Reinforcement Learning in Agent-Based Financial Market Simulation (2020) — link
  9. Portfolio Selection (1952) — link
  10. Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case (1969) — link
  11. Optimum consumption and portfolio rules in a continuous-time model (1971) — link
  12. An intelligent financial portfolio trading strategy using deep Q -learning (2020) — link
  13. Trading financial indices with reinforcement learning agents (2018) — link
  14. Multi-agent reinforcement learning approach for hedging portfolio problem (2021) — link
  15. Machine learning portfolio allocation (2022) — link
  16. Policy Gradient Methods for Reinforcement Learning with Function Approximation (1999) — link
  17. Deep Parametric Portfolio Policies (2022) — link
  18. Attention is all you need (2017) — link
  19. Research on Portfolio Optimization Based on Deep Reinforcement Learning (2022) — link
  20. Q-learning (1992) — link
  21. Model-based Reinforcement Learning for Predictions and Control for Limit Order Books (2019) — link
  22. Practical Deep Reinforcement Learning Approach for Stock Trading (2018) — link
  23. Model-based Deep Reinforcement Learning for Dynamic Portfolio Optimization (2019) — link
  24. QuantFactor REINFORCE: Mining Steady Formulaic Alpha Factors with Variance-bounded REINFORCE — → 2409.05144
  25. A Universal End-to-End Approach to Portfolio Optimization via Deep Learning (2021) — link
  26. Artificial intelligence for collective intelligence: A national-scale research strategy (2024) — link
  27. Artificial intelligence for collective intelligence: A national-scale research strategy (2024) — link
  28. Agent-based model exploration of latency arbitrage in fragmented financial markets (2018) — link

1 of 36 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.