Paper: arXiv 2412.16175

Authors: Yilie Huang, Yanwei Jia, Xun Yu Zhou

Abstract

We study continuous-time mean–variance portfolio selection in markets where stock prices are diffusion processes driven by observable factors that are also diffusion processes, yet the coefficients of these processes are unknown. Based on the recently developed reinforcement learning (RL) theory for diffusion processes, we present a general data-driven RL algorithm that learns the pre-committed investment strategy directly without attempting to learn or estimate the market coefficients. For multi-stock Black–Scholes markets without factors, we further devise a baseline algorithm and prove its performance guarantee by deriving a sublinear regret bound in terms of the Sharpe ratio. For performance enhancement and practical implementation, we modify the baseline algorithm and carry out an extensive empirical study to compare its performance, in terms of a host of common metrics, with a large number of widely employed portfolio allocation strategies on S&P 500 constituents. The results demonstrate that the proposed continuous-time RL strategy is consistently among the best, especially in a volatile bear market, and decisively outperforms the model-based continuous-time counterparts by significant margins.

Complexity vs Empirical Score

  • Math Complexity: 8.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper employs advanced continuous-time stochastic analysis and proves sublinear regret bounds, indicating high mathematical density. It also features an extensive empirical study with real S&P 500 data, comparing multiple strategies across various metrics, demonstrating substantial implementation and backtesting rigor.

Research Flowchart

  flowchart TD
  A["Research Goal:<br>Mean-Variance Portfolio<br>Selection in Unknown Markets"]
  
  B["Methodology:<br>Continuous-Time<br>Reinforcement Learning RL"]
  
  C["Inputs:<br>S&P 500 Constituents<br>Real Market Data"]
  
  D["Computational Process:<br>Direct Strategy Learning<br>No Model Estimation"]
  
  E["Key Findings:<br>Sublinear Regret Bound<br>& Superior Sharpe Ratio"]
  
  F["Outcomes:<br>Consistent Top Performance<br>Especially in Bear Markets"]
  
  A --> B
  B --> C
  C --> D
  D --> E
  E --> F

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes a data‑driven continuous‑time reinforcement‑learning approach for mean‑variance portfolio selection, including a specific algorithm for multi‑stock Black‑Scholes markets with a sublinear regret guarantee.

Source sentence
Based on the recently developed reinforcement learning (RL) theory for diffusion processes, we present a general data‑driven RL approach that learns the pre‑committed investment strategy directly without attempting to learn or estimate the market coefficients. For multi‑stock Black‑Scholes markets without factors, we f

Assumes 6

  • Stock prices and observable factors follow diffusion processes with unknown coefficients. market
    Source sentence
    stock prices are diffusion processes driven by observable factors that are also diffusion processes, yet the coefficients of these processes are unknown.
  • In the algorithmic analysis, the market is a frictionless multi‑stock Black‑Scholes market without any factor F. model
    Source sentence
    we present an RL algorithm for a frictionless, multi‑stock Black–Scholes market without any factor F, i.e., the stock prices follow multi‑dimensional geometric Brownian motions.
  • No portfolio constraints such as no‑shorting or leverage limits are imposed in the empirical comparison. execution
    Source sentence
    we compare all the methods without imposing additional portfolio constraints (such as no‑shorting or a leverage constraint).
  • Daily returns are computed from dividend‑adjusted closing prices. data
    Source sentence
    Daily returns are computed based on dividend‑adjusted closing prices.
Show 2 more
  • Transaction cost analysis assumes a flat 50 bps proportional cost per trade. other
    Source sentence
    recompute all the performance metrics assuming a 50 bps proportional transaction cost as in DeMiguel et al. (2009b).
  • The regret analysis is derived specifically for the Black‑Scholes environment and does not cover stochastic volatility or jumps. other
    Source sentence
    we are able to derive a sublinear regret only for the Black–Scholes environment, without considering more stylized facts of asset returns such as stochastic volatility and jumps.

Data

Backtest sample: 2000 – 2020 · S&P 500 constituents · equities · daily

  • Wharton Research Data Services and Yahoo Finance price data · Wharton Research Data Services; Yahoo Finance market-prices commercial
    1990-01 to 2020-01 · S&P 500 constituents, first 300 stocks alphabetically
    Source sentence
    The dataset used in this study is obtained from the Wharton Research Data Services and Yahoo Finance. Our asset universe consists of stocks that were constituents of the S&P 500 index and remained continuously listed with available daily trading data from January 1, 1990 to January 1, 2020. From this universe, we const

Recognised sources: Yahoo Finance

Costs

Modeled Costs are part of the reported headline results.

A flat 50 bps proportional transaction cost is applied to all trades when reporting net‑of‑cost performance.

Source sentence
recompute all the performance metrics assuming a 50 bps proportional transaction cost as in DeMiguel et al. (2009b).

Finds 5

Reported performance (CTRL): Sharpe 0.23 (benchmark 0.31) · annual return 12.2% · max drawdown 62.1%

  • CTRL achieves the highest average annualized return (12.211%) and Sharpe ratio (0.531) among all 14 methods over 2000‑2020. performance
    Source sentence
    CTRL 12.211% (0.219%) 0.232 (0.003) 0.531 (0.010) 0.865 (0.017) 0.200 (0.005) 0.621 (0.009) 521 (30)
  • Even after accounting for 50 bps transaction costs, CTRL still leads in return, Sharpe, Calmar ratios and has the shortest recovery time. performance
    Source sentence
    CTRL still achieves the highest average annualized return , Sharpe ratio, Calmar ratio and the shortest recovery time.
  • CTRL exhibits the lowest monthly turnover and most modest position concentrations under transaction costs. performance
    Source sentence
    CTRL continues to have the lowest monthly turnover rates and least concentrated positions. In particular, the maximum long and short positions under CTRL remain below 20% and 2%, respectively.
  • In the bear subperiod (2000‑2010), CTRL attains the highest return (9.788%) and Sharpe ratio (0.367), outperforming the next best method by a larger margin. performance
    Source sentence
    CTRL achieves the highest annualized return (9.788%) and Sharpe ratio (0.367), as well as the best Sortino and Calmar ratios.
  • The proposed algorithm enjoys a provable sublinear regret bound in terms of the Sharpe ratio for the Black‑Scholes market. theoretical
    Source sentence
    we further devise an algorithm and prove its performance guarantee by deriving a sublinear regret bound in terms of the Sharpe ratio.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 32 related papers). Coverage is partial, so this is neither replication nor refutation.

Cited by 7 papers in the archive (5 by independent authors)

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.

Cited References (166)

  1. Who are I : Time inconsistency and intrapersonal conflict and reconciliation (2022)
  2. Data-Driven M erton’s Strategies via Policy Randomization — → 2312.11797
  3. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection (2003)
  4. Optimal versus naive diversification: How inefficient is the 1/N portfolio strategy? (2009)
  5. Portfolio selection (1952)
  6. Bayes- S tein estimation for portfolio analysis (1986)
  7. Common risk factors in the returns on stocks and bonds (1993)
  8. Asset allocation: Combining investor views with market equilibrium (1990)
and 158 more
  1. Distributionally robust mean-variance portfolio selection with Wasserstein distances (2022)
  2. Capital asset prices: A theory of market equilibrium under conditions of risk (1964)
  3. Risk reduction in large portfolios: Why imposing the wrong constraints helps (2003)
  4. Estimation with quadratic loss (1992)
  5. Minimum-variance portfolios in the US equity market (2006)
  6. A simplified model for portfolio analysis (1963)
  7. Individual comparisons by ranking methods (1992)
  8. Policy evaluation and temporal-difference learning in continuous time and space: A martingale approach (2022)
  9. Policy gradient and actor-critic learning in continuous time and space: Theory and algorithms (2022)
  10. Reinforcement learning in continuous time and space (2000)
  11. Reinforcement learning in continuous time and space: A stochastic control approach (2020)
  12. Continuous-time mean–variance portfolio selection: A reinforcement learning framework (2020)
  13. Stochastic Controls: Hamiltonian Systems and HJB Equations (1999)
  14. Controlled Markov Processes and Viscosity Solutions (2006)
  15. Investment Science (1998)
  16. A closed-form solution for options with stochastic volatility with applications to bond and currency options (1993)
  17. Smart “predict, then optimize” (2022)
  18. Optimal dynamic portfolio selection: Multiperiod mean-variance formulation (2000)
  19. Continuous-time mean-variance portfolio selection: A stochastic LQ framework (2000)
  20. Portfolio selection in stochastic environments (2007)
  21. Markowitz’s mean-variance portfolio selection with regime switching: A continuous-time model (2003)
  22. Numerical solution of the Hamilton–Jacobi–Bellman formulation for continuous-time mean–variance asset allocation under stochastic volatility (2016)
  23. Mean–variance portfolio selection of cointegrated assets (2011)
  24. Mean-variance portfolio selection with random parameters in a complete market (2002)
  25. A dynamic mean-variance analysis for log returns (2021)
  26. Dynamic mean-variance asset allocation (2010)
  27. Portfolio and consumption decisions under mean-reverting returns: An exact solution for complete markets (2002)
  28. On estimating the expected return on the market: An exploratory investigation (1980)
  29. On the sensitivity of mean-variance-efficient portfolios to changes in asset means: Some analytical and computational results (1991)
  30. The sampling error in estimates of mean-variance efficient portfolio weights (1999)
  31. On portfolio optimization: Forecasting covariances and choosing the risk model (1999)
  32. A generalized approach to portfolio optimization: Improving performance by constraining portfolio norms (2009)
  33. The effect of errors in means, variances, and covariances on optimal portfolio choice (2013)
  34. Curse of optimality, and how to break it? (2023)
  35. Mean–Variance Optimization for Asset Allocation (2021)
  36. On the existence of optimal relaxed controls of stochastic partial differential equations (1992)
  37. On stochastic relaxed control for partially observed diffusions (1984)
  38. Learning equilibrium mean-variance strategy (2023)
  39. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor (2018)
  40. Maximum entropy inverse reinforcement learning. (2008)
  41. Sensitivity analysis for mean-variance portfolio problems (1991)
  42. Dynamic Nonmyopic Portfolio Behavior (1996)
  43. Consumption and portfolio decisions when expected returns are time varying (1999)
  44. Mastering the game of Go without human knowledge (2017)
  45. Human-level control through deep reinforcement learning (2015)
  46. End-to-end training of deep visuomotor policies (2016)
  47. A survey on policy search for robotics (2013)
  48. Estimating Portfolio and Consumption Choice: A Conditional E uler Equations Approach (1999)
  49. Vast portfolio selection with gross-exposure constraints (2012)
  50. Variable Selection for Portfolio Choice (2001)
  51. Strategic Asset Allocation: Portfolio Choice for Long-Term Investors (2002)
  52. Continuous-time Markowitz’s model with transaction costs (2010)
  53. Continuous-time mean-variance efficiency: the 80% rule (2006)
  54. A theory of Markovian time-inconsistent stochastic control in discrete time (2014)
  55. Mean–variance portfolio optimization with state-dependent risk aversion (2014)
  56. Myopia and inconsistency in dynamic utility maximization (1955)
  57. Being serious about non-commitment: Subgame perfect equilibrium in continuous time (2006)
  58. Time-consistent portfolio management (2012)
  59. Consumption and portfolio rules for time-inconsistent investors (2010)
  60. Investment and consumption without commitment (2008)
  61. The golden rule when preferences are time inconsistent (2010)
  62. Time-inconsistent stochastic linear-quadratic control: characterization and uniqueness of equilibrium (2017)
  63. Time-inconsistent stochastic linear–quadratic control (2012)
  64. Time-inconsistent recursive stochastic optimal control problems (2017)
  65. Time-inconsistent optimal control problems and the equilibrium HJB equation (2012) — link
  66. Mean-variance portfolio selection with dynamic targets for expected terminal wealth (2022)
  67. Portfolio analysis under uncertain means, variances, and covariances (1974)
  68. Inadmissibility of the usual estimator for the mean of a multivariate normal distribution (1956)
  69. Portfolio selection with parameter and model uncertainty: A multi-prior approach (2007)
  70. Robust portfolio selection problems (2003)
  71. Maxmin expected utility with a non-unique prior (1989)
  72. Portfolio diversification and model uncertainty: A robust dynamic mean-variance approach (2022)
  73. Robust Markowitz mean-variance portfolio selection under ambiguous covariance matrix (2019)
  74. Machine learning and portfolio optimization (2018)
  75. Robust satisficing (2023)
  76. Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets Goldilocks (2017)
  77. High dimensional covariance matrix estimation using a factor model (2008)
  78. FinBrain: when finance meets AI 2.0 (2019)
  79. Portfolio management using reinforcement learning (2016)
  80. An algorithm for trading and portfolio management using Q-learning and Sharpe ratio maximization (2000)
  81. Optimal asset allocation using adaptive dynamic programming (1996)
  82. Machine learning for trading (2017)
  83. Algorithmic aspects of mean–variance optimization in Markov decision processes (2013)
  84. The variance of discounted Markov decision processes (1982)
  85. TD algorithm for the variance of return and mean-variance reinforcement learning (2001)
  86. Variance-penalized reinforcement learning for risk-averse asset allocation (2000)
  87. Variance adjusted actor critic algorithms (2013) — link
  88. Variance-constrained actor-critic algorithms for discounted and average reward MDPs (2016)
  89. Actor-critic algorithms for risk-sensitive MDPs (2013)
  90. Reinforcement Learning: An Introduction (2018)
  91. A stochastic approximation algorithm with varying bounds (1995)
  92. General bounds and finite-time improvement for the Kiefer-Wolfowitz stochastic approximation algorithm (2011)
  93. A stochastic approximation method (1951)
  94. Stochastic Approximation (2003)
  95. Stochastic Approximation: A Dynamical Systems Viewpoint (2008) — link
  96. An overview of stochastic approximation (2014)
  97. Generalized instrumental variables estimation of nonlinear rational expectations models (1982)
  98. A practical end-to-end inventory management model with deep learning (2023)
  99. Risk parity and diversification (2011)
  100. Achieving mean–variance efficiency by continuous-time reinforcement learning (2022)
  101. On the relation between several discrete choice models (2017)
  102. Empirical asset pricing via machine learning (2020)
  103. Autoencoder asset pricing models (2021)
  104. Machine learning in the Chinese stock market (2022)
  105. Deep learning in asset pricing (2024)
  106. Bond risk premiums with machine learning (2021)
  107. (Re-) Imag (in) ing price trends (2020)
  108. Deep learning statistical arbitrage (2025)
  109. A convergence theorem for non negative almost supermartingales and some applications (1971)
  110. Numerical Solution of Stochastic Differential Equations (1992)
  111. Continuous control with deep reinforcement learning (2016) — link
  112. Proximal Policy Optimization Algorithms (2017) — link
  113. High-Dimensional Continuous Control Using Generalized Advantage Estimation (2016) — link
  114. Factors that fit the time series and cross-section of stock returns (2020)
  115. Conditional choice probabilities and the estimation of dynamic models (1993)
  116. Stochastic choice and revealed perturbed utility (2015)
  117. Machine learning in asset management—part 2: Portfolio construction—weight optimization (2020)
  118. Machine Learning for Active Portfolio Management (2021)
  119. Dynamic portfolio choice with parameter uncertainty and the economic value of analysts’ recommendations (2006)
  120. Optimal portfolio choice under incomplete information (1986)
  121. Empirical asset pricing: The cross section of stock returns (2016)
  122. The cross-section of expected stock returns (2015)
  123. Presidential address: Discount rates (2011)
  124. Evidence of predictable behavior of security returns (1990)
  125. Fads, martingales, and market efficiency (1990)
  126. Returns to buying winners and selling losers: Implications for stock market efficiency (1993)
  127. Predicting excess stock returns out of sample: Can anything beat the historical average? (2008)
  128. Policy gradient in continuous time (2006)
  129. Making deep Q -learning methods robust to time discretization (2019)
  130. Time discretization-invariant safe action repetition for policy gradient methods (2021)
  131. The properties of equally weighted risk contribution portfolios (2010)
  132. Reinforcement learning for jump-diffusions, with financial applications — → 2405.16449
  133. Estimating diffusions with discretely and possibly randomly spaced data: A general theory (2004)
  134. Likelihood inference for diffusions: A survey (2006)
  135. Maximum likelihood estimation of stochastic volatility models (2007)
  136. Maximum likelihood estimation of discretely sampled diffusions: A closed-form approximation approach (2002)
  137. Back to the Future: Generating Moment Implications for Continuous-Time Markov Processes (1995)
  138. Maximum likelihood estimation of generalized It ^o processes with discretely sampled data (1988)
  139. MCMC analysis of diffusion models with application to finance (2001)
  140. Estimating equations based on eigenfunctions for a discretely observed diffusion process (1999)
  141. Closed-Form Likelihood Expansions for Multivariate Diffusions (2008)
  142. A nonparametric model of term structure dynamics and the market price of interest rate risk (1997)
  143. The limits to learning a diffusion model (2021)
  144. Accuracy of discretely sampled stochastic policies in continuous-time reinforcement learning (2026)
  145. Regret of exploratory policy improvement and $q$-learning (2024) — link
  146. Square-root regret bounds for continuous-time episodic M arkov decision processes (2026)
  147. Unifying PAC and regret: Uniform PAC bounds for episodic reinforcement learning (2017)
  148. Optimal scheduling of entropy regularizer for continuous-time linear-quadratic reinforcement learning (2024)
  149. Optimistic Q -learning for average reward and episodic reinforcement learning (2024) — link
  150. Is Q -learning provably efficient? (2018)
  151. Breaking the sample complexity barrier to regret-optimal model-free reinforcement learning (2021)
  152. Provably efficient reinforcement learning with linear function approximation (2023)
  153. Risk-sensitive reinforcement learning: Near-optimal risk-sample tradeoff in regret (2020)
  154. Exponential B ellman equation and improved regret bounds for risk-sensitive reinforcement learning (2021)
  155. Distributionally Robust $ Q $-Learning (2022)
  156. A finite sample complexity bound for distributionally robust Q -learning (2023)
  157. Regret bounds for episodic risk-sensitive linear quadratic regulator (2025)
  158. Regret bounds for M arkov decision processes with recursive optimized certainty equivalents (2023)

2 of 166 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.