Paper: arXiv 2505.07537

Authors: Yu Li, Yuhan Wu, Shuhua Zhang

Abstract

In this paper, we study the continuous-time multi-asset mean-variance (MV) portfolio selection using a reinforcement learning (RL) algorithm, specifically the soft actor-critic (SAC) algorithm, in the time-varying financial market. A family of Gaussian portfolio selections is derived, and a policy iteration process is crafted to learn the optimal exploratory portfolio selection. We prove the convergence of the policy iteration process theoretically, based on which the SAC algorithm is developed. To improve the algorithm’s stability and the learning accuracy in the multi-asset scenario, we divide the model parameters that influence the optimal portfolio selection into three parts, and learn each part progressively. Numerical studies in the simulated and real financial markets confirm the superior performance of the proposed SAC algorithm under various criteria.

Complexity vs Empirical Score

  • Math Complexity: 8.0/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides strong theoretical foundations by proving the convergence of a policy iteration procedure for continuous-time exploratory multi-asset mean-variance portfolio selection. It effectively addresses high-dimensionality challenges in Soft Actor-Critic (SAC) reinforcement learning algorithms through parameter decoupling and verifies its approach via extensive simulation and real-world backtesting.

Research Flowchart

  flowchart TD
    A[Goal: Continuous-Time Multi-Asset Mean-Variance Portfolio Selection via RL] --> B[Formulate Exploratory Framework & Derive Gaussian Policy Family]
    B --> C[Craft Policy Iteration Process & Prove Theoretical Convergence]
    C --> D[Develop SAC Algorithm with Progressive 3-Part Parameter Learning]
    E[(Market Data: Simulated & Real Financial Equities)] --> D
    D --> F[Evaluate SAC Performance Across Various Financial Criteria]
    F --> G[Outcome: Superior Stability, Accuracy, and Portfolio Performance]

Cited References (54)

  1. Machine Learning for Continuous-Time Finance (2024) — link
  2. Entropy-Regularized Mean-Variance Portfolio Optimization with Jumps (2023) — link
  3. Learning equilibrium mean‐variance strategy (2023) — link
  4. Computing Multi-Eigenpairs of High-Dimensional Eigenvalue Problems Using Tensor Neural Networks (2023) — link
  5. Portfolio selection with exploration of new investment assets (2023) — link
  6. The reinforcement learning Kelly strategy (2022) — link
  7. Reinforcement learning with dynamic convex risk measures (2021) — link
  8. A toolkit for exploiting contemporaneous stock correlations (2021) — link
and 46 more
  1. The Sharpe Ratio (2021) — link
  2. Optimal pairs trading with dynamic mean-variance objective (2021) — link
  3. Improving Minimum-Variance Portfolios by Alleviating Overdispersion of Eigenvalues (2020) — link
  4. Entropy Regularization for Mean Field Games with Learning (2020) — link
  5. ON TIME CONSISTENCY FOR MEAN-VARIANCE PORTFOLIO SELECTION (2020) — link
  6. Multiperiod Mean Conditional Value at Risk Asset Allocation: Is It Advantageous to Be Time Consistent? (2020) — link
  7. Continuous‐time mean–variance portfolio selection: A reinforcement learning framework (2019) — link
  8. Portfolio selection in a multi-asset, incomplete-market economy (2019) — link
  9. Soft Actor-Critic Algorithms and Applications (2018) — link
  10. Exploration versus exploitation in reinforcement learning: a stochastic control approach (2018) — link
  11. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor (2018) — link
  12. Nonlinear Shrinkage of the Covariance Matrix for Portfolio Selection: Markowitz Meets Goldilocks (2017) — link
  13. Improving Policy Gradient by Exploring Under-appreciated Rewards (2016) — link
  14. Asynchronous Methods for Deep Reinforcement Learning (2016) — link
  15. A First Course in the Calculus of Variations (2014) — link
  16. MEAN–VARIANCE PORTFOLIO OPTIMIZATION WITH STATE‐DEPENDENT RISK AVERSION (2014) — link
  17. Condition Number Regularized Covariance Estimation (2012) — link
  18. Time-consistent mean-variance portfolio selection in discrete and continuous time (2012) — link
  19. It’s All in the Timing: Simple Active Portfolio Strategies that Outperform Naïve Diversification (2012) — link
  20. Mean–Variance Hedging (2010) — link
  21. Numerical solution of the Hamilton-Jacobi-Bellman formulation for continuous time mean variance asset allocation (2010) — link
  22. Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy? (2009) — link
  23. What is dynamic programming? (2004) — link
  24. A well-conditioned estimator for large-dimensional covariance matrices (2004) — link
  25. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection (2003) — link
  26. Optimal Dynamic Portfolio Selection: Multiperiod Mean‐Variance Formulation (2000) — link
  27. Continuous-Time Mean-Variance Portfolio Selection: A Stochastic LQ Framework (2000) — link
  28. Transaction costs and predictability: some utility cost calculations (1999) — link
  29. THE ECONOMETRICS OF FINANCIAL MARKETS (1996) — link
  30. Computing efficient frontiers using estimated parameters (1993) — link
  31. On the Sensitivity of Mean-Variance-Efficient Portfolios to Changes in Asset Means: Some Analytical and Computational Results (1991) — link
  32. Stochastic differential equations : an introduction with applications (1987) — link
  33. Bayes-Stein Estimation for Portfolio Analysis (1986) — link
  34. Time to Build and Aggregate Fluctuations (1982) — link
  35. Putting Markowitz theory to work (1981) — link
  36. Portfolio Selection (1971) — link
  37. Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case (1969) — link
  38. The optimization of a quadratic function subject to linear constraints (1956) — link
  39. Lest We Forget: Learn from Out-of-Sample Forecast Errors When Optimizing Portfolios (2022) — link
  40. Sampling Error and Double Shrinkage Estimation of Minimum Variance Portfolios (2011) — link
  41. Reinforcement Learning: An Introduction (2005) — link
  42. A nonparametric approach to pricing and hedging derivative securities via learning networks (1994)
  43. Calculus of variations and optimal control theory (1993) — link
  44. An analytical derivation of the efficient portfolio frontier (1972)
  45. Centre De Referència En Economia Analítica Barcelona Economics Working Paper Series Working Paper Nº 92 Honey, I Shrunk the Sample Covariance Matrix Honey, I Shrunk the Sample Covariance Matrix
  46. N (cid:16) ( e R T 0 A ( s ) ds + 2 γw o 2 γ − w )Σ − 1 ( µ − r ) , λ 2 e K ( t,T ) · ( T − t ) γ Σ − 1 (cid:17) , lim n → + ∞ V P n ( t, w ) becomes the optimal value function in (16)

0 of 54 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.