Optimal portfolio and consumption under no-shorting constraints: The exploratory formulation and reinforcement learning

Paper: arXiv 2611.00700

Authors: Xun Li, Xiang Yu, Fengyi Yuan, Yijie Zeng

Abstract

We develop a reinforcement-learning framework for the continuous-time optimal portfolio-consumption via utility maximization under no-shorting portfolio constraints. We adopt the exploratory formulation under entropy regularization and impose no-shorting constraint through a componentwise nonnegative conditional-mean constraint on the class of admissible randomized policies, while the sampled actions are allowed to be negative during the training. For CRRA utility, we derive the homothetic value function and the optimal Gaussian portfolio and Gibbs consumption policies under some model conditions and establish a verification theorem. These results allow compatible parameterizations in actor, critic, and $q$ functions for a constrained policy-iteration algorithm. Policy evaluation uses trajectory-based moment conditions, and policy improvement uses fitted coefficients to solve a convex quadratic program over the nonnegative orthant without requiring market coefficients as inputs. The mean constraint preserves tractable Gaussian exploration throughout parameter learning, while modal deployment converts the learned randomized policies into deterministic controls. In particular, the Gaussian mode coincides with its constrained mean and hence satisfies no-shorting constraint, and the Gibbs mode determines the deployed consumption rule.

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper presents a highly mathematical framework for continuous-time optimal portfolio-consumption using reinforcement learning under no-shorting constraints. While it includes numerical experiments, the core contribution is theoretical and methodological, demonstrating strong novelty in its approach to handling constraints within an RL framework.

Research Flowchart

  flowchart TD
    A[Research Goal: Optimal Portfolio-Consumption under No-Shorting Constraints] --> B{Methodology: Reinforcement Learning Framework};
    B --> C[Exploratory Formulation: Entropy Regularization & Nonnegative Conditional-Mean Constraint];
    C --> D{Computational Process: Constrained Policy-Iteration Algorithm};
    D --> E[Inputs: CRRA Utility, Trajectory-Based Moment Conditions];
    E --> F{Outputs: Homothetic Value Function, Optimal Gaussian Portfolio & Gibbs Consumption Policies};
    F --> G[Key Finding: Tractable Gaussian Exploration & Mode Deployment for Deterministic Controls];