Dynamic Programming for Value-at-risk Markov Decision Processes in Continuous Spaces

Paper: arXiv 2610.24439

Authors: Xian Chen, Erick Delage, Weikai Wang

Abstract

Value-at-Risk (VaR) is a widely adopted quantile-based risk measure in practice, yet incorporating VaR objectives into Markov decision processes (VaR MDPs) remains a longstanding challenge. Recent work has derived dynamic programming equations for VaR MDPs in finite state and action spaces but left the case of more general spaces unresolved. This paper establishes, for the first time, the validity of these equations in continuous spaces, for both finite- and infinite-horizon settings. Such cases naturally arise in many real-world dynamic decision-making problems. As an application, we study an optimal execution problem with a VaR objective and demonstrate that our framework enables the characterization of optimal dynamic strategies, resolving an open question in the field.

Complexity vs Empirical Score

• Math Complexity: 9.0/10
• Empirical Rigor: 6.0/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper presents a significant theoretical advancement by extending VaR MDPs to continuous spaces, a long-standing open problem. The application to optimal execution demonstrates practical relevance, though the empirical section could benefit from more extensive real-world data analysis.

Research Flowchart

  flowchart TD
    A[Research Goal: Validate VaR MDP Dynamic Programming in Continuous Spaces] --> B{Methodology: Extend DP Equations to Continuous Spaces};
    B --> C{Inputs: VaR MDP Framework, Continuous State/Action Spaces};
    C --> D[Computational Process: Derivation & Proof for Finite/Infinite Horizon];
    D --> E[Application: Optimal Execution Problem with VaR Objective];
    E --> F[Key Outcome: VaR MDP DP Equations Validated in Continuous Spaces];
    F --> G[Key Outcome: Optimal Dynamic Strategies Characterized for Optimal Execution];