Dynamic Programming

Optimal annuitization with labor income under age-dependent force of mortality ArXiv ID: 2510.10371 “View on arXiv” Authors: Criscent Birungi, Cody Hyndman Abstract We consider the problem of optimal annuitization with labour income, where an agent aims to maximize utility from consumption and labour income under age-dependent force of mortality. Using a dynamic programming approach, we derive closed-form solutions for the value function and the optimal consumption, portfolio, and labor supply strategies. Our results show that before retirement, investment behavior increases with wealth until a threshold set by labor supply. After retirement, agents tend to consume a larger portion of their wealth. Two main factors influence optimal annuitization decisions as people get older. First, the agent’s perspective (demand side); the agent’s personal discount rate rises with age, reducing their desire to annuitize. Second, the insurer’s perspective (supply side); insurers offer higher payout rates (mortality credits). Our model demonstrates that beyond a certain age, sharply declining survival probabilities make annuitization substantially optimal, as the powerful incentive of mortality credits outweighs the agent’s high personal discount rate. Finally, post-retirement labor income serves as a direct substitute for annuitization by providing an alternative stable income source. It enhances the financial security of retirees. ...

Optimal Consumption-Investment with Epstein-Zin Utility under Leverage Constraint ArXiv ID: 2509.21929 “View on arXiv” Authors: Dejian Tian, Weidong Tian, Jianjun Zhou, Zimu Zhu Abstract We study optimal portfolio choice under Epstein-Zin recursive utility in the presence of general leverage constraints. We first establish that the optimal value function is the unique viscosity solution to the associated Hamilton-Jacobi-Bellman (HJB) equation, by developing a new dynamic programming principle under constraints. We further demonstrate that the value function admits smoothness and characterize the optimal consumption and investment strategies. In addition, we derive explicit solutions for the optimal strategy and explicitly delineate the constrained and unconstrained regions in several special cases of the leverage constraint. Finally, we conduct a comparative analysis, highlighting the differences relative to the classical time-separable preferences and to the setting without leverage constraints. ...

Error Propagation in Dynamic Programming: From Stochastic Control to Option Pricing ArXiv ID: 2509.20239 “View on arXiv” Authors: Andrea Della Vecchia, Damir Filipović Abstract This paper investigates theoretical and methodological foundations for stochastic optimal control (SOC) in discrete time. We start formulating the control problem in a general dynamic programming framework, introducing the mathematical structure needed for a detailed convergence analysis. The associate value function is estimated through a sequence of approximations combining nonparametric regression methods and Monte Carlo subsampling. The regression step is performed within reproducing kernel Hilbert spaces (RKHSs), exploiting the classical KRR algorithm, while Monte Carlo sampling methods are introduced to estimate the continuation value. To assess the accuracy of our value function estimator, we propose a natural error decomposition and rigorously control the resulting error terms at each time step. We then analyze how this error propagates backward in time-from maturity to the initial stage-a relatively underexplored aspect of the SOC literature. Finally, we illustrate how our analysis naturally applies to a key financial application: the pricing of American options. ...

Dynamic Inverse Optimization under Drift and Shocks: Theory, Regret Bounds, and Applications ArXiv ID: 2509.14080 “View on arXiv” Authors: JINHO CHA Abstract The growing prevalence of drift and shocks in modern decision environments exposes a gap between classical optimization theory and real-world practice. Standard models assume fixed objectives, yet organizations from hospitals to power grids routinely adapt to shifting priorities, noisy data, and abrupt disruptions. To address this gap, this study develops a dynamic inverse optimization framework that recovers hidden, time-varying preferences from observed allocation trajectories. The framework unifies identifiability analysis with regret guarantees conditions are established for existence and uniqueness of recovered parameters, and sharp static and dynamic regret bounds are derived to characterize responsiveness to gradual drift and sudden shocks. Methodologically, a drift-aware estimator grounded in convex analysis and online learning theory is introduced, with finite-sample guarantees on recovery accuracy. Computational experiments in healthcare, energy, logistics, and finance reveal heterogeneous recovery patterns, ranging from rapid resilience to persistent vulnerability. Overall, dynamic inverse optimization emerges as both a theoretical contribution and a broadly applicable diagnostic tool for benchmarking resilience, uncovering hidden behavioral shifts, and guiding policy interventions in complex stochastic systems. ...

Two Stochastic Control Methods for Mean-Variance Portfolio Selection of Jump Diffusions and Their Relationship ArXiv ID: 2508.01138 “View on arXiv” Authors: Qiyue Zhang, Jingtao Shi Abstract This paper is concerned with the maximum principle and dynamic programming principle for mean-variance portfolio selection of jump diffusions and their relationship. First, the optimal portfolio and efficient frontier of the problem are obtained using both methods. Furthermore, the relationship between these two methods is investigated. Specially, the connections between the adjoint processes and value function are given. ...

Variable annuities: A closer look at ratchet guarantees, hybrid contract designs, and taxation ArXiv ID: 2507.07358 “View on arXiv” Authors: Jennifer Alonso-Garcia, Len Patrick Dominic M. Garces, Jonathan Ziveyi Abstract This paper investigates optimal withdrawal strategies and behavior of policyholders in a variable annuity (VA) contract with a guaranteed minimum withdrawal benefit (GMWB) rider incorporating taxation and a ratchet mechanism for enhancing the benefit base during the life of the contract. Mathematically, this is accomplished by solving a backward dynamic programming problem associated with optimizing the discounted risk-neutral expectation of cash flows from the contract. Furthermore, reflecting traded VA contracts in the market, we consider hybrid products providing policyholders access to a cash fund which functions as an intermediate repository of earnings from the VA and earns interest at a contractually specified cash rate. We contribute to the literature by revealing several significant interactions among taxation, the cash fund, and the benefit base update mechanism. When tax rates are high, the tax-shielding effect of the cash fund, which is taxed differently from ordinary withdrawals from the VA, plays a significant role in enhancing the attractiveness of the overall contract. Furthermore, the ratchet benefit base update scheme (in contrast to the ubiquitous return-of-premium specification in the literature) tends to discourage early surrender as it provides enhanced downside market risk protection. In addition, the cash fund discourages active withdrawals, with policyholders preferring to transfer the guaranteed withdrawal amount to the cash fund to leverage the cash fund rate. ...

Maximizing Battery Storage Profits via High-Frequency Intraday Trading ArXiv ID: 2504.06932 “View on arXiv” Authors: Unknown Abstract Maximizing revenue for grid-scale battery energy storage systems in continuous intraday electricity markets requires strategies that are able to seize trading opportunities as soon as new information arrives. This paper introduces and evaluates an automated high-frequency trading strategy for battery energy storage systems trading on the intraday market for power while explicitly considering the dynamics of the limit order book, market rules, and technical parameters. The standard rolling intrinsic strategy is adapted for continuous intraday electricity markets and solved using a dynamic programming approximation that is two to three orders of magnitude faster than an exact mixed-integer linear programming solution. A detailed backtest over a full year of German order book data demonstrates that the proposed dynamic programming formulation does not reduce trading profits and enables the policy to react to every relevant order book update, enabling realistic rapid backtesting. Our results show the significant revenue potential of high-frequency trading: our policy earns 58% more than when re-optimizing only once every hour and 14% more than when re-optimizing once per minute, highlighting that profits critically depend on trading speed. Furthermore, we leverage the speed of our algorithm to train a parametric extension of the rolling intrinsic, increasing yearly revenue by 8.4% out of sample. ...

Sequential Portfolio Selection under Latent Side Information-Dependence Structure: Optimality and Universal Learning Algorithms ArXiv ID: 2501.06701 “View on arXiv” Authors: Unknown Abstract This paper investigates the investment problem of constructing an optimal no-short sequential portfolio strategy in a market with a latent dependence structure between asset prices and partly unobservable side information, which is often high-dimensional. The results demonstrate that a dynamic strategy, which forms a portfolio based on perfect knowledge of the dependence structure and full market information over time, may not grow at a higher rate infinitely often than a constant strategy, which remains invariant over time. Specifically, if the market is stationary, implying that the dependence structure is statistically stable, the growth rate of an optimal dynamic strategy, utilizing the maximum capacity of the entire market information, almost surely decays over time into an equilibrium state, asymptotically converging to the growth rate of a constant strategy. Technically, this work reassesses the common belief that a constant strategy only attains the optimal limiting growth rate of dynamic strategies when the market process is identically and independently distributed. By analyzing the dynamic log-optimal portfolio strategy as the optimal benchmark in a stationary market with side information, we show that a random optimal constant strategy almost surely exists, even when a limiting growth rate for the dynamic strategy does not. Consequently, two approaches to learning algorithms for portfolio construction are discussed, demonstrating the safety of removing side information from the learning process while still guaranteeing an asymptotic growth rate comparable to that of the optimal dynamic strategy. ...

Exploratory Randomization for Discrete-Time Linear Exponential Quadratic Gaussian (LEQG) Problem ArXiv ID: 2501.06275 “View on arXiv” Authors: Unknown Abstract We investigate exploratory randomization for an extended linear-exponential-quadratic-Gaussian (LEQG) control problem in discrete time. This extended control problem is related to the structure of risk-sensitive investment management applications. We introduce exploration through a randomization of the control. Next, we apply the duality between free energy and relative entropy to reduce the LEQG problem to an equivalent risk-neutral LQG control problem with an entropy regularization term, see, e.g. Dai Pra et al. (1996), for which we present a solution approach based on Dynamic Programming. Our approach, based on the energy-entropy duality may also be considered as leading to a justification for the use, in the literature, of an entropy regularization when applying a randomized control. ...

Battery valuation on electricity intraday markets with liquidity costs ArXiv ID: 2412.15959 “View on arXiv” Authors: Unknown Abstract In this paper, we propose a complete modelling framework to value several batteries in the electricity intraday market at the trading session scale. The model consists of a stochastic model for the 24 mid-prices (one price per delivery hour) combined with a deterministic model for the liquidity costs (representing the cost of going deeper in the order book). A stochastic optimisation framework based on dynamic programming is used to calculate the value of the batteries. We carry out a back test for the years 2021, 2022 and 2023 for the German market and for the French market. We show that it is essential to take liquidity into account, especially when the number of batteries is large: it allows much higher profits and avoids high losses using our liquidity model. The use of our stochastic model for the mid-price also significantly improves the results (compared to a deterministic framework where the mid-price forecast is the spot price). ...