Stochastic Control

Convergence of the deep BSDE method for stochastic control problems formulated through the stochastic maximum principle ArXiv ID: 2401.17472 “View on arXiv” Authors: Unknown Abstract It is well-known that decision-making problems from stochastic control can be formulated by means of a forward-backward stochastic differential equation (FBSDE). Recently, the authors of Ji et al. 2022 proposed an efficient deep learning algorithm based on the stochastic maximum principle (SMP). In this paper, we provide a convergence result for this deep SMP-BSDE algorithm and compare its performance with other existing methods. In particular, by adopting a strategy as in Han and Long 2020, we derive a-posteriori estimate, and show that the total approximation error can be bounded by the value of the loss functional and the discretization error. We present numerical examples for high-dimensional stochastic control problems, both in case of drift- and diffusion control, which showcase superior performance compared to existing algorithms. ...

Optimal dividend payout with path-dependent drawdown constraint ArXiv ID: 2312.01668 “View on arXiv” Authors: Unknown Abstract This paper studies an optimal dividend problem with a drawdown constraint in a Brownian motion model, requiring the dividend payout rate to remain above a fixed proportion of its historical maximum. This leads to a path-dependent stochastic control problem, as the admissible control depends on its own past values. The associated Hamilton-Jacobi-Bellman (HJB) equation is a novel two-dimensional variational inequality with a gradient constraint, a type of problem previously only analyzed in the literature using viscosity solution techniques. In contrast, this paper employs delicate PDE methods to establish the existence of a strong solution. This stronger regularity allows us to explicitly characterize an optimal feedback control strategy, expressed in terms of two free boundaries and the running maximum surplus process. Furthermore, we derive key properties of the value function and the free boundaries, including boundedness and continuity. Numerical examples are provided to verify the theoretical results and to offer new financial insights. ...

Optimal Retirement Choice under Age-dependent Force of Mortality ArXiv ID: 2311.12169 “View on arXiv” Authors: Unknown Abstract This paper examines the retirement decision, optimal investment, and consumption strategies under an age-dependent force of mortality. We formulate the optimization problem as a combined stochastic control and optimal stopping problem with a random time horizon, featuring three state variables: wealth, labor income, and force of mortality. To address this problem, we transform it into its dual form, which is a finite time horizon, three-dimensional degenerate optimal stopping problem with interconnected dynamics. We establish the existence of an optimal retirement boundary that splits the state space into continuation and stopping regions. Regularity of the optimal stopping value function is derived and the boundary is proved to be Lipschitz continuous, and it is characterized as the unique solution to a nonlinear integral equation, which we compute numerically. In the original coordinates, the agent thus retires whenever her wealth exceeds an age-, labor income- and mortality-dependent transformed version of the optimal stopping boundary. We also provide numerical illustrations of the optimal strategies, including the sensitivities of the optimal retirement boundary concerning the relevant model’s parameters. ...

Optimal dividend strategies for a catastrophe insurer ArXiv ID: 2311.05781 “View on arXiv” Authors: Unknown Abstract In this paper we study the problem of optimally paying out dividends from an insurance portfolio, when the criterion is to maximize the expected discounted dividends over the lifetime of the company and the portfolio contains claims due to natural catastrophes, modelled by a shot-noise Cox claim number process. The optimal value function of the resulting two-dimensional stochastic control problem is shown to be the smallest viscosity supersolution of a corresponding Hamilton-Jacobi-Bellman equation, and we prove that it can be uniformly approximated through a discretization of the space of the free surplus of the portfolio and the current claim intensity level. We implement the resulting numerical scheme to identify optimal dividend strategies for such a natural catastrophe insurer, and it is shown that the nature of the barrier and band strategies known from the classical models with constant Poisson claim intensity carry over in a certain way to this more general situation, leading to action and non-action regions for the dividend payments as a function of the current surplus and intensity level. We also discuss some interpretations in terms of upward potential for shareholders when including a catastrophe sector in the portfolio. ...

Implementing portfolio risk management and hedging in practice ArXiv ID: 2309.15767 “View on arXiv” Authors: Unknown Abstract In academic literature portfolio risk management and hedging are often versed in the language of stochastic control and Hamilton–Jacobi–Bellman~(HJB) equations in continuous time. In practice the continuous-time framework of stochastic control may be undesirable for various business reasons. In this work we present a straightforward approach for thinking of cross-asset portfolio risk management and hedging, providing some implementation details, while rarely venturing outside the convex optimisation setting of (approximate) quadratic programming~(QP). We pay particular attention to the correspondence between the economic concepts and their mathematical representations; the abstractions enabling us to handle multiple asset classes and risk models at once; the dimensional analysis of the resulting equations; and the assumptions inherent in our derivations. We demonstrate how to solve the resulting QPs with CVXOPT. ...

Understanding the worst-kept secret of high-frequency trading ArXiv ID: 2307.15599 “View on arXiv” Authors: Unknown Abstract Volume imbalance in a limit order book is often considered as a reliable indicator for predicting future price moves. In this work, we seek to analyse the nuances of the relationship between prices and volume imbalance. To this end, we study a market-making problem which allows us to view the imbalance as an optimal response to price moves. In our model, there is an underlying efficient price driving the mid-price, which follows the model with uncertainty zones. A single market maker knows the underlying efficient price and consequently the probability of a mid-price jump in the future. She controls the volumes she quotes at the best bid and ask prices. Solving her optimization problem allows us to understand endogenously the price-imbalance connection and to confirm in particular that it is optimal to quote a predictive imbalance. Our model can also be used by a platform to select a suitable tick size, which is known to be a crucial topic in financial regulation. The value function of the market maker’s control problem can be viewed as a family of functions, indexed by the level of the market maker’s inventory, solving a coupled system of PDEs. We show existence and uniqueness of classical solutions to this coupled system of equations. In the case of a continuous inventory, we also prove uniqueness of the market maker’s optimal control policy. ...

Machine Learning-powered Pricing of the Multidimensional Passport Option ArXiv ID: 2307.14887 “View on arXiv” Authors: Unknown Abstract Introduced in the late 90s, the passport option gives its holder the right to trade in a market and receive any positive gain in the resulting traded account at maturity. Pricing the option amounts to solving a stochastic control problem that for $d>1$ risky assets remains an open problem. Even in a correlated Black-Scholes (BS) market with $d=2$ risky assets, no optimal trading strategy has been derived in closed form. In this paper, we derive a discrete-time solution for multi-dimensional BS markets with uncorrelated assets. Moreover, inspired by the success of deep reinforcement learning in, e.g., board games, we propose two machine learning-powered approaches to pricing general options on a portfolio value in general markets. These approaches prove to be successful for pricing the passport option in one-dimensional and multi-dimensional uncorrelated BS markets. ...

Over-the-Counter Market Making via Reinforcement Learning ArXiv ID: 2307.01816 “View on arXiv” Authors: Unknown Abstract The over-the-counter (OTC) market is characterized by a unique feature that allows market makers to adjust bid-ask spreads based on order size. However, this flexibility introduces complexity, transforming the market-making problem into a high-dimensional stochastic control problem that presents significant challenges. To address this, this paper proposes an innovative solution utilizing reinforcement learning techniques to tackle the OTC market-making problem. By assuming a linear inverse relationship between market order arrival intensity and bid-ask spreads, we demonstrate the optimal policy for bid-ask spreads follows a Gaussian distribution. We apply two reinforcement learning algorithms to conduct a numerical analysis, revealing the resulting return distribution and bid-ask spreads under different time and inventory levels. ...

Machine Learning and Hamilton-Jacobi-Bellman Equation for Optimal Decumulation: a Comparison Study ArXiv ID: 2306.10582 “View on arXiv” Authors: Unknown Abstract We propose a novel data-driven neural network (NN) optimization framework for solving an optimal stochastic control problem under stochastic constraints. Customized activation functions for the output layers of the NN are applied, which permits training via standard unconstrained optimization. The optimal solution yields a multi-period asset allocation and decumulation strategy for a holder of a defined contribution (DC) pension plan. The objective function of the optimal control problem is based on expected wealth withdrawn (EW) and expected shortfall (ES) that directly targets left-tail risk. The stochastic bound constraints enforce a guaranteed minimum withdrawal each year. We demonstrate that the data-driven approach is capable of learning a near-optimal solution by benchmarking it against the numerical results from a Hamilton-Jacobi-Bellman (HJB) Partial Differential Equation (PDE) computational framework. ...