Optimal Execution

Optimal Execution with Reinforcement Learning ArXiv ID: 2411.06389 “View on arXiv” Authors: Unknown Abstract This study investigates the development of an optimal execution strategy through reinforcement learning, aiming to determine the most effective approach for traders to buy and sell inventory within a finite time horizon. Our proposed model leverages input features derived from the current state of the limit order book and operates at a high frequency to maximize control. To simulate this environment and overcome the limitations associated with relying on historical data, we utilize the multi-agent market simulator ABIDES, which provides a diverse range of depth levels within the limit order book. We present a custom MDP formulation followed by the results of our methodology and benchmark the performance against standard execution strategies. Results show that the reinforcement learning agent outperforms standard strategies and offers a practical foundation for real-world trading applications. ...

Optimal execution with deterministically time varying liquidity: well posedness and price manipulation ArXiv ID: 2410.04867 “View on arXiv” Authors: Unknown Abstract We investigate the well-posedness in the Hadamard sense and the absence of price manipulation in the optimal execution problem within the Almgren-Chriss framework, where the temporary and permanent impact parameters vary deterministically over time. We present sufficient conditions for the existence of a unique solution and provide second-order conditions for the problem, with a particular focus on scenarios where impact parameters change monotonically over time. Additionally, we establish conditions to prevent transaction-triggered price manipulation in the optimal solution, i.e. the occurence of buying and selling in the same trading program. Our findings are supported by numerical analyses that explore various regimes in simple parametric settings for the dynamics of impact parameters. ...

Optimal position-building strategies in competition ArXiv ID: 2409.03586 “View on arXiv” Authors: Unknown Abstract This paper develops a mathematical framework for building a position in a stock over a fixed period of time while in competition with one or more other traders doing the same thing. We develop a game-theoretic framework that takes place in the space of trading strategies where action sets are trading strategies and traders try to devise best-response strategies to their adversaries. In this setup trading is guided by a desire to minimize the total cost of trading arising from a mixture of temporary and permanent market impact caused by the aggregate level of trading including the trader and the competition. We describe a notion of equilibrium strategies, show that they exist and provide closed-form solutions. ...

Trade execution games in a Markovian environment ArXiv ID: 2405.07184 “View on arXiv” Authors: Unknown Abstract This paper examines a trade execution game for two large traders in a generalized price impact model. We incorporate a stochastic and sequentially dependent factor that exogenously affects the market price into financial markets. Our model accounts for how strategic and environmental uncertainties affect the large traders’ execution strategies. We formulate an expected utility maximization problem for two large traders as a Markov game model. Applying the backward induction method of dynamic programming, we provide an explicit closed-form execution strategy at a Markov perfect equilibrium. Our theoretical results reveal that the execution strategy generally lies in a dynamic and non-randomized class; it becomes deterministic if the Markovian environment is also deterministic. In addition, our simulation-based numerical experiments suggest that the execution strategy captures various features observed in financial markets. ...

Price-Aware Automated Market Makers: Models Beyond Brownian Prices and Static Liquidity ArXiv ID: 2405.03496 “View on arXiv” Authors: Unknown Abstract In this paper, we introduce a suite of models for price-aware automated market making platforms willing to optimize their quotes. These models incorporate advanced price dynamics, including stochastic volatility, jumps, and microstructural price models based on Hawkes processes. Additionally, we address the variability in demand from liquidity takers through models that employ either Hawkes or Markov-modulated Poisson processes. Each model is analyzed with particular emphasis placed on the complexity of the numerical methods required to compute optimal quotes. ...

Optimal Portfolio Choice with Cross-Impact Propagators ArXiv ID: 2403.10273 “View on arXiv” Authors: Unknown Abstract We consider a class of optimal portfolio choice problems in continuous time where the agent’s transactions create both transient cross-impact driven by a matrix-valued Volterra propagator, as well as temporary price impact. We formulate this problem as the maximization of a revenue-risk functional, where the agent also exploits available information on a progressively measurable price predicting signal. We solve the maximization problem explicitly in terms of operator resolvents, by reducing the corresponding first order condition to a coupled system of stochastic Fredholm equations of the second kind and deriving its solution. We then give sufficient conditions on the matrix-valued propagator so that the model does not permit price manipulation. We also provide an implementation of the solutions to the optimal portfolio choice problem and to the associated optimal execution problem. Our solutions yield financial insights on the influence of cross-impact on the optimal strategies and its interplay with alpha decays. ...

A Note on Optimal Liquidation with Linear Price Impact ArXiv ID: 2402.14100 “View on arXiv” Authors: Unknown Abstract In this note we consider the maximization of the expected terminal wealth for the setup of quadratic transaction costs. First, we provide a very simple probabilistic solution to the problem. Although the problem was largely studied, as far as we know up to date this simple and probabilistic form of the solution has not appeared in the literature. Next, we apply the general result for the numerical study of the case where the risky asset is given by a fractional Brownian Motion and the information flow of the investor can be diversified. ...

Reinforcement Learning for Optimal Execution when Liquidity is Time-Varying ArXiv ID: 2402.12049 “View on arXiv” Authors: Unknown Abstract Optimal execution is an important problem faced by any trader. Most solutions are based on the assumption of constant market impact, while liquidity is known to be dynamic. Moreover, models with time-varying liquidity typically assume that it is observable, despite the fact that, in reality, it is latent and hard to measure in real time. In this paper we show that the use of Double Deep Q-learning, a form of Reinforcement Learning based on neural networks, is able to learn optimal trading policies when liquidity is time-varying. Specifically, we consider an Almgren-Chriss framework with temporary and permanent impact parameters following several deterministic and stochastic dynamics. Using extensive numerical experiments, we show that the trained algorithm learns the optimal policy when the analytical solution is available, and overcomes benchmarks and approximated solutions when the solution is not available. ...

Leveraging IS and TC: Optimal order execution subject to reference strategies ArXiv ID: 2401.03305 “View on arXiv” Authors: Unknown Abstract The paper addresses the problem of meta order execution from a broker-dealer’s point of view in Almgren-Chriss model under execution risk. A broker-dealer agency is authorized to execute an order of trading on some client’s behalf. The strategies that the agent is allowed to deploy is subject to a benchmark, referred to as the reference strategy, regulated by the client. We formulate the broker’s problem as a utility maximization problem in which the broker seeks to maximize his utility of excess profit-and-loss at the execution horizon, of which optimal feedback strategies are obtained in closed form. In the absence of execution risk, the optimal strategies subject to reference strategies are deterministic. We establish an affine structure among the trading trajectories under optimal strategies subject to general reference strategies using implementation shortfall (IS) and target close (TC) orders as basis. Furthermore, an approximation theorem is proposed to show that with small error, general reference strategies can be approximated by piece-wise constant ones, of which the optimal strategy is piece-wise linear combination between IS and TC orders. We conclude the paper with numerical experiments illustrating the trading trajectories as well as histograms of terminal wealth and utility at investment horizon under optimal strategies versus those under TWAP strategies. ...

Relative entropy-regularized robust optimal order execution ArXiv ID: 2311.06476 “View on arXiv” Authors: Unknown Abstract The problem of order execution is cast as a relative entropy-regularized robust optimal control problem in this article. The order execution agent’s goal is to maximize an objective functional associated with his profit-and-loss of trading and simultaneously minimize the execution risk and the market’s liquidity and uncertainty. We model the market’s liquidity and uncertainty by the principle of least relative entropy associated with the market volume. The problem of order execution is made into a relative entropy-regularized stochastic differential game. Standard argument of dynamic programming yields that the value function of the differential game satisfies a relative entropy-regularized Hamilton-Jacobi-Isaacs (rHJI) equation. Under the assumptions of linear-quadratic model with Gaussian prior, the rHJI equation reduces to a system of Riccati and linear differential equations. Further imposing constancy of the corresponding coefficients, the system of differential equations can be solved in closed form, resulting in analytical expressions for optimal strategy and trajectory as well as the posterior distribution of market volume. Numerical examples illustrating the optimal strategies and the comparisons with conventional trading strategies are conducted. ...