Dynamic Programming

Action-State Dependent Dynamic Model Selection ArXiv ID: 2307.04754 “View on arXiv” Authors: Unknown Abstract A model among many may only be best under certain states of the world. Switching from a model to another can also be costly. Finding a procedure to dynamically choose a model in these circumstances requires to solve a complex estimation procedure and a dynamic programming problem. A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem. The algorithm is shown to consistently estimate the optimal policy that may choose different models based on a set of covariates. A typical example is the one of switching between different portfolio models under rebalancing costs, using macroeconomic information. Using a set of macroeconomic variables and price data, an empirical application to the aforementioned portfolio problem shows superior performance to choosing the best portfolio model with hindsight. ...

Optimal Portfolio Execution in a Regime-switching Market with Non-linear Impact Costs: Combining Dynamic Program and Neural Network ArXiv ID: 2306.08809 “View on arXiv” Authors: Unknown Abstract Optimal execution of a portfolio have been a challenging problem for institutional investors. Traders face the trade-off between average trading price and uncertainty, and traditional methods suffer from the curse of dimensionality. Here, we propose a four-step numerical framework for the optimal portfolio execution problem where multiple market regimes exist, with the underlying regime switching based on a Markov process. The market impact costs are modelled with a temporary part and a permanent part, where the former affects only the current trade while the latter persists. Our approach accepts impact cost functions in generic forms. First, we calculate the approximated orthogonal portfolios based on estimated impact cost functions; second, we employ dynamic program to learn the optimal selling schedule of each approximated orthogonal portfolio; third, weights of a neural network are pre-trained with the strategy suggested by previous step; last, we train the neural network to optimize on the original trading model. In our experiment of a 10-asset liquidation example with quadratic impact costs, the proposed combined method provides promising selling strategy for both CRRA (constant relative risk aversion) and mean-variance objectives. The running time is linear in the number of risky assets in the portfolio as well as in the number of trading periods. Possible improvements in running time are discussed for potential large-scale usages. ...

Combining Reinforcement Learning and Barrier Functions for Adaptive Risk Management in Portfolio Optimization ArXiv ID: 2306.07013 “View on arXiv” Authors: Unknown Abstract Reinforcement learning (RL) based investment strategies have been widely adopted in portfolio management (PM) in recent years. Nevertheless, most RL-based approaches may often emphasize on pursuing returns while ignoring the risks of the underlying trading strategies that may potentially lead to great losses especially under high market volatility. Therefore, a risk-manageable PM investment framework integrating both RL and barrier functions (BF) is proposed to carefully balance the needs for high returns and acceptable risk exposure in PM applications. Up to our understanding, this work represents the first attempt to combine BF and RL for financial applications. While the involved RL approach may aggressively search for more profitable trading strategies, the BF-based risk controller will continuously monitor the market states to dynamically adjust the investment portfolio as a controllable measure for avoiding potential losses particularly in downtrend markets. Additionally, two adaptive mechanisms are provided to dynamically adjust the impact of risk controllers such that the proposed framework can be flexibly adapted to uptrend and downtrend markets. The empirical results of our proposed framework clearly reveal such advantages against most well-known RL-based approaches on real-world data sets. More importantly, our proposed framework shed lights on many possible directions for future investigation. ...