On optimal tracking portfolio in incomplete markets: The reinforcement learning approach

ArXiv ID: 2311.14318 “View on arXiv”

Authors: Unknown

Abstract

This paper studies an infinite horizon optimal tracking portfolio problem using capital injection in incomplete market models. The benchmark process is modelled by a geometric Brownian motion with zero drift driven by some unhedgeable risk. The relaxed tracking formulation is adopted where the fund account compensated by the injected capital needs to outperform the benchmark process at any time, and the goal is to minimize the cost of the discounted total capital injection. When model parameters are known, we formulate the equivalent auxiliary control problem with reflected state dynamics, for which the classical solution of the HJB equation with Neumann boundary condition is obtained explicitly. When model parameters are unknown, we introduce the exploratory formulation for the auxiliary control problem with entropy regularization and develop the continuous-time q-learning algorithm in models of reflected diffusion processes. In some illustrative numerical example, we show the satisfactory performance of the q-learning algorithm.

Keywords: optimal tracking portfolio, capital injection, incomplete markets, q-learning, HJB equation, General Portfolio Management

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 4.0/10
• Quadrant: Lab Rats
• Why: The paper presents advanced mathematical theory involving reflected diffusions, HJB equations with Neumann boundary conditions, and continuous-time q-learning extensions, resulting in high math complexity. However, the empirical component relies on a single illustrative numerical example and a comparison with MLE on real data, lacking extensive backtesting or implementation details for a trading strategy.

  flowchart TD
    A["Research Goal:<br>Optimal tracking portfolio in incomplete markets<br>with capital injection"] --> B["Model Setup<br>Benchmark: Geometric Brownian Motion<br>Formulation: Relaxed tracking"]
    B --> C{"Model Parameters Known?"}
    C -- Yes --> D["Known Model Method<br>Equivalent auxiliary control problem<br>Solve HJB with Neumann boundary condition<br>Explicit solution obtained"]
    C -- No --> E["Unknown Model Method<br>Exploratory formulation with entropy regularization<br>Continuous-time Q-learning<br>Reflected diffusion processes"]
    D --> F["Outcomes<br>Explicit analytical solution<br>Classical optimal control"]
    E --> G["Outcomes<br>Q-learning algorithm performance<br>Satisfactory numerical results<br>Model-free tracking strategy"]