Paper: arXiv 2302.07320

Authors: Mohamed Hamdouche, Pierre Henry-Labordere, Huyen Pham

Abstract

We develop policy gradients methods for stochastic control with exit time in a model-free setting. We propose two types of algorithms for learning either directly the optimal policy or by learning alternately the value function (critic) and the optimal control (actor). The use of randomized policies is crucial for overcoming notably the issue related to the exit time in the gradient computation. We demonstrate the effectiveness of our approach by implementing our numerical schemes in the application to the problem of share repurchase pricing. Our results show that the proposed policy gradient methods outperform PDE or other neural networks techniques in a model-based setting. Furthermore, our algorithms are flexible enough to incorporate realistic market conditions like e.g. price impact or transaction costs.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 6.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a rigorous mathematical framework for model-free stochastic control with exit times using policy gradients. It applies these methods to share repurchase pricing, demonstrating superiority over classic PDE/NN baselines while incorporating realistic market friction features.

Research Flowchart

  flowchart TD
    A["Research Goal:<br>Model-Free Stochastic Control<br>with Exit Time"] --> B["Input & Inputs:<br>Market Conditions<br><i>(Price Impact, Transaction Costs)</i>"]
    B --> C["Core Methodology:<br>Randomized Policy Gradients<br><i>(Overcomes Exit Time Gradient Issue)</i>"]
    C --> D["Computational Process:<br>Dual Learning Schemes<br><i>(Direct Policy vs. Actor-Critic)</i>"]
    D --> E["Application:<br>Share Repurchase Pricing Simulation"]
    E --> F["Key Findings:<br>Outperforms PDE & Neural Nets;<br>Flexible for Realistic Markets"]