Langevin algorithms for Markovian Neural Networks and Deep Stochastic control

Paper: arXiv 2212.12018

Authors: Pierre Bras, Gilles Pagès

Abstract

Stochastic Gradient Descent Langevin Dynamics (SGLD) algorithms, which add noise to the classic gradient descent, are known to improve the training of neural networks in some cases where the neural network is very deep. In this paper we study the possibilities of training acceleration for the numerical resolution of stochastic control problems through gradient descent, where the control is parametrized by a neural network. If the control is applied at many discretization times then solving the stochastic control problem reduces to minimizing the loss of a very deep neural network. We numerically show that Langevin algorithms improve the training on various stochastic control problems like hedging and resource management, and for different choices of gradient descent methods.

Complexity vs Empirical Score

• Math Complexity: 7.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper provides a mathematically solid application of Langevin dynamics to deep stochastic control and neural network parametrization. It features good numerical demonstrations on hedging and resource management problems, though additional empirical benchmarks and open-source code would further enhance reproducibility.

Research Flowchart

  flowchart TD
    A[Goal: Accelerate training for deep stochastic control problems] --> B[Parametrize control as a Deep Neural Network]
    B --> C[Input: Multi-asset market data, hedging & resource management parameters]
    C --> D[Apply Stochastic Gradient Descent Langevin Dynamics - SGLD]
    D --> E[Inject noise into gradients across many discretization time steps]
    E --> F[Findings: Improved training efficiency & convergence across SGD variants]