Paper: arXiv 2605.06570

Authors: Dmitri Goloubentsev, Natalija Karpichina

Abstract

Many real-world problems require sequential decisions under uncertainty: when to inject or withdraw gas from storage, how to rebalance a pension portfolio each month, what temperature profile to run through a pharmaceutical reactor chain. Dynamic programming solves small instances exactly but scales exponentially in state dimensions. Black-box reinforcement learning handles high-dimensional states but trains slowly and produces no sensitivities. We introduce SNAPO (Smooth Neural Adjoint Policy Optimization), a framework that embeds a neural policy inside a known, differentiable simulator, replaces hard constraints with smooth approximations, and computes exact gradients of the objective with respect to all policy parameters and all inputs in a single adjoint pass. We demonstrate SNAPO on three domains: natural gas storage (training in under a minute, 365 forward curve sensitivities at no additional cost per sensitivity), pension fund asset-liability management (6.5x-200x sensitivity speedup over bump-and-revalue, scaling with the number of risk factors), and pharmaceutical manufacturing (cross-unit sensitivities through a 4-unit process chain, with 20 ICH Q8 regulatory sensitivities from 5 adjoint passes in 74.5 milliseconds). All sensitivities are produced by the same backward pass that trains the policy, at a cost proportional to one reverse pass regardless of how many sensitivities are computed.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper combines sophisticated adjoint sensitivity methods and smooth constraint relaxation with high-performance, real-world industrial implementations for gas storage, pension funds, and manufacturing. By delivering both theoretical framework and empirical proof-of-work across three distinct fields with impressive speedups, it bridges advanced control theory with practical, high-throughput quantitative application.

Research Flowchart

  flowchart TD
    A[<b>Goal:</b> Efficient Optimal Control in High-Dimensions] --> B[<b>Framework:</b> SNAPO (Neural Policy + Differentiable Simulator)]
    B --> C[<b>Process:</b> Smooth Approximation & Adjoint Sensitivity Analysis]
    C --> D[<b>Data/Inputs:</b> Natural Gas, Pension Funds, Pharma Reactor States]
    D --> E{<b>Compute:</b> Single Backward Pass vs. Black-box RL}
    E --> F[<b>Findings:</b> Simultaneous Policy Optimization & Massive Sensitivity Speedups]