Towards Sobolev Pruning

ArXiv ID: 2312.03510 “View on arXiv”

Authors: Unknown

Abstract

The increasing use of stochastic models for describing complex phenomena warrants surrogate models that capture the reference model characteristics at a fraction of the computational cost, foregoing potentially expensive Monte Carlo simulation. The predominant approach of fitting a large neural network and then pruning it to a reduced size has commonly neglected shortcomings. The produced surrogate models often will not capture the sensitivities and uncertainties inherent in the original model. In particular, (higher-order) derivative information of such surrogates could differ drastically. Given a large enough network, we expect this derivative information to match. However, the pruned model will almost certainly not share this behavior. In this paper, we propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the pruned neural network based surrogate model. We experimentally underpin the method on an example of pricing a multidimensional Basket option modelled through a stochastic differential equation with Brownian motion. The proposed method is, however, not limited to the domain of quantitative finance, which was chosen as a case study for intuitive interpretations of the sensitivities. It serves as a foundation for building further surrogate modelling techniques considering sensitivity information.

Keywords: Surrogate modeling, Sensitivity analysis, Sobolev training, Interval Adjoint Significance Analysis, Neural network pruning, Equity Derivatives

Complexity vs Empirical Score

• Math Complexity: 8.5/10
• Empirical Rigor: 4.0/10
• Quadrant: Lab Rats
• Why: The paper introduces advanced mathematical concepts like Sobolev spaces and interval adjoint significance analysis, supported by formal definitions and derivations, indicating high math complexity. However, it relies on a single experimental case study (basket option pricing) without detailed backtesting protocols or production-ready data pipelines, resulting in moderate empirical rigor.

  flowchart TD
    Start["Research Goal<br/>'How to create accurate surrogate models<br/>that preserve sensitivity information?'"] --> Problem["Identify Problem<br/>Standard pruning loses sensitivity/Uncertainty info"]
    Problem --> Method["Proposed Methodology<br/>Sobolev Training + IASI Pruning"]
    Method --> Data["Inputs & Data<br/>Basket Option Model<br/>(SDE, Brownian Motion)"]
    Data --> Process["Computational Process<br/>1. Train large network w/ sensitivities<br/>2. Prune via IASI retaining sensitivity info"]
    Process --> Findings["Key Findings/Outcomes<br/>- Surrogate preserves derivatives & uncertainties<br/>- Effective for SDE based models<br/>- Foundation for sensitivity-aware surrogate modeling"]