Paper: arXiv 2312.03510
Authors: Neil Kichler, Sher Afghan, Uwe Naumann
Abstract
The increasing use of stochastic models for describing complex phenomena warrants surrogate models that capture the reference model characteristics at a fraction of the computational cost, foregoing potentially expensive Monte Carlo simulation. The predominant approach of fitting a large neural network and then pruning it to a reduced size has commonly neglected shortcomings. The produced surrogate models often will not capture the sensitivities and uncertainties inherent in the original model. In particular, (higher-order) derivative information of such surrogates could differ drastically. Given a large enough network, we expect this derivative information to match. However, the pruned model will almost certainly not share this behavior. In this paper, we propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the pruned neural network based surrogate model. We experimentally underpin the method on an example of pricing a multidimensional Basket option modelled through a stochastic differential equation with Brownian motion. The proposed method is, however, not limited to the domain of quantitative finance, which was chosen as a case study for intuitive interpretations of the sensitivities. It serves as a foundation for building further surrogate modelling techniques considering sensitivity information.
Complexity vs Empirical Score
- Math Complexity: 8.5/10
- Empirical Rigor: 4.0/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper introduces advanced mathematical concepts like Sobolev spaces and interval adjoint significance analysis, supported by formal definitions and derivations, indicating high math complexity. However, it relies on a single experimental case study (basket option pricing) without detailed backtesting protocols or production-ready data pipelines, resulting in moderate empirical rigor.
Research Flowchart
flowchart TD Start["Research Goal<br>'How to create accurate surrogate models<br>that preserve sensitivity information?'"] --> Problem["Identify Problem<br>Standard pruning loses sensitivity/Uncertainty info"] Problem --> Method["Proposed Methodology<br>Sobolev Training + IASI Pruning"] Method --> Data["Inputs & Data<br>Basket Option Model<br>(SDE, Brownian Motion)"] Data --> Process["Computational Process<br>1. Train large network w/ sensitivities<br>2. Prune via IASI retaining sensitivity info"] Process --> Findings["Key Findings/Outcomes<br>- Surrogate preserves derivatives & uncertainties<br>- Effective for SDE based models<br>- Foundation for sensitivity-aware surrogate modeling"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes to create surrogate models by pruning a large neural network using Interval Adjoint Significance Analysis and then fine‑tuning the pruned model with Sobolev Training to recover first‑ and second‑order derivative accuracy.
Source sentence
We propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the prun
Assumes 4
- Basket assets are jointly normally distributed so that the basket option price remains Gaussian and can be modeled with the Bachelier model. model
Source sentence
We will consider basket models with assets being jointly normal distributed. As a result, the price of the basket option will remain Gaussian and the Bachelier model computes suitable prices.
- Pathwise (first‑ and second‑order) derivatives of the pricing function can be obtained via Algorithmic Differentiation. model
Source sentence
derivative information can, in many cases, easily be obtained through Algorithmic Differentiation (AD) [23], but is often neglected.
- A sufficiently large initial neural network can approximate the reference model’s sensitivities before pruning. other
Source sentence
Given a large enough network, we expect this derivative information to match.
- The synthetic Gaussian basket option pricing model is an appropriate test case for evaluating the method. data
Source sentence
We provide an experimental comparison of the presented methods on a Gaussian basket option pricing model.
Data
- Gaussian basket option pricing model (synthetic) synthetic unknown
Source sentence
We provide an experimental comparison of the presented methods on a Gaussian basket option pricing model.
Costs
Not applicable
Finds 3
- After iterative pruning the network retained similar R² score with only three nodes in the first and second dense layers, and one node in the third layer, achieving comparable value accuracy with far fewer parameters. performance
Source sentence
R2 score (i.e., the coefficient of determination) with only three nodes at both the first and second dense layers. The number of nodes at the third layer was reduced to one
- Sobolev fine‑tuning restored derivative accuracy: Delta R² improved to 0.999479 and Gamma R² to 0.987393, surpassing the original large network for Delta. predictive-accuracy
Source sentence
Gamma values previously dropped from 0.997033 to 0.902470 and after Sobolev fine-tuning has again an accuracy of
- When ground‑truth Bachelier model derivatives are available, fine‑tuning further improves accuracy to Delta R² 0.999863 and Gamma R² 0.997374. predictive-accuracy
Source sentence
If we furthermore have access to the original model, here the Bachelier basket model, the accuracy of the predicted derivatives improves further (Delta: 0.999863, Gamma: 0.997374).
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 12 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.
Cited References (47)
- UK List of \TeX Frequently Asked Questions (2019)
- The \textsf amsart , \textsf amsproc , and \textsf amsbook document~classes (2004)
- a lgorithm2e.sty—package for algorithms (2015)
- The algorithms bundle (2009)
- The Listings Package (2015)
- Publication quality tables in \LaTeX (2005)
- ACM Visual Identity Standards (2007)
- The subcaption package (2013)
and 39 more
- A package to create a nomenclature (2005)
- User Manual for glossaries.sty v4.44 (2019)
- The \textsl textcase package (2004)
- Babel (2022)
- Hyperparameter Search in Machine Learning (2015) — link
- Snip: Single-shot network pruning based on connection sensitivity (2018) — link
- Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science (2018)
- Significance analysis for numerical models (2015)
- Learning Both Weights and Connections for Efficient Neural Network (2015)
- NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm (2019)
- Designing Energy-Efficient Convolutional Neural Networks Using Energy-Aware Pruning (2017)
- Learning to Prune Deep Neural Networks via Layer-Wise Optimal Brain Surgeon (2017)
- Learning Structured Sparsity in Deep Neural Networks (2016)
- Scalpel: Customizing DNN Pruning to the Underlying Hardware Parallelism (2017)
- Channel Pruning for Accelerating Very Deep Neural Networks (2017)
- Optimal Brain Damage (1989)
- Optimal Brain Surgeon and General Network Pruning (1993)
- Methods and Applications of Interval Analysis (1979)
- Introduction to Interval Analysis (2009)
- Automation of significance analyses with interval splitting (2016)
- Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation (2008)
- The Art of Differentiating Computer Programs: An Introduction to Algorithmic Differentiation (2012)
- Interval adjoint significance analysis for neural networks (2020)
- Hidden Neuron Pruning of Multilayer Perceptrons Using a Quantified Sensitivity Measure (2006)
- A New Pruning Heuristic Based on Variance Analysis of Sensitivity Information (2001)
- Interval-Adjoint Significance Analysis: A Case Study (2016)
- Differential Machine Learning (2020) — link
- On the option pricing formula based on the bachelier model (2019) — link
- Monte Carlo Methods in Financial Engineering (2003) — link
- Knowledge Transfer with J acobian Matching (2018) — link
- Second-Order Differential ML (2023)
- How close are the option pricing formulas of Bachelier and Black–Merton–Scholes? (2008) — link
- A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options (2015) — link
- Adam: A Method for Stochastic Optimization (2015) — link
- Valuing American Options by Simulation: A Simple Least-Squares Approach (2001) — link
- Pattern Recognition and Machine Learning (2006) — link
- Sobolev training for neural networks (2017) — link
- The art of differentiating computer programs: an introduction to algorithmic differentiation (2011)
- Estimating the Hessian by Back-propagating Curvature (2012) — link
0 of 47 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2596 linked papers.