{"paper_id":"2312.03510","title":"Towards Sobolev Pruning","authors":["Neil Kichler","Sher Afghan","Uwe Naumann"],"paper_date":"2023-12-06","paper_type":"empirical","paper_url":"https://arxiv.org/pdf/2312.03510v2","page_url":"https://thequant.space/flowcharts/towards-sobolev-pruning/","scores":{"empirical_rigor":4,"math_complexity":8.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The paper proposes to create surrogate models by pruning a large neural network using Interval Adjoint Significance Analysis and then fine‑tuning the pruned model with Sobolev Training to recover first‑ and second‑order derivative accuracy.","quote":"We propose to find surrogate models by using sensitivity information throughout the learning and pruning process. We build on work using Interval Adjoint Significance Analysis for pruning and combine it with the recent advancements in Sobolev Training to accurately model the original sensitivity information in the prun","summary":"The increasing use of stochastic models for describing complex phenomena warrants surrogate models that capture the reference model characteristics at a fraction of the computational cost, foregoing potentially expensive Monte Carlo simulation. The predominant approach of fitting a large neural…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Basket assets are jointly normally distributed so that the basket option price remains Gaussian and can be modeled with the Bachelier model.","type":"model","stated":"explicit","quote":"We will consider basket models with assets being jointly normal distributed. As a result, the price of the basket option will remain Gaussian and the Bachelier model computes suitable prices."},{"assumption":"Pathwise (first‑ and second‑order) derivatives of the pricing function can be obtained via Algorithmic Differentiation.","type":"model","stated":"explicit","quote":"derivative information can, in many cases, easily be obtained through Algorithmic Differentiation (AD) [23], but is often neglected."},{"assumption":"A sufficiently large initial neural network can approximate the reference model’s sensitivities before pruning.","type":"other","stated":"implied","quote":"Given a large enough network, we expect this derivative information to match."},{"assumption":"The synthetic Gaussian basket option pricing model is an appropriate test case for evaluating the method.","type":"data","stated":"explicit","quote":"We provide an experimental comparison of the presented methods on a Gaussian basket option pricing model."}]},"datasets":{"status":"available","items":[{"name":"Gaussian basket option pricing model (synthetic)","provider":null,"kind":"synthetic","access":"unknown","period":null,"universe":null,"quote":"We provide an experimental comparison of the presented methods on a Gaussian basket option pricing model."}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"pricing-calibration","findings":[{"finding":"After iterative pruning the network retained similar R² score with only three nodes in the first and second dense layers, and one node in the third layer, achieving comparable value accuracy with far fewer parameters.","kind":"performance","quote":"R2 score (i.e., the coefficient of determination) with only three nodes at both the first and second dense layers. The number of nodes at the third layer was reduced to one"},{"finding":"Sobolev fine‑tuning restored derivative accuracy: Delta R² improved to 0.999479 and Gamma R² to 0.987393, surpassing the original large network for Delta.","kind":"predictive-accuracy","quote":"Gamma values previously dropped from 0.997033 to 0.902470 and after Sobolev fine-tuning has again an accuracy of"},{"finding":"When ground‑truth Bachelier model derivatives are available, fine‑tuning further improves accuracy to Delta R² 0.999863 and Gamma R² 0.997374.","kind":"predictive-accuracy","quote":"If we furthermore have access to the original model, here the Bachelier basket model, the accuracy of the predicted derivatives improves further (Delta: 0.999863, Gamma: 0.997374)."}],"reported_performance":null,"out_of_sample":{"method":"none","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":12,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":1,"extracted_at":"2026-11-14T18:18:20Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}