{"paper_id":"2609.26445","title":"A Practical Guide on Graphical Model Validation","authors":["Mario V. Wüthrich"],"paper_date":"2026-09-22","paper_type":"theoretical","paper_url":"https://arxiv.org/pdf/2609.26445v1","page_url":"https://thequant.space/flowcharts/a-practical-guide-on-graphical-model-validation/","scores":{"empirical_rigor":3,"math_complexity":7.5,"quadrant":"Lab Rats"},"proposes":{"status":"available","statement":"The manuscript formalizes popular model validation tools for general insurance actuarial pricing, including graphical tools (calibration plots, lift charts, Murphy diagrams) and statistical tools (Bregman losses, deviance losses, Murphy's decomposition, Gini scores).","quote":"This manuscript formalizes the most popular model validation tools used in general insurance actuarial modeling.","summary":"This manuscript formalizes the most popular model validation tools used in general insurance actuarial modeling. These include graphical tools like calibration plots, actual-vs-expected plots, lift charts, Murphy diagrams, as well as classical statistical tools such as Bregman losses, deviance…","strategy":null},"assumptions":{"status":"available","items":[{"assumption":"Conditional mean independence: E[Y | Π, V] = E[Y | Π] a.s.","type":"model","stated":"explicit","quote":"Assumption 3.6 (Conditional mean independence) Assume that E [Y | Π, V ] = E [Y | Π] a.s. (3.5) Under this additional assumption (3.5) we have the following equivalences.","support":"full"},{"assumption":"All considered quantities have finite means and the exposure is strictly positive a.s.","type":"statistical","stated":"explicit","quote":"Note that cur-rently we have not made any model assumptions, except that all considered quantities have finite means and that the exposure is strictly positive, a.s. Thus, Proposition 3.5 holds in full generality.","support":"full"}]},"datasets":{"status":"available","items":[{"name":"French MTPL dataset","provider":null,"kind":"other","access":"public","period":null,"universe":null,"quote":"We highlight that both of the two unit premium rules are strong claim frequency forecast models on the French MTPL dataset. 2The cleaned data is available from https://aitools4actuaries.com/.","support":"full"}],"known":[],"sample":{"start":null,"end":null,"market":null,"asset_class":null,"frequency":null},"data_resolution":"synthetic"},"transaction_costs":{"status":"not_applicable","modeled":"n/a","headline_net_of_costs":null,"detail":null,"quote":null,"source":"front_matter","rule":"R3","candidates":{"front_matter":"n/a","evidence_card":"n/a","evidence_card_quoted":false},"meaning":"the paper reports no trading or execution result"},"results":{"status":"available","evaluation_type":"unclear","findings":[{"finding":"Recalibrated premiums (m_Q(Π)) match actual outcomes better than raw premiums (Π), indicating a clear preference for recalibrated premiums.","kind":"pricing-calibration","quote":"conclusion of this plot is that the expected values mQ(Π) b,κ match the actuals Y κ k much better than the expected values Π a,κ k , thus, there is a clear preference for the recalibrated premiums from this double lift chart.","support":"full"},{"finding":"Both unit premium rules (PIN and LightGBM) are strong claim frequency forecast models on the French MTPL dataset.","kind":"predictive-accuracy","quote":"We highlight that both of the two unit premium rules are strong claim frequency forecast models on the French MTPL dataset.","support":"full"},{"finding":"Poisson deviance losses on the test sample show that the PIN model has a loss of 44.615 and the LightGBM model 44.372, both outperforming the global mean and GLM benchmarks.","kind":"performance","quote":"This allows us to compute the sample versions of the Poisson Bregman 49 Bregman Bregman global premium rules divergence score mean global mean model bµQ = bµQ,L 47.967 – Y Q,T = 7.35% GLM unit premium 45.435 2.532 7.40% PIN unit premium ΠPIN 44.615 3.352 7.31% LightGBM unit premium ΠGBM 44.372 3.595 7.38% Table 3: MTPL Bregman divergences bEQ[Lφ(Y, ·)] and Bregman scores bSφ(Y, ·) of the considered premium rules","support":"full"}],"reported_performance":null,"out_of_sample":{"method":"unclear","headline_out_of_sample":null}},"independent_evidence":{"status":"none_found","supports":[],"contradicts":[],"qualifies":[],"compared_with":15,"citations":{"cited_by_in_corpus":0,"by_independent_authors":0,"how_cited":{"used_as_baseline":0,"uses_method_or_data":0,"extends":0,"discusses_findings":0,"mentions":0,"unclassified":0},"independent_papers":[],"scope":"Citations between papers in this corpus only; citations from outside it are not counted."}},"implementation":{"status":"available","public":false,"code_url":null,"note":"No repository presented by the authors as their own was found in the paper. Absence here is not proof that no code exists."},"replication":{"status":"none_found","card":null,"note":"No replication of this paper has been run by thequant.space yet."},"extraction":{"text_source":"fulltext","model":"openai/gpt-oss-120b","version":2,"extracted_at":"2026-11-14T19:39:23Z"},"provenance":{"claims":"Everything here is what the authors state in the paper. Nothing was re-run or reproduced by the platform.","quotes":"Assumptions, datasets, cost detail and findings were extracted by a language model. Every quote is verbatim paper text. Where an item has `support`, \"full\" means every figure in it appears in the quoted sentences and \"partial\" means a figure is stated elsewhere in the paper; items with a figure found nowhere in the paper were discarded.","datasets_known":"datasets.known lists well-known vendors and public datasets by a stable id, found by a dictionary over the full text where the sentence says the data is used; it is independent of the model and covers only names in the dictionary.","flags":"transaction_costs.modeled, results.evaluation_type and results.out_of_sample.method are platform classifications (structured-decision models); see https://thequant.space/score-guide/#penalty-flags","comparisons":"supports / contradicts / qualifies are a language model's reading of two papers' quoted findings, restricted to closely related papers that share no author. They are pointers to read, not verified replications, and coverage is partial.","implementation":"code_url is a repository the paper itself presents as its own (rule-scored, ambiguous cases judged by a model), checked for liveness when it was recorded."},"docs":"https://thequant.space/agents/"}