Paper: arXiv 2610.24586

Authors: Agus Sudjianto, Denis Burakov

Abstract

Traditional weight of evidence assigns a credit characteristic a value based on its association with default across a population. Credit analysis also requires an assessment of that characteristic in the context of an individual applicant’s record. We study conditional weight of evidence for this purpose. It measures the change in default log odds from observing a characteristic after the remaining credit information is known. Its sign identifies whether the characteristic adds adverse or favorable information for that applicant, and its magnitude measures the strength of that information. The same characteristic value can therefore receive different assessments for applicants with different payment histories or other credit attributes. We develop estimators using gradient-boosted trees and evaluate their accuracy in a bureau-style simulation. Regressions of the full model’s predicted probabilities recover conditional evidence more accurately than separate models fitted to default outcomes, provided the full model is accurate. They also recover a deliberately constructed file-age effect whose direction varies with utilization and is often missed by marginal weight of evidence. Local class-frequency estimates improve on marginal weight of evidence but remain inaccurate for some weak effects. An application to FICO HELOC data illustrates estimation when the population evidence is unknown. Conditional weight of evidence provides an applicant-specific interpretation of the information in a credit characteristic; its population summary, a conditional information value, splits the joint information in the record across blocks when averaged over orderings. Neither is an adverse action reason, which explains the fitted score difference between the applicant and chosen passing records.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 8.0/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: This paper introduces a novel concept of conditional weight of evidence, addressing a critical gap in credit risk interpretation. It combines advanced mathematical concepts with a strong empirical evaluation using a bureau-style simulation and real-world data. The methodology is well-defined, contributing significantly to interpretable AI in finance.

Research Flowchart

  flowchart TD
    A[Research Goal: Applicant-Specific Credit Characteristic Assessment] --> B{Key Concept: Conditional Weight of Evidence (CWOE)};
    B --> C{Methodology: Estimating CWOE using Gradient-Boosted Trees};
    C --> D[Data/Inputs: Bureau-Style Simulation, FICO HELOC Data];
    D --> E[Computational Process: Model Training (GBM), Regression for CWOE Recovery, Local Class-Frequency Estimates];
    E --> F[Key Findings: CWOE Provides Applicant-Specific Interpretation, Accurate Recovery via Full Model Regressions, CWOE != Adverse Action Reason];