Enhancing ML Models Interpretability for Credit Scoring

Paper: arXiv 2509.11389

Authors: Sagi Schwartz, Qinling Wang, Fang Fang

Abstract

Predicting default is essential for banks to ensure profitability and financial stability. While modern machine learning methods often outperform traditional regression techniques, their lack of transparency limits their use in regulated environments. Explainable artificial intelligence (XAI) has emerged as a solution in domains like credit scoring. However, most XAI research focuses on post-hoc interpretation of black-box models, which does not produce models lightweight or transparent enough to meet regulatory requirements, such as those for Internal Ratings-Based (IRB) models. This paper proposes a hybrid approach: post-hoc interpretations of black-box models guide feature selection, followed by training glass-box models that maintain both predictive power and transparency. Using the Lending Club dataset, we demonstrate that this approach achieves performance comparable to a benchmark black-box model while using only 10 features - an 88.5% reduction. In our example, SHapley Additive exPlanations (SHAP) is used for feature selection, eXtreme Gradient Boosting (XGBoost) serves as the benchmark and the base black-box model, and Explainable Boosting Machine (EBM) and Penalized Logistic Tree Regression (PLTR) are the investigated glass-box models. We also show that model refinement using feature interaction analysis, correlation checks, and expert input can further enhance model interpretability and robustness.

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper provides a practical, industry-oriented framework combining post-hoc SHAP feature selection with lightweight glass-box models (EBM, PLTR) to satisfy regulatory requirements in credit scoring. While mathematically light, focusing on standard standard formulations, it presents a solid empirical evaluation using the Lending Club dataset to show a massive feature reduction with minimal performance loss.

Research Flowchart

  flowchart TD
    A["Research Goal: Balance Interpretability & Performance for Credit Scoring"] --> B["Data Input: Lending Club Dataset"]
    B --> C["1. Train Black-Box Model (XGBoost)"]
    C --> D["2. Apply SHAP Analysis for Feature Selection"]
    D --> E["3. Refine Features: Interactions, Correlation & Expert Input"]
    E --> F["4. Train Glass-Box Models (EBM & PLTR)"]
    F --> G["Key Outcome: 88.5% Feature Reduction (10 Features) with Comparable Accuracy & Regulatory Transparency"]

Cited References (35)

1. Machine Learning for IRB Models: Follow-up Report from the Consultation on the Discussion Paper on Machine Learning for IRB Models (2023) — link
2. Accurate intelligible models with pairwise interactions (2013) — link
3. Intelligible models for classification and regression (2012) — link
4. An Interpretable Model with Globally Consistent Explanations for Credit Risk (2018) — link
5. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (2022)
6. Explainable Boosting Machines with Sparsity – Maintaining Explainability in High - Dimensional Settings (2023) — link
7. InterpretML : A Unified Framework for Machine Learning Interpretability (2019) — link
8. Explainable Automated Machine Learning for Credit Decisions : Enhancing Human Artificial Intelligence Collaboration in Financial Engineering (2024) — link

0 of 35 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.