Assessment of creditworthiness models privacy-preserving training with synthetic data

Paper: arXiv 2301.01212

Authors: Ricardo Muñoz-Cancino, Cristián Bravo, Sebastián A. Ríos, Manuel Graña

Abstract

Credit scoring models are the primary instrument used by financial institutions to manage credit risk. The scarcity of research on behavioral scoring is due to the difficult data access. Financial institutions have to maintain the privacy and security of borrowers’ information refrain them from collaborating in research initiatives. In this work, we present a methodology that allows us to evaluate the performance of models trained with synthetic data when they are applied to real-world data. Our results show that synthetic data quality is increasingly poor when the number of attributes increases. However, creditworthiness assessment models trained with synthetic data show a reduction of 3% of AUC and 6% of KS when compared with models trained with real data. These results have a significant impact since they encourage credit risk investigation from synthetic data, making it possible to maintain borrowers’ privacy and to address problems that until now have been hampered by the availability of information.

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper provides a practical empirical evaluation of privacy-preserving synthetic data for credit scoring models, showing modest performance drops (AUC/KS metrics) compared to models trained on real data. While mathematical formulation is minimal, the empirical application directly addresses the critical bottleneck of data privacy in quantitative credit risk research.

Research Flowchart

  flowchart TD
    A[Goal: Evaluate credit scoring models<br/>trained on synthetic vs. real data] --> B[Input: Real Borrower & Credit Data]
    B --> C[Generate Synthetic Data<br/>via Generative Models]
    C --> D[Evaluate Synthetic Data Quality<br/>vs. Number of Attributes]
    D --> E[Train Credit Models on<br/>Real vs. Synthetic Data]
    E --> F[Test Models on Real-World Data]
    F --> G[Outcome: Quality drops with high attributes,<br/>but synthetic models retain performance<br/>-3% AUC, -6% KS for privacy-preserving research]