Paper: arXiv 2402.00299

Authors: Sahab Zandi, Kamesh Korangi, María Óskarsdóttir, Christophe Mues, Cristián Bravo

Abstract

Whereas traditional credit scoring tends to employ only individual borrower- or loan-level predictors, it has been acknowledged for some time that connections between borrowers may result in default risk propagating over a network. In this paper, we present a model for credit risk assessment leveraging a dynamic multilayer network built from a Graph Neural Network and a Recurrent Neural Network, each layer reflecting a different source of network connection. We test our methodology in a behavioural credit scoring context using a dataset provided by U.S. mortgage financier Freddie Mac, in which different types of connections arise from the geographical location of the borrower and their choice of mortgage provider. The proposed model considers both types of connections and the evolution of these connections over time. We enhance the model by using a custom attention mechanism that weights the different time snapshots according to their importance. After testing multiple configurations, a model with GAT, LSTM, and the attention mechanism provides the best results. Empirical results demonstrate that, when it comes to predicting probability of default for the borrowers, our proposed model brings both better results and novel insights for the analysis of the importance of connections and timestamps, compared to traditional methods.

Complexity vs Empirical Score

  • Math Complexity: 6.5/10
  • Empirical Rigor: 7.5/10
  • Quadrant: Holy Grail — high math complexity, high empirical rigor

Why this score: The paper introduces an innovative approach to credit default prediction by combining Graph Attention Networks (GAT) and LSTMs to capture dynamic, multilayer connections between borrowers. Empirical rigor is strong, utilizing a large-scale real-world mortgage dataset from Freddie Mac to validate performance against baseline models.

Research Flowchart

  flowchart TD
    A[Research Goal: Predict Loan Default Risk using Network Connections & Time Dynamics] --> B[Data Inputs: Freddie Mac Mortgages<br/>Borrower Location & Mortgage Provider Networks]
    B --> C[Dynamic Multilayer Graph Construction<br/>Multiple Relationship Types over Time]
    C --> D[Hybrid Neural Network Architecture<br/>GAT + LSTM + Custom Attention Mechanism]
    D --> E[Key Findings: Superior Probability of Default Prediction & Insights on Temporal/Network Importance]

Cited References (112)

  1. Network science (2016)
  2. Explainable AI for operational research: A defining framework, methods, applications, and a research agenda (2023)
  3. Social network analysis: An approach and technique for the study of information exchange (1996)
  4. A review on graph neural network methods in financial applications (2022)
  5. Graph learning based recommender systems: A review (2021)
  6. Graph Neural Network-based Node Classification with Hard Sample Strategy (2021)
  7. Link prediction based on graph neural networks (2018)
  8. Graph clustering with graph neural networks (2020)
and 104 more
  1. Multilayer networks (2014)
  2. Credit scoring and its applications (2017)
  3. Evolution of credit risk using a personalized pagerank algorithm for multilayer networks (2020)
  4. Multilayer network analysis for improved credit risk prediction (2021)
  5. Graph neural networks: A review of methods and applications (2020)
  6. Semi-Supervised Classification with Graph Convolutional Networks (2017)
  7. Graph Attention Networks (2018)
  8. Representation learning for dynamic graphs: A survey. (2020)
  9. Foundations and modeling of dynamic networks using dynamic graph neural networks: A survey (2021)
  10. Obligations with physical delivery in a multilayered financial network (2019)
  11. Loan default correlation using an A rchimedean copula approach: A case for recalibration (2015)
  12. A survey of the issues in consumer credit modelling research (2005)
  13. Improving credit scoring by differentiating defaulter behaviour (2015)
  14. Credit scoring for a microcredit data set using the synthetic minority oversampling technique and ensemble classifiers (2019)
  15. An empirical comparison of classification algorithms for imbalanced credit scoring datasets (2019)
  16. Development and application of consumer credit scoring models using profit-based classification measures (2014)
  17. Cost-based feature selection for support vector machines: An application in credit scoring (2017)
  18. The value of big data for credit scoring: Enhancing financial inclusion using mobile phone data and social network analytics (2019)
  19. Super-app behavioral patterns in credit risk models: Financial, statistical and regulatory implications (2021)
  20. What does your F acebook profile reveal about your creditworthiness? U sing alternative data for microfinance (2019)
  21. Long short-term memory (1997)
  22. Granting and managing loans for micro-entrepreneurs: New developments and practical experiences (2013)
  23. Finding structure in time (1990)
  24. Multi-dimensional recurrent neural networks (2007)
  25. Syllable-based acoustic modeling with CTC-SMBR-LSTM (2017)
  26. An LSTM network for highway trajectory prediction (2017)
  27. Deep sentence embedding using long short-term memory networks: Analysis and application to information retrieval (2016)
  28. Deep canonically correlated LSTM s (2018) — link
  29. Selective recurrent neural network (2013)
  30. On the properties of neural machine translation: Encoder-decoder approaches (2014)
  31. Empirical evaluation of gated recurrent neural networks on sequence modeling (2014)
  32. LSTM and GRU neural network performance comparison study: Taking Y elp review dataset as an example (2020)
  33. Neural machine translation by jointly learning to align and translate (2015)
  34. Multilingual hierarchical attention networks for document classification (2017)
  35. Attentional factorization machines: Learning the weight of feature interactions via attention networks (2017)
  36. Show, attend and tell: Neural image caption generation with visual attention (2015)
  37. Effective approaches to attention-based neural machine translation (2015)
  38. Attention is all you need (2017)
  39. A structured self-attentive sentence embedding (2017)
  40. Long short-term memory-networks for machine reading (2016)
  41. Geometric deep learning on graphs and manifolds using mixture model cnns (2017)
  42. Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting (2018)
  43. Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting (2018)
  44. Structural- RNN : Deep learning on spatio-temporal graphs (2016)
  45. Dynamic graph convolutional networks (2020)
  46. Evolve GCN : Evolving graph convolutional networks for dynamic graphs (2020)
  47. Sequence to Sequence Learning with Neural Networks (2014)
  48. Long-Term Recurrent Convolutional Networks for Visual Recognition and Description (2017)
  49. Backpropagation through time: what it does and how to do it (1990)
  50. Approximating real-time recurrent learning with random kronecker factors (2018)
  51. A learning algorithm for continually running fully recurrent neural networks (1989)
  52. The recurrent cascade-correlation architecture (1990)
  53. A survey on the application of recurrent neural networks to statistical language modeling (2015)
  54. Credit risk measurement: Developments over the last 20 years (1997)
  55. Review of the literature on credit risk modeling: Development of the past 10 years (2010)
  56. The pricing of options and corporate liabilities (1973)
  57. On the pricing of corporate debt: The risk structure of interest rates (1974)
  58. A comparative analysis of current credit risk models (2000)
  59. Credit risk: Drawing the analogy (1992)
  60. Correlation between intensity and recovery in credit risk models (2005)
  61. A M arkov model for the term structure of credit risk spreads (1997)
  62. A comparative anatomy of credit risk models (2000)
  63. Default rates in the syndicated bank loan market: A mortality analysis (2000)
  64. Machine learning-driven credit risk: A systemic review (2022)
  65. A ResNet-LSTM Based Credit Scoring Approach for Imbalanced Data (2022)
  66. Graph convolutional network-based credit default prediction utilizing three types of virtual distances among borrowers (2021)
  67. T- GCN : A temporal graph convolutional network for traffic prediction (2020)
  68. Inductive representation learning on temporal graphs (2020)
  69. Predicting dynamic embedding trajectory in temporal interaction networks (2019)
  70. Dy R ep: Learning representations over Dynamic Graphs (2019)
  71. Dynamic graph representation learning via self-attention networks (2019)
  72. On the combination of graph data for assessing thin-file borrowers’ creditworthiness (2023)
  73. Gr AMME : Semisupervised learning using multilayered graph attention models (2019)
  74. MELL : Effective embedding method for multiplex networks (2018)
  75. m GNN : Generalizing the Graph Neural Networks to the Multilayer Case (2021) — link
  76. Deep learning for credit scoring: Do or don’t? (2021)
  77. A survey on long short-term memory networks for time series prediction (2021)
  78. Long short-term Memory Recurrent neural network for Automatic speech recognition (2022)
  79. English grammar detection based on LSTM - CRF machine learning model (2021)
  80. Human action recognition using attention based LSTM network with dilated CNN features (2021)
  81. Credit scoring methods: Latest trends and points to consider (2022)
  82. State initialization for recurrent neural network modeling of time-series data (2017)
  83. Single Family Loan-Level Dataset (2022) — link
  84. Zahlentheorie: th. Die analytische Zahlentheorie (1894) (1894)
  85. Handbuch der Lehre von der Verteilung der Primzahlen (1909)
  86. A unified approach to interpreting model predictions (2017)
  87. Deep neural networks and tabular data: A survey (2022)
  88. Attention-based recurrent neural network for multistep-ahead prediction of process performance (2020)
  89. Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research (2015)
  90. “ B irds of a feather” fail together: exploring the nature of dependency in SME defaults (2019)
  91. The effect of interfirm financial transactions on the credit risk of small and medium-sized enterprises (2019)
  92. Bankruptcy prediction for SME s using relational data (2017)
  93. Graph Neural Network with Self-attention and Multi-task Learning for Credit Default Risk Prediction (2022)
  94. Financial Default Prediction via Motif-preserving Graph Neural Network with Curriculum Learning — → 2403.06482
  95. Financial risk analysis for SME s with graph-based supply chain mining (2021)
  96. Understanding Dropout for Graph Neural Networks (2022)
  97. GUIDE : Training Deep Graph Neural Networks via Guided Dropout Over Edges (2022)
  98. An improved LSTM model for correcting grammatical errors in E nglish text (2023)
  99. Evaluating credit risk models (2000)
  100. Deep learning (2016)
  101. Machine learning: A probabilistic perspective (2012)
  102. Impact of financial factors on the economic cycle dynamics in selected E uropean countries (2023)
  103. Strictly proper scoring rules, prediction, and estimation (2007)
  104. A window of opportunity: Assessing behavioural scoring (2013)

1 of 112 cited works are in the corpus; resolved via arXiv source bundles, Semantic Scholar and OpenAlex. Corpus-wide: 2518 linked papers.