The Impact of Feature Selection and Transformation on Machine Learning Methods in Determining the Credit Scoring

Paper: arXiv 2303.05427

Authors: Oguz Koc, Omur Ugur, A. Sevtap Kestel

Abstract

Banks utilize credit scoring as an important indicator of financial strength and eligibility for credit. Scoring models aim to assign statistical odds or probabilities for predicting if there is a risk of nonpayment in relation to many other factors which may be involved in. This paper aims to illustrate the beneficial use of the eight machine learning (ML) methods (Support Vector Machine, Gaussian Naive Bayes, Decision Trees, Random Forest, XGBoost, K-Nearest Neighbors, Multi-layer Perceptron Neural Networks) and Logistic Regression in finding the default risk as well as the features contributing to it. An extensive comparison is made in three aspects: (i) which ML models with and without its own wrapper feature selection performs the best; (ii) how feature selection combined with appropriate data scaling method influences the performance; (iii) which of the most successful combination (algorithm, feature selection, and scaling) delivers the best validation indicators such as accuracy rate, Type I and II errors and AUC. An open-access credit scoring default risk data sets on German and Australian cases are taken into account, for which we determine the best method, scaling, and features contributing to default risk best and compare our findings with the literature ones in related. We illustrate the positive contribution of the selection method and scaling on the performance indicators compared to the existing literature.

Complexity vs Empirical Score

• Math Complexity: 3.5/10
• Empirical Rigor: 6.5/10
• Quadrant: Street Traders — practical and empirical, lighter on theory

Why this score: The paper presents a standard empirical evaluation comparing common machine learning classifiers, feature selection methods, and data scalers on benchmark credit scoring datasets. While methodologically structured and clear in its evaluation metrics, it lacks advanced mathematical formulations and offers limited methodological novelty beyond benchmarking existing techniques.

Research Flowchart

  flowchart TD
    A[Research Goal: Evaluate ML, Feature Selection & Scaling for Credit Default Risk] --> B[Data Inputs: German & Australian Credit Scoring Datasets]
    B --> C[Preprocessing: Data Scaling Methods]
    C --> D[Feature Selection: Wrapper Feature Selection]
    D --> E[Model Training: 8 ML Algorithms e.g., SVM, XGBoost, RF, LR]
    E --> F[Evaluation: Performance Comparison Accuracy, Type I/II Errors, AUC]
    F --> G[Outcomes: Identified Best Pipeline Combinations & Improved Performance vs Literature]