Credit risk is an inherently unbalanced classification problem, as good loans easily outnumber risky loans. Therefore, you’ll need to employ different techniques to train and evaluate models with unbalanced classes. I'll be using imbalanced-learn and scikit-learn libraries to build and evaluate models using resampling.
"Using the credit card credit dataset from LendingClub, a peer-to-peer lending services company, you’ll oversample the data using the RandomOverSampler and SMOTE algorithms, and undersample the data using the ClusterCentroids algorithm. Then, you’ll use a combinatorial approach of over- and undersampling using the SMOTEENN algorithm. Next, you’ll compare two new machine learning models that reduce bias, BalancedRandomForestClassifier and EasyEnsembleClassifier, to predict credit risk. Once you’re done, you’ll evaluate the performance of these models and make a written recommendation on whether they should be used to predict credit risk".
-Step 1: Use Resampling Models to Predict Credit Risk
-Step 2: Use the SMOTEENN Algorithm to Predict Credit Risk
-Step 3: Use Ensemble Classifiers to Predict Credit Risk
Naive Random Oversampling
Balanced Accuracy: %64.7
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .66/.67
SMOTE Oversampling
Balanced Accuracy: %62.5
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .62/.67
Undersampling
Balanced Accuracy: %52.1
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .59/.46
Combination Under-Over Sampling
Balanced Accuracy: %62.5
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .71/.54
Balanced Random Forest Classifier
Balanced Accuracy: %78.7
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .67/.91
Easy Ensemble AdaBoost Classifier
Balanced Accuracy: %92.5
Precision: low for High-risk loans, high for Low-risk loans
Recall Score: High-Low risk = .91/.94
Based off all the models ran, the Easy Ensemble AdaBoost Classifier shines above all the others with a %92.5 accuracy at detecting if someone is a low-credit risk, the next highest only having ~%78. None of the models hold well at determining whether someone is high-risk, that's something to take into account if what youre seeking is pinpointing high-risk individuals. For this particular dataset at least, it makes the most sense to go with EEAC algorithm.