Skip to content
Tutorials Center
My learning

Topics

Lesson 99 of 111 ยท Projects

Project: Loan Approval Prediction

Predict loan approvals from mixed applicant data with missing values: pipelines, engineered ratios, repeated cross-validation, costs and explanations.

A bank receives thousands of loan applications. Each one has numbers (income, loan amount, term), categories (education, property area, marital status) and gaps (the applicant left a field empty). The task: predict whether an application will be approved, so that obvious cases can be decided quickly and borderline cases go to a human.

This project is a compact tour of the "typical business table" problem: a few hundred rows, mixed data types, missing values, skewed incomes and an outcome that depends on a mix of factors. It complements the heavier credit risk project, which focuses on defaults, leakage and fairness audits. Here you will build a clean preprocessing pipeline, engineer two or three useful ratios, compare models fairly on small data, and translate the result into decisions with costs.

The table is synthetic (generated by the code, with realistic gaps), so it runs anywhere. Related lessons: missing values, column transformer, pipelines and cross-validation.

Step 1: Inspect the table

Always start by looking. How many rows and columns? Which columns have gaps, and does being missing seem to mean something? How is the target distributed, and how does it relate to the most obvious columns?

Python Step 1: look before you model

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

There are 900 applications and 73.8% were approved, so the classes are imbalanced but not extremely. Six columns have gaps (59 missing credit histories, 60 missing employment types, 30 missing dependents counts, and so on). Credit history is clearly the strongest signal: applicants with a good history were approved 83.6% of the time against 19.6% for those with a bad one. Applicants whose history is unknown look like good-history applicants (83.1%). In this table the gaps were created at random, but in real data missingness can itself carry information (people with a bad history may skip the question), so it is worth keeping a record of it. Income is right-skewed: the mean is above the median and the maximum is far above both, a sign that a log transform is useful.

Step 2: One pipeline from raw table to prediction

All preparation is part of the model so that it is learned only from training data in every cross-validation fold (leakage):

  1. Engineer features. Total household income (applicant plus co-applicant), its logarithm, and the monthly payment share (loan amount spread over the term, divided by total income): a heavily indebted application is riskier than a large one. The dependents count 3+ becomes the number 3.
  2. Numeric columns: fill gaps with the median and add a missing indicator column (add_indicator=True) so the model can see which values were filled; scale for linear models.
  3. Categorical columns: fill gaps with the most frequent value, then one-hot encode.

Because the dataset is small, a single train/test split would be noisy. We use repeated stratified 5-fold cross-validation (10 fits per model) and report the mean and the standard deviation. The "always approve" model is the baseline.

Python Step 2: compare four models with repeated cross-validation

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

"Always approve" is already 73.8% accurate and has an F1 of 0.849, which shows why accuracy alone is a weak yardstick here. The logistic regression leads on every score (accuracy 0.861, F1 0.909, AUC 0.862), the random forest and gradient boosting trail slightly (AUC about 0.84), and the standard deviations (0.02 to 0.03) are about as large as the gaps between models. So with this much data the honest conclusion is "the simple model is at least as good as the complex ones", which is also the model that is easiest to explain to an applicant and to a regulator. In general, do not pay for complexity that cross-validation does not reward.

Step 3: Explain the model and set the decision rule

A loan decision must be justified. We fit the logistic regression on 75% of the data and examine it in three ways:

  • Permutation importance shuffles one raw input column at a time and measures how much the AUC falls. This respects the full pipeline, including the engineered features.
  • Costs. Approving a loan that goes bad costs far more than refusing a good applicant. With a cost of 5 for a bad approval and 1 for a good refusal, we compare decision thresholds.
  • A single-applicant explanation: for a linear model, the contribution of each (standardized) feature to the log-odds is coefficient ร— value.
Python Step 3: importance, costs and one explained decision

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Credit history is the most important input by a wide margin (AUC drops by 0.17 when it is shuffled), followed by the loan amount and income, which together drive the monthly payment share. The cost table shows that the default threshold of 0.5 is not the cheapest choice: raising the threshold to 0.8 refuses more good applicants (30, instead of 5) but approves 12 fewer bad loans, and the expected cost drops from 155 to 120. Whether this is the right trade depends on real costs and on how many applications people can review by hand, so treat the threshold as a business parameter. The last block explains one applicant: the heavy-weight factors are the payment share and the good credit history.

Good practice for lending models

  • Do not use protected attributes such as gender, age or ethnicity, and check indirect proxies. Compare approval and error rates across groups before launch (fairness, credit risk project).
  • Keep humans in the loop for borderline scores, and keep an audit trail.
  • Give reasons. Many jurisdictions require lenders to explain a refusal; linear models and monotonic models make that straightforward.
  • Beware of selection bias: you only see repayment outcomes for applicants who were approved in the past, so the model learns from a filtered population.
  • Monitor approval rates and score distributions after launch (monitoring).

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why is a missing-indicator column (add_indicator=True) added next to a median-filled numeric column?

    A filled value looks like a real one. The indicator preserves the information that it was unknown, which sometimes predicts the outcome.
  2. 2. With 900 rows, why use repeated stratified cross-validation instead of one train/test split?

    On small data the score depends on which rows land in the test set. Averaging many splits and reporting the standard deviation is more honest.
  3. 3. The models differ by about 0.02 AUC and the standard deviation is about 0.025. What is the sensible conclusion?

    When differences are smaller than the variation between folds, other criteria (simplicity, speed, explainability) should decide.
  4. 4. Why might the best threshold for approving loans be above 0.5?

    With asymmetric costs, the cheapest decision rule demands a higher probability of a good outcome before approving.

Summary

  • Business tables mix numbers, categories and gaps; handle all of it inside one pipeline so that cross-validation stays honest.
  • Impute, add missing indicators, scale only where needed, and engineer meaningful ratios such as the monthly payment share.
  • On small data use repeated cross-validation, report the spread, and prefer the simplest model when scores are within the noise.
  • Explain the model with permutation importance and per-applicant contributions, and choose the decision threshold from real costs.
  • Lending decisions need fairness checks, reasons for refusals and monitoring.

Next: a regression project with outliers and skewed targets: Box Office Revenue Prediction.