January seminar: Fighting sampling bias—A novel framework for training and evaluating credit scorecards

The seminar highlighted the use of Scoring models to support resource allocation decisions in finance. Credit scorecards were seen as a prominent example. As they are trained using data from past accepted applicants whose repayment conduct has been observed. This results in sampling bias because the training data only represent a subset of the borrowers to whom the model is applied in operations.

The presentation also emphasised how the research contributed to the field of reject inference in the following ways. First, the report quantified loss due to sample bias using a simulated study and demonstrated how the bias affects scorecard training and evaluation. Second, the research proposed a shallow self-learning approach that addresses training bias and mitigates performance loss by inferring labels from selected rejected applications. Finally, the article developed a new evaluation tool known as the kick-out measure. This measure lowers the bias associated with evaluating a scorecard using a biased training sample of accepted clients and a test sample indicative of the scorecard's operating conditions. Experiments using real credit scoring data proved the proposals' superiority in terms of prediction performance and profitability over existing bias-correcting approaches.

View the full paper