Fairness Goals

How Can Incorporating Group-Specific Thresholds Affect Fairness?

We find that applicants living in (LMI) areas have less access to credit than applicants not living in LMI areas, even when more sophisticated models are used (see Model Improvement). One solution to create more equal access to credit between groups is to create a different credit score requirement for specific groups of applicants (i.e., group-specific thresholds) that equalize credit access. This translates to lowering the credit score threshold in LMI areas rather than using a single credit score threshold for all applicants. This approach requires the use of sensitive information — in this case, neighborhood — to identify the LMI- and non-LMI-area applicant groups. The following interactive graphic can be used to switch the single and group-specific threshold options to highlight a key benefit and cost of this approach.

Group-specific thresholds reduce disparities in credit access between the groups by allowing creditworthy applicants in LMI communities to have more access to credit; however, the number of applicants who get loans and then default will also increase.

Here are several key ideas that define fairness within this research and how group-specific thresholds are determined.

A loan applicant is approved or rejected for a loan. Approved borrowers will either repay or default on the loan. For every applicant who applied for a loan but did not receive one, the two hypothetical outcomes are the same: They could have repaid, or they could have defaulted.¹ Each of these scenarios has a name:

• True Positive (TP): An applicant is granted a loan and successfully repays it. The model made the correct classification.
• False Positive (FP): An applicant is granted a loan but defaults. The model made an incorrect classification.
• True Negative (TN): An applicant is not granted a loan; however, if they had been granted a loan, they would have defaulted. The model made the correct classification.
• False Negative (FN): An applicant is not granted a loan; however, if they had been granted a loan, they would have successfully repaid it. The model made an incorrect classification.

We focus on two key metrics related to these scenarios: the true positive rate (TPR) and the false positive rate (FPR).

The TPR is the percentage of applicants who are granted a loan and will repay. Ideally, we would want the TPR to be as high as possible. We can think about increasing TPR as “giving credit where it’s due”:

The FPR is the percentage of applicants who are granted a loan and will default. Ideally, we would want the FPR to be as low as possible. Default is costly to the lender and the borrowers who default:

The best scenario is one in which as many creditworthy applicants are granted a loan as possible (TPR is highest) and few applicants who would default are granted loans (FPR is lowest). Try using the following interactive graphic to see how TPR and FPR change with different thresholds.

Applicants who are capable of repaying their loan should have equal access to credit, regardless of whether they live in an LMI neighborhood. In that case, the TPR for LMI- and non-LMI-area applicants should be roughly equal. In our research, we examine fairness by comparing the TPR between LMI- and non-LMI-area applicants. If the difference in TPR is large, creditworthy individuals of LMI and non-LMI areas are getting unequal access to credit. If the difference is small, they are getting equal access and the situation is fairer.

Under a single-threshold approach, we find that fairness (i.e., equal TPRs) is not achieved. Applicants from LMI areas have a lower TPR at any given threshold than applicants from non-LMI areas (as shown next). Group-specific thresholds can help to achieve this fairness goal by equalizing — or at least reducing — the difference in TPR. Try using the following interactive graphic to swap between different fairness goals to see how group-specific thresholds are set to equalize or reduce the TPR disparity.

Group-specific thresholds can mechanically improve fairness by equalizing — or reducing — the disparity in TPR, allowing for more equal access to credit to creditworthy applicants in LMI and non-LMI areas. This approach, however, doesn’t come without a cost. Group-specific thresholds increase the FPR for LMI-area applicants. A higher FPR for LMI applicants means more LMI borrowers will default on loans, which is costly to the lender and the applicants who default. To make a profit, lenders need to make multiple good loans for every defaulter because losses on defaults are much higher than profits from a good loan.² For a borrower, defaulting on a loan can trigger aggressive collections efforts, including judicial action, and can significantly reduce a borrower’s credit score, making it more difficult for them to access credit in the future. The following graph shows the change in TPR and FPR across different fairness goals.

Next: Learn about Model Improvement with Fairness Goals