Notes: Limitations

Our study has a number of limitations:

• Discussing the basis of the current regulatory standard is beyond the scope of this paper, but there are ethical, legal, and practical reasons behind the present approach. The existing policy, for instance, is naturally concerned with lenders using protected classes, such as race, gender, or age, or information highly correlated with these classes, such as neighborhood, to discriminate against loan applicants. Thus, implementing our approach on a large scale under current law raises complicated questions. For a discussion, see our paper.
• The complexity of ML models, including transparency issues, creates additional risks for lenders that are beyond the scope of this research.
• Our analysis represents a stylized version of the lending-decision process and does not fully reflect the dynamics of a particular credit market or lender. For example:
• We assume that repayment behavior for simulated loans is the same as for loans the individual already has.
• We assume that all lenders are using the same models and threshold policy.
• Evaluating the long-term effects of any policy is challenging; both applicant and lender behavior can change in response to the policy.

For more on scope and limitations, see our paper.