Published by VantageScore Oct 2017:
Credit scores have been a ubiquitous and valuable component of the underwriting process for decades. Credit bureau risk scoring models have been available since the late 1980s and have been used for almost three decades by credit card issuers and auto lenders. Further, it’s been almost 20 years since both Fannie Mae and Freddie Mac endorsed the use of credit scores in their respective automated underwriting systems, known as Desktop Underwriter and Loan Prospector.
The measure of a good credit score model is its ability to rank order. This means that the model effectively identifies those who will remain current on their payments versus those who will not.
So-called “generic credit scoring models” are built using millions of credit files to represent the enormous number of patterns of credit management and payment behaviors for a set of products during a specific time frame and economic condition. As long as those patterns and economic conditions remain fairly stable, the model will continue to rank order as well as when it was first developed.
However, when economic conditions change dramatically (i.e., as they did during the recession), models deteriorate and fail to rank order as well.
For a lender, failing to rank order means consumers who will ultimately default (i.e., fall 90 days or more past due) receive high enough scores to result in credit and loan approvals. In the end, lenders experience higher losses, causing their businesses to be less profitable.
A core responsibility for credit score model developers and users is to validate their models in order to assess whether the models are still performing strongly
For the developer, these validation procedures should assess whether the scoring model effectively rank orders populations and key subpopulations, whether the model has implicit bias that could cause a disparate impact, and whether scores reflect the appropriate level of risk.
A substantial deterioration in the performance of any of these tests, the availability of newly developed data or modeling techniques that could materially improve a model’s predictive performance or ability to score more people should cause the developer to consider developing a new model. To aid lenders and regulators, VantageScore Solutions transparently posts the results of its validations publicly on our website.
Similarly, model users – such as credit and risk functions within lending institutions – should periodically consider their own validation processes. These processes, as outlined in the
Office of the Comptroller of the Currency’s (OCC’s) 2011-12 guidelines, should be implemented to determine how effectively the scoring model they are using identifies and measures the risk of their lending strategy on their particular customer base.
Typically, when lenders validate their incumbent scoring model, they also evaluate a number of “challenger” scoring models to determine whether more effective risk management tools are available than the one being used. In the event that a challenger model is more predictive or scores a larger population of consumers with equivalent accuracy, the lender will begin a process of replacing the incumbent model with the new model.
This can be an involved process, requiring redevelopment of strategies with new score cutoffs, rebuilding internal models and decision processes, coordinating and aligning reason codes, as well as initiating thorough audit and compliance reviews. Ultimately, the cost of converting to the new model must be offset by the opportunity to enhance profitability through loss reduction and customer revenue.
Quite interestingly, there is a substantial yet generally invisible benefit of these ongoing validation exercises. Today, as a lesson learned through the recession, credit scores are now often tested on an almost continual basis to determine their effectiveness. As soon as a score fails to deliver sufficient value, it is replaced by newer models that leverage more advanced credit data and sophisticated modeling techniques, which are more representative of the current credit environment.
As such, the industry standard for superior predictive performance continues to improve as the competition for better predictive performance and a larger, scoreable population intensifies with the introduction of each new model.
