Model validation is a routine we’re happy to do
We follow many annual routines out of a sense of duty or responsibility: car inspections, yearly physicals, eye exams and so on. While a rare few may eagerly await those annual trips to the dentist, for most of us, these are chores we’re glad to be done with, but they aren’t especially rewarding in and of themselves. In fact, when performing such a job turns out to be a truly a pleasurable experience, it’s often worth talking about.
At VantageScore Solutions, one of our biggest annual checkups is model validation. We take a look at each of our model’s predictions of consumer loan-default behavior and compare them to actual behaviors as reflected in credit files at the three major credit reporting companies (CRCs): Equifax, Experian and TransUnion. We also evaluate the models’ consistency across the three CRCs, their performance in different economic quadrants of the United States, and performance on originations and account management for three key lender categories — auto, bankcard, and real estate.
We publish the results of our yearly validations on our website for everyone to see. We have always done this, and with model governance now at the forefront of lender and regulator concerns, providing this kind of transparency is especially important.
Happily, the 2014 validation process for VantageScore 3.0 was, as with past validations, a fulfilling exercise. The results are summarized below, but suffice it to say that they affirm the hard work and continual vigilance of our analytics team.
Each version of the VantageScore model represents countless hours of testing and statistical analysis aimed at understanding and encapsulating the factors that predict future loan defaults. Those factors shift over time as consumer behaviors change, along with the economic conditions that drive them, and each new version of the model is tuned to reflect those changes.
Speaking of adapting to changing conditions, you’ll find an article in this issue summarizing my recent opinion column in American Banker, explaining why it makes sense for lenders to lower their credit score cut-offs as default risk declines across the entire U.S. population.
You’ll also find a great article by John Ulzheimer on the growing trend of free credit scores (and how to get the most out of it), and a “Did You Know” explanation on why credit scores are three-digit numbers.
In closing, you’ll find an incisive “Five Questions with” interview featuring my friend Debra Still, former chairman of the Mortgage Bankers Association and CEO of Pulte Mortgage, the financing arm of America’s largest homebuilder. I think you’ll find her thoughts on how mortgage companies affiliated with established homebuilders have a deeply vested interest in responsible lending and how their approach can foster deep, sustainable customer relationships.
As always, this edition of The Score offers a little something for everyone. Enjoy, and please feel free to share with your colleagues, friends, and family members.