Collateral Damage — Landing Credit

Collateral Damage, Landing Credit is Chapter 8 of Cathy O’Neil’s book Weapons of Math Destruction: How Big Data increases Inequality and Threatens Democracy. This section of the book details how credit software impacts people’s ability to get a job, low interest loans, and housing.

Cathy O’Neil describes how small towns used to bank. Bankers had predetermined models of you based upon race, sex, and class. Your family and contribution to the community also were weighed. Earl Isaac and Bill Fair “devised a model they called FICO to evaluate the risk that an individual would default on a loan.” This invention has a clear feedback loop and is colorblind. Therefore eliminating much of the human bias used in previous systems.

E-scores go beyond FICO scores and operate in the realm of pseudoscience. Financial livelihoods are determined by browser cookies and zip codes. E-scores create instant feedback loops. You can easily get denied service or fall into the trap of high interest loans. The chapter on for-profit colleges and payday loans are based upon e-scores. Advertising companies pray on exploitable data in order to turn a profit. Modelers are trying to answer the question “How have people like you behaved in the past?” when ideally they would ask, “How have you behaved in the past?” Grouping people in this way is not working. Models must shift towards individual behavior.

FICO credit scores aren’t perfect. The model may be fair but companies use it as a proxy for determining which candidates they’d like to hire. One missed payment during a recession is all it takes to create a feedback loop. This perpetuates the cycle of poverty and is far too easy to get sucked into.

My biggest take away is noticing how computer-generated crime lists constantly need human monitoring. More people should be hired to regulate and stabilize these systems because one false calculations can cause a plethora of problems for anyone. We need to make models more individualistic, not use credit scores in the application or promotion process, and increase transparency. “Few of us get a chance to see these details. If we did, and the FTC is pushing for more accountability, the brokers would likely find themselves besieged by consumer complaints — millions of them.”