Can AI-Based Credit Decisions Be Explained?

The use of AI in business has been moot for a while now. One of the biggest points of the debate has been the explainability of decisions by AI. AI may or may not explain all the decisions arrived at but one could mimic an AI process and develop an understanding of the decisions made. 

Consider the following recent event – a New York financial regulator has reportedly initiated an investigation into the allegations of gender discrimination against the Apple Card and its issuer Goldman Sachs. American businessman David Heinemeier Hansson had called out Apple Card for extending him 20 times the credit limit of his wife, even though they file joint tax returns and she has a higher credit score.

In a series of tweets where he called the Apple Card a “sexist program”, Hansson said that even after his wife completely paid off her limit, the card didn’t approve any spending until the next billing period. He blamed Apple’s “black box” algorithm for the disparity in credit limit. A black box algorithm is an artificial intelligence system whose decisions cannot be explained.

He also shared his interactions with Apple customer service agents who couldn’t explain the anomaly and blamed it on the algorithm that determines users’ credit-worthiness. Hansson said that the customer service was quick to respond, but nobody is authorized to discuss the credit assessment process and there is no opportunity to present evidence.

Even Apple co-founder Steve Wozniak commented on the thread, sharing a similar predicament as Hansson. “The same thing happened to us. I got 10x the credit limit. We have no separate bank or credit card accounts or any separate assets. Hard to get to a human for a correction though. It’s big tech in 2019,” he commented on Hansson. (Source).

The above-reported event points towards a ‘black box’ algorithm ostensibly used for credit decisions by Apple and Goldman Sachs. As a Data Scientist, one should be curious to know what might be inside this ‘black box’ algorithm. Since this particular algorithm is private to Apple and Goldman it would make sense to try and mimic it by considering the factors that would normally go into a credit scoring model, to develop an understanding of the functioning of this algorithm.  

What are the factors that would be considered by retail credit scoring models such as FICO, TransUnion CIBIL, amongst others? Fair Isaac, the deviser of the FICO score seems to use the following weightages — 35% for payment history, 30% for the amount outstanding, 15% for length of history, 10% for new credit, and 10% for types of credit used. CIBIL score seems to consider — timely payments, length of card accounts, if using EMI payments thereof, credit utilization ratio, increases in credit limits, types of credit, amongst others. 

Now the question to explore is differential decisions by an AI-based credit model. The constraints in the above case are we do not know the full financial profiles of the customers involved, their total secured and unsecured credit. However, let us use the factors and weightages used by standard credit models to delve deeper. It is clear that 80% of the credit score from either of the two models – FICO, CIBIL are easy to explain and understand. These are payment history, length of history, and the amount outstanding. Thus David’s assertion about his wife having paid off the outstanding amount at a certain point in time does not inform us about her past payment history – was it always consistent, were their periods of default etc. Any reported incidence of default in any manner would affect the credit score and hence credit denial from banks and others. 

The other 20% of factors affecting a credit score or credit advance would include offers of increase in credit limit, types of credit — secured, unsecured, and credit utilization ratio. If a certain customer has been offered increases in credit limits over a period of time, then the bank or the franchise is clearly impressed with the creditworthiness of this particular customer. If a customer has managed to keep the credit utilization at a certain threshold that points to a less risky customer. Banks and franchisees also have access to most if not all the credit usage of most customers through credit agencies that would allow to size up a customer’s total credit usage and hence their creditworthiness.  

A case can be made for a primary card customer versus a secondary card customer. It is very likely that Steve Wozniak had built creditworthiness that his wife may or may not have with independent cards / accounts. Most credit models in the financial services sector seem to use logistic regression as a base reference model that is used for bucketing customers. Data availability of a particular customer could determine the treatment a bank or franchise might give to a customer. Does mortgage payments affect credit card issuance and limits? Most likely yes. Does new credit issuance affect a credit score? It would appear so. 

Thus we can unravel the credit decisions under the question to a large extent by delving into the process of decision making. A specific decision made for a certain customer might still escape us however it would not be difficult to follow the general trend. 

This article is presented by AIM Expert Network (AEN), an invite-only thought leadership platform for tech experts. Check your eligibility.