The Curious Case Of Paradoxes In Business Analytics

“All generalisations are false, including this one,” wrote Mark Twain. The human mind has the meta ability to yoke two contradicting thoughts to birth a paradox. For instance, simple tasks on machines demand more computation. Computers can outsmart chess grandmasters but still fall short on trivial human tasks. Moravec’s paradox is all about this. “Abstract thought, though, is a new trick, perhaps less than 100 thousand years old. We have not yet mastered it. It is not all that intrinsically difficult; it just seems so when we do it,” observed Moravec.

At MLDS 2021, David Hauser, Chief Science Officer at Genpact, offered a refreshing perspective on paradoxes. In his talk, titled “Paradoxes and Unexpected Counter-Intuition in Business Analytics and Mathematics”, gave the audience a sneak peek into how business decision making can be heavily influenced by counterintuitive thinking.

Paradoxes, Business Analytics and Mathematics

(Image credits: David Hauser, Genpact)

When it comes to paradoxes, we tend to miss the trees for the forest. The more paradoxes there are, the less chances of them getting noticed. And if they are noticed, their influence on the outcome is disregarded.

Subscribe to our Newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Paradoxes populate all kinds of common and complex mathematical applications, including business analytics, machine learning, econometric modeling, statistical reporting, and business inferential and strategic development processes. Some paradoxes are comical, while others are odd, and a few others are highly counter-intuitive. Yet all have tremendous implications. And in some (many) contexts, the paradoxes can lead to diametrically opposite outcomes than intended.

The more sophisticated and realistic a model, the more of a blackbox it becomes.”

-Bonini’s paradox

As a complex system grows in sophistication and completeness, the less understandable it becomes. As a model grows more realistic,it becomes more difficult to understand. Clients want the model to be understandable and realistic. So, here is a paradox that a data scientist has to live with. Hauser also stressed on how making solutions intuitive means to give up accuracy (eg:pruning the network). The moment we introduce non linearities or interactions, the answers become less intuitive. Business decisions are all about making the right choices. But, we live in a world of abundance. There are so many tools and services to choose from. On similar lines, Fredkin’s paradox states the more two alternatives seem similar, the harder it is to choose and the more time/effort required to decide. Hauser presented the audience with two scenarios:

  • where two variables highly correlated in a model
  • many explanatory variables are highly correlated in a model

If Fredkin’s paradox is not dealt with, the organisation would be looking at a potential “buyer’s remorse” as more time is wasted on meaningless decisions and unimportant differences become more critical.

In AI, the easier the task, the more computational effort is required. A business looking to implement AI usually picks up complex projects than works towards easier goals. For example, AI search engines, driverless cars are complex tasks whereas matching disparate datasets and real-time outlier detection are the easy kind. Another premise of poor decision making in business analytics is the problem of overconfidence bias:

  •  Overconfident about some feature being wrong.
  • Overconfident about some feature being right.

“ There is more benefit from investing in customer service than in reducing risk of failure.”

David Hauser

In both cases, the data scientist stands to lose a lot of information. Sometimes different domains can present open with almost identical variables. This is why Hauser underlines the significance of perfectly capturing the underlying mathematics of business analytics. Hauser also advocated the need for objectively questioning and challenging assumptions around data. “There is a need for Analytic Literacy,” says Hauser. In the end it all boils down to one thing–implementation. Some ML models might look great to a team of data scientists but a sub standard model’s results might impress the client and that is all one would need; this is the solution paradox!

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Download our Mobile App

MachineHack | AI Hackathons, Coding & Learning

Host Hackathons & Recruit Great Data Talent!

AIMResearch Pioneering advanced AI market research

With a decade of experience under our belt, we are transforming how businesses use AI & data-driven insights to succeed.

The Gold Standard for Recognizing Excellence in Data Science and Tech Workplaces

With Best Firm Certification, you can effortlessly delve into the minds of your employees, unveil invaluable perspectives, and gain distinguished acclaim for fostering an exceptional company culture.

AIM Leaders Council

World’s Biggest Community Exclusively For Senior Executives In Data Science And Analytics.

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox