Now Reading
IBM Launches FreaAI That Automatically Finds Weaknesses In ML models

IBM Launches FreaAI That Automatically Finds Weaknesses In ML models

kumar Gandharv
  • IBM FreaAI will be able to automatically discover inaccuracies in an AI model and suggest course corrections.

IBM’s Research team at Haifa built FreaAI to detect flaws in machine learning models by automatically scrutinising human-interpretable chunks of data to predict when algorithms are working and when they are not. 

The researchers addressed the challenge of finding under-performing data slices to validate the ML performance of ML solutions. They developed automated slicing heuristics and implemented them in FreaAI, such that the resulting slices are correct, statistically significant and explainable.

Register for Analytics Olympiad 2021>>

“It might be acceptable in some cases to have an incorrect answer from time to time, but it’s vital that we are able to understand and control the extent of a mistake and the circumstances under which it could occur,” the IBM blog said.

For example, take a health insurance firm. Here, FreaAI could be used to uncover the error rate in insurance approval or rejection and fix the issue. Going forward, FreaAI will be able to automatically discover inaccuracies in an AI model and suggest course corrections.

Image Credits: IBM

IGNITE is an IBM platform that pools automated testing services. The Big Blue incorporated FreaAI to IGNITE, along with additional testing technologies from their collaborators in the India Research Lab, to provide IBM clients with Full AI Cycle Testing (FACT) capabilities.

Traditional testing 

Today, machine learning models are the key drivers of business decisions. And, these ML models must be updated over time to neuter ‘Model Drift’ risks. Model drift can be classified into two broad categories.

  • Concept drift occurs when the statistical properties of the target variable changes leading to a broken model.
  • Data drift happens when the statistical properties of the predictors change.

Testing has traditionally been an exercise in optimising detection based strategies for correcting business processes. Machine learning and neural networks offer obvious advantages in software testing compared to traditional testing:

  • Traditional testing tools can test and provide results automatically, but they still require human oversight.
  • Traditional test automation technologies can’t identify which tests to run without human supervision; thus, they wind up performing every test or a predetermined set of tests.
  • AI in test automation tools can intelligently pick which tests to run and then trigger them automatically based on test status, recent code modifications, available data and associated code metrics.
What Do You Think?

Subscribe to our Newsletter

Get the latest updates and relevant offers by sharing your email.
Join our Telegram Group. Be part of an engaging community

Copyright Analytics India Magazine Pvt Ltd

Scroll To Top