Algorithmic bias in AI and machine learning models is a problem that many researchers are trying to fix by creating tools and frameworks to identify them and eventually mitigate them. The common biases that exist are, for instance, gender-bias, racial-bias, among others. As machine learning models are trained on human-generated data, eliminating bias entirely is impossible. However, researchers are actively working on preventing it by developing tools to identify and work on them.
Recently, researchers at Princeton University developed a tool that identifies potential biases in image datasets that are used to train AI systems such as computer vision models. The open-source tool called REVISE can automatically uncover potential bias in visual datasets. The findings by researchers at the Princeton Visual AI Lab is a more effective way to mitigate bias which was suggested by them earlier.
REVISE or REvealing VIsual biaSEs use statistical methods to study the dataset and identify potential bias across three dimensions — object-based, gender-based and geography-based. As the researchers mentioned, it works by filtering and balancing a dataset’s images in a way that requires more direction from the user. “It uses existing image annotations and measurements such as object counts, the co-occurrence of objects and people, and images’ countries of origin to study the bias,” the research citing noted.
AIM Daily XO
Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.
In research dating back to Feb 2020, researchers from Princeton and Stanford University, researchers addressed bias in AI by developing methods to obtain fairer datasets containing images of people. It worked by suggesting improvements in ImageNet, which is a database of more than 14 million pictures used extensively for developing computer vision models. It could identify non-visual concepts and offensive categories, such as racial and sexual characteristics, among ImageNet’s person categories and proposed removing them from the database.
While these developments to identify bias in the image datasets are revolutionising the the area of computer vision, we bring five more such tools and frameworks that are being extensively used to identify and remove bias in AI and ML models.
Download our Mobile App
FairML
A framework to identify bias in ML models, FairML works by finding relative significance and importance of features used in the machine learning model to detect bias in the linear and non-linear model. It can work upon attributes such as gender, race, religion and others to find out data that may be biased. It works by auditing the predictive models by quantifying the relative significance of the model’s input, which helps in assessing the fairness of the model.
Know more about it here.
IBM AI Fairness 360
This open-source toolkit by IBM helps mitigate bias from massive datasets as it is developed on more than 70 fairness metrics and 10 bias mitigation algorithms. These bias algorithms work on areas such as re-weighting, optimised preprocessing, among others. A developer can apply these bias mitigation algorithms to identify fairness and compare with the original model. An open-source toolkit, it can be used to examine, report, and mitigate discrimination in ML models throughout the AI application lifecycle.
Know more about it here.
Accenture’s “Teach and Test” Methodology
Launched in 2018, this framework by Accenture ensures that AI systems are producing the right decisions in two phases — teach and test. While the prior focus on the choice of data, models and algorithms used to train machine learning, the latter works on AI model scoring and evaluation. It experiments and statistically evaluates different models to select the best performing model to be deployed into production while overcoming bias or risks of any form. Mostly used in financial services, it achieves 85% accuracy rate on customer recommendations.
Read more about it here.
Google’s What-If Tool
This interactive open-source tool by Google allows a user to investigate machine learning models visually. A part of open-source TensorBoard, it can analyse datasets in addition to trained TensorFlow models. It provides an understanding of how models work under different scenarios and build rich visualisations to explain model performance. Its bias detecting feature allows the user to manually edit samples from a dataset and study the effect of these changes through the associated model. Its algorithmic fairness analysis can detect features and discover patterns that were previously not identifiable.
Explore the tool here.
Microsoft’s Fairlearn
An open-source toolkit by Microsoft, it allows AI researchers and data scientists to detect and correct the fairness of their AI systems. With two components — an interactive visualisation dashboard and bias mitigation algorithm — this tool works on improving the fairness and model performance quite drastically. As the company notes, prioritising fairness in AI systems is a sociotechnical challenge and that the goal of this tool is to mitigate fairness-related harms as much as possible.
Know more about it here.
