According to research, it is estimated that AI could contribute $15.7 trillion to the global economy by 2030. As AI-enhanced products diffuse into markets and households, service providers have a gigantic task to facilitate interpretable algorithmic decision making. Over the past couple of years, many fairness tools have been introduced to make ML models more fair and explainable. In the next section, we list popular tools by the likes of Google and Microsoft, which are designed to imbue fairness into ML pipeline building.
1| Google’s Model Card Toolkit
The Model Card Toolkit is designed to streamline and automate the generation of Model Cards. These model cards are nothing but machine learning documents that provide context and transparency into a model’s performance. Integrating them into ML pipeline enables one to share model metadata and metrics with researchers, developers, etc. If your machine learning pipeline uses the TensorFlow Extended (TFX) platform or ML Metadata, you can automate model card generation.
2| Microsoft’s Responsible Innovation Toolkit
RI toolkit provides a set of practices for addressing the potential negative impacts of technology on people. Here are a couple of tools from the toolkit:
- Harms Modeling is a framework for product teams to examine how people’s lives can be negatively impacted by technology.
- Community Jury is a technique that brings together diverse stakeholders.
3| IBM’s AI Fairness 360
A Python toolkit for algorithmic fairness, AI Fairness 360 (AIF360) is designed to help facilitate the transition of fairness research algorithms to use in an industrial setting and to provide a common framework for fairness researchers to share and evaluate algorithms. This extensible open-source toolkit can help examine, report and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle.
4| Google’s What-If Tool
Researchers and designers at Google’s PAIR (People and AI Research) initiative created the What-If visualisation tool as a practical resource for developers of machine learning systems.
The What-If Tool supports:
- binary classification
- multi-class classification
- regression tasks
However, fairness optimisation strategies are available only with binary classification models due to the nature of the strategies themselves.
Try What-If Tool here.
5| Lime
Local interpretable model-agnostic explanations or Lime is based on the paper titled, ‘Why Should I Trust You?’, by a team of researchers at the University of Washington. Lime helps explain black-box classifiers. The classifier implements a function that takes in raw text or a NumPy array and outputs a probability for each class. LIME also has built-in sci-kit-learn classifiers.
Installation: pip install lime
Check Lime.
6| Microsoft’s Fairlearn
Microsoft’s Fairlearn is an open-source toolkit to assess and improve the fairness of AI systems. It primarily consists of two components: an interactive visualisation dashboard and unfairness mitigation algorithms. These components are designed to understand the trade-offs between fairness and model performance.
Check more here.
7| PwC’ Responsible AI Toolkit
PwC’s Responsible AI Toolkit is a suite of customisable frameworks, tools and processes designed to help you harness the power of AI ethically and responsibly — from strategy through execution. PwC’s team designed this toolkit with various stakeholders such as regulators, board members and many more. Not only that but the frameworks and tools available in this toolkit also help address the regulatory and compliance aspect of AI-based businesses as lwell.
8| audit-AI
Pymetrics’ audit-AI is a tool to measure and mitigate the effects of potential biases in training data. The predictions made by ML algorithm are trained for socially sensitive decision processes. According to the Pymetrics team, the overall goal of this research is to come up with a reasonable way to think about how to make machine learning algorithms fairer. Audit-ai determines whether groups are different according to a standard of statistical significance (within a statistically different margin of error) or practical value (whether a difference is large enough to matter on a practical level).
