Recently, researchers from the University of Toronto and LG AI Research developed an Explainable AI algorithm (XAI) to help identify and eliminate defects in display screens. Researchers claimed the algorithm outperformed the existing approaches and benchmarks.
Last year, LG invested in the University of Toronto’s artificial intelligence research to focus on building business-to-business applications. The university has been doing extensive research in deep learning. Meanwhile, LG has been leading the efforts in artificial intelligence with its AI research centre in Toronto and LG Science park, the global research and development arm of the LG group of companies.
While the LG and U of T team have tapped AI to improve the functionality of LG products such as TV and smartphones, it’s the first time they are venturing into explainable AI territory.
The research team included U of T Engineering graduate Mahesh Sudhakar and master’s candidate Sam Sattarzadeh and researchers led by Jongseong Jang at LG AI Research, Canada.
Context
The new XAI algorithm addresses the black-box problem in machine learning models. Researchers said, in a black-box model, training data could be provided in the form of millions of labelled images. The algorithm learns from this data and eventually associates certain input features with certain output features. The machine decides for itself which aspect of the image to pay attention to.
For industries that rely on sensitive data like healthcare, law, and insurance, such black-box models may present challenges. Any actions based on biased information may lead to severe consequences. For instance, machine learning models may determine a patient has a 90 percent chance of having a tumour. The margin for error on such a critical diagnosis is low. And the black-box nature of the model means doctors have no clue how the algorithm arrived at the diagnosis, which can have serious consequences.
Researchers, therefore, proposed a glass-box approach that makes the decision-making transparent. They run the XAI algorithms with traditional algorithms to audit the validity and the level of training performance. It also allows carrying out tasks such as debugging and finding training efficiencies.
Methodology
Researchers used two methodologies to develop XAI algorithms — backpropagation and perturbations.
For instance, in backpropagation-based methods, only the local attributions are represented, making them unable to measure global sensitivity. This drawback is addressed by image perturbation techniques used in recent works such as RISE and Score-CAM.
Perturbation methods sacrifice speed for accuracy and involve changing data inputs and tracking the corresponding outputs to determine the necessary compensation. However, researchers said feedforwarding several perturbed images in these works make them very slow. Explanation maps produced by CAM-based methods suffer from a lack of spatial resolution as they are formed by combining the feature maps in the last convolutional layer of CNNs, which lack spatial information on the captured attributions.
“In this work, we delve deeper into providing a solution for interpreting CNN-based models by analysing multiple layers of the network. Our solution concentrates on mutual utilisation of features represented inside a CNN in different semantic levels, achieving class discriminability and spatial resolution simultaneously,” the researchers stated.
Borrowing productive ideas from different approaches, the researchers formulated a four-phase explanation method. In the first three phases, information extracted from multiple layers of the CNN is represented in their accompanying visualisation maps. These maps are then combined via a fusion module to form a unique explanation map in the last phase. They also used techniques such as block-wise feature explanation, feature map selection and attribute-based input sampling to get results.
Researchers hope XAI algorithms could be used in areas where it’s critical to understand how machine learning makes its decision. The algorithm has a lot of application potential in product features, manufacturing, supply chain, etc.
