This New Semi-Supervised Learning Method Is Gaining Traction

FixMatch

Deep neural networks are the most used model for computer vision applications, largely because of their scalability. Deep neural networks generally derive their superior performance through underlying supervised learning mechanisms. 

Supervised learning is a type of deep learning methods which uses labelled datasets. While supervised learning offers superior performance benefits, it comes at a high cost, as labelling data requires human labour. Further, the cost is significantly higher when a data labelling has to be done by an expert, such as a medical practitioner. 

In such a scenario, semi-supervised learning (SSL) proves to be a powerful alternative. SSL is a method where learning takes place with a small number of labelled data and a relatively larger set of unlabelled data. This method mitigates the need for labelling all the data as in the case of supervised learning. 

Recently, a paper accepted by the NeurIPS 2020 conference, speaks of using an SSL method called FixMatch to achieve state-of-art performance across various SSL benchmarks such as CIFAR-10 even with very few labelled data.

What is FixMatch Algorithm?

FixMatch Algorithm is essentially an SSL method that combines diverse mechanisms to produce artificial labels for unlabelled data. In particular, this algorithm uses consistency regularisation and pseudo labelling for this purpose, as well as a separate set of weak and strong augmentation.

First, in the FixMatch process, the predicted value of an unlabelled image with weak augmentation is calculated by the FixMatch algorithm. Introducing weak augmentation to the image means changing slightly by methods such as rotation and flipping. Only the image with prediction confidence above a certain threshold is treated as pseudo labels. Next, the same model is used to generate predicted values for generating the image with strong augmentation, where larger changes such as changing the temperature of the image are applied.

An artificial label is then computed on the weak augmented image, and the calculated loss is applied against the model’s output for the strongly augmented image. This introduces a form of consistency. This consistency is then regularised.

Consistency regularisation is a method where the weak and strong augmented data are used separately, and the end goal is to force the SSL model to learn to produce the same output for different ‘versions’ of an image. It increases the accuracy of the temporary label by limiting the predicted value. So even if the data is converted, the predicted value is not changed. Further, the reliability of this predicted value is ensured by incorporating the difference between predicted values of both augmented images into the objective function.

FixMatch’s Performance Against Its Counterparts

The paper (referenced above) showed that the FixMatch performed well across standard benchmarks such as CIFAR-10 and CIFAR-100. For example, on CIFAR-10 with four labels per class, FixMatch achieved a 99.43% accuracy on CIFAR-10 with 250 labels and 88.61% accuracy with 40 samples, with four labels per class.

FixMatch’s performance in most cases was found to be better than its counterparts such as Π Model, Mean Teacher, Pseudo-Label, MixMatch, UDA. Only ReMixMatch managed to outperform FixMatch in terms of its performance on CIFAR-100 benchmark.

Looking Ahead

Off late, there has been rapid progress in SSL. As opposed to other complicated learning algorithms, FixMatch is much simpler and achieves state-of-the-art results across different datasets and on several standard benchmarks.

This algorithm can also obtain high-accuracy with just one label per class. Further, by using standard cross-entropy losses on labelled and unlabelled data, the training object of FixMatch can be written in a few lines of code.

It is believed that such a simple yet high-performance SSL algorithm can allow the convenient deployment of machine learning in several domains where obtaining a complete set of labelled data is expensive.

More Great AIM Stories

Shraddha Goled
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at shraddha.goled@analyticsindiamag.com.

More Stories

OUR UPCOMING EVENTS

8th April | In-person Conference | Hotel Radisson Blue, Bangalore

Organized by Analytics India Magazine

View Event >>

30th Apr | Virtual conference

Organized by Analytics India Magazine

View Event >>

MORE FROM AIM
Yugesh Verma
All you need to know about Graph Embeddings

Embeddings can be the subgroups of a group, similarly, in graph theory embedding of a graph can be considered as a representation of a graph on a surface, where points of that surface are made up of vertices and arcs are made up of edges

Yugesh Verma
A beginner’s guide to Spatio-Temporal graph neural networks

Spatio-temporal graphs are made of static structures and time-varying features, and such information in a graph requires a neural network that can deal with time-varying features of the graph. Neural networks which are developed to deal with time-varying features of the graph can be considered as Spatio-temporal graph neural networks. 

Yugesh Verma
A guide to explainable named entity recognition

Named entity recognition (NER) is difficult to understand how the process of NER worked in the background or how the process is behaving with the data, it needs more explainability. we can make it more explainable.

Yugesh Verma
10 real-life applications of Genetic Optimization

Genetic algorithms have a variety of applications, and one of the basic applications of genetic algorithms can be the optimization of problems and solutions. We use optimization for finding the best solution to any problem. Optimization using genetic algorithms can be considered genetic optimization

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM