GIPHY, one of the Internet’s biggest GIF sharing and creation platforms, recently open-sourced a model that allowed for the facial recognition of celebrities. The company reported that artificial intelligence could discern over 2,300 distinct faces of celebrities, with an accuracy of 98%. The database also includes the data of Indian celebrities like Shah Rukh Khan and Virat Kohli, among others.

The model was trained based on GIFs submitted to the platform and was created with an intention of reproducing similar offerings which were blocked off by a paywall.

GIPHY also revealed that a majority of the traffic across all of their sources came from celebrity searches. In order to make the process more efficient and provide a new way of searching on the platform, they created this algorithm.

The Need For A Facial Recognition Algorithm

GIPHY found the need to create this so that they could find and annotate the content on their platform. Since the platform has over 100 million daily active users, the throughput for GIFs being sent is said to be over 1 billion every day.

Moreover, the platform also collaborates closely with multiple celebrities to create reactions, GIFs and stickers for their library. Along with the abundance of content regarding celebrities on the platform, GIPHY also makes promotional GIFs for movies.

This adds to their already vast library of celebrity content, providing fodder for the AI to be trained on. The need for a new search method, along with the easy availability of labelled data made creating a model the natural way forward for the site to serve its queries.

How The Model Was Trained To Pick Out Faces

The first step was to organize the data on the platform, which was done by extracting all celebrity names from the top 50,000 searches on their platform. This resulted in a dataset of above 2,300 celebrities being returned, along with labelled data for determining who the celebrities were.

This was the basement for the data involved in creating the model. However, some data was not as clean as they would have expected, especially for the representative images of non-popular celebrities. To tackle this, the team then grouped these images by similarity and measured the uniformity of each data set.

This led them to find that the noisy data were similar in distribution to the clean ones, marking that it was safe to train the AI on it. Proceeding this, the image processing module of the model was split into two parts. The first was detected where the face was in the image or GIF.

This was done through the use of a Multi-Task Cascade Convolutional Network or MTCCN. This can detect faces across frames of the GIFs. The detected faces are then captured and then sent to a deep convolutional neural network.

This CNN is based on the Resnet-50 model and is trained on the dataset. This is where most of the recognition occurs, with the model being a facial features extractor. As the name suggests, the model identifies key facial features and creates a vector space of faces. These are grouped together by using centre loss, and then each face is given a celebrity prediction along with a unique vector.

Post this step, a Gaussian Mixture Model is used to generate from the mixture of Gaussian distributions. In this use-case, it is used to cluster faces by vector representations. These clusters are used to compute an aggregate prediction for all the presented faces, along with a confidence score.

Accuracy For An Integral Part Of Internet Culture

GIPHY is one of the companies that prides itself on being an important part of Internet culture, as GIFs become more and more widely used across the net. They open-sourced this algorithm to contribute to this, even concluding their blog post by asking the users to come up with newer use cases or extend it for their own needs.

This feature is also sure to increase the usability of the platform among Indian users, with celebrities such as Shah Rukh Khan, Salman Khan, Virat Kohli, and Imran Khan being a part of the roster picked by GIPHY to train this model. All in all, it presents a valuable addition to the site’s arsenal of tools to enable the rise of further Internet culture.