How Has Facial Recognition Technology Adapted To The Pervasive Use Of Masks?

As millions don masks across the globe, the propitious future of facial recognition technology may be left hanging in the balance. In a departure from WHO guidelines amid the Covid-19 pandemic, people have started adopting the practice of wearing masks in public places. Although difficult to ascertain its potency against coronavirus, this move may compromise facial recognition technology.

A fundamental application in many routine activities in countries like China, partially covering one’s face can affect related functions, including unlocking smartphones, entering certain establishments, or even unlocking bank accounts.

This is because most artificial intelligence (AI) systems are designed to factor in areas around the nose, mouth – and even shape of the face – and not just the eyes. Even companies like Google and Apple, with an arsenal of advanced technology, have been unsuccessful at surmounting the challenges posed by masks. For instance, Pixel 4’s facial recognition system is rendered useless against masked faces, although it can reportedly recognize people with beards and sunglasses. The same may be true for iPhones as well, based on growing complaints by users.

Unmasking The Challenge

This problem is not new – as indicated above, most facial recognition systems will be ineffectual against partially covered faces. However, this technology has been evolving at the epicenter of most debates around this technology – China.

China has some of the world’s most comprehensive and sophisticated electronic surveillance infrastructure in place, including facial recognition. But with most people wearing masks amid Covid-19, it has posed a unique problem for the nation.

Download our Mobile App

While few companies have made strides in developing the appropriate technology, some solutions have emerged from China in recent weeks. These companies have introduced new algorithms to its face detection tech stack for people wearing masks. 

According to a report, Chinese electronics manufacturer Hanwang – pronounced Hanvon – claims to have developed the country’s first facial recognition technology that works on masked users as well. Ready in just a month’s time after the outbreak gathered pace, the Beijing-based firm’s 20-member team used a sample database of about 6 million unmasked faces – and a smaller database of masked faces – to develop its new FaceGo technology.

ALSO READ: Why We Should Embrace Facial Recognition

The company claims that it can identify everyone in a crowd of up to 30 people with a recognition rate of about 95%. In fact, China’s Ministry of Public Security is reportedly using Hanwang’s technology to screen people and cross-reference images with its own database to identify and track people. How does this work? The facial recognition system compares certain vectors of facial features to the faces in the database. It finds a match if both meet a level of accuracy. Simply put, the company’s system tries to guess what the faces in the database would look like if they were masked. 

However, this is not without its challenges and these kinds of systems still have limitations. Hanwang’s technology struggles to identify people who are wearing both a mask and another accessory that would result in losing key facial information. This may include sunglasses, beards, or even winter wear that partially covers the face.

Moreover, these technologies – as they exist – might be limited to small-scale applications because expanding it to a wider group of people would be even more challenging. One of the reasons for this may be difficulties in identifying people with similar eyes once a population reaches a certain scale.

But it is a promising development and several other companies are also experimenting with this technology. Chinese AI startup SenseTime had also recently announced that it had adapted its product to identify masked people.

Future Of Facial Recognition With Research

Facial recognition that targets masked-wearers have been in development in academic circles for a while now. A postdoctoral fellow from Stanford University and his team had published a research around Disguised Face Identification (DFI) that made a breakthrough in 2017. 

Although, admittedly, it would be less accurate, the team had established that a match could still be conducted.

This is because face recognition works by locating certain key points around the eyes, nose and lips and connecting them together to form a unique signature. But advanced systems, like the one developed by this team, could accomplish something similar with fewer key points.

Since then, more research has been done in this space. A team from the University of Bradford was able to achieve 90% accuracy with their facial recognition model. In a study published in 2019, they demonstrated that they could train a program to accurately recognise masked faces by deleting parts of the photos they used to train the software.More recently, as the Covid-19 outbreak was gaining global momentum, researchers from Wuhan University released the Real World Masked Face Recognition data set, which they claimed had the potential to train AI to identify masked people with an accuracy rate of 95%. With the objective of enabling existing facial recognition approaches that heavily rely on all facial feature points to recognize people in masks, the researchers have open-sourced this data set.