Surgical masks have transitioned from playing the role of protective equipment to being a societal norm to becoming a fashion statement within a couple of months due to COVID-19.
As the pandemic engulfed the world, there has been an increased deployment of AI-based solutions to keep the spread in check. Computer vision is being used to identify patients, track their movements and for other contactless precautionary measures.
When it comes to identifying people using facial recognition technology, the algorithms take cues from the landmarks around eyes, nose and mouth. Now since the bottom half of the facial features are hidden by the masks, facial recognition technology is facing new challenges.
To make things even more challenging for the algorithms, people are using masks that can fool these systems.
Bruce MacDonald, a software developer from Canada, has designed a mask that could act as adversarial attacks on the facial recognition models.
Fashion As An Adversarial Attack
For this experiment, Bruce used a HOG or Histogram of Gradients, which is a facial detection method that divides a processed input image into cells with gradient orientations before feeding the result into a support vector machine.
For finding an input that corresponded to the most faces detected by dlib, Bruce used simulated annealing with random optimisation. Dlib is a modern C++ toolkit containing machine learning algorithms and tools.
Whereas, simulated annealing is a technique that is used to find the best solution for either a global minimum or maximum, without having to check every single possible solution that exists.
For optimisation, mlrose, a Python package was used. This package is typically used for applying common randomised optimisation and search algorithms to a range of different optimisation problems, over both discrete- and continuous-valued parameter spaces.
Here is a snippet of code:
fitness = mlrose.CustomFitness(detected_max)
problem = mlrose.DiscreteOpt(length=24, fitness_fn=fitness,maximize=True, max_val=scale_factor)
<<First the fitness and the initial state are set and then ‘mlrose’ was configured for simulated annealing. Then the inputs are assigned, and this step is run repeatedly until an optimal result is found as shown in the picture above.>>
schedule = mlrose.ExpDecay()
best_state, max_faces = mlrose.simulated_annealing(problem, schedule=schedule, max_attempts=10, max_iters=1000,init_state=initial_state, random_state=1)
get_img_from_state(best_state)
len(detect_faces(cv2.imread(OUTPUT))))
Check the full code here.
Future Direction
The earliest large scale use of masks that fooled surveillance systems was seen during the Hong Kong protests last year. The use of these fashionable masks exposes a couple of things. First, that the public has realised the extent of surveillance, and secondly, the computer vision models have a long way to go. Imagine a scenario where a self-driving car comes across a signal or an advertisement that has these confusing imagery. The consequences can be fatal.
There have been works already to identify people even with masks. A Chinese company, Hawang, claims that it can identify everyone in a crowd with great accuracy, and it has also been deployed by the Chinese Ministry of Public Security to screen people and cross-reference images with its own database to identify and track people.
As cities became centres of mass surveillance, the yearning for anonymity is on the rise. So far, the argument for surveillance has been to make the streets safer. People have been sceptical about this, and many regulations have been passed against the inappropriate use of facial recognition, and now, the pandemic has given a touch of benevolence and made these systems mainstream.
