Active Hackathon

How Facebook Is ‘Opening’ Their Users’ Eyes With The Help Of ExGANs

facebook eyes open

Every day, a large number of pictures are captured and shared in social networks with a large percentage of them featuring people-centric content. Facebook is making sure that you don’t make your friends take endless photos, posing and reposing at important occasions. They are also helping you cut out the time you spend on Photoshop or similar image-correction apps and products by making their own photo uploading platform more imaginative.

According to a recent paper published by Facebook, the Menlo Park-based social media giant is working on an “in-painting technology” that replaces closed eyes with open ones in a remarkably convincing manner. They are working on achieving this by using Exemplar Generative Adversarial Networks (ExGANs) — a type of conditional GAN that utilise exemplar information to produce high-quality, personalised in-painting results.


Sign up for your weekly dose of what's up in emerging technology.

The paper says, “We propose using exemplar information in the form of a reference image of the region to in-paint, or a perceptual code describing that object. Unlike previous conditional GAN formulations, this extra information can be inserted at multiple points within the adversarial network, thus increasing its descriptive power. We show that ExGANs can produce photo-realistic personalised in-painting results that are both perceptually and semantically plausible by applying them to the task of closed-to-open eye in-painting in natural pictures. A new benchmark dataset is also introduced for the task of eye in-painting for future comparisons.”

Basically, with data collected from the target people when they kept their eyes open, the GAN system learns what eyes should go on the person on the basis of shape, colour and other features of the eyes of a particular person.

ex gan facebook

Created by Brian Dolhansky and Cristian Canton Ferrer from Facebook added in the paper, “…Because Exemplar GANs are a general framework, they can be extended to other tasks within computer vision, and even to other domains. In the future, we wish to try more combinations of reference-based and code-based exemplars, such as using a reference in the generator but a code in the discriminator.”

Last year, researchers at Facebook’s AI lab had developed an expressive bot, the animations of which were controlled by an artificially intelligent algorithm. The algorithm was trained by making it watch hundreds of videos of Skype conversation and was now capable of mimicking the way humans adjust their expressions while talking.

More Great AIM Stories

Prajakta Hebbar
Prajakta is a Writer/Editor/Social Media diva. Lover of all that is 'quaint', her favourite things include dogs, Starbucks, butter popcorn, Jane Austen novels and neo-noir films. She has previously worked for HuffPost, CNN IBN, The Indian Express and Bose.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM

Data Science Skills Survey 2022 – By AIM and Great Learning

Data science and its applications are becoming more common in a rapidly digitising world. This report presents a comprehensive view to all the stakeholders — students, professionals, recruiters, and others — about the different key data science tools or skillsets required to start or advance a career in the data science industry.

How to Kill Google Play Monopoly

The only way to break Google’s monopoly is to have localised app stores with an interface as robust as Google’s – and this isn’t an easy ask. What are the options?