In the 2006 movie ‘X Men: The Last Stand’, actors Patrick Steward and Ian McKellan were “de-aged” using visual effects (VFX) to play the younger selves of their characters, Professor X and Magneto. This was the first time such technology was used to alter the age of actors rather than rely solely on makeup and prosthetics. What followed thereafter was a trend where VFX was heavily used to adjust onscreen age. Since then, however, production houses have advanced to new age tech and are shifting towards AI-based softwares to tackle de-ageing. 

But how exactly is AI employed in post production? 

AI Takes on the Big Screen

The recent development of AI technology in films is most evident in an upcoming movie, “Here”, which is an adaptation of Richard McGuire’s graphic novel. Tom Hanks and Robin Wright, who will play the lead roles, will be de-aged using AI software. Production house ‘Miramax’ has tied up with Metaphysic, a machine learning and visual effects company known for the Deepfake Tom Cruise . Metaphysic’s CEO Tom Graham said that the company will produce a younger version of the actors which will then be superimposed on their live performances. He believes that this AI technique is cost effective when compared to visual effects (VFX) and computer-generated images (CGI) and the output is also “more realistic”. This is the first time AI has been employed to play a central role in a Hollywood production. Graham also believes that the biggest problem to tackle when using such technology is to ensure that the original performance of the actors is preserved. In films such as ‘Benjamin Button’ or ‘Gemini Man’ where 3D, VFX/CGI was used, the output looked “uncanny”. 

The actors are usually dressed according to the age they want to depict, such as hair and costumes, which will solely allow the face to be de-aged. In cases where the age is to be reduced by a huge number, for example an old actor depicting a young person, or even a child, then the actors of that required age will be cast wherein only the face is altered. In Disney’s ‘Obi Wan Kenobi’ series, there is a lightsaber scene where the younger version of Hayden Christensen has a duel with Obi Wan. Christensen’s face was de-aged using CGI. However, Lucas Films’ another tv series, “The Book of Boba Fett”, features a young Luke Skywalker where a young actor was cast to replace Mark Hamill and his face was altered using deepfake technology. 

De-Aging with AI in ‘The Irishman’

In the 2019 movie, ‘The Irishman’, all the lead actors were de-aged to play their younger selves. Director Martin Scorsese was against traditional motion capture methods, which he believed to be intrusive in an actor’s performance during filming. Visual effects company Industrial Light and Magic (ILM), founded by George Lucas, was employed to de-age actors Robert DeNiro, Al Pacino and Joe Pesci. 

A three-camera rig, i.e., a primary camera with two infrared “witness cameras” on either side was used. The infrared cameras record multiple points of view, which results in better 3D translations of geometry and texture. A huge library of image grabs of the actors’ previous performances across a wide range of their ages was created. ILM developed a software called “Facefinder” that used AI to sift through those images. 

“Medusa” capture technology was used by ILM to record facial expressions. The task of de-ageing is not a mere superimpose of images of the actors’ face grabs from their specific ages. The challenge is to make it look as natural as possible because facial expressions are governed by facial muscles that twitch every second, which makes the overlaying work extremely complex. ILM’s AI tool locates frames that match camera angle, lighting, framing and expression of the scene that needs to be rendered which is then used as reference points to transform actors into their target age for each scene. NVIDIA Quadro RTX GPUs were used for powering the production pipeline. Pablo Helman, VFX supervisor at ILM, said that AI and machine learning is now increasingly part of “everything in VFX.” 

Disney’s FRAN

Disney had recently introduced FRAN (Facial Re-aging Network) that works on neural network architecture ‘U-net’ that performs image-to-image translation tasks. Adjustments are made to the U-Net architecture to control re-aging and match the output and specified target age. A five-channel tensor is used as input to the U-Net, which consists of the RGB image that needs to be re-aged, and two single-channel age maps consisting of input and output age for each pixel. With machine learning, the process uses a combination of three different types of loss functions : L1 loss, perceptual loss and adversarial loss, which are put back to the input image to get a re-aged result. 

It is evident that AI is now increasingly being used in film production. Harrison Ford’s upcoming film, ‘Indiana Jones and the Dial of Destiny’ will see him de-aged to make him look the way he did in his first Indiana Jones movie. The trend to move to realism with the help of AI is something that needs to be considered. Artistic creation with tech collaboration will go a long way with innovations at every step. It would be exciting to watch the extent to which filmmakers will go to achieve their dream, but one must always remember that AI and technology can only go as far as the performers who can push the story forward. 

While the ease of using technology to de-age an actor is going to trend, a looming question arises. Does this process take away opportunities from young actors? Production houses are increasingly adopting AI technology to give ample screen space to their aged lead actors or use their figures (deep fake) to trigger viewer nostalgia, all in a bid to attract and retain viewers. By doing so, they are saving production costs by limiting the cast. A move that may seem detrimental for young actors who could have ideally played the younger selves. In Godfather 2, Robert De Niro played young Don Vito Corleone which Marlon Brando played in the first part. Come to think of it, isn’t it better if an actual “young” actor played the role of a young person than using technology to seal the “fakeness” that Hollywood is often branded with?