The silent movie, ‘Le Voyage dans la Lune’ (A Trip to The Moon), released in 1902, was considered to be the first science fiction movie in the world. Unlike today, the movie’s ‘special effects’ were all human-made. Over 120 years later, we are standing at the crux of creative exploration where AI can pretty much make a movie. 

Though there’s a remarkable surge in AI video generation tools, the actual usage of these tools is yet to be measured for big production houses. Tools such as Stable Diffusion, Runway and the recent, Pika, may look promising at the moment, but there still are limitations that can restrict a creator from fully utilising them.

However, production houses have already started experimenting with it, benefitting not just creatively but also for better operational efficiency. 

AI Upgrades Pre-Production 

While AI has been used at various stages of movie production, post-production (editing) is where the latest advancements in AI tools are mostly implemented. From analysing clips for finding the best shot, splicing, de-ageing and other visual effects, including deepfakes, AI has found a crucial place to save time and effort.

And now, it has found its way in the pre-production stage (planning) as well, where the biggest use case is storyboarding.  One of the crucial steps at the planning stage of a movie production is storyboarding. A storyboard visually outlines the sequence of shots that will unfold in a video. If done incorrectly, production houses not only lose precious man-hours, but also face huge financial losses. 

Realising the potential for simplifying storytelling and breaking barriers for those who wish to enter filmmaking with financial constraints and technical dearth, d.cult studio , a startup in Trivandrum, is out to ‘democratise the art of filmmaking’ through AI. The company is already working with studios in the Malayalam movie industry, including Festival Studios and major production house Prithviraj Productions. 

“What used to be sketches and infographic-based, is now changing into a cinematic-style storyboarding,” said Gokul A, founder of d.cult and co-founder of Accubits Technologies. 

​​https://twitter.com/dcultstudio/status/1706578659195146533

Essence of Time and Cost 

Accurate storyboarding saves time and cost during actual shoots. There’s a 50-80% overall reduction in time for the entire storyboarding process, and 40-70% reduction in costs, after factoring in reduced labour and material expenses.

“For a 50-day shoot, where the per day shoot cost is Rs 5 lakh, even if five days are reduced owing to AI storyboarding, Rs 25 lakh can be straightaway saved from the production cost,” said Gokul. For a storyboard creator, such a tool is believed to augment their work. “Initially, an experienced storyboard creator could make only two images in a day, but now, AI tools can help the creator make upto 100 images in a day.“

The company uses a mix of models including proprietary and open-source ones, along with NLP algorithms. Idea to story/screenplay generation uses BUD LLMs — GenZ model, with natural intelligence algorithms. For auto music suggestion and background music integration, music generation models such as OpenAI’s Jukebox is used. Similarly, deep learning and computer vision models are used for AI video analysis and storyboard visualisation.  

Treading Around Copyrighting 

Storyboarding is only one part of the AI solution that d.cult studio offers. It also provides the option to convert scripts to storyboard, collaborative whiteboard for idea discussion, video analysis for story ideas, pre-visualisation, and photo generation, among other features.  

With copyright issues causing a churn in companies producing AI-generated content, especially AI images, d.cult studio ensures strict measures on the same. “We are diligent in using only those materials that are either originally created by our team, fall within the public domain, or are available under creative commons licences for AI inputs,” said Gokul. 

Animation Studios Are Not All Yay

Hayao Miyazaki- Studio Ghibli.  Source: LA Times

While companies are experimenting with AI to ease movie production, there are some who do not wish to jump on the wagon just yet. Signature style production houses such as Studio Ghibli, a Japanese animation studio that still uses traditional hand-drawn animation methods, wouldn’t embrace AI all too quickly.

Co-founder and director Hayao Miyazaki has previously said that AI art is ‘utterly disgusting’ and that it will never succeed in giving artwork the genuine feel of a human being. He even said that he will never incorporate such technology in his work as he feels that it will be “an insult to life itself”. 

On one end when you have animation studios shying away from embracing AI, there are others who are refining animation styles with the same. The LoRA training model simplifies the process of training Stable Diffusion on various concepts, including characters or specific styles. Once trained, these models can be exported for use in generating content by others, thereby enabling control levels for AI animation. 

Furthermore, many are experimenting with a mix of models to create their own anime. Entrepreneur Varun Mayya said that production cost for an episode of anime can be anything from $140,000 to $180,000. He believes that making anime on their own with a mix of tools can probably drop the price by a 100 or 1,000. He created one with a string of 20 different AI tools.

While people are experimenting with generative AI tools to create their own anime, movies, and more, the scaling of these techniques is yet to be seen. Meanwhile, human creativity will continue to play a significant role in movie production houses, thereby limiting its complete adoption. The implementation of AI for operational and cost efficiency is where the actual use-case will lie.