Meta Unveils Movie Gen; its state-of-the art Video Editor

Meta AI launched the Movie Gen, a 13B parameter model designed for video and text to audio generation - one of the most advanced media foundation models to-date. Its chief features include generating videos from text, editing videos with text, producing personalised videos, and creating sound effects. While the model is not publicly available yet, the demos and use cases are already taking over the internet.

“We’re continuing to work closely with creative professionals from across the field to integrate their feedback as we work towards a potential release. We look forward to sharing more on this work and the creative possibilities it will enable in the future,” read their statement hinting on its potential release, hopefully next year. 

To this end, Meta's Chief, Mark Zuckerberg, shared a video on Instagram of himself doing leg press, created by Movie Gen. He teased that the feature would come to Instagram by 2025. 

True to its open-research roots, Yann LeCun and team have released a technical paper, spanning 90 pages, explaining the methodology behind the foundational model. It is important to note that Meta hasn’t mentioned whether this model would be open-source yet. 

Race to Release First

The capabilities are at par, or some suggest even better than Sora, which is yet to be released. OpenAI’s flagship model is not publicly available yet, nor is there any update from the company regarding its release - eight months since its announcement. The timing of this is interesting, as Tim Brooks, a member of the Sora team, announced his departure from OpenAI today to join its rival, Google DeepMind.

Beyond Sora

Needless to say, the video generation space is growing increasingly competitive. Last month, an NYC based AI video startup, Runway, partnered with Lionsgate to introduce AI into filmmaking. Runway aims to bring these tools for artists, and by extension bring their stories to life. This deal would open the doors for many of these stories to appear on the big screen eventually. Launched earlier this year in SF, Luma's Dream Machine was freely accessible for experimentation on its website, marking a significant step in AI-driven video creation. Competition from China, Kling, recently announced more features to its model, including a new lip sync feature which works on faces that aren't directly facing the camera.