Article complete

ByteDance Unveils Goku to Take on Google’s Luma and OpenAI’s Sora

While the models may be less powerful than the anime character Goku, they do look pretty impressive.

A large, humanoid robot walks down a red carpet, surrounded by cameras and onlookers at a formal event.

Key takeaways

  • ByteDance launches Goku, a new family of image-video generation models to compete with Google and OpenAI.
  • Goku models leverage advanced techniques like rectified flow and Transformer networks for high-quality video generation.
  • Research shows Goku sets new benchmarks against competitors in qualitative and quantitative video evaluations.
  • Goku's capabilities include creating marketing avatars, product videos, and visualizing poetry.
  • The introduction follows ByteDance's OmniHuman-1, a video AI model that generates videos from images.

Takeaways drafted with AI from the article text.

ByteDance, the parent company of TikTok, has dropped a family of joint image-video generation models called Goku. The models seem to be named after the popular anime character ‘Goku’ from the Dragon Ball series. 

This comes right after the company teased a video AI model that generates videos from images, dubbed OmniHuman-1.

The research paper attributes the model's ability to generate high-quality videos to several key factors. One is the implementation of a rectified flow (RF) formulation for joint image and video generation and the employment of a 3D joint image-video VAE to compress inputs into a shared latent space. 

Moreover, the architecture features a Transformer network with full attention, enhanced with techniques like FlashAttention, sequence parallelism, Patch n’ Pack, 3D RoPE position embedding, and Q-K normalisation.

The paper also states that the Goku models demonstrate superior performance in both qualitative and quantitative evaluations, setting new benchmarks when compared to competitors like Luma, Open-Sora, Mira, and Pika.

Goku achieved 0.76 on GenEval, 83.65 on DPG-Bench for text-to-image generation, and 84.85 on VBench for text-to-video tasks. You can see the benchmark results below. 

(Credit: GitHub page)
(Credit: GitHub page)

“We believe that this work provides valuable insights and practical advancements for the research community in developing joint image-and-video generation models,” the researchers said.

The model’s ability to generate high-quality product videos featuring AI-generated influencers and other realistic visuals could hugely benefit content creators, influencers, marketers, and others.

Published Updated

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

AI Video Generation Models · 5 stories
  1. PhysDreamer Study Reveals Breakthrough in Video Generation for Dynamic 3D Object Interactions
  2. Databricks Ventures Invests in Twelve Labs to Unlock Video Intelligence Capabilities
  3. You are here: ByteDance Unveils Goku to Take on Google’s Luma and OpenAI’s Sora
  4. AI Video Generator Invideo to Integrate NVIDIA’s Compute to Make Films
  5. Ryanair is Using AI-Generated Videos for Staff Training & Passenger Updates
About the author

Editor

I am a tech aficionado and a computer science graduate with a keen interest in AI, Coding, Open Source, Global SaaS, and Cloud. Have a tip? Reach out to

All stories
Belamy · Daily

The day’s AI news, before your first meeting.