Article complete

ByteDance's DreamActor-M1 Turns Images into Stunningly Real Human Videos

The framework aims to overcome the limitations of current image-based human animation methods.

A man with a red beard and blue eyes stands before various paintings, resembling an artist in a gallery setting.

Key takeaways

  • ByteDance launches DreamActor-M1 for creating realistic human animations from reference images.
  • Utilizes Diffusion Transformer architecture for enhanced control and adaptability in animations.
  • Addresses limitations of existing models with improved consistency and precision in facial and body movements.
  • Trained on diverse datasets to manage various body poses and image scales effectively.
  • Competes with leading animation models, demonstrating superior performance in human animation tasks.

Takeaways drafted with AI from the article text.

ByteDance has introduced DreamActor-M1, a new framework designed to generate realistic human animations from reference images. This framework addresses key issues in current animation models to achieve finer control, greater adaptability, and better consistency.

This comes in right after ByteDance announced Goku and InfiniteYou AI models.

DreamActor-M1 is based on a Diffusion Transformer (DiT) architecture and uses a hybrid guidance approach to achieve its results. The model employs a combination of implicit facial representations, 3D head spheres, and 3D body skeletons to control facial expressions and body movements with greater precision.

To handle various body poses and image scales, the model is trained using a progressive strategy on a dataset with varying resolutions and scales. DreamActor-M1 integrates motion patterns from sequential frames with complementary visual references to ensure consistency over extended periods, addressing challenges with unseen regions during complex movements.  

The research paper compares DreamActor-M1 with several state-of-the-art human image animation models. For body animation, DreamActor-M1 was compared against Animate Anyone, Champ, MimicMotion, and DisPose.

In portrait animation, the model was evaluated alongside LivePortrait, X-Portrait, SkyReels-A1, and Runway Act-One. 


The results of these comparisons demonstrate that DreamActor-M1 outperforms existing methods in generating more expressive and consistent animations. 

The researchers also acknowledged that these AI models can be misused. They stated, “To reduce these risks, clear ethical rules and responsible usage guidelines are necessary. We will strictly restrict access to our core models and codes to prevent misuse. Images and videos are all from publicly available sources.”

While DreamActor-M1 represents a significant advancement, the researchers acknowledge certain limitations. The model struggles with controlling dynamic camera movements and generating physical interactions with environmental objects. Their future work aims to address these challenges and further enhance the model's capabilities.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.

More in this story

ByteDance · 5 stories
  1. The Topper of IIT-JEE Advanced is…Gemini 2.5 Pro!
  2. ByteDance Drops 'InfiniteYou', an AI Model for Photo Recrafting
  3. You are here: ByteDance's DreamActor-M1 Turns Images into Stunningly Real Human Videos
  4. China Offers Cheaper Power to Local AI Firms Ditching NVIDIA Chips
  5. China’s ByteDance is Training a 10 Trillion-Parameter AI Model: Report
About the author

Editor

I am a tech aficionado and a computer science graduate with a keen interest in AI, Coding, Open Source, Global SaaS, and Cloud. Have a tip? Reach out to

All stories
Belamy · Daily

The day’s AI news, before your first meeting.