Deepmind Launches SOTA Video Generation Framework, ‘Transframer’

The framework uses U-net and Transformer components to condition on annotated context frames, and generate a sequence of spare, compressed image features.
Deepmind Launches SOTA Video Generation Framework, ‘Transframer’
Listen to this story

Recently, Deepmind researchers announced the launch of Transframer—a new general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. This new model unifies a broad range of tasks, including image segmentation, view synthesis and video interpolation. 

This latest framework uses U-Net and Transformer components to condition on annotated context frames, and outputs sequences of sparse, compressed image features. 

What does Transframer do

Subscribe to our Newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Developed by Deepmind, Transframer unifies a range of image modelling and vision tasks and has the ability to create videos or image features based on a single image with one or more context frames.

Transframer works on a variety of video generation benchmarks. The research team claims that it is a state-of-the-art model which is expected to be the strongest and most competitive on few-shot view synthesis, and can generate coherent 30-second videos from a single image.

The proposed model also showed promising results on eight tasks in total, some of which are semantic segmentation, image classification, and optical flow prediction with no task-specific architectural components. 

Transframer can also be used in various applications that require learning conditional structure using text or a single image, and will be able to predict and generate video models, novel view synthesis and multi-task vision.

Backed by Google, Deepmind has been researching in the field of AI since 2010 and focusing on building computer models that can solve building and generative problems on their own.

Click here to read the research paper.

Mohit Pandey
Mohit dives deep into the AI world to bring out information in simple, explainable, and sometimes funny words. He also holds a keen interest in photography, filmmaking, and the gaming industry.

Download our Mobile App

MachineHack | AI Hackathons, Coding & Learning

Host Hackathons & Recruit Great Data Talent!

AIMResearch Pioneering advanced AI market research

With a decade of experience under our belt, we are transforming how businesses use AI & data-driven insights to succeed.

The Gold Standard for Recognizing Excellence in Data Science and Tech Workplaces

With Best Firm Certification, you can effortlessly delve into the minds of your employees, unveil invaluable perspectives, and gain distinguished acclaim for fostering an exceptional company culture.

AIM Leaders Council

World’s Biggest Community Exclusively For Senior Executives In Data Science And Analytics.

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox