Deepmind Launches SOTA Video Generation Framework, ‘Transframer’

The framework uses U-net and Transformer components to condition on annotated context frames, and generate a sequence of spare, compressed image features.
Deepmind Launches SOTA Video Generation Framework, ‘Transframer’
Listen to this story

Recently, Deepmind researchers announced the launch of Transframer—a new general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. This new model unifies a broad range of tasks, including image segmentation, view synthesis and video interpolation. 

This latest framework uses U-Net and Transformer components to condition on annotated context frames, and outputs sequences of sparse, compressed image features. 

What does Transframer do


Sign up for your weekly dose of what's up in emerging technology.

Developed by Deepmind, Transframer unifies a range of image modelling and vision tasks and has the ability to create videos or image features based on a single image with one or more context frames.

Transframer works on a variety of video generation benchmarks. The research team claims that it is a state-of-the-art model which is expected to be the strongest and most competitive on few-shot view synthesis, and can generate coherent 30-second videos from a single image.

Download our Mobile App

The proposed model also showed promising results on eight tasks in total, some of which are semantic segmentation, image classification, and optical flow prediction with no task-specific architectural components. 

Transframer can also be used in various applications that require learning conditional structure using text or a single image, and will be able to predict and generate video models, novel view synthesis and multi-task vision.

Backed by Google, Deepmind has been researching in the field of AI since 2010 and focusing on building computer models that can solve building and generative problems on their own.

Click here to read the research paper.

More Great AIM Stories

Mohit Pandey
Mohit is a technology journalist who dives deep into the Artificial Intelligence and Machine Learning world to bring out information in simple and explainable words for the readers. He also holds a keen interest in photography, filmmaking, and the gaming industry.

AIM Upcoming Events

Early Bird Passes expire on 3rd Feb

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox