Kubric, a scalable dataset generator, is the python framework that is used for generating photo-realistic computer-generated images and videos. The main advantage of this python tool is that it helps in creating datasets very cheaply, and gives us full control over them. In this article we will go through the Introduction of Kubric, then we will see what is the need of this framework. We will also try to understand its working mechanism. The major points to be discussed in this article are listed below. 

Table of contents

1. Introduction to Kubric
2. Architecture and sample images
3. Kubric Workflow
4. Why do we need this framework?
5. Working of Worker

Introduction to Kubric

After several groundbreaking NLP, computer vision libraries and frameworks, Google has released another framework named Kubric. On 07 Mar 2022 Google researchers Klaus Greff, Francois Belletti, Lucas Beyer released their research paper on Kubric: A scalable dataset generator.

It is an open-source Python framework that allows you to create photo-realistic scenes by combining the functions of PyBullet and Blender. PyBullet is the Python module that is used for creating physics simulations, and Blender is used for rendering. These generated data are referred to as Synthetic data. These synthetic datasets provide high-quality labels for various image tasks such as object detection, segmentation, classification etc. 

Architecture and sample images

The below image shows the scene generated by Kubric along with some of the automatically generated annotations.

(Image source)

There are many synthetic datasets available on the internet such as ScanNet, SYNTHIA etc, but they do not contain all possible annotations for all image tasks (instance and semantic segmentation masks, optical flow) but Kubric generates the images with annotations and supports different viewing angles and lighting conditions. For generating synthetic datasets there are also pipelines available like Blender and Unity3D, these pipelines fulfil the lighting and different viewing angles limitations, but for creating additional annotation they need in-depth knowledge of the underlying rendering engine.

The architecture of Kubric is given below.

Kubric is a high-level python library that fuses the functionality of PyBullet and Blender. We see a blue box named Worker, the main component of Kubric that is responsible for creating synthetic data. Like in web scraping the code which extracts data from the website we call it Crawler or Spider, In the same way the code which is responsible for generating synthetic data we call it as Worker. 

Now, In the figure Assets are loaded from the external source in order to create a scene by the worker, PyBullet creates physics simulation, Blender renders the frames, and finally, images, annotations layers and other metadata are exported.

Kubric Workflow

Kubric workflow is very simple, just write the worker script that creates a single scene. In order to create a full dataset, you have to run a worker script many times. After this, collect the generated dataset.

Why do we need this framework?

The most important question is why we need this data generator framework. As we know machine learning and artificial intelligence how much we rely on data. As it is mentioned in the research paper:

“Data is the driving force of machine learning”

Data is necessary but, How about good data?. Good data is the key to the performance of a machine learning system than the anatomy of models and training details. But collecting, annotating, and cleaning real data at a large scale is a laborious and expensive job and frequently raises privacy, fairness of use and legal concerns. To handle these concerns Google introduces Kubric, with this tool we can generate data very cheaply, automate the ground-truth annotation, give full control over the data, and reduce the bias, privacy and licensing problems.

Working of Worker

A Worker is the main component of Kubric as we discussed above. So each worker sets up a Scene object, which keeps track of global settings like resolution, the number of frames to render, gravity, a Camera, and all the objects, including lights, material, animations etc. are called Assets. When we put the Asset into the Scene, it creates corresponding objects of different views, as we discuss above PyBullet creates the physics simulation and bpy module for using the Blender functionality, Blender is a powerful 3-D graphics renderer. Below the image is the output of the Worker. A simple environment with a floor, pointing light, a perspective camera and 8 KuBasic objects placed on the floor with velocity. It’s an animation video but we put the still image.

                                        (Image source)

Final words

In this article, We learned about Kubric, from its definition to its architecture and we get to know why this framework is necessary. We could also understand how it addresses the problems of creating the real dataset, and how easily Kubric tackles those problems.

 References

• Official research paper