Jumping On The Metaverse Bandwagon: NVIDIA Launches Range Of Omniverse Capabilities

Launched officially in April this year, NVIDIA has been speeding up its metaverse efforts or what it calls -- the NVIDIA Omniverse.
NVIDIA Omniverse

“How do astronomers look for exoplanets?” – after pondering over this question for what seems like a split second, the avatar of NVIDIA CEO Jensen Huang responds, “Most exoplanets are found via the transit method. That was the case for the TRAPPIST-1 planets.”

This avatar, also nicknamed Toy Jensen, is one of the latest offerings from NVIDIA. It was demonstrated at the GTC 2021 Fall event recently. These conversational avatars called Toy-Me are trained for speech understanding and natural language processing on large models. These avatars can help synthesise users voices and character animation. Toy Me is just one of the many developments of NVIDIA in the metaverse space.

Launched officially in April this year, NVIDIA has been speeding up its metaverse efforts or what it calls — the NVIDIA Omniverse. It was hardly surprising that a major chunk of Huang’s keynote speech revolved around developments in this field, subtly indicating that the company is ready for the transition in the tech world, which is levitating towards the new wonder word — metaverse. Good news for Omniverse! It has been ranked by TIME magazine as one of the top 100 innovations of 2021.


Sign up for your weekly dose of what's up in emerging technology.

NVIDIA Omniverse is a scalable, multi-GPU real-time reference development platform that helps users build 3D simulations and collaborate for designing. This Omniverse is based on Pixar’s Universal Scene Description and NVIDIA RTX technology. The Omniverse is in the open beta stage and is being used by 70,000 users. Talking about the technology that was utilised in building this Omniverse, Huang said, “USD is the HTML of the 3D world.”

NVIDIA Omniverse Avatar

The Omniverse Avatar is an amalgamation of the company’s technologies in speech AI, NLU, recommendation engine, computer vision, and simulation technologies. These 3D avatars are interactive characters that can see and interact with a wide variety of subjects, capable of understanding naturally spoken intent. Its range of applications is very diverse and can be used in several customer service interactions in the field of hospitality, banking, and appointment making. 

Download our Mobile App

One of the most important announcements about Omniverse Avatars, Huang, spoke about Project Tokkio. Toy Me is part of this project. This toy version can hold a conversation with real people on subjects of climate change, astronomy, human biology, and much more. For speech and voice, Project Tokkio uses a software called the Riva Speech AI from NVIDIA. This software helps create a virtual assistant with a unique voice. Apart from its applications in customer support roles, Riva can also prove to be a boon for those with speech and language deficiencies.

NVIDIA Riva will now also be integrated into Project Maxine to enable better communication and understanding. It will help in real-time translation and text-to-speech with photo animation and eye contact in real-time. Project Maxine was first announced at the 2020 GTC event. It is a platform that provides a cloud-based suite of GPU-accelerated AI video conferencing software.

The Omniverse Avatar also DRIVE Concierge to serve as a digital assistant to make everyday driving stress-free and simplistic. Passengers can have natural conversations with the vehicle and use voice to control functions that previously required physical control or touch screens.

Omniverse Replicator

Omniverse Replicator is the latest offering from NVIDIA. It is a synthetic data generator that produces physically simulated synthetic data to train deep neural networks. Under its initial offering, NVIDIA introduced two applications of Omniverse Replicator — NVIDIA DRIVE Sim, a virtual world that hosts the digital twin of autonomous vehicles and NVIDIA Isaac Sim, a virtual world for the digital twin for robot manipulation. In both the applications, the Replicators help developers bootstrap AI models, fill data gaps, and label the ground truth like velocity, depth, adverse weather conditions, etc. It is particularly useful for dangerous situations which are not regularly and safely experienced in real life.

Other new features

Other major features announced with Omniverse are:

  • NVIDIA CloudXR: It is an enterprise-class streaming framework integrated into Omniverse Kit (a toolkit for building native Omniverse applications and microservices). CloudXR allows users to interactively stream the Omniverse experience onto their mobile AR and VR devices.
  • Omniverse Farm: It is a system layer to orchestrate multiple computing resources to execute batch and interactive tasks, performing tasks like rendering, streaming, file conversion, simulation, etc. It can work across typical workstations, bare metal servers, advanced cloud scheduling platforms like Kubernetes or VMs. Developers can build tools to leverage Farm in batch mode or interactive mode. 
  • Omniverse VR is the world’s first full-image, real-time ray-traced VR. It helps developers in building their own VR-capable tools on the platform for end-users to utilise them directly.
  • Omniverse XR Remote offers augmented reality capabilities and virtual cameras. Designers can leverage it to view their assets fully ray-traced through both iOS and Android devices.
  • Omniverse Showroom: As an app in Omniverse Open Beta, Omniverse Showroom lets even non-technical users play with tech demos that demonstrate the platform’s real-time physics and rendering technologies.

More Great AIM Stories

Shraddha Goled
I am a technology journalist with AIM. I write stories focused on the AI landscape in India and around the world with a special interest in analysing its long term impact on individuals and societies. Reach out to me at

AIM Upcoming Events

Regular Passes expire on 3rd Mar

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 17th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, Virtual
Deep Learning DevCon 2023
27 May, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

A beginner’s guide to image processing using NumPy

Since images can also be considered as made up of arrays, we can use NumPy for performing different image processing tasks as well from scratch. In this article, we will learn about the image processing tasks that can be performed only using NumPy.

RIP Google Stadia: What went wrong?

Google has “deprioritised” the Stadia game streaming platform and wants to offer its Stadia technology to select partners in a new service called “Google Stream”.