Thanks to NVIDIA, the era of humanoids becoming a reality is not far. At the highly-anticipated NVIDIA GTC event, with some unexpected showmanship, NVIDIA chief Jensen Huang announced Project GR00T, a general-purpose foundation model for humanoid robots.

Robots powered by GR00T, short for Generalist Robot 00 Technology, are engineered to understand natural language and mimic human movements by observing actions. This allows them to quickly learn coordination, dexterity, and other skills required to navigate, adapt, and interact effectively in the real world.

The highlight of the keynote was Huang posing with nine humanoids about the same size as him. While wrapping up his keynote, he was accompanied by Orange and the infamous Green BDX robots from Disney Research, which kept interrupting and disrupting the flow of his keynote presentation. With NVIDIA, Disney is now starting to look more like a robotics company. 

“The next generation of robotics will likely be humanoid robotics,” Huang said, calling it easier due to the availability of much more imitation training data for these robots. This is because they are constructed much like humans, according to Huang. “It is very likely that human robotics will be much more useful in our world because we created the world to be something that we can interoperate in and work well in,” he said.  

Interestingly, GR00T may be the first foundational model designed specifically for humanoids. It takes multimodal instructions and past interactions as input and produces the next action for the robot to execute. Emphasising its multimodality, Huang said that GR00T learns from human examples, which could be in ‘video or virtual reality form’.

NVIDIA has developed Isaac Lab, a robot learning application to train GR00T on Omniverse Isaac Sim, alongside Osmo, a new compute orchestration service that coordinates workflows across DGX systems for training, and OVX systems for simulation. With these tools, NVIDIA can train GR00T in simulation and transfer zero-shot learning to the real world.

“Today is the beginning of our moonshot to solve embodied AGI in the physical world,” wrote Jim Fan, lead of Embodied AI at NVIDIA, on X.

NVIDIA is partnering with top humanoid robot companies like 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, XPENG Robotics, and more. Recently it also invested in Figure AI alongwith Microsoft, OpenAI and others. 

A few days ago, the robotics startup Figure shared a video demonstration of its first humanoid engaging in real-time conversations with humans. The robot was powered by an OpenAI model, possibly GPT-5 with Vision, showcasing high-level visual and language intelligence.

In a recent podcast with Lex Fridman, OpenAI chief Sam Altman announced OpenAI’s return to robotics: “I think it’s sort of depressing if we have AGI and the only way to get things done in the physical world is to make a human go do it.”

Unleashing the Power of Thor 

Possibly inspired by Marvel and named in line with GR00T, NVIDIA has developed a new computing platform called Jetson Thor, designed specifically for humanoid robots. It is capable of efficiently handling complex tasks and interacting safely and seamlessly with both people and machines.

The platform’s SoC features a new-generation GPU based on NVIDIA’s Blackwell architecture, equipped with a Transformer engine that delivers 800 teraflops of 8-bit floating-point AI performance. This enables the platform to run multimodal generative AI models like GR00T efficiently.

Not just that, NVIDIA has also announced a collection of robotics pretrained models, libraries and reference hardware, called Isaac Manipulator and Isaac Perceptor. The latter will assist humanoids in navigating physical environments autonomously without pre-programmed routes between two points. For example, in a warehouse scenario where a box falls, the humanoid can find its own path without waiting for humans to clear the way.

“With the Isaac Perceptor, we have incredible state-of-the-art vision odometry, 3D reconstruction, and depth perception,” said Huang. On the other hand, the Isaac Manipulator offers advanced dexterity and flexible AI features for robotic arms.

NVIDIA is democratising the robotics field, much like it did for LLMs with its GPUs. The concept of ‘2024 being the year of Embodied AI’ resonated with Figure founder Brett Adcock, who believes that advanced AI capable of complex tasks will likely develop in parallel with, or even slightly ahead of, reliable humanoid robot hardware.