Earlier this week, Google DeepMind released resources for general purpose robots and took it a step further. This is touted to be the ImageNet moment for robotics, where it won’t be necessary to individually train robotic models to each specific task as it explores knowledge transfer between robots. 

In line with this, let’s take a look at other models and robots for general purpose tasks and operations.  

RT-X 

RT-X, or the Robotics Transformer X, is a general-purpose robotics model developed by Google DeepMind to enhance robotics research and performance across a variety of robots and tasks.

The key goal was to create a model that can generalize and transfer its skills to various robots and tasks. It can be used for actions, vision, and language understanding, making it a comprehensive tool for general robotics research.

RT-X is built upon the foundation of two previous robotics transformer models: RT-1 and RT-2. RT-1 is a model developed for real-world robotic control at scale, and RT-2 is a vision-language-action (VLA) model that learns from both web and robotics data. By incorporating the architectures and knowledge from these models, RT-X achieves higher performance, especially due to the extensive and diverse cross-embodiment data it is trained on.

The development of RT-X involves training the model on a vast and diverse dataset, known as the Open X-Embodiment dataset. This dataset is a collection of data from 22 different robot embodiments, encompassing a wide range of skills and tasks, a total of 150,000 tasks in more than 500 distinct skills according to the paper.

RoboCat

RoboCat developed by Google DeepMind is an AI agent in robotics that can learn various tasks across different robotic systems. What sets RoboCat apart is its ability to not only master multiple tasks but also generate its own training data to enhance its skills.

With access to a broad and varied dataset, it can quickly grasp a new task in as little as 100 demonstrations. This rapid learning potential is a promising advancement in robotics research, diminishing the need for extensive human-guided training. 

RoboCat is a product of merging Google’s Gato multimodal model, proficient in processing language, images, and actions in both simulated and real-world contexts. Gato’s architecture was integrated with an extensive training dataset comprising image-action sequences from diverse robot arms engaged in a myriad of tasks.

Dactyl

Dactyl, built by OpenAI, was trained in a computer simulation within, with its learnings then applied in the real world, despite some discrepancies in simulation accuracy. The specialised Shadow Dexterous Hand enables Dactyl to manipulate objects like blocks or prisms, employing a learning approach akin to that used for OpenAI Five. 

Dactyl specifically addresses the task of altering an object’s position while held by the robotic hand, such as rotating a block. For precise grips, like the Tip Pinch grasp, Dactyl cleverly uses its thumb and little finger, which is quite like how humans use their thumb along with either the index or middle finger. 

However, Dactyl’s little finger on the robot hand is more flexible due to an extra way it can move. That’s why Dactyl often opts for this finger. Essentially, Dactyl can independently discover grips like humans do but adjusts them to suit its own hand’s capabilities and limitations.

Jaco Arm

JACO arm built by Kinova Robotics is a lightweight assistive robot designed to compensate for lost arm movements. It consists of six linked segments, including a three-fingered hand. Users control hand movements in 3D space and object grasping/releasing with an adaptable controller, utilizing two or three fingers. 

The arm can be mounted on a wheelchair, workstation, or table, fitting under the wheelchair’s armrest without widening it. JACO enhances daily activities for individuals with physical disabilities without hindering wheelchair mobility.

Key components include actuators driven by DC brushless motors with Harmonic Drive tech and various sensors for precise control. Grippers are designed to grasp diverse daily objects, with underactuated grippers simplifying control and adapting to object shapes. 

The interface options include a Software Development Kit (SDK), a ROS package, and a joystick, providing users with flexible control over the robot’s movements.

Panda Arm

Panda Arm was built by Franka Emika, a German robotics platform company. The robot has a range of features and is designed with sensors to stop moving when there’s unexpected interference to prevent accidents. It has a special hand that can hold things tightly, like exerting a strong grip. This hand can hold up to 3 kg of weight. This allows the robot to pick up various objects almost like a human arm.

The arm is used in production lines and its uses are non-exhaustive. It is being used in nursing homes, laboratories and universities, logistics platforms, etc.