Article complete

Deepmind Releases New Framework For Robots' Object Stacking Capabilities

DeepMind presents a new standard for improving robots' ability to stack objects

Deepmind Releases New Framework For Robots' Object Stacking Capabilities

Key takeaways

  • DeepMind introduces a benchmark to enhance robots' object stacking capabilities.
  • Robotic stacking requires a mix of motor, sensory, and analytical skills.
  • The RGB-Stacking benchmark involves grasping and balancing multiple objects.
  • The research combines simulation with real-world data for better learning.
  • DeepMind is open-sourcing tools and environments to support further robotics research.

Takeaways drafted with AI from the article text.

DeepMind suggests a new benchmark for improving robots' ability to stack objects. For the majority of people, stacking an object on top of another is a simple task. However, even the most complicated robots fail to perform more than one of these tasks concurrently. Stacking needs a variety of motor, sensory, and analytical abilities, as well as the capacity to interact with a variety of items. 

Source: DeepMind RGB-Stacking

Objective

The goal of RGB-Stacking is to train a robotic arm to stack objects of various shapes using reinforcement learning. The researchers suspend a parallel gripper coupled to a robot arm above a basket with three objects — one red, one green, and one blue — in the basket, hence the name RGB. The objective is straightforward: stack the red object on top of the blue object in less than 20 seconds, while the green object acts as an obstruction and diversion. Through training on a variety of object sets, the learning process ensures that the agent gains generic skills. The researchers varied the grip and stack affordances — the characteristics that dictate how the agent can grasp and stack each object — deliberately. This design principle compels the agent to engage in behaviours other than simply pick-and-place.

Challenge

RGB-Stacking features two challenge variants with varying degrees of difficulty:

  • Skill Mastery
  • Skill Generalisation

The objective of "Skill Mastery" is to train a single agent to be proficient at stacking a specified set of five triplets. They employ the same triplets for evaluation in "Skill Generalisation," but train the agent on a vast set of training items — totaling more than a million possible triplets. To assess generalisation, these training objects are chosen to be distinct from the group of objects where the test triplets were selected.

The researchers separate their learning pipeline into three phases in both versions:

  • First, they train in simulation using a commercial reinforcement learning technique called Maximum a posteriori Policy Optimization (MPO). At this step, they take advantage of the simulator's state, which enables rapid training by providing the agent with the object positions directly, rather than the agent having to learn to detect the objects in images. Because this information is not available in the real world, the generated policy cannot be directly transferred to the real robot.
  • They then simulate a new policy using solely realistic observations: visuals and the robot's proprioceptive state. They improve transfer to real-world visuals and dynamics by using a domain-randomised simulation. The state policy acts as a teacher, providing corrections to the learning agent's behaviours, which are then turned into the new policy. 
  • Finally, they gather data using this approach on real robots and train an improved policy offline using a learnt Q function to weight good transitions, as done in Critic Regularised Regression (CRR). This enables us to leverage data acquired passively throughout the project rather than executing a time-consuming online training algorithm on real robots.

As researchers continue to work on the open problem of generalisation in robotics, let us hope that this new standard contributes to the development of new concepts and approaches that make manipulation easier and robots more capable.

Published

Spotted an error? Tell our editors. Read our editorial standards and corrections.