Active Hackathon

Can Uber’s POET Address AI’s Open-Ended Endeavours?

Embracing the open-ended is considered by many futurists and experts, to be the last grand frontier for the artificial intelligence to invade. The concept of the open-ended can be explained in terms of natural evolution. Just like the way a species that has survived over millennia, makes course correction under new environments, machines too will be made to learn about the problems which they haven’t been trained to do. Not only that, considering open-endedness at the algorithmic stage, will enable AI to generate the new problems as well.

Just like a species corrects itself with suitable strategies with millions of years of evolution open-ended computation systems will make systems more powerful. These are processes that produce not just a single positive result, but multiple unplanned advances rolling ahead in parallel without any final destination.


Sign up for your weekly dose of what's up in emerging technology.

In this paper the researchers at Uber’s AI labs demonstrate  POET (Paired Open-Ended Trailblazer) an open-ended algorithm, to tackle problems in a space of two-dimensional landscapes and a solution space of robotic behaviours that aim to traverse them.

POET is designed to facilitate an open-ended process of discovery within a single run. It maintains a population of environments.

POET has three main tasks that it performs at each iteration of its main loop:

  1. generating new environments E(·) from those currently active,
  2. optimizing paired agents within their respective environments, and
  3. attempting to transfer current agents θ from one environment to another.

These steps are to make sure that the curriculum that emerges from adding new environments is calibrated to the behaviour of agents. So, whenever an agent is exposed to a new environment, the corrections made will make the model versatile and robust.

Experimental Setup

The agent’s hull is supported by two legs. The hips and knees of each leg are controlled by two motor joints, creating an action space of four dimensions.

The domain of this experiment has been adopted from OpenAI’s “Bipedal Walker hardcore”. In this modified version, simulation is quicker, which enables the researchers to conduct experiments for longer duration in environments of a complex scenario.

The movement of the agent on this highly varying terrain is guided by the LIDAR rangefinders, whose measurements are included in the state variables such as the hull angle, hull angular velocity, horizontal and vertical speeds, positions of joints and joint angular velocities, and whether legs touch the ground.

Source: Paper by Rui Wang, Joel Lehman, Jeff Clune,Kenneth O. Stanle

The generated terrain consists of one or more types of obstacles.  These can include stumps, gaps, and stairs on a surface with a variable degree of roughness and the agent’s only aim is to keep on moving without tumbling.

The reward function to keep the hull straight is as follows:

If the agent falls, the reward is – 100

The episode immediately terminates when the time limit (2,000 time steps) is reached, when the

agent falls, or when it completes the course.

All controllers in the experiments are implemented as neural networks with 3 fully-connected layers with than activation functions.

Check the full algorithm here.

What Does POET Have To Offer For The Future

In this domain, POET builds out a diverse collection of solved landscapes through experiments that modify previously-solved landscapes and apply and refine previously-discovered successful robotic behaviours. The=is makes it easy for the robot to traverse through rugged obstacle-filled landscapes that are not solvable when approached in isolation with standard optimization techniques. The success of POET hints at a step towards truly open-ended machine learning algorithms that continually invent and solve new challenges.

Implementing such algorithms will lead us to problems which were thought to be solvable or those problems we never knew existed. While the road remains long, the rewards for machine learning of beginning to capture the character of open-ended processes is potentially high. In addition, a diversity of such results can be generated in a single run, and the problems and solutions can both increase in complexity over time.

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
How Data Science Can Help Overcome The Global Chip Shortage

China-Taiwan standoff might increase Global chip shortage

After Nancy Pelosi’s visit to Taiwan, Chinese aircraft are violating Taiwan’s airspace. The escalation made TSMC’s chairman go public and threaten the world with consequences. Can this move by China fuel a global chip shortage?

Another bill bites the dust

The Bill had faced heavy criticism from different stakeholders -citizens, tech firms, political parties since its inception

So long, Spotify

‘TikTok Music’ is set to take over the online streaming space, but there exists an app that has silently established itself in the Indian market.