Evolution strategy (ES) is an optimisation technique that has been applied to multiple challenging decision-making problems, such as power system control and legged locomotion. 

However, one of the major weaknesses of ES-based algorithms lies in the complexity in scaling to issues that require high-dimensional sensory inputs for encoding dynamics, such as training of robots with complex vision inputs.

To combat this challenge, researchers at Google AI have proposed a learning algorithm that combines ES and representation learning to solve high dimensional problems effectively. 

The paper, ‘PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representations’, says that the primary idea is to leverage predictive information in order to obtain a representation of the high-dimensional environment dynamics.

The same innovation would be applied on Augmented Random Search (ARS) – a well-known ES algorithm – transforming the learned compact representation into robot actions. 

Initially, the team tested PI-ARS on the challenging issue of visual-locomotion for legged robots. The algorithm enables fast training of performant vision-based locomotion controllers and can traverse a variety of difficult environments. 

Source: Overview of PI-ARS Data Flow, Google AI

The researchers then combined PI (Predictive information) with Augmented Random Search (ARS), an algorithm with excellent optimization capabilities for challenging decision-making tasks. The blog read, “At each iteration of ARS, it samples a population of perturbed controller parameters, evaluates their performance in the testing environment, and then computes a gradient that moves the controller towards the ones that performed better.”
Read the whole blog here.