In the field of robotics, stable locomotion has been one of the fundamental challenges because the traditional robots which have reliable locomotion, often need high expertise and manual efforts to design. These traditional hand-engineered controlled robots are only effective for a small range of environments, and therefore, becoming hard to scale for the real world. To resolve this issue, Google has taken the help of deep reinforcement learning as it can learn to control policies automatically without the knowledge of the environment or the robot. And with this, one doesn’t have to train the robot again for a different environment.

The fastest anyone has learned to walk is in six months, which is a world record, which also means it takes six months minimum for a human being to get from crawling to walking. A baby, usually, takes around 10 minutes after its birth; however, this robot, depending upon the three terrains it was tested on, takes an average of around 3.5 hours to learn to walk forwards, backwards, and to turn right and left. 

Google’s Robot

Google has made a significant advancement towards reliable and stable locomotion of four-legged robots, not only the locomotion but also, these robots can navigate without any help.

There are previous researches done on this, where researchers looked for a way to get the robot to learn the real-world environment through a simulation. A virtual body of the robot first interacts with the virtual environment in the simulation. And then, the algorithm takes this data in, and once it is robust enough to operate safely, it is imported into the physical robot. This method helps in avoiding any damage to the robot and its surroundings during the trial and error process. However, the problem is that the environment should also be easy to model. The ultimate goal of this study is to make the robot ready for real-world scenarios, but the real-world environment is full of unexpected things, from sticks and stones on the path to slippery surfaces, the robot takes a very long time to simulate to environments like this. In fact, it’s so long that there is no upside to waiting for the results.

In this recent study, the researchers have avoided the hassle of modelling the real-world environment by training the model in a real-world environment from the beginning. What was required was to minimise the expected damage that training could be done with fewer iterations of trial and error methods. The researchers came up with an algorithm that requires fewer trails, which resulted in fewer errors.

The physical environment provided the unexpected natural variations for the model, and the robot could easily adapt to other similar environments, like inclines, flat terrains and steps. With the new and better algorithm, the environmental problems were solved, and the robot was able to walk in two hours.

But even if the robot adapted to the new environment, it still needed human intervention. So, to solve this problem, the team of researchers first bounded the terrain of the robot where it was allowed to move, and then the researchers trained the robot in multiple manoeuvres. So, at a time when the robot reaches the edge of the bounds by moving forward, the robot would automatically reverse its direction and start to walk backwards. Once that is set, the robot’s movements were then constrained, which also reduced the trial movements, in turn, reducing the damages from repeated falling. When the robots inevitably fell anyway, the researchers added another hard-coded algorithm to help it stand back up. 

Through exposing the robot and the model to so many variations and tweaks, the robot learned to walk autonomously. The robot, because of the deep reinforcement learning learned to walk autonomously on different surfaces, including flat ground, doormat with cervices, and a memory foam mattress. The study shows how well robots can learn to walk on unknown terrains without any human intervention.

Once the robot had learned to walk, the researchers connected a video game controller to it that allowed them to move the robot using the movements and techniques that were learnt.

The Future Of The Study

Although the setup currently can’t be used for the real world because it relies on motion capture system which is fitted above it, says one of the co-authors of the paper. In future, the researchers plan to extend this algorithm’s application to different kinds of robots and all of them learning at the same time.

Outlook

On the flat ground, the robot learned to walk in 1.5 hours, on the mattress, around 5.5 hours, and on the doormat, it took about 4.5 hours. Making a robot learn how to walk on their own on different terrains will prove to be a lot more beneficial than it looks. These robots could be used to explore different terrains and unexplored areas in the earth where it would be nearly impossible for humans to penetrate. Even space exploration might become easier in case the robot encounters some pitfalls or unusual terrains.