DeepMind Introduces bsuite To Evaluate Performance Of Reinforcement Learning Algorithms

Reinforcement Learning has been widely used by machine learning researchers to achieve the purpose of artificial general intelligence and is one of the hottest topics in ML. RL has been applied in several ground-breaking advancements – beating StarCraft champions to defeat the world champion in the ancient Chinese game of Go, checkers, and DOTA among others. 

Reinforcement Learning technique helps an agent to interact and learn in an environment from its actions and experiences. This technique is different in the learning aspect from the supervised as well as an unsupervised learning technique. It has been applied in various applications, for instance, in online stock trading, text summarization engines, dialogue agents, among others. 

In order to solve a specific challenge or a problem, a researcher needs to understand the system better which includes several measures like scalability, the interactive environment and the key issues prevailing in the AGI system. However, many times, due to the limitations of RL algorithms, researchers find it difficult to implement this algorithm in applications such as in autonomous cars and other important applications which are interconnected with the risk of human lives.

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Now, researchers from DeepMind introduced the Behaviour Suite for Reinforcement Learning or bsuite which is the collection of experiments designed to highlight key aspects of RL agent scalability. bsuite, which is completely written in Python, aims to provide a bridge between theory and practice of RL algorithms which will benefit both the sides.

Each experiment in bsuite mainly consisted of three parts as mentioned below

Download our Mobile App

  • Environments: a fixed set of environments determined by some parameters.
  • Interaction: a fixed regime of agent/environment interaction (e.g. 100 episodes).
  • Analysis: a fixed procedure that maps agent behaviour to results and plots.

The analysis part of bsuite includes “score” which maps the performance of the RL agent on the task to [0, 1] and allows to visualise how the RL agents are behaving in a specific environment. All the experiments in bsuite have the capability to only measure the behavioural aspects of RL agents which means that they can only measure the properties which can be observed in the environment and are not internal to the agent.

Each experiment in bsuite requires the five key qualities as mentioned below

  • Targeted: performance in this task corresponds to a key issue in RL.
  • Simple: strips away confounding/confusing factors in research.
  • Challenging: pushes agents beyond the normal range.
  • Scalable: provides insight on scalability, not performance on one environment.
  • Fast: iteration from launch to results in under 30min on standard CPU. 

The experiments in bsuite are a set of environments and a number of episodes of interaction. In bsuite, all the plots and analysis are generated through the automated bsuite Jupyter notebook. In order to generate the plots and analysis via the notebook, the user will require the path to the logged data.

Advantages of bsuite

  • According to researchers, one of the most valuable uses of bsuite is as a diagnostic ‘unit-test’ for large-scale algorithm development
  • Another benefit is that it can disseminate a developer’s results more easily and engage with the research community


The Behaviour Suite for Reinforcement Learning is not a permanent replacement for the major challenges being undertaken in the AI domain. Instead, it is basically a collection of diagnostic experiments which are designed to provide meaningful insights into key aspects of RL agent behaviour. The researchers compared this project with the MNIST for reinforcement learning and stated that similar to MNIST dataset, bsuite aims to instantiate targeted experiments for the development of key RL capabilities. The Behaviour Suite for Reinforcement Learning is open-sourced in GitHub which will eventually provide significant value to the RL research community.

Read the paper here.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Ambika Choudhury
A Technical Journalist who loves writing about Machine Learning and Artificial Intelligence. A lover of music, writing and learning something out of the box.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Council Post: From Promise to Peril: The Pros and Cons of Generative AI

Most people associate ‘Generative AI’ with some type of end-of-the-world scenario. In actuality, generative AI exists to facilitate your work rather than to replace it. Its applications are showing up more frequently in daily life. There is probably a method to incorporate generative AI into your work, regardless of whether you operate as a marketer, programmer, designer, or business owner.