Last week, Alphabet subsidiary, DeepMind open-sourced Lab2D, which the researchers explained to be a scalable environment simulator for artificial intelligence research, helping them to create 2D environments for AI and ML research. Researchers claim that it facilitates researcher-led experimentation with environment design while also helping them understand the influence of environments in multi-agent reinforcement learning.
While it was built with the specific needs of multi-agent deep reinforcement learning researchers in mind, it can be used beyond that particular subfield. In this article, we take a deeper look into what DeepMind Lab2D is all about and how it can help AI researchers.
Diving Deeper Into Lab2D
As researchers explain in the paper, DeepMind Lab2D (or “DMLab2D” for short) is a platform for the creation of two-dimensional, layered, discrete “grid-world” environments, in which the pieces — which can be compared to chess pieces on a chessboard — move around. This system is particularly tailored for multi-agent reinforcement learning.
The use of AI in reinforcement learning (RL) is increasing at a rapid pace, and there is a need to have rigorous standards in terms of correctness, scale, reproducibility and ethicality in order to ensure continued research in the space. Simulation environments for RL play a crucial role and providing a robust simulation platform enables in-silico exploration of agent learning, skill acquisition, and careful measurement. DMLab2D promises to ensure just that while supporting an extensive range of research projects.
Further explaining its core features, researchers said that it is a computationally-intensive engine which is written in C++ for efficiency, whereas, most of its level-specific logic is scripted in Lua. It is not only user-friendly but ensures performance while supporting multiple simultaneous players interacting in the same environment — which can be controlled by human or computer.
“Each player can have a custom view of the world that reveals or obscures particular information. This can be used for imperfect information games, where players don’t share common knowledge, as well as for human behavioural experiments where the experimenter can see the global state of the environment as the episode is progressing,” the research paper noted.
Moreover, DMLab2D provides other features, such as:
- Observations: It allows researchers to add specific information from the environment to the observations that are produced at each time step.
- Events: They are similar to observations but are not tied to time steps, instead are triggered on specific conditions.
- Properties API: It provides a way to read and write parameters of the environment, typically parameters that change rarely.
Researchers believe that unlike other frameworks such as GVGAI, Griddle, Pommerman, BabyAI and microRTS that have existed for years, Lab2D is a step toward robust simulation platforms that might enable learning, skill acquisition, and measurement of AI systems at scale.
How Will It Help AI Researchers
Working in a 2D environment is inherently easier compared to 3D ones — both in terms of complexity and expressiveness — making it easier to capture the essence of problems and concepts in AI.
One of the significant advantages is that it provides an easy to design and program compared to 3D counterparts. This is particularly true when the 3D world actually exploits the space or physical dynamics beyond the capabilities of 2D ones. 2D worlds do not require complex 3D assets to be evocative, nor do they require reasoning about shaders, lighting, and projections.
“Rich complexity along numerous dimensions can be studied in 2D just as readily as in 3D, if not more so,” noted researchers in the paper. It also allows studying phenomena such as navigation, abstract reasoning and exploration with much ease. Moreover, the paper suggested that researchers in RL need to discretise the interactions and observations so that they become tractable. A 2D scenario can also capture the relevant complexity at hand without the need for continuous-time physical environments.
Some of the other benefits are that 2D worlds are less resource-intensive and do not require specialised hardware such as GPUs to attain performance. Better scalability and cost-effectiveness are other advantages.
Wrapping Up
It is interesting to note that DeepMind has been working rigorously in the field of reinforcement learning such as introducing methods for choosing the best policy in offline reinforcement learning and proposing new approaches to speed up solution development in complex reinforcement problems.
Now, open-sourcing the Lab2D technology will help enterprises that are conducting AI research based on reinforcement learning to inch towards success more efficiently. It provides a step towards bringing robust simulation platforms to facilitate learning, skill acquisition, and AI systems measurement at scale. The fact that it can support a wide range of projects further makes it likely to be used by researchers in their projects. “We are excited to see what the research community uses it to build in the future,” said researchers on a concluding note.
Read the complete paper here.
