Active Hackathon

Facebook Introduces New Platform For Building Robots

Droidlet is capable of building embodied agents that can recognise, react, and navigate their surroundings.

Facebook has introduced Droidlet, an open-source, modular, heterogeneous embodied agent architecture. The Droidlet platform can be used to build embodied agents using natural language processing, computer vision, and robotics. Now with Facebook Droidlet platform, researchers can build more intelligent real-world robots. In addition, it simplifies the integration of a wide range of state-of-the-art machine learning algorithms and robotics to facilitate rapid prototyping.


A droid agent is considered to be made up of a collection of components, which are both heuristic and learned. Droidlet allows for component swapping and customisable designs. This software platform gives researchers a debug dashboard as well as a markup interface for correcting errors and annotation. Robotic-focused features such as robot-specific wrappers and environments and models built specifically for robots are also included.


Sign up for your weekly dose of what's up in emerging technology.

Not only does the Droidlet platform have different tools and advancements in artificial intelligence, like PyRobot, AllenNLP, and Detectron, but it also connects these iterations of tools to provide a better and unified experience.

Droidlet lets researchers use different computer vision or NLP algorithms with their robots. In addition, they can use Droidlet to accomplish complex tasks in both real world or within a simulated environment like Minecraft or Habitat. 

Droidlet is capable of building embodied agents that can recognise, react, and navigate their surroundings. It simplifies the integration of various cutting-edge machine learning algorithms in these systems, allowing users to prototype new ideas faster than ever before.

According to the research paper, “Droidlet: modular, heterogenous, multi-modal agents”, The objective of the platform is to build intelligent agents that can learn continuously from their encounters with the real world.

The researchers hope that the platform for Droidlets may help to further their understanding of various areas of research including self-supervised learning, multi-modal learning, interactive learning, human-robot interaction, and lifelong learning.

Droidlet provides “battery-included” systems for researchers and hobbyists that have access to trained object detection and pose estimation models to collect their observations and then store them in the robot’s memory. To convert a natural language statement like “Go to the red chair” into a programme, the system activates a pre-trained neural semantic parser.


What’s Droidlet made of? 

Droidlet comprises a collection of various components, some based on statistical heuristics, and others trained via machine learning. The user interface of the Droidlet is made up of the following elements:

  • Memory system: An information storage and retrieval system that operates across various modules. 
  • Perceptual modules: A set of perceptual modules which are typically used to obtain data from the environment and store it in memory. 
  • Lower-level tasks: Tasks that, at the lower level, affect the agent’s environment include things like “move three feet forward” and “place item in hand at given coordinates.”
  • Controller: A controller that decides which tasks to perform depending on the current state of the memory. 


Summing up 

This platform is capable of delivering robust and sophisticated results while operating as an independent component. In addition, the Droidlet system will be able to grow even more robust over time, as they introduce new functions and capabilities based on sensory modalities or hardware setups others have added to the system. 

Meanwhile, Robotics software development is increasingly becoming a large portion of Facebook’s tech operations. It recently teamed up with Carnegie Mellon and Berkeley to teach robots how to adjust in response to different environments in real-time. It will be interesting to see how Facebook’s latest open-source platform works out for researchers in the coming days.

More Great AIM Stories

Ritika Sagar
Ritika Sagar is currently pursuing PDG in Journalism from St. Xavier's, Mumbai. She is a journalist in the making who spends her time playing video games and analyzing the developments in the tech world.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan, 2023

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

Conference, in-person (Bangalore)
MachineCon 2023
23rd Jun, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM

Data Science Skills Survey 2022 – By AIM and Great Learning

Data science and its applications are becoming more common in a rapidly digitising world. This report presents a comprehensive view to all the stakeholders — students, professionals, recruiters, and others — about the different key data science tools or skillsets required to start or advance a career in the data science industry.

How to Kill Google Play Monopoly

The only way to break Google’s monopoly is to have localised app stores with an interface as robust as Google’s – and this isn’t an easy ask. What are the options?