CEO Mark Zuckerburg spoke about a few breakthroughs in AI research geared for the metaverse, alongside his vision for AI assistants (aka Project CAIRaoke) and more during his keynote speech at Meta’s Inside the Lab: Building for Metaverse for AI event.
Highlighting the challenges in building the metaverse, Zuckerburg said the key to unlocking a lot of advances relies on AI. Further, he said, as these worlds will be dynamic and always changing, AI will need to understand the context and learn in the way that humans do. “When we have glasses on our faces, that will be the first time that an AI system will be able to really see the world from our perspective,” said Zuckerberg.
He said computing is becoming more contextual. “As devices have gotten better at understanding and anticipating what we want, they have also gotten more useful. I expect that these trends will only increase in the future,” he added.
Zuckerberg said the metaverse would consist of immersive worlds the user can create and interact with, including the position in 3D space, body language, facial gestures, etc. “So, you experience it and move through it as if you are really there,” he added.
Project CAIRoke
Meta announced project CAIRaoke, an end-to-end neural model for building on device systems, to help deliver better dialogue capabilities, true world creation, and exploration.
He said Meta AI is exploring two research areas to make this possible – egocentric perception, which is about seeing the world from a first-person perspective, and a whole new class of generative AI models to help users create anything they can imagine.
Further, he also demonstrated BuilderBot, a new tool that enables users to generate or import things into the virtual environment via voice commands.
Universal Speech Translator
Nearly half of the world’s population does not have access to online content in their preferred languages. To address this challenge, Meta is coming up with Universal Speech Translator, an AI tool that offers real-time speech-to-speech translation across all languages.
Jérôme Pesenti, who currently leads Facebook AI, said the responsible AI team, in collaboration with the Instagram equity team, is planning to publish a prototype system card for Instagram feed ranking.
Meta also revealed TorchRec, a library for building SOTA recommendations systems for the open-source PyTorch machine learning framework. The system will power personalisation across their products.
“TorchRec demonstrates Meta’s commitment to AI transparency and open science. It is available as Pytorch memory and provides common sparsity and parallelism, primitives, enabling researchers to build the same state of the art personalisation that is used by Facebook Newsfeed and by Instagram Reels today,” Pesenti added.
Check out TorchRec here.
According to Zuckerberg, there are four basic pillars of work on AI. Foundational research, AI for products, responsible AI and AI infrastructure. “In each of these areas, we have some pretty ambitious hiring goals,” he said.
Inside the Lab offered a sneak peek into Meta’s metaverse roadmap, alongside its breakthroughs in the area of self-supervised learning and AI supercomputers.
