CEO Mark Zuckerberg announced Project CAIRaoke, a fully end-to-end neural model for building on-device assistants, at Meta’s first virtual event since the rebranding.
We’re excited to share details on Project CAIRaoke, a breakthrough in conversational AI. With this end-to-end system, we’ll be able to have much more personal, contextual conversations than we can with the systems people are familiar with today, said Meta AI.
Alborz Geramifard, a senior research manager at Meta AI, expanded on how the company is taking conversational AI to a different level. "At Meta AI, we're working on a system that could be personalised, embedded, embodied, and can interact with you in a contextual multimodal fashion. That way your interactions are as frictionless as possible,” he said.
In the future, the assistant may even follow the person in the metaverse. But currently, the focus is on voice-only interactions.
"It can see what you see from your first-person perspective, hear what you hear and, most importantly, understand the context of the situations you are in," he said.
Despite advancements in natural language understanding, the supercharged assistants have yet to become a reality. The Meta AI team has combined the modules from natural language understanding to the natural language generation in a single model. For building the contextual aspect, Meta is relying on the AI model called BART.
Meta is using the model in its video-calling Portal device. “We plan to augment the project CAIRaoke model to handle multilingual and then multimodal input and outputs as we hope its single model architecture allows for a scooter upgrade process,” said Geramifard.
Meta demonstrated the potential of Project CAIRaoke in a cooking video.
https://twitter.com/QWongSJ/status/1496555874764083201?t=daK6J3nOw9Uod0vrr4S8Fg&s=08
Meta has plans to integrate it in augmented and virtual reality devices to enable even more immersive and multimodal interactions.
- Why AI is Eating Google Search’s Ad Revenue While Boosting Meta’s
- Anthropic Makes $2.08 in Revenue for Every $1 Spent on Compute — Here’s the Full IPO Story
- OpenAI Says AGI is Here. But Not Everyone is Buying It
- Why Oracle is Bringing Post-Quantum Security to Java
- What Really Happens When You Turn Your Selfie Into a 1980s AI Pic?
- AI Agents Are Making Data Security A Bigger Enterprise Problem