Google is reportedly rolling out new features for Gemini to analyse a user’s surroundings in real-time using the device’s camera or screen, providing instant answers to questions. These features are a part of Project Astra, which was announced by the company last year.
A user on Reddit claimed to have received the update on Gemini and shared a video demonstrating how the feature works. This feature is a part of Gemini Live, which lets users converse with the AI tool in real-time and using natural language.
Later, a spokesperson from Google confirmed to The Verge that the company has indeed started rolling out the features. Recently, at the Mobile World Congress in Barcelona, the company announced that ‘live video’ and ‘screen sharing’ capabilities will start rolling out to Gemini Advanced subscribers as part of the Google One Premium plan on Android devices this month.
In December last year, the company announced more improvements and features to Project Astra, such as better dialogue, latency and memory capabilities and the ability to use external tools. During this time, the company also unveiled Multimodal Live API with Gemini 2.0, which could read information from the user’s screen and offer real-time advice.
OpenAI already has a feature for ChatGPT Plus and Pro subscribers to use the camera to feed real-time visual information for the advanced voice mode feature.
Recently, Google announced Audio Overviews for Gemini, which brings NotebookLM’s capabilities to the AI assistant. The company also announced native image generation and image editing features for the Gemini 2.0 Flash model.
Meanwhile, Google announced Gemma 3, the next iteration in the Gemma family of open-weight models—a successor to the Gemma 2 model released last year. In the Chatbot Arena, Gemma 3 27B outperformed DeepSeek-V3, OpenAI’s o3-mini and Meta’s Llama 3-405B model. Models in Chatbot Arena are evaluated against each other through side-by-side evaluations by humans.
- AI Productivity to Add $28 Billion to $84 Billion to India's GDP: Cognizant Study
- OpenAI's Codex Helps in Aerodynamic Engineering for Mazda
- Bengaluru Accounts for 34% of India's FDE Talent: Report
- 84% See AI Productivity Gains, but Only 10% Allow Agent Autonomy: TCS Study
- Snowflake Introduces Business Ontology, Cortex AI Gateway to Bring AI Agents Into Enterprise Workflows
- Three Members Behind Claude Code Leave Anthropic Within 10 Days