Key takeaways
- Bard's integration with Google Lens enhances its image recognition capabilities, allowing users to extract insights and generate captions from images.
- The chatbot can interpret complex artworks and explain their meanings, offering detailed descriptions and various interpretations.
- Bard demonstrates versatility by analysing casual images, like a pug in a graduation cap, and suggesting multiple scenarios related to the image.
Ever since Google integrated the powers of Google Lens into Bard enabling users to inquire the chatbot for insights about an image or create descriptive captions, the image recognition game went through a massive makeover. Not only can it identify the things in the picture, but also extract text and make sense of the image. Even GPT-4 is multimodal, but it is still in the research phase. Several users have taken to Twitter to express their excitement about the new update.
Let’s take a look at some interesting use cases of the same.
Read and Understand Images
Let’s start with the basics. Bard is now able to understand and explain a picture. We uploaded a picture of Salvador Dali’s masterpiece ‘The Persistence of Memory’ and asked Bard to explain its meaning. The chatbot quickly gave detailed descriptions of the paintings, and the story behind them and also told us that it can be interpreted in different ways.
Another user gave an image of a pug wearing a graduation cap and asked Bard what was happening. It gave three potential instances of how the puppy could be graduating from obedience school or therapy program and more.
Create a Website from Sketch
Add an image of your sketch with the ‘+’ option and provide your prompt, for example, "Compose a concise HTML/JS script to transform this mock-up into a vibrant website, wherein the jokes are substituted with two genuine jokes". Since the initial outcome may not meet user expectations, Bard offers additional choices by selecting "View other drafts". Alternatively, you can regenerate the output. If any specific modifications are desired, you can make another request in a separate prompt. To execute the script, the HTML code should be copied into a text editor or saved as a text file.
Understand Complex Graphs
Another user used Google Bard's image input function to read text and graphs on the GPT-4 demo slides and perform calculations -- in Japanese. They explained that they used to rely on ChatGPT or Perplexity for their tasks, but Bard has now become a powerful tool for them, depending on the specific application.
Cooking Gets Easier
Stumbled upon a picture of pasta on your feed and now have a craving for it? Well, now you can upload the image of a meal and ask for a full recipe. Bard will do it for you. That is what AI influencer Rowan Cheung did.
Create iPhone App from a Screenshot
Ammar Barshi, the design manager at Brex, used Bard to replicate a basic timer application for the iPhone in just 4 minutes from a screenshot without any explicit cues about the app's functionality. Bard generated the necessary code, although it did commit a few errors, but they were easily rectified.
Brain CT Diagnoses
Another user added the image of a CT scan and asked Bard to understand it. The user gave inputs in Japanese. Bard was able to list the potential causes despite lacking specialised expertise in the field.
Social Media Caption
Bard is also a great buddy if you want to have great captions for your social media posts. It can generate captions as per your needs.
Personal Expense Manager
If you need to compile your expenditure for an expense report but you have too many bills to handle, you can now use Bard’s image recognition to take photos of the receipts and feed them into the chatbot. Bard organises the receipts into a table with details like date, time, category, description, and amount and this table can be exported to Google Sheets, eliminating the need for separate expense report apps.
