AI Will Redefine How We Search the Web

The reasoning engine can provide a human-like conversational response backed and reasoned by source through citations
Listen to this story

Traditional search engines have existed for over two decades now. Google, which is synonymous with search, launched its search engine in 1998 and over the years, has become the most-used search engine worldwide.

Nothing stirred for years, and search engines continued to work in the same fashion – indexing and extracting on a keyword hit. But, things are about to change. Microsoft is set to change the rules of the game with the integration of ChatGPT with its search engine Bing

Although Bing won’t be the first search engine to integrate a large language AI model since You.com, Neeva AI and Perplexity AI have already done it successfully so far, a bigger player initiative would mark the evolution of search engines. 

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Traditional search engines have also been using AI for a few years now. Google Search has already been leveraging the power of AI since 2015 when it introduced the RankBrain algorithm. Over the years, other algorithms such as Neural matching, BERT and MUM, among others, have been incorporated into Google Search.

However, recent approaches have been very different. Large Language Models (LLMs) are rapidly developing and mastering various applications. Now, the capabilities of LLMs are being used to define the next generation of search engines. 


Download our Mobile App



Rise of reasoning engines

Traditional search engines do not address information needs directly, and instead provide references. Therefore, by leveraging the capabilities of LLMs like GPT-3, startups are building search engines which provide pointed answers to your search query, along with the indexed web pages. 

Privacy-focused search engine You.com recently launched youChat, a ChatGPT-like chatbot that provides answers upfront along with citations. You.com has added the bot to its search engine as a feature.

Similarly, Neeva AI, a paid search engine, goes a step further. Once a user types his query, the AI-generated response is the first thing you see.

(Source: Twitter)

“Neeva AI is leading a new frontier of AI search that leverages cutting-edge LLMs and an independent search stack to create a unique and transformative search experience,” the company said.

The extractive nature of traditional search engines and the abstractive nature of the generative approach found in LLMs that underpin ChatGPT have to come to terms with each other to provide a better experience for the user. 

“This will result in a new generation of the search engine – the ‘reasoning engine’ – that can provide a human-like conversational response backed and reasoned by source through citations,” Raj Neervannan, co-founder and CTO, AlphaSense, told AIM

Neervannan explained how AlphaSense has been leveraging the latest AI technologies to provide search results in a more natural, native way that is seamless to the user. “We use language models to teach semantics to our search engines and have been combining it with the ability to provide precise and stateful search results that tie the results back to the source at the snippet level in any document or source. For our users conducting market research, this technology helps instantly identify trends and summarise long reports, as well as more easily gather, digest, and analyse large amounts of information across multiple languages,” he added.

What happens to traditional search engines?

We believe traditional search engines like Google Search and Bing will continue to exist and evolve and leverage the capabilities of LLMs. Microsoft is already planning to integrate ChatGPT with Bing. Google, on the other hand, published a research paper titled ‘Rethinking Search: Making Domain Experts out of Dilettantes’  in this regard in 2021. Researchers at Google proposed a radical redesign that throws out the ranking approach and replaces it with a single large AI language model—a future version of BERT or GPT-3

This approach not only changes how search engines work but also how we interact with them. “Traditional search engines will have to learn to rank better with a stateful engine in a native way, and generate a response that summarises the search results. This is a big jump from the current models that just index and extract on a keyword hit,” Neervannan concluded.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Pritam Bordoloi
I have a keen interest in creative writing and artificial intelligence. As a journalist, I deep dive into the world of technology and analyse how it’s restructuring business models and reshaping society.

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox
MOST POPULAR

Council Post: The Rise of Generative AI and Living Content

In this era of content, the use of technology, such as AI and data analytics, is becoming increasingly important as it can help content creators personalise their content, improve its quality, and reach their target audience with greater efficacy. AI writing has arrived and is here to stay. Once we overcome the initial need to cling to our conventional methods, we can begin to be more receptive to the tremendous opportunities that these technologies present.