IBM Sets Its NLP Ambitions High With New Capabilities In Watson

Boosting its NLP capabilities, IBM has launched new innovative capabilities in IBM Watson Discovery and IBM Watson Assistant, which will empower businesses to deploy and scale sophisticated AI systems. It will leverage NLP with accuracy and efficiency, all while requiring fewer data and training time. It is another significant step by the tech giant to offer advanced ability to understand the language of business. 

With an aim to bring better NLP and NLU offerings to users in its enterprise products, the company has yet again shown its drive to take NLP efforts to a newer height. 

What’s New With IBM’s NLP Game

While recent announcements by IBM focus around language, explainability, and workplace automation, the update around its language capabilities include reading comprehension, FAQ extraction and improving interactions in Watson Assistant. All these products aim to bring resilience, productivity and value for enterprises. Let’s break these features in detail.

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Reading comprehension, which is currently available in beta in IBM Watson Discovery, will let users access and search enterprise documents using natural language. Built on top of the question answering system by IBM Research, it is designed to help identify more precise answers in response to natural language queries. The company stated that it also provides scores that indicate how confident the system is in each answer.

Another feature by IBM called FAQ Extraction will enable businesses to keep virtual assistants up to date more efficiently by automatically extracting relevant data from preselected sources. The idea is to keep virtual assistants updated with latest answers while reducing the need for manual intervention. It is to be noted that FAQ Extraction will use NLP technique to automate the extraction of Q&A pairs from FAQ documents.

Download our Mobile App

Apart from these two, the company also announced a new intent classification model in IBM Watson Assistant, which is aimed at understanding an end user’s goal or intent behind engaging with the virtual assistant. It will then be used to train the systems accordingly while enabling greater accuracy in virtual assistants. 

Furthermore, Watson discovery has extended its capabilities in terms of language support as it now supports ten new languages — including Hindi. As the company stated, it can understand Devanagari script and extract information such as business keywords, named entities, perform sentiment analysis, among others. To bring about this capability, IBM Research India collaborated with AI Horizon Network — IIT Bombay — that works on a series of advanced research in AI, ML and NLP applications. 

Talking about the partnership, Prof Pushpak Bhattacharyya, Professor – Department of Computer Science and Engineering, IIT- Bombay said that the complexity of Indian languages makes its adoption and adaption into English-centric NLP quite difficult. He said that through this partnership, they have been able to use machine learning for India language NLP and address challenges such as low resource, understanding of Hindi language sense, intent, sentiment and more. The collaboration has witnessed some interesting developments over the last two years.  

Further stressing on their focus on NLP developments, Gargi Dasgupta –Director, IBM Research India said that they used a combination of AI, deep learning and shallow semantic understanding models to make Watson understand Hindi. 

IBM Improvising Its NLP Capabilities Since Earlier This Year

Earlier this year, IBM made some big improvements to the natural language processing capabilities of its IBM Watson platform. These innovations which were born out of IBM Research’s Project Debater led Watson to understand and analyse challenging aspects of the English language. 

With a capability to debate with humans on complex topics, Debater was involved in a popular TV show — “That’s Debatable” on the topic “Is it time to redistribute the world’s wealth?” earlier this year. Using a feature of NLP called the key point analysis, it could categorise and summarise thousands of public opinions into concrete points. The technique essentially summarises the most significant points from statements and complex documents to produce a concise list of information. 

Having worked on Project Debator’s NLP capabilities throughout the year, IBM is now bringing these into larger IBM products. Not just the language understanding and processing, but the company is building upon sentiment analysis, intent analysis, summarisation and more, to further make the models more accurate. The idea is to work continuously in these projects to improve natural language offering for clients. All these efforts combined are directed towards IBM being a leader in the NLP space, not far away in the future.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Srishti Deoras
Srishti currently works as Associate Editor at Analytics India Magazine. When not covering the analytics news, editing and writing articles, she could be found reading or capturing thoughts into pictures.

Our Upcoming Events

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023

21 Jul, 2023 | New York
MachineCon USA 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

The Great Indian IT Reshuffling

While both the top guns of TCS and Tech Mahindra are reflecting rather positive signs to the media, the reason behind the resignations is far more grave.

OpenAI, a Data Scavenging Company for Microsoft

While it might be true that the investment was for furthering AI research, this partnership is also providing Microsoft with one of the greatest assets of this digital age, data​​, and—perhaps to make it worse—that data might be yours.