Microsoft has open sourced its Grounded Open Dialogue Language Model (GODEL). The tech giant is open-sourcing three versions of GODEL: small (GODEL_B) and large (GODEL_L) initiated from T5 and T5-Large, respectively, and a very large model (GODEL_XL^GPT-J) trained off GPT-J. The codes are available on Github.

GODEL

GODEL evolved out of Microsoft‘s 2019 DialoGPT project, the first large-scale pretrained language model designed specifically for dialogue. The language model is pre-trained in three phases, successively folding in data from web text, publicly open dialogues from sources such as Reddit and a collection of existing corpora that support grounded dialogue tasks.

GODEL combines two functionalities in a single model. The first is task-oriented, and the second is making the dialogue more realistic and social. Most chatbots are focused on being one or the other. The agent can recommend a restaurant and at the same time engage in chit chat about sports or weather games, and then bring the conversation back on track. 

GODEL is parameterised with a Transformer-based encoder-decoder model and trained for response generation grounded in external text, which allows more effective fine-tuning on dialogue tasks that require conditioning the response on information that is external to the current conversation (e.g., a retrieved document).

(Source: Microsoft)

Conversational AI

The conversational AI market was worth USD 6.9 billion in the year 2021. It is estimated to grow at a CAGR of 23.4 percent, earning revenue of around USD 29.9 billion by the end of 2028. Over the years, we have seen the emergence of the Bots-as-a-service (BaaS) segment. More and more organisations are now relying on Conversational AI to optimise customer experience.

“Tech giants and open source initiatives are trying their best to push conversational AI solutions and products to enterprise customers, most of them are limited to supporting or assisting teams to a certain level,” said Ashwin Swarup, VP of Data Science and Data Engineering at Digité Inc.

This is where language models like GODEL come in. What if the AI bot is able to talk to you about any current world events beside solving your queries? According to Microsoft, such days are not far-off.

A conversation with GODEL

(Source: Microsoft)

“Many companies today are looking at developing chatbots to address end customers’ problems in real-time and offer them a seamless experience. The customisation of chatbots plays a critical role in enhancing the end-user experience,” Swarup added.

Microsoft too offers AI chatbots as a service on the Azure cloud service marketplace called Azure Bot Service. 

Microsoft and open source

Companies such as Hugging Face are doubling down on its efforts to democratise AI and ML through open source and open science. Today, the platform offers 100,000 pre-trained models and 10,000 datasets for NLP, computer vision, speech, time-series, biology, reinforcement learning, chemistry and more.

Open-sourcing AI models is important because it makes the technology accessible to all the members of the community. Taking the cue, Google, Meta and Amazon are now open-sourcing their models. Microsoft was anti open source up until 2010. Later, the tech giant joined the Open Source Initiative in 2016.

In 2019, Microsoft open-sourced the algorithm that powers its Bing search engine. Last year, the company open-sourced ‘mixture of experts’ library to train huge AI models. Similarly, in 2020, it open-sourced a library called DeepSpeed for training the Turing Natural Language Generation (T-NLG) model, the largest natural language processing(NLP) model then.

With regard to open-sourcing GODEL, Microsoft said: “We hope GODEL helps numerous academic research teams advance the field of conversational AI with innovative dialogue models while eliminating the need for significant GPU resources. We plan to continuously improve GODEL and make more models available to the research community.”