Microsoft Launches BioGPT, the ChatGPT of Lifescience

BioGPT achieves SOTA results on end-to-end relation extraction tasks.
Listen to this story

From food recipes to medical solutions to even Jesus, there literally exists one or the other versions of ChatGPT. Adding to the band is Microsoft’s BioGPT, a generative pre-trained Transformer language model in a biomedical domain. It can be used for lifescience literature text generation and mining. 

The team studied the prompt design and target sequence design when applying BioGPT to downstream tasks and found that target sequences with natural language semantics are better than structured prompts explored in previous works.

The team designed and examined the prompt and the target sequence format while applying pre-trained BioGPT to downstream tasks based on GPT-2 and pre-trained on 15 million PubMed abstracts corpus. It performs better than earlier models on most of the six biomedical NLP tasks it evaluates.

The pre-trained model was used for biomedical NLP tasks such as question answering, document classification, text generation, and end-to-end relation extraction.

BioGPT achieves SOTA on three end-to-end relation extraction tasks and one question-answering task  Additionally, it outperforms GPT-2 on the text generation task in terms of lifescience text-generating ability.

It sets a new record with F1 scores of 44.98 percent, 38.42 percent, and 40.76 percent on the BC5CDR, KD-DTI, and DDI end-to-end relation extraction tasks, respectively, and 78.2 percent accuracy on PubMedQA.

On PubMedQA, the larger model ‘BioGPT-Large’ obtains 81.0 percent accuracy. their case study on text generation further highlights the benefit of using BioGPT to create easy definitions for biological terms. 

Read the full paper here. Check out the source code here

Why Does It Matter?

Pre-trained language models have gained significant attention in the biomedical field due to their success in the general natural language domain. Within this domain, there are two main branches of pre-trained language models: BERT (and its variants) and GPT (and its variants). BioBERT and PubMedBERT are examples of the first branch that have received the most attention in the biomedical field. While they have performed well in many discriminative downstream biomedical tasks, their use is limited by their inability to generate text.

Microsoft’s Healthcare Initiatives

In March 2022, Apollo launched ProHealthDeepX, which shows how a patient’s risk factors for heart disease affect their heart through an immersive mixed reality experience using the Microsoft HoloLens 2.  

At the leadership summit 2022 in India, Microsoft CEO Satya Nadella spoke about the tech giant partnering with Apollo to develop a cardiac prognosis model that will be trained on data from South Asians. 

Microsoft is also working closely with the OpenAI team to employ GPT-3 to facilitate collaboration between employees and clinicians and improve healthcare teams’ efficiency.

Download our Mobile App

Shritama Saha
Shritama is a technology journalist who is keen to learn about AI and analytics play. A graduate in mass communication, she is passionate to explore the influence of data science on fashion, drug development, films, and art.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Upcoming Events

15th June | Online

Building LLM powered applications using LangChain

17th June | Online

Mastering LangChain: A Hands-on Workshop for Building Generative AI Applications

Jun 23, 2023 | Bangalore

MachineCon 2023 India

26th June | Online

Accelerating inference for every workload with TensorRT

MachineCon 2023 USA

Jul 21, 2023 | New York

Cypher 2023

Oct 11-13, 2023 | Bangalore

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Is Sam Altman a Hypocrite? 

While on the one hand, Altman is advocating for the international community to build strong AI regulations, he is also worried when someone finally decides to regulate it