ChatGPT clears the United States Medical Licensing Examination (USMLE)

Listen to this story

What takes a medical student close to four years and over two years of clinical rotations to clear the United States Medical Licensing Examination (USMLE), OpenAI’s ChatGPT has successfully cleared all three parts of the USMLE in a single go, as per the results of a new experiment.

The researchers said that “ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations.”

The paper also cites that “these results suggest that large language models (LLMs) may have the potential to assist with medical education, and potentially, clinical decision-making.”

The USMLE is a high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ knowledge spanning basic science, clinical reasoning, medical management, and bioethics. The difficulty and complexity of questions are highly standardized and regulated, making it an ideal input substrate for AI testing. 

However, this is not the first time the chatbot has aced an examination. A few days back, professors at the University of Pennsylvania’s Wharton School of Business discovered that the ChatGPT could easily successfully complete examinations on a typical MBA core course, Operations Management.

As per a report in the Fortune, Professor Christian Terwiesch released a paper this week which evaluates ChatGPT’s performance on the Operations Management paper. According to him, the chatbot “does an amazing job at basic operations management and process analysis questions, including those based on case studies.”

Terwiesch further said that the chatbot had its fair share of shortcomings, wherein the AI bot failed to answer “more advanced process analysis questions.” The professor also noted that ChatGPT “would have received a B to B- grade on the exam.”

Most recently, researchers also tested GPT-3.5  using questions from the US Bar Exam. They predict that GPT-4 and similar models have the potential to pass the exam very soon. In addition, the researchers found that hyper parameter optimisation and prompt engineering positively impacted GPT-3.5’s zero-shot performance. 

Download our Mobile App

Aparna Iyer
Aparna Iyer has covered various sectors spanning education, wildlife, culture and law for close to a decade. She now writes on technology and is keen to unearth its capability for public good.

Subscribe to our newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day.
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

Our Recent Stories

Our Upcoming Events

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox