ChatGPT clears the United States Medical Licensing Examination (USMLE)

Listen to this story

What takes a medical student close to four years and over two years of clinical rotations to clear the United States Medical Licensing Examination (USMLE), OpenAI’s ChatGPT has successfully cleared all three parts of the USMLE in a single go, as per the results of a new experiment.

The researchers said that “ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations.”

The paper also cites that “these results suggest that large language models (LLMs) may have the potential to assist with medical education, and potentially, clinical decision-making.”


Sign up for your weekly dose of what's up in emerging technology.

The USMLE is a high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ knowledge spanning basic science, clinical reasoning, medical management, and bioethics. The difficulty and complexity of questions are highly standardized and regulated, making it an ideal input substrate for AI testing. 

However, this is not the first time the chatbot has aced an examination. A few days back, professors at the University of Pennsylvania’s Wharton School of Business discovered that the ChatGPT could easily successfully complete examinations on a typical MBA core course, Operations Management.

Download our Mobile App

As per a report in the Fortune, Professor Christian Terwiesch released a paper this week which evaluates ChatGPT’s performance on the Operations Management paper. According to him, the chatbot “does an amazing job at basic operations management and process analysis questions, including those based on case studies.”

Terwiesch further said that the chatbot had its fair share of shortcomings, wherein the AI bot failed to answer “more advanced process analysis questions.” The professor also noted that ChatGPT “would have received a B to B- grade on the exam.”

Most recently, researchers also tested GPT-3.5  using questions from the US Bar Exam. They predict that GPT-4 and similar models have the potential to pass the exam very soon. In addition, the researchers found that hyper parameter optimisation and prompt engineering positively impacted GPT-3.5’s zero-shot performance. 

More Great AIM Stories

Aparna Iyer
Aparna Iyer has covered various sectors spanning education, wildlife, culture and law for close to a decade. She now writes on technology and is keen to unearth its capability for public good.

AIM Upcoming Events

Regular Passes expire on 3rd Mar

Conference, in-person (Bangalore)
Rising 2023 | Women in Tech Conference
16-17th Mar, 2023

Early Bird Passes expire on 17th Feb

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
27-28th Apr, 2023

Conference, Virtual
Deep Learning DevCon 2023
27 May, 2023

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

A beginner’s guide to image processing using NumPy

Since images can also be considered as made up of arrays, we can use NumPy for performing different image processing tasks as well from scratch. In this article, we will learn about the image processing tasks that can be performed only using NumPy.

RIP Google Stadia: What went wrong?

Google has “deprioritised” the Stadia game streaming platform and wants to offer its Stadia technology to select partners in a new service called “Google Stream”.