Jivi's Medical LLM Beats OpenAI at the Open Medical LLM Leaderboard

A purpose-built medical Large Language Model (LLM) developed by Jivi, an Indian healthcare AI startup co-founded by former BharatPe Chief Product Officer Ankur Jain and GV Sanjay Reddy, Chairman, Reddy Ventures, has ranked number 1 on the Open Medical LLM Leaderboard. 

Jivi’s LLM, Jivi MedX, has beaten established LLMs, including OpenAI’s GPT-4 and Google’s Med-PaLM 2 with an average score of 91.65 across the leaderboard’s nine benchmark categories.

Hosted by leading AI platform Hugging Face, the University of Edinburgh, and Open Life Science AI, the leaderboard ranks medical-specific LLMs based on their performance in answering medical questions from exams and research.

The evaluation covers medical exams such as Indian medical entrance exams (AIIMS and NEET), US Medical License Exams (USMLE), and detailed assessments in clinical knowledge, medical genetics, and professional medicine, among others.

"Jivi is revolutionizing primary healthcare through generative AI, making top-quality care accessible 24/7 at a fraction of the cost. Our mission at Jivi is to harness artificial intelligence to enhance patient care. Our platform accelerates diagnostics and ensures higher accuracy, enabling timely and precise treatment for all," said Ankur Jain, Co-founder and CEO, JIVI.

Jivi currently operates with a lean, 20-member team of physicians, surgeons, AI engineers, and data scientists. It is developing technology to transform accessibility, affordability, and quality of healthcare globally.

Jivi uses its large proprietary medical dataset consisting of millions of medical research papers, journals, clinical notes, and other sources to train its Jivi MedX LLM. This dataset is among the largest in the world. Jivi MedX was trained using an instruction fine-tuning algorithm called Odds Ratio Preference Optimisation (ORPO).