EPFL Researchers Develop Open Source LLM for Healthcare

EPFL researchers have unveiled Meditron, an open-source and open access large language model (LLM) tailored to the medical field, trained on top of Meta’s Llama 2.

Check out the GitHub repository here.

EPFL's MEDITRON, available in 7B and 70B versions, stands out for its open-source nature. The models were carefully trained on curated medical data, including literature from PubMed and clinical guidelines from diverse sources, as the paper reads.

The model comes with a Llama 2 community licence agreement and Apache 2.0 licence for commercial use. The model is also available on Hugging Face.

Evaluation against medical benchmarks revealed superior performance compared to existing open-source models and closed models like GPT-3.5 and Med-PaLM.

Zeming Chen, lead author and doctoral candidate, highlighted MEDITRON-70B's competitiveness, being within 5% of GPT-4 and 10% of Med-PaLM-2. Professor Martin Jaggi stressed the significance of MEDITRON's transparency, providing the code for training and model weights. The open-source approach allows researchers to enhance the model's reliability and robustness through stress testing.

Professor Mary-Anne Hartley, a medical doctor, emphasised MEDITRON's safety design, encoding medical knowledge from transparent and high-quality sources. Collaboration with the International Committee of the Red Cross integrates their clinical practice guidelines into the model, catering to humanitarian contexts.

Dr. Javier Elkin from the International Committee for the Red Cross expressed excitement about the initiative, noting the rare sensitivity of health tools to humanitarian needs. A workshop in Geneva, funded by the Humanitarian Action Challenge grant, will explore the potential, limitations, and risks of MEDITRON, focusing on its unique features.

Professor Antoine Bosselut, the principal investigator, outlined the goal of MEDITRON—to make access to medical knowledge a universal right. The release aligns with the EPFL AI Center's mission, emphasising responsible and effective AI for societal benefit. The centre fosters multidisciplinary engagement in AI research, education, and innovation, promoting partnerships across various sectors.

While generalist models serve diverse tasks, specialised models, such as those in the medical domain, can be more accessible. Previous attempts at medical LLMs, like Med-PaLM 2 and GPT-4, were either closed source or limited in scale. While there was medAlpaca that was open source, but only for medical question answering.