Allen Institute for AI Unveils AI2 OLMo, An Open Source Language Model

Allen Institute for AI (abbreviated as A12) recently announced AI2 OLMo (Open Language Model), an open, state-of-the-art generative language model, comparable in scale to other state-of-the-art large language models at 70 billion parameters. 

A12 OLMo, expected to be released in early 2024, is a platform that will allow AI researchers to work directly on language models for the first time. The project will provide accessibility to all its elements, including the available data and the code used to generate it. Furthermore, the model, training code, training curves, model weights, and evaluation benchmarks will be available as open-source materials.

“With the scientific community in mind, OLMo will be purpose-built to advance the science of language models,” says Hannaneh Hajishirzi, an OLMo project lead and a Senior Director of NLP Research at AI2. “OLMo will be the first language model specifically designed for scientific understanding and discovery.”

The purpose of this truly open language model is to benefit the research community by providing access and education on all aspects of model creation. AI2 is developing OLMo in collaboration with AMD and CSC, using the new GPU portion of the all-AMD processor-powered LUMI pre-exascale supercomputer, one of the greenest supercomputers in the world. 

Apart from collaborating on hardware and computing resources with AMD and LUMI, A12 is also partnering with organisations such as Surge AI and MosaicML for data and training code. The OLMo model and API is touted to be a powerful new resource for the broader community to participate in the generative AI revolution. Additionally, A12 has opened the gates for more such partnerships from organisations for building responsible and beneficial AI technologies. 

Founded by late Microsoft co-founder Paul Allen, the Seattle-based institute has over 200 AI researchers, engineers, professors, and staff, and benefits from an annual funding of $100 million. They have published over 1000 research papers in areas of artificial intelligence such as natural language processing, computer vision, and common sense reasoning. 

Recently, the institute also raised a fresh $30 million pre-seed fund to invest in “AI-first” startups born from its incubator program. The fund is backed by venture capital heavyweights including Madrona Venture Group, Sequoia Capital, and Two Sigma Ventures, which all previously invested in the first fund.