273 Ventures Introduces KL3M, a Family of Legal LLMs for Enterprises

Illinois-based 273 Ventures-owned Kelvin Legal Data OS has released KL3M, a family of LLMs designed for legal purposes. It is trained from scratch on legally permissible data for enterprise applications. KL3M1 is the largest legal model trained on over two trillion tokens of proprietary clean data, focusing on legal, financial, and general domains. 

This dataset, the Kelvin Legal DataPack, is commercially available and scored and filtered for continuous improvement using a custom pipeline.

KL3M targets enterprise use in legal, regulatory, and financial workflows. Based on the performance metrics, including perplexity, toxicity, and human preference, KL3M demonstrates superiority over peer models. The initial models, kl3m-170m and kl3m-1.7b, outperform peers in perplexity and toxicity, with applications for tasks such as answering regulatory questions, drafting contracts, and extracting structured information, surpassing other models in detail and stylistic realism. Larger Mixture-of-Experts (MoE) models are also in development for release in the late first quarter. 

The models’ training data is ethically collected, avoiding fair use interpretations or contract breaches. The release aligns with the success of small language models (SLMs), prompting an accelerated roadmap. The models are designed to run efficiently on consumer-grade hardware like a MacBook Air or a $300 NVIDIA GPU.

KL3M's availability is tied to the Kelvin Legal Data OS, and interested parties can sign up for notifications. The creators seek collaboration for fine-tuning KL3M for other domains and testing it against existing SLM workflows. KL3M's training involves a high-quality, curated English subset of the Kelvin Legal DataPack, and the models are based on the GPT-3 architecture with specific modifications.