Gyan AI Research, a firm focused on developing powerful language models, has introduced PARAMANU-AYN, a new legal natural language processing (NLP) model.

The model is based exclusively on case documents from the Supreme Court of India, the Indian Constitution, and the Indian Penal Code. It is the first generative language model built from scratch specifically for the Indian legal domain, with a context size of 8192.

After pretraining on the legal corpus, the researchers instruction-tuned PARAMANU-AYN on over 10,000 instructions covering legal tasks like drafting contracts, summarizing cases, and answering constitutional questions.

To evaluate the model, the researchers used GPT-3.5-Turbo to assess the quality of PARAMANU-AYN’s responses across metrics like clarity, relevance, completeness, and legal reasoning ability. The model achieved scores around 7-8 out of 10 on these metrics for legal instructions.

PARAMANU-AYN can run efficiently on CPUs, with an inference speed of over 42 tokens per second on an 8-core AMD processor. The researchers found their model could learn the legal domain knowledge required for tasks like contract drafting from a relatively small amount of instruction data, without pretraining on full legal books.

The researchers believe this is the first attempt at an Indian legal language model from scratch. They plan to release PARAMANU-AYN publicly at https://www.bharatgpts.com

The work represents a novel approach to developing specialised legal language models for jurisdictions like India. 

However, the researchers note some limitations, such as potential hallucinations from the instruction data, lack of human evaluation, and absence of guardrails in the current system.