Rakuten Releases Suite of RakutenAI-7B Models

Tokyo-based tech giant Rakuten recently released RakutenAI-7B, a suite of LLMs in the Japanese language. This includes base, instruction, and chat models, which have been made freely available to the open-source community.

Check it out on Hugging Face. 

The foundation model, RakutenAI-7B, is a 7 billion-parameter model trained on a vast corpus of English and Japanese text data. The researchers said the model was developed by continually training weights from Mistral AI, an open-source model by a French-based AI startup. 

The researchers also extended the tokeniser vocabulary from 32,000 to 48,000 tokens to handle Japanese characters, which fares better than Mistral 7B-v0.1.

RakutenAI-7B outperformed other open Japanese language foundation models on the Japanese Language Model Evaluation Harness benchmarks, achieving an average score of 62.83.

Further, instruction tuning the foundation model resulted in performance gains. RakutenAI-7B-instruct achieved an average score of 68.74, leading by almost 2 points over Youri-7B-instruction, the second-best model on Hugging Face 

"At Rakuten, we want to leverage the best tools to solve our customers' problems," said Ting Cai, Chief Data Officer of Rakuten Group. With RakutenAI-7B, we have reached an important performance milestone and are excited to share our learnings with the open-source community and accelerate the development of Japanese language LLMs."

In addition to Rakuten's AI models, NEC and Mitsui have made significant strides in AI. NEC has developed a 13-billion-parameter Japanese language model focusing on efficiency and high Japanese language proficiency. Mitsui, collaborating with NVIDIA, launched Tokyo-1, a supercomputer to accelerate drug discovery with AI models. 

Both models are tailored for specific applications, with NEC's model being a general-purpose LLM and Mitsui's Tokyo-1 focusing on the pharmaceutical industry.