Bhaiyabot’s LLM Outperforms OpenHathi, Llama 2, and Mistral 7B

Bhaiyabot’s latest AI model, rohansolo/BB-L-01-7B, trained on a corpus of 360k conversational samples in Hindi, English, and Hinglish, has outperformed Sarvam AI’s OpenHathi-7B, Meta Llama-7B, and even Mistral AI’s own fine-tuned model on all benchmarks while retaining the performance of the base model, according to the founder Rohan Shiralkar’s LinkedIn post.

This model is a fine-tuned version of mistralai/Mistral-7B-v0.1 on the HuggingFaceH4/ultrachat_200k and the rohansolo/BB_HindiHinglish datasets. It achieves the following results on the evaluation set:

Shrialkar said that Indian AI is far behind and companies working in AI are too busy marketing non-achievements. This includes activities such as fine-tuning a model and labeling it as a pre-trained model (as seen with Sarvam AI), claiming to be India’s first AI chatbot despite no product launch or release (as with BharatGPT - India's First Gen AI (LLM) in 14 Indian Languages - Text, Voice, Video), or even fabricating facts (as with Krutrim) and more.

Shiralkar even questioned Ola’s recently launched Krutrim. “Ola’s Krutrim claims to have trained a 2 trillion token LLM already. And they’ve been alive for all of 2 weeks. Is that even long enough to train a tiny model on 2 trillion tokens?”he wrote on Linkedin.

Furthermore he said that it's hilarious that the news claims it’s already better than GPT-4. “I want an Indian LLM. I have been crying for one for so long. It’s a strategic imperative.I just want a real one, not fabrications to raise capital.  he added.