Meet The New Marathi RoBERTa

The duo unveiled the model at Hugging Face’s community week.

Shraddha Goled

Published 2021-07-19T05:30:00.000+05:30Updated 2021-07-19T05:30:00.000+05:30

Key takeaways

The new Marathi RoBERTa model enhances natural language processing for the Marathi language.
It was unveiled by researchers Nipun Sadvilkar and Haswanth Aekula at Hugging Face’s community week.
The model is suitable for tasks involving whole sentences, like sequence and token classification.
The training dataset, sourced from Marathi newspapers, may contain biases that affect the model's performance.

5 min

FollowPreferred Source Google News WhatsApp Telegram

In July 2019, the Facebook research team introduced the Robustly Optimized BERT Pretraining Approach (RoBERTa)--an improvement over the Bidirectional Encoder Representations from Transformers (BERT), a self-supervised method for NLP tasks released by Facebook in 2018.

Two researchers, Nipun Sadvilkar and Haswanth Aekula, have now pretrained the RoBERTa model on Marathi language using a masked language modelling (MLM) objective in a self-supervised manner. The duo unveiled the model at Hugging Face’s community week.

https://twitter.com/NipunSadvilkar/status/1416464257890152448?s=20

The model is primarily aimed at tasks that use the whole sentences (potentially masked) to make decisions, such as sequence classification, token classification or question-answer. The duo used this model to fine-tune text classification tasks like iNLTK and indicNLP. Since the Marathi mc4 dataset is made up of text from Marathi newspapers, it might involve biases that can affect all fine-tuned versions of the model, the team warned.