A New AI Tool To Detect & Remove Caste-Based Abuse From Social Media Platforms

A New AI Tool To Detect & Remove Caste-Based Abuse From Social Media Platforms

In the wake of increasing cyberbullying to fake news, Social Media Matters has partnered with Spectrum Labs to launch a Behaviour Identification Model in order to detect caste discrimination within online communities. 

According to the company, the social media platforms of users usually contain a spectrum of information ranging from personal to mundane to sharing political opinions and building communities. And that’s why the online communities have become a fertile ground for groups based on ethnicity or castes. Social Media Matters, that’s why designed this model — The Behaviour Identification model in order to detect the same.

The company stated in their release that the designed model had been made available through Spectrum’s behaviour identification solution. The model has been designed to be flexible and fit into any workflow. Spectrum’s deployment options include a SaaS offering or an on-premise binary. 

AIM Daily XO

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.

According to the company, the model is currently trained to detect caste discrimination within all forms of text data, including users’ status updates, messages, tweets, comments, etc. Both deployments have a streaming API or batch mode for data processing.

Spectrum’s behaviour models are all designed to be updated continuously over time. The company is currently working with customers to iterate on the baseline model on a regular cadence in order to ensure that the content is flagged that their customers need. Such a process ensures that the results are customised for each customer in order for them to build trust in it.

Download our Mobile App

Some of the critical features of Behaviour Identification Model in order to detection caste discrimination includes real-time recognition and response to toxicity before it evolves into an even bigger problem; multi-languages detection, where the tool can scale across regions; and secure deployment of the device, which offers the power to understand the community while maintaining the data privacy requirements.

The company stated the results produced by the model are highlighted for customers in a way that they can plug into existing moderation efforts. This includes webhooks into internal systems for customers to manage users, manage content, send alerts, send for moderator review, and even pipe into analytics platforms to see trends over time. 

According to Amitabh Kumar, the founder of Social Media Matters, caste discrimination is one of the oldest forms of evils still existing in Indian society; and sadly, it is also reflected in the cyberspaces. And that’s why Social Media Matters together with Spectrum has created an artificial intelligence tool to help social media platforms, like Facebook, TikTok, Twitter, Instagram, to detect and remove caste-based abuse from their platforms. 

“It will decrease the time taken for detection, and also decrease the constant stress that human moderators have to go through constantly dealing with abuse. Initially, the model is trained to work with several languages English, Hindi, and Hindi-English mix and we’ll continue to upgrade it further,” said Kumar.

Spectrum is also offering a set of moderator tools through a UI called Guardian, which includes four main features — moderation queue, automation builder, retraining, and analytics. And with this tool, the produced results can either be put into this Spectrum offering or can be plugged into existing moderation efforts.

Speaking on the collaboration Justin Davis, CEO at Spectrum Labs said that the hardest part of building an AI model to detect caste discrimination online effectively is to define and understand what caste discrimination is, and also what it is not. 

“The team of Social Media Matters have taken bold strides in raising awareness about injustice and discrimination in many forms, so we could not have asked for better partners. Spectrum’s insights and expertise helped in navigate the nuances, history, and politics of caste discrimination, to build a tool that can combat it effectively and inclusively. We were humbled and honoured to work with them,” concluded Davis.

Sign up for The Deep Learning Podcast

by Vijayalakshmi Anandan

The Deep Learning Curve is a technology-based podcast hosted by Vijayalakshmi Anandan - Video Presenter and Podcaster at Analytics India Magazine. This podcast is the narrator's journey of curiosity and discovery in the world of technology.

Sejuti Das
Sejuti currently works as Associate Editor at Analytics India Magazine (AIM). Reach out at sejuti.das@analyticsindiamag.com

Our Upcoming Events

24th Mar, 2023 | Webinar
Women-in-Tech: Are you ready for the Techade

27-28th Apr, 2023 I Bangalore
Data Engineering Summit (DES) 2023

23 Jun, 2023 | Bangalore
MachineCon India 2023 [AI100 Awards]

21 Jul, 2023 | New York
MachineCon USA 2023 [AI100 Awards]

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox

Council Post: Evolution of Data Science: Skillset, Toolset, and Mindset

In my opinion, there will be considerable disorder and disarray in the near future concerning the emerging fields of data and analytics. The proliferation of platforms such as ChatGPT or Bard has generated a lot of buzz. While some users are enthusiastic about the potential benefits of generative AI and its extensive use in business and daily life, others have raised concerns regarding the accuracy, ethics, and related issues.