For most of us from the Doordarshan generation, the first introduction to the community of People with Disabilities was DD news: The sign language bit in the inset of the morning news had inclusivity written all over it. Then came the cable revolution, and the old news format was traded in for soundbyte-ish, high decibel, anchor-driven shows. The inclusivity took a back seat.
Taking the thought of inclusivity ahead, Priyanjali Gupta, a VIT third-year computer science student, built an AI model that turns sign language into English in real-time and went viral with it on LinkedIn.
Currently, her model can translate 6 ASL signs to English such as “Hello”, “Thank you”, “Please”, “I love you”, “Yes”, and “No”. The 20-year-old Delhite got the idea from her mother, a professor at Delhi University, who had a hearing-impaired student in her class who found it difficult to communicate with her. Priyanjali understood the assignment!
Subscribe to our Newsletter
Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.
In an exclusive interview with Analytics India Magazine, she said that people should not feel intimidated by coding and come out of their zone and build creative solutions.
Excerpts:
AIM: What was the need to make an AI model that turns sign language into English in real-time?
PG: I believe that we don’t have enough inclusivity in tech for the specially-abled people. What if the deaf and the mute want to access voice assistants like Google, Echo or Alexa? We really don’t have any solutions for that. I could also not stop thinking about one of my mother’s students who is hearing impaired and how it would be difficult for her to express her thoughts. I decided to make this small-scale model.
I believe that our education system must consider introducing sign language and braille in the curriculum to make the world more inclusive. The issue is so big that when I posted the video on LinkedIn, from the comments, I realised that there is not one universal sign language. People really said that there should be a solution for the British sign language, Canadian sign language, Indian sign language, among others.
AIM: How did you build it?
PG: This AI-powered model is inspired by data scientist Nicholas Renotte and is built using TensorFlow object detection API. It has many additional capabilities, which makes it a lot easier to build object detection models. I used a small data set of 250 to 300 images, gave the annotations manually, and used a pre-trained SSD model for accuracy. The code is written in Python, and I have run everything in Jupyter Notebook.
AIM: What is the scope of your model?
PG: I believe that the issue of understanding sign language is really big and hard, and a small-scale object detection model cannot solve it. I believe that sign languages include facial expressions, shoulder movements and many more things, and it requires a very well trained deep neural network to understand everything. There is research being done on it, and I believe that we will have a solution, maybe in the next three to five years—not the perfect solution, but certainly the one that can be implemented in real life and has the flexibility to be tuned with advancing technology. With my solution going viral, I believe that it can make experts and professionals think more about the issue and find better solutions to improve inclusivity.
AIM: What has changed in your life after the post went viral on LinkedIn?
PG: I would really thank LinkedIn because it has helped me come out of my zone. I have received a lot of help from people who want to contribute to this project and extend it. The one great thing that I feel happened with my post is that the idea of inclusivity is getting normalised. I have been able to solve some doubts of people who are trying to make similar models. I find it very amazing and feel really happy helping them. If more and more minds keep working on this, we will surely have a solution.
AIM: When did your interest in computer science begin?
PG: I was in class 11 when I decided to choose computer science. I had my uncle inspire me to opt for it and inform me of the many opportunities in the field. I did my first project in class 12 and was thrilled when I saw that some lines of code could help develop so much. With so many libraries and open-source codes to help us through, I started developing my interest in the subject. My uncle also advised me to choose data science as my major. So I started developing an interest in managing, cleaning and processing data. I was completely thrilled when I heard about machine learning and artificial intelligence. With the help of open-source platforms like GitHub, I learned a lot. A major part of my learning also came from the internet, like YouTube videos, Coursera, and Udemy. The documentation is so perfect about all the open-source libraries that you can be guided from scratch. Also, the communities are there to solve your doubts. For example, I got stuck many times and received great help from experienced professionals in the open-source community.
AIM: What do you think is the potential of AI?
PG: AI gives us the power to solve so many issues. I was also in awe of the solution for the visually impaired that automatically tells you what is happening around you just by wearing a camera. I do hear that AI and ML are hyped up, but I think that it’s worth the hype because it can solve many problems. There is so much more to data and what we can do with it.
AIM: What are your future plans?
PG: After my graduation, I’m planning to work for two years to understand how problems are being solved in real life, how things work, how companies work, and how people manage their businesses. Then, I’ll pursue my master’s as I believe I will need that proper and deep knowledge, and then, maybe will have my startup.
AIM: What are your views on the potential of what students can do?
PG: There are so many college students I really respect. They are already an inspiration because of their ideas, passion, and energy. If given proper guidance, I think college students can help the youth of India to achieve so many things.
For learning, there are a lot of resources that are open-source and free. There are many online resources to which we can refer to learn and build projects. With proper guidance, every student can solve any world problem.
AIM: According to you, what makes a good data scientist student?
PG: Other than the technical knowledge, which we are always learning, an ideal student has curiosity, creativity, and a great passion for solving problems and analysing data.
