The session “Intercepting Youth At Risk Using Computer Vision On Social Media Platforms” was presented at the first-of-its-kind virtual conference on Computer Vision, CVDC 2020 by Dr Murphy Choy, who is the Executive Director at MC EduTech.
The primary aspect of the talk is how computer vision can be used to augment the social worker’s ability to identify youth at risk using Computer Vision combined with the Knowledge Graph.
The session started with discussing the current situation for youth during the COVID-19 pandemic. In the COVID 19 lockdown period, the bulk of the population was forced to stay indoors at proximity for a prolonged period. There were reports about family feuds and violence happening at a much higher frequency compared to the pre-COVID 19 periods.
According to Dr Choy, this is a cause of major concerns to many social workers as younger members of the population generally take to social media to air their frustrations and pain which provides a channel for social workers to reach out. However, social workers can only screen a limited amount of information on social media.
This limited amount of information can be further augmented with the help of Computer Vision and Knowledge Graph techniques.
Dr Choy discussed four significant challenges faced during the COVID-19 pandemic. They are-
- During the lockdown period, no youth could escape from the source of violence.
- The overarching environment of COVID-19 resulted in an influx of information and drowning of the youth’s call for help.
- The lockdown restrictions forced the youths at risk to continually face the tormentor, which created a negative environment.
- The proximity and inability to leave the common space resulted in a lack of avenue to seek help when needed openly.
According to the speaker, during the lockdown period, youths had minimal sources to seek help. One of the foremost ways is posting through online social media platforms. These posts consist of several valuable information which can help social workers to identify who is going through depression or who is suffering from violence.
Dr Choy suggested that to be able to identify the emotions behind these posts; one must develop a good Computer Vision Algorithm. He explained in detail how he and his team built a Computer Vision model that can detect people in distress. The CV algorithm includes three major pieces of information like scenic, colours and text.
Choy explained the tricks of tackling scenics. He said, “Tackling scenic is perhaps easier than what most people would assume. One should never attempt to use brute force techniques to solve a problem with infinite possibilities.” He added, “The trick here is to understand the context of scenic and how it relates to the state of mind for an individual”
He mentioned a research paper named “Understanding and Mapping Natural Beauty” as a reference, which focuses on what constitutes beauty and its impact on the overall health. According to him, the results of this research provides the basis for subsequent work on youth-at-risk detection.
The speaker elaborated the methods of scenics that are explained in the above-mentioned paper and explained various other related topics, such as the meaning of the term “scenicness”, what categories one should look out for, distributions of the colours and frequencies in a scenic image and other such.
Dr Choy explained how one can build a CNN model specifically for scenic images. He said that colours play a crucial role in scenic images as well as a guide to understand the emotions of humans. Brighter colours such as red and yellow are represented as powerful, bold and creative. While colours like black, grey and other dull colours represent disdainful, unconcerned, depressed, lonely, etc.
Besides this, the speaker also shed light on the Computer Vision algorithm, which he built to detect text in images through Optical Character Recognition (OCR). To construct this algorithm, Dr Choy and his team extracted texts from images using OCR. They used the PyTesseract library, cognitive AI and emotion analysis.
According to the speaker, the problem with the OCR is the accuracy of the words identified. Sometimes, the words get jumbled up and require additional correction. Due to this problem, they further built an auto-correction and missing word prediction engine.
Given the probability involved in the auto-correction and missing word, the sentiment of the sentence is a multiple imputation problem that created a range of confidence level. To be able to understand the secondary meaning, the team built a CNN model that identifies the closest group of words related to the words in the text.
After building all the models, they developed an additional CNN model that uses the outputs from the scenic model, colour model and text models to determine whether the social media post indicated a person in distress. The accuracy of the model is claimed to be over 86 per cent in detecting at-risk situations.
