Between May and July 2020, there have been 30 recorded industrial accidents in India, killing at least 75 workers. These figures, along with regular reports of similar incidents around the country and the world, bring into sharp focus the need to enhance industrial safety on site.
While safety has always been a primary concern for manufacturing and logistics organisations, forward-looking companies have, in recent years, tried to leverage emerging technologies to monitor and improve safety protocols and training methods on site. With the possibility of using machine learning to detect and flag violations, artificial intelligence (AI) can help organisations around the world make strides in reducing the incidence of injuries and fatalities at work.
One crucial aspect of industrial safety is the need for people on the site to wear personal protective equipment (PPE). At a typical construction site or industrial shop floor, a large number of people work in shifts, necessitating round-the-clock safety monitoring to avoid accidents. The current monitoring system involves a combination of direct human supervision and human monitoring of CCTV footage to detect breaches in safety protocols. The use of AI under these circumstances could enable an automated scanning of CCTV footage in real-time, triggering alerts at any instance of a safety protocol breach.
Automation is an attractive proposition, but how do we make it work? The two most pressing demands for an AI safety solution are a high degree of accuracy in detecting every violation and the speed to detect violations in real-time. A typical solution for such a problem uses an object-detection algorithm to detect both people and PPE in the video footage and then map them against safety compliance mandates.
These executional imperatives pose challenges on multiple fronts. For instance, an AI system that needs to monitor PPE compliance among workers on-site can face the following challenges:
- Pre-trained networks for object-detection are configured to identify common objects like animals, automobiles, and humans. No appropriate trained model exists for detecting PPE.
- The size of objects to be detected in the same frame are vastly different. Typically, AI models use Anchor Boxes of comparable sizes for detection. However, the model must be able to detect a person and the industrial safety component (like a hard hat) he is wearing at the same time. This necessitates deploying Anchor Boxes of widely varying sizes, which adds to the training complexity.
- CCTV cameras are usually placed well above ground level to keep it safe from damage. However, this results in construction equipment often obstructing the view of the cameras. Pre-trained networks are effective in detecting people at short distances but fail when used to detect objects from high mast cameras. Moreover, dust, rain and other pollutants compromise the quality and performance of an object detection model.
- An important drawback is that the existing infrastructure and quality of equipment produce video footage in very low resolution, resulting in grainy images that the system finds difficult to read. With the current infrastructure, poor quality imagery makes it infeasible to zoom into parts of the video stream to facilitate better detection.
Despite these challenges, it is possible to build a deep learning solution to detect personnel without PPE at construction sites and shop floors. At Tiger Analytics, we build solutions for such problems by training deep nets to detect two classes – person class and PPE. Training data is prepared by pooling images from various sources like ImageNet, CoCo dataset and publicly available images of personnel with safety gear.
The PPE class, which has comparatively limited images available, can be augmented with graphically recreated 3D geometry files. Open-source tools can then be used to annotate the images and mark the two classes.
An initial solution is built using a basic neural net architecture – VGG19 with Faster-RCNN algorithm that offers training speed. To improve the accuracy of the two classes, new images are continuously added to the training data set. Here we face a trade-off between a more complex deep learning architecture versus accuracy. If the algorithm’s performance has an accuracy of 70% or less, it is considered unsatisfactory. In that case, a more complex approach is likely required. This involves shifting to deeper neural networks and creating an architecture that extracts more features. Usually, this should boost accuracy to 90% or more. In real-life deployment scenarios, we achieved an accuracy of 100% for the person class and more than 98% for the PPE class.
While PPE compliance is widespread and critical, it is not the only aspect of industrial safety that is monitored. In addition to PPE, sites have other protocols, including permissions to enter specific zones, special PPE for different tasks, etc. AI can well be used in these areas as well. The system would need to use available video frames to implement zoning and mark areas that are barred from entry. Personnel entering a barred zone or walking in without PPE can then be marked as a violation of safety procedure. The violations can be flagged by highlighting the person and logging the coordinates. Similar solutions can be evolved for a variety of safety protocol needs.
The advent of AI and machine learning has enabled the creation of new ways to solve traditional business problems that can significantly enhance the quality of work life. Progressive organisations would do well to closely watch these developments to understand how technology can benefit their systems and processes.