International Journal of Scientific Research in Computer Science, Engineering

and Information Technology

ISSN : 2456-3307 Available Online at : www.ijsrcseit.com
doi : https://doi.org/10.32628/CSEIT2490216

Safety Measure Detection Using Deep Learning

Tejas Bagthaliya*, Vaidehi Shah, Shubham Shelke, Devang Shukla, and Yatin Shukla
Department of Computer Science and Engineering, PIET, Vadodara, Gujarat, India

ARTICLEINFO ABSTRACT

This implementation is for a computer vision application that detects individuals

Article History:
and verifies their compliance with safety gear regulations, such as safety jackets
Accepted: 05 March 2024 and hard-hats. The system counts the number of individuals violating safety
Published: 15 March 2024 standards and keeps track of the total number of individuals detected. The system
uses advanced image processing techniques, including object detection and
classification, to accurately identify the presence or absence of safety gear. The
Publication Issue user interface provides real-time analysis of the data, with the option to alert the
Volume 10, Issue 2 user of any violations. This implementation is a valuable tool for organizations
March-April-2024 looking to ensure the safety of their employees and customers, providing a
comprehensive solution for monitoring compliance with safety regulations. It
Page Number can also be used to analyze trends and identify areas for improvement, making it
148-155 an essential tool for safety professionals and facilities managers.

Keywords: Deep Learning, Convolutional Neural Network (CNN), Safety

Measures , Object Detection, Image Classification

I. INTRODUCTION machine-to-machine communication. In addition,

research is being done on how to employ
The industry has a high accident rate and can be an contemporary technologies to boost safety. One
extremely dangerous place to work. The high accident example of how technology may be applied to properly
rate is a result of the industries use of hazardous train personnel and lower hazards is the use of Virtual
machinery and heavy weights. In the sector, safety Reality (VR) for training and education.
concerns are treated seriously and have always been a
top focus. Nonetheless, given the frequency of
incidents, there is always a necessity for development.
New techniques are utilized to improve safety in
manufacturing sectors as technology advances. One of
the main goals of industry is to automate and simplify
repetitive tasks by utilizing artificial intelligence and

Copyright © 2024 The Author(s): This is an open-access article distributed under the terms of the Creative 148
Commons Attribution 4.0 International License (CC BY-NC 4.0) which permits unrestricted use, distribution, and
reproduction in any medium for non-commercial use provided the original author and source are credited.
 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

In [2] discussed the integration of Realtime Locating

Systems (RTLS) in enhancing safety measures at
construction sites, offering valuable insights into RTLS
technology's impact on safety protocols. The review
emphasized the importance of real-time location data
in improving worker safety and accident prevention on
construction sites.
In [3] proposed a real-time monitoring system
Figure 1. Image Recognition designed to control interference in large-scale
construction projects, contributing to improved
It has been demonstrated that proactive measures
operational efficiency and safety. The system addresses
are very important for accident mitigation and accident
critical challenges in construction site management,
prevention. Examples of preventative measures
such as real-time interference control, leading to
include safety training for employees to learn and
smoother project execution and reduced safety risks.
practice the workplace’s safety principles and standard
In [4] investigated the effectiveness of data
emergency procedure. The use of personal protective
augmentation techniques in deep learning for image
equipment (PPE), such as gloves, hard hats, safety
classification tasks, evaluating various augmentation
glasses, and vests, is another form of proactive action.
methods' impact on model performance. The study
Researchers have investigated automated methods of
provided valuable guidance on selecting appropriate
observing and picking up harmful conditions, like the
data augmentation strategies to enhance the robustness
absence of personal protective equipment (PPE) in
and generalization of deep learning models.
manufacturing and on building sites. Due to significant
In [5] explored innovative data augmentation
developments in the computer science and machine
methods to improve the accuracy of image
learning. workers’ perceptions of such technology
classification using convolutional neural networks
might change because of their awareness of it. The
(CNNs), contributing to advancements in image
level of privacy in monitoring has also been suggested
recognition technologies. The research focused on
to be increased in many ways, for as by blurring areas
enhancing CNN performance through effective data
like faces that can identify a person.
augmentation, resulting in more reliable and accurate
To investigate the potential for increasing the use of
image classification results.
safety devices through picture recognition to improve
In [6] introduced a real-time monitoring system for
worker safety. This is accomplished by offering a
personal protective equipment (PPE), ensuring worker
platform for the development of a system that employs
safety by continuously monitoring PPE usage and
image recognition to find safety gear.
compliance. This system plays a crucial role in
preventing workplace accidents and ensuring
II. LITERATURE SURVEY
regulatory compliance in hazardous work
environments.
In [1] introduced deep residual learning techniques
In [7] discussed the application of deep learning
for image recognition tasks, significantly advancing the
techniques for real-time detection of PPE at
field of computer vision and pattern recognition. This
construction sites, contributing to the development of
work has had a substantial impact on improving the
automated safety systems. The research improved
accuracy and efficiency of image classification models,
hazard detection and risk prevention on construction
especially in complex recognition tasks.
sites, enhancing overall safety protocols.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 149

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

In [8] delved into the application of machine In [14] discussed the integration of Realtime
learning in predictive maintenance for smart Locating Systems (RTLS) in enhancing safety measures
manufacturing, highlighting machine learning at construction sites, offering valuable insights into
algorithms' role in optimizing maintenance processes. RTLS technology's impact on safety protocols. The
The study emphasized the importance of predictive review emphasized the importance of real-time
maintenance in ensuring equipment reliability and location data in improving worker safety and accident
reducing downtime in smart manufacturing prevention on construction sites.
environments. In [15] proposed a real-time monitoring system
In [9] proposed a comprehensive safety management designed to control interference in large-scale
system for construction sites based on computer vision construction projects, contributing to improved
and deep learning technologies, automating hazard operational efficiency and safety. The system addresses
detection and monitoring to enhance safety protocols. critical challenges in construction site management,
The system integrated advanced technologies to such as real-time interference control, leading to
improve safety measures and reduce the likelihood of smoother project execution and reduced safety risks.
workplace accidents. These papers collectively contribute to advancements
In [10] demonstrated the effectiveness of the in safety management, deep learning, computer vision,
Haarcascade algorithm in detecting safety equipment and predictive maintenance, addressing critical
across diverse work environments, contributing to challenges and improving safety measures in various
improved safety management systems. The research industrial and construction environments.
enhanced safety protocols by accurately identifying
safety equipment usage and compliance in various III.PROJECT FLOW AND METHODOLOGY
working environments.
In [11] presented a method for detecting safety gear
A. Image Enhancement
using advanced deep learning techniques, promoting
Image enhancement involves adjusting digital images
safer working conditions. Their approach leveraged
to make them more suitable for display or further
deep learning algorithms to improve safety gear
analysis. This can include tasks such as noise reduction,
detection accuracy, ensuring workers' adherence to
sharpening, or brightening an image to improve the
safety protocols and regulations.
visibility of important features.
In [12] presented enhancements for detecting
industrial safety gear using a re-ID conditioned B. Feature Extraction
detector, contributing to the development of more Feature extraction methods identify distinct features in
accurate safety gear detection systems. The research images, such as edges and corners, which can then be
focused on improving the efficiency and reliability of used to distinguish or match images with similar
safety gear detection, enhancing overall workplace features. An algorithm was developed in this study to
safety. extract color features from samples. Typically, color,
In [13] introduced deep residual learning techniques shape, and texture features are extracted from images
for image recognition tasks, significantly advancing the to aid in image recognition.
field of computer vision and pattern recognition. This
work has had a substantial impact on improving the C. Image acquisition

accuracy and efficiency of image classification models, Images were acquired through two methods: the first
especially in complex recognition tasks. system utilized a scanner (hp 1370) and a Pentium IV
Computer unit, while the second system used a digital

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 150

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

video camera. Key point descriptors were generated for

both training and test images. These key points serve
as reference points to identify objects in the training
image, which may also exist in the test image alongside
other objects. The matches were evaluated based on
the Euclidean distance between matched points in the
training and test images.

Figure 3. ER Diagram

D. Training application
The training application begins by receiving a video
stream from a connected camera. Frames are extracted
from this stream to capture images at an adjustable rate.
The application continues this process until it reaches
a default target of 30 images, uploading these images to
the designated Custom Vision resource. During this
image capture phase, the camera can be exposed to the
specific
objects you want the model to recognize. Once the
images are uploaded, they are available within the
Custom Vision resource. Alternatively, images can be
directly uploaded to Custom Vision from a local folder
via the Custom Vision web portal.

E. Training the model in Custom Vision

Figure 2. Workflow In Custom Vision, models are constructed using

training images. When creating a model, specific
objects you want the model to detect are determined.
This is accomplished by defining class labels. For
instance, when building a model to detect a person
with and without a reflective vest, class labels
like ”PersonWithVest” and ”PersonWithoutVest” are
defined. The most effective approach involves
simultaneously detecting both the person and the vest,
as it has been demonstrated to yield the best
performance.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 151

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

IV.MATERIAL METHODOLOGY the camera are pre-processed and fed into the model
for safety gear detection. The output of the model can
A safety gear detection system using deep learning be displayed on a monitor or integrated with other
methodology typically follows the following steps: safety equipment for real-time alerts.

A. Data Collection:
V. RESULTS
The first step is to collect a large dataset of images that
contain workers wearing safety gear, such as hard hats
The toolkit for Intel® Distribution of OpenVINOTM
and safety vests. The dataset should include images
contains an inference engine that is used by the
captured in different lighting conditions, angles, and
application. First, a trained neural network finds
orientations.
persons in the frame and shows a bounding box over
B. Data Pre-processing: them in a green colour. The programme checks
The collected data is pre-processed by resizing the everyone it finds to see if they are donning a hard
images, removing irrelevant features, and normalizing helmet and safety gear. If they are not, the system
the pixel values. The pre-processed data is then divided registers an alarm.
into training, validation, and test sets.

C. Model Development:
The next step is to develop a deep learning model using
a suitable framework, such as TensorFlow or PyTorch.
The model can be a convolutional neural network
(CNN) or a hybrid model that combines CNN with Figure 4. Working System
other deep learning architectures. The model is trained
To prepare the environment for utilizing the Intel®
on the training set using backpropagation and gradient
Distribution of OpenVINO™ toolkit, configureit by
descent algorithms.
exporting the necessary environment variables:
D. Model Evaluation: command: source /opt/intel/openvino/bin/setupvars.sh
Once the model is trained, it is evaluated on the
validation set to assess its performance. The evaluation
metrics used can include accuracy, precision, recall,
and F1 score.

E. Model Optimization:
Figure 5. Environment Setup
The model is optimized by fine-tuning the
hyperparameters, such as learning rate, number of This command is essential for setting up the required
layers, environment variables. The system can access crucial
and batch size. The optimized model is then tested on OpenVINOTM libraries, tools, and dependencies by
the test set to evaluate its generalization performance. sourcing this script. In order to enable for easy
interaction with OpenVINOTM components, it makes
F. Deployment:
sure that the system environment is configured
Once the model is trained and optimized, it can be correctly.
deployed in the field using suitable hardware and Begin by changing the current directory to the specific
software components. The input images captured by location of the code on your system.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 152

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

Figure 6. Run Code

Our application’s interoperability with a variety of Figure 8. Input Stream

input sources contributes to its versatility. The system The system’s versatility is further seen in Figures 3.6
adjusts fluidly to various input sources, whether using and 3.7. Figure 3.6 shows how the programme analyses
pre-recorded video footage or providing real-time video material that has been uploaded, showcasing its
monitoring through a linked camera. Users’ freedom to capacity to analyse previously recorded information
switch between several video sources improves the and provide precise violation counts and person
application’s capacity to adapt to various surroundings. tracking. The system’s smooth transition to real-time
Additionally, the device has the ability to dynamically monitoring is shown in Figure 3.7. Our programme
activate the camera, enabling real-time runtime utilises the potential of instant data capture by
analysis. Real-time monitoring is ensured by this connecting to a live camera stream. The technology
feature, enabling prompt answers to safety compliance can identify safety infractions in real time thanks to
problems. Our programme is able to respond to a this film, ensuring prompt reactions and improving
variety of settings thanks to its versatility, making it a overall safety procedures. The application’s
reliable option for safety gear identification in dynamic adaptability is demonstrated by these numbers, which
contexts. make it a dependable option for both static and
dynamic contexts.

VI.CONCLUSION & FUTURE SCOPE

The implementation of safety gear detection systems

using deep learning technologies has paved the way for
numerous advancements in various industries. As we
move forward, there are several promising future
scopes and potential applications for these systems,
Figure 7. Video Detection
driven by ongoing advancements in artificial
The bottom left corner of the screen in the visual intelligence and machine learning. Here are some key
depiction shown in Figure 3.6 prominently displays areas where the future of safety gear detection systems
important metrics. These metrics include” Total using deep learning appears particularly promising
Violation Count,” which shows instances of safety Safety gear detection system using deep learning has
equipment non-compliance,” Current People Count,” great potential in various industries and can lead to
which shows the number of people currently visible in numerous future scopes. Some of the possible future
the frame, and” Total People Count,” which shows the scopes are:
total number of people seen. These metrics offer
A. Industrial safety
insightful information briefly and act as real-time
indicators of safety compliance and human presence.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 153

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

The safety gear detection system using deep learning The safety gear detection system can be
can be used in industries to detect the usage of safety implemented in firefighting settings to detect the usage
gear such as helmets, gloves, safety shoes, goggles, etc. of safety gear such as helmets, boots, gloves, and
This can prevent accidents and ensure the safety of the breathing apparatus. This can prevent injuries and
workers. ensure the safety of the firefighters.

B. Construction safety
VII. REFERENCES
Construction sites are considered high-risk areas,
and the safety of the workers is of utmost importance.
[1] K. He, X. Zhang, S. Ren and J. Sun, ”Deep
The safety gear detection system can be implemented
Residual Learning for Image Recognition,” 2016
in construction sites to detect the usage of safety gear
IEEE Conference on Computer Vision and
such as helmets, safety harnesses, safety shoes, etc.
Pattern Recognition (CVPR), Las Vegas, NV,
C. Healthcare safety USA, 2016, pp. 770-778, doi :
The safety gear detection system can be used in 10.1109/CVPR.2016.90.
healthcare settings to detect the usage of personal [2] Soltanmohammadlou, N., Sadeghi, S., Hon, C.K.,
protective equipment (PPE) such as masks, gloves, Mokhtarpour-Khanghah, F. (2019). Realtime
goggles, and gowns. This can prevent the spread of locating systems and safety in construction sites:
infectious diseases and ensure the safety of healthcare A literature review. Safety Science.
workers. [3] Naticchia , Berardo et al. “A monitoring system
for real-time interference control on large
D. Sports safety construction sites.” Automation in Construction
The safety gear detection system can be 29 (2013):148-160.
implemented in sports settings to detect the usage of [4] Perez, L., Wang, J. (2017). The Effectiveness of
safety gear such as helmets, pads, and mouthguards. Data Augmentation in Image Classification using
This can prevent injuries and ensure the safety of Deep Learning. ArXiv, abs/1712.04621.
athletes. [5] Shijie, Jia Ping, Wang Peiyi, Jia Siping, Hu.
(2017). Research on data augmentation for image
E. Transportation safety
classification based on convolution neural
The safety gear detection system can be used in
networks. 4165-4170.
transportation settings such as aviation, rail, and
10.1109/CAC.2017.8243510.
maritime industries to detect the usage of safety gear
[6] Barro-Torres, Santiago Fern´andez-Caram´es,
such as helmets, life jackets, and harnesses. This can
Tiago P´erez-Iglesias, H´ector Escudero, Carlos.
prevent accidents and ensure the safety of the
(2012). Real-Time Personal Protective
passengers and crew.
Equipment Monitoring System. Computer
F. Military safety Communications. 36. 42-50.
The safety gear detection system can be used in 10.1016/j.comcom.2012.01.005.
military settings to detect the usage of protective gear [7] Nath, Nipun Behzadan, Amir Paal, Stephanie.
such as helmets, body Armor, and gas masks. This can (2020). Deep learning for site safety: Realtime
ensure the safety of the soldiers during combat. detection of personal protective equipment.
Automation in Construction. 112. 103085.
G. Firefighter safety
10.1016/j.autcon.2020.103085.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 154

 Tejas Bagthaliya et al Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., March-April-2024, 10 (2) : 148-155

[8] C¸ ınar, Z.M.; Abdussalam Nuhu, A.; Zeeshan,

Q.; Korhan, O.; Asmael, M.; Safaei, B. Machine
Learning in Predictive Maintenance towards
Sustainable Smart Manufacturing in Industry
4.0. Sustainability 2020, 12,8211. 32.
[9] Lee J, Lee S. Construction Site Safety
Management: A Computer Vision and Deep
Learning Approach. Sensors (Basel). 2023 Jan
13;23(2):944. doi: 10.3390/s23020944. PMID:
36679738; PMCID: PMC9863726.
[10] Le, Billy Phuc Jeon, HyeJun Truong, Nguyen
Hak, Jung. (2019). Applying the Haarcascade
Algorithm for Detecting Safety Equipment in
Safety Management Systems for Multiple
Working Environments. Electronics. 8. 1079.
10.3390/electronics8101079.
[11] Cinar, Zeki Nuhu, Abubakar Zeeshan, Qasim
Korhan, Orhan Asmael, Mohammed Safaei,
Babak. (2020). Machine Learning in Predictive
Maintenance towards Sustainable
SmartManufacturing in Industry 4.0.
Sustainability. 12. 8211. 10.3390/su12198211.
[12] Lee J, Lee S. Construction Site Safety
Management: A Computer Vision and Deep
Learning Approach. Sensors (Basel). 2023 Jan
13;23(2):944. doi: 10.3390/s23020944. PMID:
36679738; PMCID:PMC9863726.
[13] X. Wang, D. Niu, P. Luo, C. Zhu, L. Ding and K.
Huang, ”A Safety Helmet and Protective
Clothing Detection Method based on Improved-
Yolo V 3,” 2020 Chinese Automation Congress
(CAC), Shanghai China, 2020, pp. 5437-5441,
doi:10.1109/CAC51589.2020.9327187.
[14] Ravikiran, Manikandan. (2019). Improving
Industrial Safety Gear Detection through Re-ID
conditioned Detector.
[15] le, Billy Phuc Jeon, HyeJun Truong, Nguyen
Hak, Jung. (2019). Applying the Haar-cascade
Algorithm for Detecting Safety Equipment in
Safety Management Systems for Multiple
Working Environments. Electronics. 8. 1079.
10.3390/electronics8101079.

Volume 10, Issue 2, March-April-2024 | http://ijsrcseit.com 155