See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/360732255

Fatigue detection using artificial intelligence

Conference Paper in AIP Conference Proceedings · May 2022

DOI: 10.1063/5.0074121

CITATIONS READS
4 410

5 authors, including:

Dr.Pravin R. Kshirsagar Pankaj Dadheech

J D College of Engineering Swami Keshvanand Institute of Technology Management and Gramothan (SKIT)
185 PUBLICATIONS 1,584 CITATIONS 168 PUBLICATIONS 937 CITATIONS

SEE PROFILE SEE PROFILE

Yuvaraj Thangaraj Makarand Upadhyaya

Chennai Institute Of Technology University of Bahrain
83 PUBLICATIONS 1,459 CITATIONS 70 PUBLICATIONS 519 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Yuvaraj Thangaraj on 23 May 2022.

The user has requested enhancement of the downloaded file.

Fatigue detection using artificial intelligence
Cite as: AIP Conference Proceedings 2393, 020080 (2022); https://doi.org/10.1063/5.0074121
Published Online: 19 May 2022

Pravin R. Kshirsagar, Pankaj Dadheech, T. Yuvaraj, et al.

AIP Conference Proceedings 2393, 020080 (2022); https://doi.org/10.1063/5.0074121 2393, 020080

© 2022 Author(s).
 Fatigue Detection Using Artificial Intelligence
Pravin R. kshirsagar1, a), Pankaj Dadheech2, T.Yuvaraj3, C. A Sathiya Moorthy4,
Makarand Upadhyaya5
1
Department of Electronics and Communication Engineering, AVNIET, Hyderabad, India,
2
Computer science & Engineering, Swami Keshvanand institute of technology, Jaipur, Rajasthan, India
3
Department of Electrical and Electronics Engineering, Saveetha School of Engineering, Chennai, India
4
Department of Electronics and Communication Engineering, CMS College of engineering, India.
5
Department of management & marketing, University of Bahrain, Bahrain
a)
Corresponding author: [email protected]
Abstract.Over the years the development of computer technology has benefited drivers, often in the form of smart
systems.In a significant number of car collisions, tiredness is a major factor.Road collisions are the world's most frequent
types of injuries and fatalities, and the main causes are typically drunken, sleepy, and rudely behavioral. The main
purpose of this article is to define common sources of knowledge for the detection of drowsiness to evaluate when a
specified degree of drowsiness is achieved.Fatigue was one of the key techniques for detecting or tracking augmented
reality.In particular, driver drowsiness identification can protect riders from drowsy riding collisions.This technology
relies on face detection to warn the driver of somnolence or to avoid accidents in traffic.
Keywords:Driver Drowsiness, Artificial Intelligence, Image Processing, Eye Detection, Face detection

INTRODUCTION
The symbol of human intellectual processes, including learning, thinking, and self-correction, is artificial
intelligence (AI) [14][15][16][17]. It enables the thinking and learning of computer programs or computers.
Computers will fulfill tasks like visual insight, strategic thinking, adaptation, sensory awareness, and communication
through the advancement of growth and technology [1]. Artificial intelligence has numerous fields that are essential
and advantageous for humanity such as language comprehension, learning languages and optimum efficiency,
problem resolution, sensory perceptions, robots, technology, design, etc. In a significant number of car collisions,
sleepiness is a major factor. Due to driver-related incidents like distraction, exhaustion, and sleeplessness, deaths
have occurred. Self-standard devices for the study of tiredness and driver drowsiness may be an important part of the
development of the smart car to avoid sleep accidents. Various strategies for weakness and exhaustion identification
were employed by vehicle drivers [2].Sensors are mounted on typical vehicle parts, e.g., electric motor, gear shifter,
and evaluated for the identification of drowsiness signals from these sensors.It is essential to adjust these approaches
to the driver because As well about and his subordinates notice that the drivers have noticeable variations in their
use of the gas pedal [3].Physiologic parameters such as heart speeds, pulse rates and electroencephalography are
based in the second technical category.The alertness levels of the EEG power of the alpha and theta bands have
decreased.It therefore provides indices of somnolence.However, this type has practical drawbacks since a person is
obliged to wear an EEG signal cap when driving.The third range of strategies concentrates on computer systems for
visualization that can identify and detect the changes in face and presence during somnolence.The study of blinks
and hand gestures has based much of the scientific literature on machine learning approaches to fatigue
measurement [9].

LITERATURE SURVEY
In this paper, an accurate assessment of the origin, variables, and features that affected fatigue recognition was
made available using artificial neural networking, the transformation of the wavelets, data analytics for keyboard
and mouse trends, image processing, kernel machine learning, tiredness, and anxiety connections, and cardiac data
inspection studies [1]This paper provides an algorithm for the identification of driver somnolence with
representational education.The leveraging of multi-layer fully convolutional neural networks offer a new perspective
for driver sleep detection, as the decisional features [2]This paper showed a method to detect driver sleepiness

Recent Trends in Science and Engineering

AIP Conf. Proc. 2393, 020080-1–020080-5; https://doi.org/10.1063/5.0074121
Published by AIP Publishing. 978-0-7354-4198-9/$30.00

020080-1
dynamically from the video.A method has been used to assess spontaneous actions during real somnolence episodes
to automatically measure emotional expressions [3].This study explains the separation of a lightweight model from a
heavy baseline model.Also, the minimum device structure is developed to examine whether the operator is drowsy
or not based on the main point of facial identification [4].This paper examines in depth the different techniques used
to recognize driver fatigue, thus increasing the driver's alertness and warning him of fatigue [5].This paper gives an
idea to recognize somnolence utilizing machine learning techniques to alert the driver to a collision in real-time.In
combination with OpenCV, the model uses Haar Cascade's algorithm to track the driver's video on the spot and
detect the driver's eyes [6].This study illustrates the main techniques for the identification of driver somnolence by
examining face photos taken by a camera mounted on the dashboard [7].This paper proposes a compact system that
provides tracking in actual environments.To provide greater performance than older iterations, the method utilizes
conventional and modern, and advanced approaches such as computer training [8].This paper focuses on the
identification of the iris of the eye for protection.The goal of this study is to use stationary wavelet transformation to
improve the image processing algorithm to get the normalized cropping images that would be suitable for processing
and detecting inputs [9]

MULTILAYER NEURAL NETWORK MODEL

Designers have used Neural Network Classifier16 for data acquisition that can also be called MLP.MLP is an
unpretentious neural network consisting of interlinked points (neurons) that map out the output class input
[9][10][11][12[[13]. The artificial neuron selects one or more inputs, applies the accepted input based on the weight,
and generates an output class afterward.The multi-layered neural network model is shown in Figure 1.

FIGURE 1.Multilayered neural network model

In other words, the corresponding variable fu(x) is given to one-hidden layer MLP
݂௨ (‫ ܣ = ) ݔ‬ቀܾଶ + ‫ݓ‬ଶ ൫‫ܾ(ݏ‬ଵ + ‫ݓ‬ଵ.௫ )൯ቁ 1
"W2" and "W1" is the matrix weight and "A" is represented by the kernel function, where "b2" and "b1" is the
bias objects.Moreover, the hidden state of the h variable is defined as
݄(‫ܾ(ݏ = ) ݔ‬ଵ + ‫ݓ‬ଵ.௫ ) 2

Multilayer Perceptrons are capable of learning through preparation.During this method iteration are used to
ensure the minimum number of potential errors before the necessary input-output mapping has been achieved; a
collection of training data, including certain input and associated output vectors, is required here.We learn all model
parameters to train an MLP. Let theta=W2 ,b2 ,W1,B1 is the set of parameters for learning.

DROWSINESS DETECTION SYSTEM

Drowsiness is alertness or diminished perception of a desire or urge to fall asleep.Therefore, drowsiness is the
last phase of the brain before sleep.The drowsiness detection system, which records, processes, identifies, and
results from the consumer may take action in events Tiredness detection requires an image-section of the face and
eye motion observation and blink pattern observation as shown in Figure 2.It is suspected that driver fatigue signs
can be identified at an early stage to prevent accidents by eye-tracking [5].

020080-2
 FIGURE 2. Block diagram of the drowsiness detection system

Image Acquisition System

Visual behavior comprehension begins with the acquisition of an image.The aim of the images acquired is to
obtain the following facial image of the driver in real-time.A video camera is used for taking a video of several
drivers and converting their images into multiple images.The picture must be 92*112 pixels in size. The driver
drowsiness rates with the machine learning algorithms are established using new materials. Develops automated
classification models for 30 facial acts, using a machine learner process, to generalize drowsiness by several random
words, which allow automatic facial measurements to find new indicators of sleepiness in facial characteristics and
head movements during real drowsiness [6].
Image Processing and Extract Visual Cues
Image analysis is used to evaluate images using some kind of signals processing and ROI extracting (Facial
Orientation and Moment of Eyes) to extract relevant features from data and to process images using the hybrid
system of arithmetic operations [4]
Face 'etection
The most commonly used tool for the coding of facial gestures in the conduct sciences is the face action coding
scheme (FACS).The process shows the facial movements in 46 element movements which correspond
approximately to the amount of activity of the face muscle. FACS presents an objective and systematic way of
evaluating expressions in elementary elements, similar to phoneme decomposition. FACS has proven to be useful in
discovering facial movements that indicate cognitive and affective states, because of their comprehensively [3].
Action units (AUs) such as chin elevators (AU17), nasolabial furrow elevators (AU11), external (AU2), and inner
braw elevations (AU1) predict drowsiness levels observed before sleep fall subjects.
The detection approach was suggested as follows: first faces and eyes could be detected on a generative boosting
technique in real time.The faces that are automatically classified are aligned according to the observed eye positions
and cut and scaled to 96 / 96 pixels.72 Gabor systems cover 9 space levels and 8 directions. The method
incorporates 45.These filter outputs are standardized and transferred to an understanding better.For each video test
frame [7], the device output is a constant value, the range to each hyperplane.
Methods to 'etect Drowsiness
The method of offering driver information is altered by the conscience level or by the awareness of a vehicle
owner, and the media or their method of providing the information is altered according to the information approval
or urgency level. Within this method, the drivers' monitoring system is adjusted substantially. The early design of
the sleepiness of the driver was extracted by blink estimation insignificant to the encircling luminosity and particular
qualities with film images [7]. As a result, the increase in long eyelid closing time was the main factor in the
estimation of the initial phase of driving somnolence. And the dullness could be assumed by verifying the frequency
bands per unit period of lengthy eyelid closing.The goal method for perceiving the somnolence of the driver was
examined using the CCD camera method to monitor the opening of the eyelid, which shows a strong sleeve view.

020080-3
 FIGURE 3.Flow chart showing step by step working of the system
The camera starts acquiring the grayscale frame by frame video stream that is analyzed for face points.The EAR
(Eye Aspect Ratio), which then is checked for the predefined threshold value, is calculated using the eye coordinates
removed when face characteristics are detected.If the threshold is lower, the counter is increased as shown in Figure
3.It is determined that perhaps the driver is drowsy and the warning system is triggered when we found the counter
to have exceeded 15 [8].

RESULTS
The dataset uses the face characteristics of the human face to train the model, and it can detect the driver's eyes
in real-time as shown in Figure 6.The EAR or the eye look ratio is monitored in the top hand corner of the
window.The "openness" of the eyes is determined and the value falls below 0.17 (the threshold to measure the
system's somnolence) when the system detects a closed eye.The EAR is calculated by each frame and plotted in
graph as shown in Figure 4 and a counter variable tracks the amount of advanced authentication where the driver's
eyes are closed.Once the counter reaches the model's threshold value, the driver is considered drowsy and sends the
digital output to the alarm system.

FIGURE 4.Multilayer Neural Convolutionary Standard deviation

CONCLUSION
This paper presents the procedure for driver somnolence detection by analyzing facial pictures of a camera on
the dash board.First, the eye detection is carried out in two stages, and then eye drowsiness is detected.The image
processing technique is used for the detection of the eye.In the second stage, we use different artificial techniques
such as the neural network, the sensing of different corporeal movements, etc.The convolutional neural network-
based representation approach offers an automated and effective set of features that enable us to accurately classify
the driver as being drowsy or not.Artifacts based on intelligence are well-known to use latent relations in image data
and to learn excellent features to better represent raw data.

020080-4
 REFERENCES
1. Parekh, V., Shah, D. & Shah, Augment Hum Res (2020).
2. KartikDwivedi and Kumar Biswaranjan,IEEE International Conference 2, 1765- 1770 (2005).
3. EsraVural and MüjdatÇetinl, IEEE International Conference 2,1765- 1770 (2005).
4. RatebJabbara and Khalifa Al-Khalifa, 9th International Conference on Ambient Systems, Networks, and
Technologies (2018).
5. M. Ramachandran, Benildus, International Journal of Applied Engineering Research10(11), 10230-
10233(2015).
6. T.Edison and K.Ulagapriya, Journal of Critical Reviews7(6), (2020).
7. Nidhi Sharma and V. K. Banga,World Academy of Science, Engineering and Technology (2010).
8. GauriThakare and VaishnaviShirbhate, International Journal of Advanced Research in Computer and
Communication Engineering 7(3), (2018).
9. PravinKshirsagar, SudhirAkojwar, International Journal of Computer Applications 146 (7), (2016).
10. Neerajkumars sathawane1 &Pravinkshirsagar, International Journal of Research in Engineering &
Technology2(5), 199-206 (2014).
11. Dr. Sudhir G. Akojwar, PravinKshirsagar, VijetalaxmiPai, National Conference on Research Trends in
Electronics, Computer Science & Information Technology And Doctoral Research Meet.
12. PravinKshirsagar and SudhirAkojwar, International Conference on Signal Processing, Communication, Power
and Embedded System (SCOPES) (2016).
13. P. Kshirsagar, S. Akojwar, Proceedings of the International Conference on Communication and Signal
Processing (2016).
14. Pravin R. Kshirsagar, Pranav P. Chippalkatti, Swagat M. Karve ,Journal of Advanced Research in Dynamical
and Control Systems (2018).
15. Pravin R. Kshirsagar, Pranav P. Chippalkatti, Swagat M. Karve, Journal of Advanced Research in Dynamical
and Control Systems (2018).
16. P. Kshirsagar, S. Akojwar, Nidhi D. Bajaj ,International Journal of Biomedical Engineering and
Technology 28(4), (2018).
17. P. Kshirsagar, S. Akojwar, IEEE International Conference on Communication Networks (ICCN) (2015).

020080-5
View publication stats