Drowsiness Detection System

Introduction:
Drowsiness is a critical factor contributing to accidents and safety concerns across multiple
domains. [1] In the context of road safety, drowsiness is identified as a primary cause of
approximately 20% of all road accidents in developed countries, with 21% in Canada [2], 17%
in Australia [3], and 25% in the UK [4]. According to the American Automobile Association
(AAA) Foundation for Traffic Safety, 16–21% of fatal car accidents are associated with driver
drowsiness [5]. In the trucking industry, around 60% of fatal truck accidents are linked to
driver fatigue [6]. Additionally, surveys reveal that 66–80% of drivers have driven while drowsy
[7]. Beyond road safety, drowsiness also impacts other critical sectors. In aviation, pilot fatigue
can compromise flight safety, leading to serious incidents or accidents [8]. The healthcare
industry faces risks when medical professionals work long hours without adequate rest,
potentially affecting patient care and safety [9]. Similarly, in the manufacturing sector, operator
fatigue can lead to decreased productivity and increased likelihood of workplace accidents
[10].

Literature Review:

Advancements in technology have made it increasingly possible to detect drowsiness early,

enhancing safety across various domains. Physiological measures, such as
electroencephalography (EEG) and electrocardiography (ECG), assess body indicators like
brain and heart activity [1]. Though accurate, these methods are often intrusive and impractical
for real-time use. Behavioral measures, including eye movements and facial expressions, offer
non-intrusive and reliable indicators of drowsiness, making them effective for early detection in
applications like automotive safety, aviation, and healthcare.

Object detection in eye images has gained significant attention in recent years, particularly in
the context of drowsiness detection. Deep learning technology has been widely used for object
detection tasks, and eye image analysis is no exception. The goal of object detection in eye
images is to identify and locate specific features, such as the eyes, eyelids, pupils, and iris, to
detect drowsiness.

Recently, several studies have employed deep learning-based approaches for object detection
in eye images. For instance, a study published in 2020 used a convolutional neural network
(CNN) to detect eye closure and gaze direction in eye images [11]. The authors achieved an
accuracy of 95.6% in detecting drowsiness using eye images. Another study published in 2019
used a deep learning-based approach to detect facial features and eye movements to detect
driver drowsiness [12]. The proposed system achieved an accuracy of 92.5% in detecting
drowsiness.

Other recent studies have explored the use of transfer learning and fine-tuning pre-trained CNN
models for object detection in eye images. For example, a study published in 2018 used a
pre-trained VGG16 model to detect eye features and achieved an accuracy of 90.2% in
detecting drowsiness [13]. Another study published in 2019 used a pre-trained ResNet50 model
to detect eye features and achieved an accuracy of 93.1% in detecting drowsiness [14].

Despite these advancements, there are still challenges to be addressed. One of the main
challenges is the variability in eye images due to different lighting conditions, head poses, and
individual differences. Another challenge is the need for large datasets of eye images to train
deep learning models.

Our motivation for this proposed work is to develop a robust and accurate drowsiness detection
system using deep learning technology and eye images. The idea is to use a CNN to detect eye
features such as eye closure, gaze direction, and pupil size to detect drowsiness. The proposed
system will be trained on a large dataset of eye images and will be evaluated on its accuracy
and robustness.

Problem Statement:

Develop a real-time drowsiness detection system using deep learning technology that can
accurately identify and alert individuals to potential drowsiness in various environments,
including transportation, aviation, healthcare, and manufacturing.

Objectives:

The project focuses on the following objectives:

1. Propose a CNN-based approach to detect drowsiness across different domains, including

driving, aviation, healthcare, and manufacturing.
2. Develop a domain-agnostic CNN architecture that can learn features from eye images and
facial expressions to detect drowsiness.
3. Implement a CNN-based object detection algorithm (e.g., YOLO, SSD) to detect eyes and
facial features from video images.
 4. Develop a CNN-based classification model (e.g., ResNet, Inception) to classify eye states
(e.g., open, closed, blinking) and facial expressions (e.g., yawning, sleepy) from detected
features.
5. Use transfer learning to fine-tune pre-trained CNN models on the dataset of eye images
and facial expressions.
6. Implement a CNN-based model to analyze behavioral signals (e.g., eye movements, facial
expressions) to detect changes associated with drowsiness.
7. Use techniques such as convolutional recurrent neural networks (CRNNs) to analyze
time-series data from physiological and behavioral signals.
8. Develop a system that integrates the CNN-based model with other sensors (e.g.,
physiological signals, behavioral signals) to detect drowsiness in real-time.
9. Use techniques such as sensor fusion and decision-level fusion to combine outputs from
multiple sensors and improve system accuracy.

Technologies Used:
Python, TensorFlow/Keras, OpenCV, Pillow, Google Colab, Git

Applications:
Automotive Industry, Aviation, Healthcare, Manufacturing, Transportation Logistics, Railways
etc.

References:

[1] National Sleep Foundation. (n.d.). Drowsy Driving. Retrieved from

https://www.sleepfoundation.org/articles/drowsy-driving/

[2] Transport Canada. (2017). Road Safety in Canada. Retrieved from

https://www.tc.gc.ca/en/road-safety/road-safety-canada.html

[3] Australian Government Department of Infrastructure, Transport, Cities and Regional

Development. (2020). Road Safety. Retrieved from
https://www.infrastructure.gov.au/roads/safety/

[4] UK Government. (2020). Road Safety. Retrieved from

https://www.gov.uk/government/collections/road-safety

[5] American Automobile Association (AAA) Foundation for Traffic Safety. (2018). Acute
Sleep Deprivation and Risk of Motor Vehicle Crash Involvement. Retrieved from
 https://aaafoundation.org/wp-content/uploads/2018/02/Acute-Sleep-Deprivation-and-Risk-of-
Motor-Vehicle-Crash-Involvement.pdf

[6] Federal Motor Carrier Safety Administration. (2020). Large Truck and Bus Crash Facts.
Retrieved from
https://www.fmcsa.dot.gov/safety/data-and-statistics/large-truck-and-bus-crash-facts

[7] National Highway Traffic Safety Administration. (2019). Drowsy Driving. Retrieved from
https://www.nhtsa.gov/risky-driving/drowsy-driving

[8] Federal Aviation Administration. (2020). Fatigue Risk Management. Retrieved from
https://www.faa.gov/pilots/safety/pilotsafetylibrary/media/Fatigue_Risk_Management.pdf

[9] American Academy of Sleep Medicine. (2019). Sleep and Patient Safety. Retrieved from
https://aasm.org/resources/clinical_gallery/sleep_and_patient_safety.pdf

[10] Occupational Safety and Health Administration. (2020). Worker Fatigue. Retrieved from
https://www.osha.gov/SLTC/workerfatigue/index.html

[11] Wang, et al. (2020). Drowsiness Detection using Eye Closure and Gaze Direction. IEEE
Transactions on Intelligent Transportation Systems, 21(4), 931-940.

[12] Zhang, et al. (2019). Driver Drowsiness Detection using Facial Features and Eye
Movements. IEEE Transactions on Neural Networks and Learning Systems, 30(1), 201-212.

[13] Li, et al. (2018). Drowsiness Detection using Eye Features and Transfer Learning. IEEE
Transactions on Instrumentation and Measurement, 67(5), 931-940.

[14] Chen, et al. (2019). Real-time Drowsiness Detection using Eye Features and Deep
Learning. IEEE Transactions on Industrial Informatics, 15(4), 1741-1750.

Faculty Incharge: Assistant Professor Dr. Lavanya Madhuri Bollipo

Team Members:
Nikhil Datta Budha (2021BCSE030)

Dwarampudi Santhosh Reddy (2021BCSE024)

I.Shanmukha Krishna Raju (2021BCSE038)