Team 1: Course Instructor:
Nguyen Le Quoc Anh _ EEEEIU180003 Dr N.D.Uyen
Nguyen Duong Song Toan _ EEEEIU18098
Dao Ngoc Quoc Khanh _ EEEEIU18044
Truong Hoang Bao Long _ EEEEIU18050
Pham Vinh Phu _EEEEIU18085
Background Research Plan Worksheet
Project Name:
Developing AI Based Automatic Door to Detect Mask & Measure
Temperature
Solving Problem: Manual COVID-19 checkup process at public places may
indirectly cause spreading in society.
▪ The public places are crowded places so the security is very strict. With
the widespread of the coronavirus, it is mandatory to check temperature
and mask before walking into any places. But manually temperature
checkups can be inefficient, impractical and may cause the infection
which is very dangerous.
▪ We need a solution that can reduce the close-touch between people at the
public places.
▪ Contactless Body Temperature Measurement and Mask Detection is one
of the solutions for this problem. A system using an IR temperature
sensor for measuring body temperature and deep learning neural network
for mask detection.
People at any places can keep distance and still complete the security
process without close-touch to people.
Keywords:
- COVID-19, prevention method, temperature checking method, mask
detection, automatic entrence kiosk, social distancing
Questions:
1. Target User
a. Who needs
▪ The public places, where COVID virus can easily be
infected.
b. Who wants
▪ School, Airport, Big company, …
c. Who buys
▪ The security staffs
d. Target user
▪ The project targets to replace the human workforce in
temperature checkup process. Thus, target users could be
security agency or authority of the public places (School,
Airport, Big company, …)
e. Size
▪ The size will be similar to a normal door.
2. How it works, how to make it
• Facial Landmarks :
Facial landmark detection is the task of detecting key landmarks on the face and
tracking them (being robust to rigid and non rigid facial deformations due to head
movementsand facial expressions)
Facial landmarks are used to localize and represent salient regions of the face, such
as:
• Eyes
• Eyebrows
• Nose
• Mouth
• Jawline
Detecting facial landmarks is a subset of the shape prediction problem. Given an
input image (and normally an ROI that specifies the object of interest), a shape
predictor attempts to localize key points of interest along the shape.
In the context of facial landmarks, our goal is detect important facial structures on
the face using shape prediction methods.
Dlib’s facial landmark detector:
The pre-trained facial landmark detector inside the dlib library is used to estimate
the location of 68 (x, y)-coordinates that map to facial structures on the face.
The indexes of the 68 coordinates can be visualized on the image below:
Regardless of which dataset is used, the same dlib framework can be leveraged to
train a shape predictor on the input training data — this is useful if you would like
to train facial landmark detectors or custom shape predictors of your own.
Link : Facial landmarks with dlib, OpenCV, and Python - PyImageSearch
Mask Detection :
Open CV (Open Source Computer Vision Library is a library of programming
functions mainly aimed at real-time computer vision ) : create dataset and process
image.
Tensor Flow (TensorFlow is an end-to-end open source platform for machine
learning. It has a comprehensive, flexible ecosystem of tools, libraries and
community resources that lets researchers push the state-of-the-art in ML and
developers easily build and deploy ML powered applications.) : training face mask
classifier.
First we have to create the dataset include 2 classes : with mask and without mask.
Technique to create dataset :
1. Taking normal images of faces.
2. Creating a custom computer vision Python script to add face masks to ,
thereby creating an artificial dataset.
For making dataset face mask with facial landmark include these steps:
• Step 1 : Start with an image of a person not wearing a face mask.
• Step 2: Apply face detection to compute the bounding box location of the
face in the image.
• Step 3 : Once we know where in the image the face is, we can extract the
face Region of Interest (ROI).
• Step 4 : We need an image of a mask. This mask will
be automatically applied to the face by using the facial landmarks (namely
the points along the chin and nose) to compute where the mask will be
placed.
• Step 5 : We can then repeat this process for all of our input images, thereby
creating our artificial face mask dataset.
Phase 1 : Training
Here we’ll focus on loading our face mask detection dataset from disk,
training a model (using Keras/TensorFlow) on this dataset, and then serializing the
face mask detector to disk.
Tensor flow import some set library for :
• Data Augmentation.
• Loading the MobileNetV2 classifier(we will fine-tune this model with pre-
trained ImageNet weights).
• Building a new fully-connected head.
• Pre-processing.
• Loading image data.
We’ll use scikit-learn(sklearn) for binarizing class labels, segmenting out
dataset and printing a classification report.
Using imutils paths implementation will help us to find and list images in our
dataset.
Phase 2 : Deployment
Once the face mask detector is trained, we can then move on to loading the mask
detector, performing face detection, and then classifying each face as with mask or
without mask.
Raspberry Face Mask Detection project :
When a user approaches your webcam, the Python code
utilizing TensorFlow, OpenCV, and imutils packages will detect if a user is
wearing a face mask or not. Users not wearing a face mask will be designated with
a red box around their face, and users wearing a face mask will see a green box
around their face with the text, “Thank you. Mask On.” Users not wearing a face
mask will see a red box around their face with, “No Face Mask Detected.”
Link : COVID-19: Face Mask Detector with OpenCV, Keras/TensorFlow, and
Deep Learning - PyImageSearch
OpenCV - Wikipedia
TensorFlow
How to Build a Face Mask Detector with Raspberry Pi | Tom's Hardware
Temperature Checkup
Using the Contactless IR Temperature Sensor for taking the temperature. But to get
the right temperature on the face, we be using the facial landmarking method to
spot the forehead and measure it.
Contactless Infrared Temperature Sensor measures the surface temperature of
an object depending on the emitted IR waves of the target without touching it and
also measures the average temperature over an area. It is a Contact-less, high
precision, high resolution and a fast response sensor.
For body temperature measurement using facial landmarking :
• The system using DLIB Module for Facial Landmarking to find the best
Spot on the forehead of the person to take temperature from.
• Then by using the PID control system with servo motors, the system tried to
align the Selected Spot on the Forehead with the Sensors. Once aligned the
system takes Temperature Reading using the Contactless IR Temperature
Sensor.
Link : TouchFree: Automated Temperature Checkup and Mask Detection -
Hackster.io
Automated Door :
For this part, we’ll using the raspberry to control the servo motor which attach with
a bar. We will code for the motor to turn the bar when 2 conditions such as : the
body temperature between 37-38 and the mask be detected are met.
Link : Servo Motor Control With Raspberry Pi : 4 Steps - Instructables
4. Questions help understand products or programs that fill similar needs
- Product filling similar need: contactless thermal meter is the used widely at
public places to measure temperature.
- Strength & weakness:
+ Strength: able to detect people with high temperature
+ Weakness: Inefficiency
• Required people to operate
• Close-distance contact
o Key feature: Measure temperature
o Our project improvement:
o Develop an automatic process (automatic door)
o Operating independently from human (replace human workforce)
o Able to detect no-mask wearing people
