A PROJECT REPORT ON

MUSIC RECOMMENDATION WITH EEG SIGNALS USING MACHINE

LEARNING AND DEEP LEARNING TECHNIQUE

SUBMITTED TO THE SAVITRIBAI PHULE PUNE UNIVERSITY, PUNE

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS
FOR THE AWARD OF THE DEGREE

OF

BACHELOR OF ENGINEERING
(Computer Engineering)

BY

Aadit Bagga Exam Seat No: B151004204

Sahil Bora Exam Seat No: B151004211
Akash Deshmukh Exam Seat No: B151004217
Pinak Wadekar Exam Seat No: B151004279

Under The Guidance of

Prof. Ms. M.S Namose

DEPARTMENT OF COMPUTER ENGINEERING

JSPM Narhe Technical Campus
Rajarshi Shahu School of Engineering & Research
Narhe, PUNE 411041
(2021-22)

JSPM NTC, Department of Computer Engineering 2021-22

1
 JSPM Narhe Technical Campus
DEPARTMENT OF COMPUTER ENGINEERING

CERTIFICATE
This is to certify that the BE Project Report entitled

“MUSIC RECOMMENDATION WITH EEG SIGNALS USING MACHINE

LEARNING AND DEEP LEARNING TECHNIQUE”

Submitted by

Aadit Bagga Exam Seat No: B151004204

Sahil Bora Exam Seat No: B151004211
Akash Deshmukh Exam Seat No: B151004217
Pinak Wadekar Exam Seat No: B151004279

is a bonafide work carried out by them under the supervision of Prof. Ms. M.S
Namose and it is submitted towards partial fulfillment of the requirement of Savitribai
Phule Pune University, Pune, for the award of the degree of Bachelor of Engineering
(Computer Engineering).

Prof. Ms. M.S Namose Name:

Internal Guide Signature:
External Examiner

Dr. N. A. Auti Dr. S. A. Choudhari

HOD, Computer Engineering Director

Place: Pune
Date: 18/05/22

JSPM NTC, Department of Computer Engineering 2021-22

2
 ACKNOWLEDGEMENT

We would like to express our gratitude and appreciation to all those who gave
us the possibility to complete this report. Special thanks is due to our supervisor Prof.
Ms. M.S. Namose whose help, stimulating suggestions and encouragement helped us
in all time of fabrication process and in writing this report. We also sincerely thank
you for the time spent proofreading and correcting our many mistakes.

We would also like to extend our gratitude to the Head of department Dr. N.A.
Auti and also our respected Director Dr. S.A. Choudhari for providing us with all the
facilities that were required.

Lastly we would like to thank our parents and friends who have helped us with
their valuable suggestions and guidance, it has been helpful in various phases of the
completion of the project.

Sincerely,

Aadit Bagga
Exam Seat No: B151004204
Sahil Bora
Exam Seat No: B151004211
Akash Deshmukh
Exam Seat No: B151004217
Pinak Wadekar
Exam Seat No: B151004279

JSPM NTC, Department of Computer Engineering 2021-22

3
 ABSTRACT

It is quite difficult for those who are paralysed or on bed rest to express them.
They experience feelings like regular individuals, yet it is difficult for them to enjoy
their time. In India, millions of individuals are afflicted with various ailments. There is
no way to keep them entertained.
The EEG (electroencephalogram) gadget is used to track a person's brain
activity. It assists in identifying the emotion of the paralysed and bed rest sick person
by obtaining electroencephalogram signals using EEG devices, utilizing those signals
as data, and applying the circumplex model. The music recommendation algorithm
enters the picture after the system has identified the emotion. Music recommendations
are made by the system based on the feeling of the crippled and bedridden patient. So
that people who are paralysed or on bed rest may enjoy and relax with the use of
entertainment as treatment. The music recommendation system is based on two
approaches: the first is expert-based music, and the second is a characteristic of the
music.

JSPM NTC, Department of Computer Engineering 2021-22

4
 INDEX

Chapter No. Content Page No.

List of Figures 7
List of Tables 8

Chapter 1 Synopsis 9
1.1 Project Title 10
1.2 Technical Keywords 10
1.3 Problem Statement 10
1.4 Abstract 10
1.5 Objectives 10
Chapter 2 Technical Keywords 12
2.1 Area of Project 13
2.2 Technical Keywords 14
Chapter 3 Introduction 15
3.1 Introduction 16
3.2 Motivation 16
3.3 Problem Definition 17
3.4 Objectives 17
Chapter 4 Literature Survey 18
Chapter 5 Problem Definition and Scope 22
5.1 Problem Definition 23
5.2 Project Scope 23
Chapter 6 Project Plan 24

Chapter 7 Software Requirement Specification 26

Chapter 8 Methodology & Algorithms 28
Chapter 9 High Level Design Document 33
Chapter 10 Experimentation & Results 40
Chapter 11 Summary and Conclusion 43
Chapter 12 Annexure

JSPM NTC, Department of Computer Engineering 2021-22

5
 Annexure A: Laboratory assignments on Project Analysis of
Algorithmic design
Annexure B: Laboratory assignments (OOMD) on Project
Quality and Reliability testing of Project Design
Annexure C: Project planner
Annexure D: Published Paper as per author guidelines and
format given by conference, Certificate of Publication
Annexure E: References (In IEEE Format)

JSPM NTC, Department of Computer Engineering 2021-22

6
 LIST OF FIGURES

Figure No. Name of Figure Page No.

8.1 Proposed system architecture 29
8.2 Logical workflow of proposed classifier (CNN) 30
9.1 DFD 0 Level 34
9.2 DFD Multi Level 34
9.3 Class Diagram 35
9.4 Sequence Diagram 36
9.5 Use Case diagram 37
9.6 Activity Diagram 38
9.7 Component Diagram 39
10.1 The error of the training set of the artificial neural 41
network
10.2 Accuracy of the training set of the artificial neural 42
network
10.3 User Interface and Result 42

JSPM NTC, Department of Computer Engineering 2021-22

7
 LIST OF TABLES

Table No. Name of Table Page No.

6.1 System Implementation Plan 25

JSPM NTC, Department of Computer Engineering 2021-22

8
JSPM NTC, Department of Computer Engineering 2021-22

9
1.1 Project Title:
Music Recommendation with EEG signals using Machine Learning and Deep
Learning Technique

1.2 Technical Keywords

EEG classification, classification, music recommendation, machine learning, feature
extraction, feature selection.

1.3 Problem Statement

Propose and Implement a system to detect and process emotions of Speechless
handicapped / differently abled people to recommend different entertainments to them
to make them stress free and access to basic utilities for entertainment.

1.4 Abstract
It is quite difficult for those who are paralysed or on bed rest to express them. They
experience feelings like regular individuals, yet it is difficult for them to enjoy their
time. In India, millions of individuals are afflicted with various ailments. There is no
way to keep them entertained.
The EEG (electroencephalogram) gadget is used to track a person's brain activity. It
assists in identifying the emotion of the paralysed and bed rest sick person by
obtaining electroencephalogram signals using EEG devices, utilizing those signals as
data, and applying the circumplex model. The music recommendation algorithm enters
the picture after the system has identified the emotion. Music recommendations are
made by the system based on the feeling of the crippled and bedridden patient. So that
people who are paralysed or on bed rest may enjoy and relax with the use of
entertainment as treatment.
The music recommendation system is based on two approaches: the first is expert-
based music, and the second is a characteristic of the music.

1.5 Objectives
● First, Study of DEAP dataset to understand basics of EEG data
● Study of working of EEG scanner devices.

JSPM NTC, Department of Computer Engineering 2021-22

10
● Extracting different features of dumb person through scanner.
● Preprocessing data captured through the device.
● Classification of all signals and captured Arousal and Valence Values.
● Using Arousal and Valence values, Detect the emotion via Circumplex model
● Study the music recommendation system which recommends music according to
emotion.

JSPM NTC, Department of Computer Engineering 2021-22

11
JSPM NTC, Department of Computer Engineering 2021-22

12
2.1 Area of Project

● Machine Learning- Machine learning (ML) is a type of artificial intelligence

(AI) that allows software applications to become more accurate at predicting
outcomes without being explicitly programmed to do so. Machine learning
algorithms use historical data as input to predict new output values.
Machine learning is important because it gives enterprises a view of trends in
customer behavior and business operational patterns, as well as supports the
development of new products. Many of today's leading companies, such as
Facebook, Google and Uber, make machine learning a central part of their
operations. Machine learning has become a significant competitive
differentiator for many companies.

Supervised machine learning requires the data scientist to train the algorithm with both
labeled inputs and desired outputs. Supervised learning algorithms are good for the
following tasks:

● Binary classification: Dividing data into two categories.

● Multi-class classification: Choosing between more than two types of answers.
● Regression modeling: Predicting continuous values.
● Ensembling: Combining the predictions of multiple machine learning models
to produce an accurate prediction.
● Deep Learning Technique- Deep learning is a machine learning technique
that teaches computers to do what comes naturally to humans: learn by
example. Deep learning is a key technology behind driverless cars, enabling
them to recognize a stop sign, or to distinguish a pedestrian from a lamppost. It
is the key to voice control in consumer devices like phones, tablets, TVs, and
hands-free speakers. Deep learning is getting lots of attention lately and for
good reason. It’s achieving results that were not possible before.

In deep learning, a computer model learns to perform classification tasks

directly from images, text, or sound. Deep learning models can achieve state-
of-the-art accuracy, sometimes exceeding human-level performance.

JSPM NTC, Department of Computer Engineering 2021-22

13
 Models are trained by using a large set of labeled data and neural network
architectures that contain many layers.

Deep learning applications are used in industries from automated driving to

medical devices.

Automated Driving: Automotive researchers are using deep learning to

automatically detect objects such as stop signs and traffic lights. In addition,
deep learning is used to detect pedestrians, which helps decrease accidents.

Aerospace and Defense: Deep learning is used to identify objects from

satellites that locate areas of interest, and identify safe or unsafe zones for
troops.

2.2 Technical Keywords

● EEG classification
● Classification
● Music recommendation
● Music recommendation
● Machine learning
● Feature extraction
● Feature selection

JSPM NTC, Department of Computer Engineering 2021-22

14
JSPM NTC, Department of Computer Engineering 2021-22

15
3.1 Introduction
Emotion awareness is one of the most important subjects in the field of affective
computing. Using nonverbal behavioral methods such as recognition of facial
expression, verbal behavioral methods such as recognition of speech emotion, or
physiological signals based methods such as recognition of emotions based on
electroencephalogram (EEG) can predict human emotion. However, it is notable that
data obtained from either nonverbal or verbal behaviors are indirect emotional signals
suggesting brain activity. Unlike the nonverbal or verbal actions, EEG signals are
reported directly from the human brain cortex and thus may be more effective in
representing the inner emotional states of the brain. Consequently, when used to
measure human emotion, the use of EEG data can be more accurate than data on
behavior. For this reason, the identification of human emotion from EEG signals has
become a very important research subject in current emotional brain-computer
interfaces (BCIs) aimed at inferring human emotional states based on the EEG signals
recorded. Monitoring the patient's mental state to prevent stress-related illnesses.
Suicide is a common cause of death for both teenagers and adults. We can avoid the
emotions of suicides through user identity. Help people with mental issues by offering
them advice. To create a training model that contains physiological signals (EEG
signals from the Synchronized Dataset) useful for detecting elevated levels over long
periods of time or abrupt changes in mental exhaustion, emotional response, facial
expression and user speech to detect user emotions. Solutions to monitor signs related
to the development of stress-induced occupational diseases or prompt detection of
acute rises in stress levels in specifically hazardous work scenarios may be needed.

3.2 Motivation
In all over India, the highest number of disabled people live in the state of Uttar
Pradesh (approximately 3.6 million). There are other states like Bihar (approximately
1.9 million), West Bengal (approximately 1.8million), Tamil Nadu and Maharashtra
(approximately 1.6 million each). Tamil Nadu is the one and only state, which has
more population of disabled females than males. Other states, like Arunachal Pradesh,
have the most population of disabled males (approximately 66.6%) and lowest
population of female disabled.

JSPM NTC, Department of Computer Engineering 2021-22

16
 ● Disable people in India by age group: -
● 5% children are in the age group of 0 to 4 years
● 7% children are in the age group of 5 to 9 years
● 17% children or teenager are in the age group of 10 to 19 years
● 16% teenager or young people are in the age group of 20 to 29 years
● 13% young or middle age people are in the age group of 30 to 39 years
● 12% middle age people are in the age group of 40 to 49 years
● 9% old people are in the age group of 50 to 59 years
● 10% old people are in the age group of 60 to 69 years
● 7% old people are in the age group of 70 to 79 years
● So, for them it is very important to stay positive and live a happy and healthy
life. For them, create a system that help them to stay away from stress because
they are physically disable not mentally disable. So for them using EEG
Signal, First detect the emotion of the handicap Dumb person and then
according to the emotion suggest them music/movie as a therapy to stay in
positive mindset.

3.3 Problem Definition

Propose and Implement a system to detect and process emotions of Speechless
handicapped people to recommend different entertainments to them to make them
stress free.

3.4 Objectives
● First, Study of DEAP dataset to understand basics of EEG data
● Study of working of EEG scanner devices.
● Extracting different features of dumb person through scanner .
● Preprocessing data captured through the device.
● Classification of all signals and captured Arousal and Valence Values.
● Using Arousal and Valence values, Detect the emotion via Circumplex
model
● Study the music recommendation system which recommends music
according to emotion.

JSPM NTC, Department of Computer Engineering 2021-22

17
JSPM NTC, Department of Computer Engineering 2021-22

18
4.1 Literature Survey

According to [1] a framework for learning a user's mood utilizing data from a
wearable device paired with physiological sensor signals such as galvanic skin
response (GSR), photoplethysmography (PPG), and electroencephalography (EEG), as
well as data from a camera. This data is added to the music recommendation engine as
a complement. Sensor and facial expression data may therefore increase the
recommendation engine's usefulness and accuracy.

According to [2] a commercially available EEG Bluetooth headset that uses

sensors to detect changes in brain waves (such as alpha and beta waves). Data may be
safely transferred using the mobile device using the Bluetooth transmission
mechanism. The EEG signal may provide a wealth of information concerning a broad
variety of cognitive disorders and illnesses. In addition to providing medical
professionals with a reference for treatment, the EEG signal, categorization, and
classification of depression level may be utilized as an assessment foundation for
music therapy. As a result, in this article, a music recommendation system will be
implemented to give users a list of relaxing feelings based on various symptoms and
degrees.

According to [3] Due to the development of an attention device, a select

electroencephalographic (EEG) was utilized to record and evaluate attention according
to the individual's preferences for music content. The gathered brainwave data values
are then filtered for incorrect data before being integrated with an SVM classifier. For
computation and analysis, the SVM classifier distinguishes between two kinds of
brainwave data values (attention vs. non-attention). Due to the development of an
attention device, a hybrid music recommendation model based on an exciting device
that includes the user's electroencephalographic (EEG) information and records the
individual's selections for music material has been developed.

According to [4] Using brain signals, the influence of music tracks in English and
Urdu on human stress levels. Twenty-seven people, 14 men and 13 women, with Urdu
as their first language and ages ranging from 20 to 35, volunteered to take part in the

JSPM NTC, Department of Computer Engineering 2021-22

19
research. The subjects' electroencephalograph (EEG) signals are captured as they listen
to several music tracks using a four-channel MUSE headgear. The state and trait
anxiety questionnaire asks participants to subjectively rate their stress level. The four
genres of English music that were utilized in this research are rock, metal, electronic,
and rap.

According to [5] a comparison of different machine learning and deep learning

techniques, both with and without feature selection, for binarily classifying the six
basic emotions, namely anger, disgust, fear, joy, sadness, and surprise, into two
symmetrical categorical classes (emotion and no emotion), using physiological
recordings and subjective ratings of valence, arousal, and dominance from the DEAP
(Dataset for Emotion Analysis using EEG, Physiological, and Video Signals). The
greatest categorization accuracies for each emotion were 98.02 % for anger, 100
percent for pleasure, 96 percent for surprise, 95 percent for contempt, 90.75 percent
for fear, and 90.08 percent for sorrow.

According to [6] uses ScientoPy to do a scientometric analysis of the main

recommendation approach publications discovered in scientific databases (Elsevier's
Scopus and Clarivate Web of Science). The past two decades have seen a lot of
research on the recommenders of emotion-based tourism. To identify emotions, the
review focuses on the collection, processing, and feature extraction of data from
sensors and wearables. Recommendation systems, emotion detection, wearable
technology, and machine learning are proposed as topic areas in the research.

According to [7] music's potential as a therapeutic component in digital treatment

programmes to promote mental health and well-being Music elicits an emotional
response in the listener, which manifests itself in changes in brain activity that may be
measured by electroencephalography. The most current relevant studies relating to the
influence of music on brain activity and emotional state in digital treatment
programmes were identified via a scoping review. Six relevant papers were chosen
among 585 identified publications that met all of the study's parameters..

JSPM NTC, Department of Computer Engineering 2021-22

20
 According to [8] a noninvasive EEG-based BCI for a robotic arm control system
that allows users to perform multitarget reach and grip tasks while avoiding obstacles
via hybrid control Seven individuals' findings showed that motor imagery (MI)
training may change brain rhythms, and six of them performed the online tasks using
the hybrid-control-based robotic arm system. The suggested system performs well
thanks to a combination of MI-based EEG, computer vision, gaze recognition, and
partly autonomous guiding, which greatly improves online task accuracy and reduces
the brain load generated by long-term mental activity.

According to [9] after preprocessing, the preprocessed data are utilized to train the
LSTM network, and then the Softmax function is used to categorize the input data into
normal and seizure data. Normalization of EEG data, application of suitable filters to
pick the useful sections of the data, and data management are the three phases of
preprocessing. The normalization of EEG data, the use of suitable filters to pick the
important sections of the data, and data management are the three phases of
preprocessing. After preprocessing, the preprocessed data are utilized to train the
LSTM network, which is then used to categorize the input data into normal and seizure
data using the Softmax function.

According to [10] they get the data from the user's text on social media using an
IoT. The text data will be analyzed for emotion detection. They offered two
approaches for music suggestion after emotion was identified. The first strategy is an
expert-based approach, in which certain experts are used to distribute music depending
on emotion. The second technique is a feature-based strategy, which does not need the
assistance of an expert. They employ music's rhythm and articulation to distribute
songs according to emotion. For the music suggestion, they devised a feedback
mechanism. As a result, the algorithm will provide music recommendations based on
user response.

JSPM NTC, Department of Computer Engineering 2021-22

21
JSPM NTC, Department of Computer Engineering 2021-22

22
5.1. Problem Definition

Propose and Implement a system to detect and process emotions of Speechless

handicapped people to recommend different entertainments to them to make them
stress free.

5.2 Project Scope

● This system will be limited to recommending only music for entertainment.

● It will be limited to single channel EEG scanner devices.
● The system can be enhanced to incorporate wider sets of data for
recommendation of music to increase options.
● The system can be enhanced to recommend content apart from music, visual
and hearing aid depending on the type of user.
● The system will ease and bring in an added source of happiness in the people
who cannot access basic utilities.

JSPM NTC, Department of Computer Engineering 2021-22

23
JSPM NTC, Department of Computer Engineering 2021-22

24
6.1 Project Plan

• Phase- 1 Requirement Gathering

• Phase- 2 Analysis
• Phase- 3 Research Design
• Phase- 4 Requirement Gathering
• Phase- 5 Development of System
• Phase- 6 GUI and Front End Development
• Phase- 7 Part Testing
• Phase- 8 Testing of hypothesis and performance analysis of system.
• Phase- 9 Submission of Project

JSPM NTC, Department of Computer Engineering 2021-22

25
JSPM NTC, Department of Computer Engineering 2021-22

26
7.1 Software Requirements (Platform Choice)

Technologies and tools used in Policy system project are as follows Technology used:

Front End

● Internet Explorer 6.0/above- Microsoft Internet Explorer 6 (IE6) is a

graphica lweb browser developed by Microsoft for Windows operating
systems. Released on August 24, 2001, it is the sixth, and by now discontinued,
version of Internet Explorer and the successor to Internet Explorer 5. It was the
default browser in Windows XP 5 (later default was Internet Explorer 8) and
Windows Server 2003 and can replace previous versions of Internet Explorer
on Windows NT 4.0 SP6a, Windows 98, Windows 2000 and Windows ME but
unlike version 5, this version does not support Windows 95 or an earlier
version. IE6 SP2+ and IE7 were only included (IE6 SP2+) in or available (IE7)
for Windows XP SP2+.

● Tool : PyCharm on words- PyCharm is an integrated development

environment (IDE) used in computer programming, specifically for the Python
programming language. It is developed by the Czech company JetBrains
(formerly known as IntelliJ).[5] It provides code analysis, a graphical debugger,
an integrated unit tester, integration with version control systems (VCSes), and
supports web development with Django as well as data science with
Anaconda.[6]

● Programming Language: Python 2.7 onwards

7.2 Hardware Requirements

● Processor:- Intel Pentium 4 or above

● Memory:- 2 GB or above

● Other peripheral:- Printer

● Hard Disk:- 500gb

JSPM NTC, Department of Computer Engineering 2021-22

27
JSPM NTC, Department of Computer Engineering 2021-22

28
8.1 Proposed system design

The recommended system model architecture contains the design. The user
will enter text or voice into a text or speech format, which will convert the speech into
a text, transmit the text in json for analysis to a server, compute emotions, and search
for keywords to return millions of songs, which will be filtered again under the
cooperation filter. In the first step, emotions and keywords are considered, and based
on the most likely extracted data, which other users listened to, the cold start problem
is reduced because we already have all data, which is sorted by the most popular
genres, type of music, music year, length, singer, and feelings.

Figure 8.1 : Proposed system architecture

8.1.1 Preprocessing

It is a method for transforming raw data into a format that is both usable and
efficient. Preprocessing is a procedure that comprises several processes that are
applied to input pictures. Images are altered throughout this process. The
reorganization of data's shape without affecting the data's substance. In this procedure,
several types of arrangements are made based on the criteria that are required to
continue to the next step.

JSPM NTC, Department of Computer Engineering 2021-22

29
 Figure 8.2: Logical workflow of proposed classifier (CNN)

8.1.2 Build a Classifier CNN

Deep Learning includes the Convolutional Neural Network (CNN). To far,
CNN has shown to be a highly efficient and successful method of achieving
handwriting recognition. Convolutional Neural Networks are neural networks that
employ multiple layers of filters to extract information from pictures.

1: Convolutional Layer: It's the foundation for constructing a CNN model. This layer
conducted mathematical calculations on the picture that was used as input, as well as
resizing the image into the M* M format. This layer's output describes the image's
features, such as edge and corner mapping, also known as a feature map. The
information was then added to the following layer.

2: Pooling Layer: This is the layer that connects the convolutional and fully
connected layers. This layer is used to decrease the network's parameters and
computation. The max pooling and average pooling methods are provided by this
layer. The most frequent method is max pooling. The output of the preceding layer, the

JSPM NTC, Department of Computer Engineering 2021-22

30
pooling layer, is sent to the fully connected layer. This layer is where the
categorization process takes place.

3: Classification and Recommendation: This layer test classifier has been utilized
for recommendation. The input features and trained modules have fed to the test
classifier and recommended the specific song for the end user accordingly.

8.2 Algorithm Design

8.2.1. Artificial Neural Network

Input : Training Rules Tr[], Test Instances Ts[], Threshold T.
Output : Weight w=0.0
Step 1 : Read each test instance from (TsInstnace from Ts)
𝑛𝑛
Step 2 : 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 = ∑ {𝐴𝐴𝐴𝐴…𝐴𝐴𝐴𝐴}
𝑘𝑘=0

Step 3 : Read each train instance from (TrInstnace from Tr)

𝑛𝑛
Step 4 : 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 = ∑ {𝐴𝐴𝐴𝐴……𝐴𝐴𝐴𝐴}
𝑗𝑗=0

Step 5 : w = WeightCalc(TsIns, TrIns)

Step 6 : if (w >= T)

Step 7 : Forward feed layer to input layer for feedback FeedLayer[] {Tsf,w}

Step 8 : optimized feed layer weight, Cweigt FeedLayer[0]

Step 9 : Return Cweight

8.2.2. Convolutional Neural Network (CNN)

Training

Input: Training dataset TrainData[], Various activation functions[], Threshold Th

Output: Extracted Features Feature_set[] for completed trained module.
Step 1: Set input block of data d[], activation function, epoch size,
Step 2 : Features.pkl ExtractFeatures(d[])

Step 3 : Feature_set[] optimized(Features.pkl)

Step 4 : Return Feature_set[]

JSPM NTC, Department of Computer Engineering 2021-22

31
Testing
Input: Test Dataset which contains various test instances TestDBLits [], Train
dataset which is built by training phase TrainDBLits[] , Threshold Th.
Output: HashMap <class_label, SimilarityWeight> all instances which weight
violates the threshold score.

Step 1: For each read each test instances using below equation
𝑛𝑛
𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡(𝑚𝑚) = ∑ ( 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓[𝐴𝐴[𝑖𝑖]…….. 𝐴𝐴[𝑛𝑛] 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 )
𝑚𝑚=1
Step 2 : extract each feature as a hot vector or input neuron from 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡(𝑚𝑚)
using the below equation.
𝑛𝑛
Extracted_FeatureSetx[t……n] = ∑ (𝑡𝑡) 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 (m)
𝑥𝑥=1
Extracted_FeatureSetx[t] contains the feature vector of respective domain
Step 3: For each read each train instances using below equation
𝑛𝑛
𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡(𝑚𝑚) = ∑ ( 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓[𝐴𝐴[𝑖𝑖]…….. 𝐴𝐴[𝑛𝑛] 𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇 )
𝑚𝑚=1
Step 4 : extract each feature as a hot vector or input neuron from 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡(𝑚𝑚)
using the below equation.
𝑛𝑛
Extracted_FeatureSetx[t……n] = ∑ (𝑡𝑡) 𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡𝑡 (m)
𝑥𝑥=1
Extracted_FeatureSetx[t] contains the feature vector of the respective domain.
Step 5 : Now map each test feature set to all respective training feature set
𝑛𝑛
𝑤𝑤𝑤𝑤𝑤𝑤𝑤𝑤ℎ𝑡𝑡 = 𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐𝑐 (𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹 || ∑ 𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹𝐹[𝑦𝑦])
𝑖𝑖=1
Step 6 : Return Weight

JSPM NTC, Department of Computer Engineering 2021-22

32
JSPM NTC, Department of Computer Engineering 2021-22

33
9.1 Data Flow Diagrams

9.1.1 DFD ZERO

Figure 9.1: DFD 0 Level

9.1.2 DFD MULTI LEVEL

Figure 9.2: DFD Multi Level

JSPM NTC, Department of Computer Engineering 2021-22

34
9.2 UML Diagrams

9.2.1 CLASS DIAGRAM

A class diagram is a type of static structure diagram that describes the structure of a
system by showing the system's classes, their attributes, operations (or methods), and
the relationships among objects.

Figure 9.3 : Class Diagram

9.2.2 SEQUENCE DIAGRAM

A sequence diagram is an interaction diagram that shows how processes

operate with one another and in what order. It is a construct of a Message
Sequence Chart. A sequence diagram shows object interactions arranged in
time sequence. It depicts the objects and classes involved in the scenario and
the sequence of messages exchanged between the objects needed to carry out
the functionality of the scenario. Sequence diagrams are typically associated
with use case realizations in the Logical View of the system under
development. Sequence diagrams are sometimes called event diagrams, event
scenarios.

JSPM NTC, Department of Computer Engineering 2021-22

35
 Figure 9.4 : Sequence Diagram

9.2.3 USE CASE DIAGRAM

A use case diagram at its simplest is a representation of a user's interaction with the
system that shows the relationship between the user and the different use cases in
which the user is involved. A use case diagram can identify the different types of users
of a system and the different use cases and will often be accompanied by other types of
diagrams as well.

JSPM NTC, Department of Computer Engineering 2021-22

36
 Figure 9.5 : Use Case diagram

9.2.4 ACTIVITY DIAGRAM

Activity diagram is a graphical representation of workflows of stepwise
activities and actions with support for choice, iteration and concurrency. In the
Unified Modeling Language, activity diagrams are intended to model both
computational and organizational processes (i.e. workflows). Activity diagrams
show the overall flow of control.

JSPM NTC, Department of Computer Engineering 2021-22

37
 Figure 9.6 : Activity Diagram

9.2.5 COMPONENT DIAGRAM

Component diagram is a special kind of diagram in UML. Component diagram does
not describe the functionality of the system, but it describes the components used to
make those functionalities.

JSPM NTC, Department of Computer Engineering 2021-22

38
 Figure 9.7 : Component Diagram

JSPM NTC, Department of Computer Engineering 2021-22

39
JSPM NTC, Department of Computer Engineering 2021-22

40
10.1 Results

The results of both neural network architectures, specifically, the ANN, are presented
and discussed in this section. After pre-processing, the ECG signals were fed into both
networks and, in the case of the CNN, after performing the filtrations and
transformation. The findings show how much a CCN may enhance a classification task
when compared to a normal feed forward neural net. The performance of the nets was
evaluated by repeating the whole training procedure ten times and evaluating after
each time. In this way, we may evaluate the mean value of each neural network's
ultimate performance.

Figure 10.1: The error of the training set of the artificial neural network

On the prediction set, after ten cycles of full training, the neural net had an average
accuracy of 86.4 percent for appropriate music selection. Figure 2 (Left Structure)
shows the accuracy of both the training and prediction sets in each run, with the
prediction set achieving the highest accuracy of 89.3 percent. Figures 2 and 3 show the
training set MSE and accuracy, respectively, across the 220 training epoc of the
highest accuracy run. The neural network had been learning at a decent pace until the
final cycles, when it stopped learning.

JSPM NTC, Department of Computer Engineering 2021-22

41
 Figure 10.2: Accuracy of the training set of the artificial neural network.

Two artificial neural network topologies for ECG signal categorization were designed
and evaluated. For the ANN, input signals were supplied in time series format, while
for the Feed Forward ANN, input signals were fed in picture format. The first
technique clearly has difficulty generalizing what it has learned; however, the second
method generalized rather well, with an accuracy of 99.0 percent in the best run.

Figure 10.3: User Interface & Result

JSPM NTC, Department of Computer Engineering 2021-22

42
JSPM NTC, Department of Computer Engineering 2021-22

43
11.1 Conclusion
The Emotion detection will be done by using an EEG headset; the system
demonstrates the music and movie will be recommended according to the emotion of
the handicap people. Music and movies provide a soothing effect on human
temperament because of these, it will also reduce the stress level of the handicap
people. This study examines how single and machine learning applied classification
methods for taking the windowed data from four points on the scalp and quantifies that
data into an emotional representation of what was felt by the respondent at the time.
The comparisons combine that a low resolution, commonly produced EEG headband
can be efficient in categorizing the psychological response of a participant. The
possibilities for this are considerable product classification techniques with functional
value for the systems to help real-world decision taking. Emotional reactions states
should enhance engagement, especially for programs of mental health that contribute
to overall evaluation of the problems and how to solve such issues.

11.2 Future Work

As future work, we will consider different combinations of sensors that handle the
failures of wearable sensors and additional sensor usage to increase performance of the
emotion detection.

JSPM NTC, Department of Computer Engineering 2021-22

44
References
[1] Wei Tao, Chang Li, Rencheng Song, Juan Cheng, Yu Liu, Feng Wan, Xun Chen.
"EEG-based Emotion Recognition via Channel-wise Attention and Self Attention."
"Transactions on Affective Computing." IEEE 2020: 3025777
[2] MD. Rabiul Islam, Mohammed Ali Moni, MD. Milon Islam, MD. Rashed-Al-
Mahfuz, MD. Saiful Islam, MD.Kamrul Hasan, MD Sabir Hossain, Mohiuddin
Ahmad, Shahadat Uddin, AKM Azad, Salim A. Alyami, MD. Atiqur Rahman Ahad,
and Pietro Leo. "Emotion Recognition From EEG Signal Focusing on Deep Learning
and Shallow Learning Techniques." IEEE Access 2021: Vol 9: 94601-94624
[3] Shih-Hsiung Lee, Tzu-Yu Chen, Yu-Ting Hsien, and Lin-Roung Cao. "A music
recommendation system for depression therapy based on EEG." 2020 IEEE
International Conference on Consumer Electronics-Taiwan (ICCE-Taiwan). IEEE,
2020.
[4] Esha Dutta, Ananya Bothra, Theodora Chaspari, Thomas Ioerger, Bobak J.
Mortazavi. "Reinforcement Learning using EEG signals for Therapeutic Use of Music
in Emotion Management." 2020 IEEE: 5553-5556
[5] Meetkumar Patel, Dr. Sharmishta Desai. "Entertainment suggestion for paralyzed
and bed rest patient using EEG Signals." International Engineering Research Journal
(IERJ) 2020: 327-330.
[6] Santamaria-Granados, Luz, Juan Francisco Mendoza-Moreno, and
Gustavo Ramirez-Gonzalez. "Tourist recommender systems based on emotion
recognition—a scientometric review." Future Internet 13.1 (2020): 2.
[7] Cunha, Joana, et al. "The Effect of Music on Brain Activity an Emotional
State." Engineering Proceedings 7.1 (2021): 19.
[8] Xu, Baoguo, et al. "Continuous Hybrid BCI Control for Robotic Arm Using
Noninvasive Electroencephalogram, Computer Vision, and Eye Tracking."
Mathematics 10.4 (2022): 618.
[9] Jaafar, Sirwan Tofiq, and Mokhtar Mohammadi. "Epileptic seizure detection using
deep learning approach." UHD Journal of Science and Technology 3.2 (2019): 41-50.
[10] Kang, Dongwann, and Sanghyun Seo. "Personalized smart home audio system
with automatic music selection based on emotion." Multimedia Tools and Applications
78.3 (2019): 3267-3276.
[11] Dellaert, F.; Polzin, T.; Waibel, A. Recognizing emotion in speech. In Proceeding
of the Fourth International Conference on Spoken Language Processing, ICSLP’96,
Philadelphia, PA, USA, 3–6 October 1996; Volume 3, pp. 1970–1973.
[12] Mustaqeem; Kwon, S. A CNN-Assisted Enhanced Audio Signal Processing for
Speech Emotion Recognition. Sensors 2020, 20, 183.
[13] Zhao, J.; Mao, X.; Chen, L. Speech emotion recognition using deep 1D & 2D
CNN LSTM networks. Biomed. Signal Process. Control. 2019, 47, 312–323.
[14] Kalsum, T.; Anwar, S.M.; Majid, M.; Khan, B.; Ali, S.M. Emotion recognition
from facial expressions using hybrid feature descriptors. IET Image Process. 2018, 12,
1004–1012.
[15] Qayyum, H.; Majid, M.; Anwar, S.M.; Khan, B. Facial Expression
Recognition Using Stationary Wavelet Transform Features. Math. Probl. Eng. 2017,
2017.

JSPM NTC, Department of Computer Engineering 2021-22

45