INTELLIGENT MOOD DETECTION AND MUSIC RECOMMENDATION

Disha Saraiya - LY B 1512102

Saad Ansari - LY A 1512062

Project Guide : Dr. Jagannath H. Nirmal

Department of Electronics Engineering

K. J. Somaiya College of Engineering, Mumbai-77 1
 CONTENTS
1) MOTIVATION
2) INTRODUCTION
3) METHODOLOGY
4) DATABASE
5) IMPLEMENTATION
6) RESULTS
7) ADDITIONAL WORK
8) FUTURE WORK
9) REFERENCES
2
1. MOTIVATION
● A shared interest in Music.

● Growth in the field of machine learning.

● The changing world of Advertisements.

● Future scope of the technology.

3
 2. INTRODUCTION
Need ●
●
An automated Music Player
Does not shuffle Music randomly

● Emotion based Music player - Viola Jones Algorithm, Haar

Implementation ●
Classifier, FisherFaces algorithm and CK+ Database
Weather based Music player - OpenWeatherMap API,
Database

Result A Python based Project that can be turned into a website

or cross platform application.

4
SURVEY

5
2.1 EMOTION BASED MUSIC PLAYER
4. ACCEPT 1. RECOGNIZE
FEEDBACK AND HUMAN
IMPROVISE EMOTIONS

INTELLIGENT MOOD
DETECTION AND
MUSIC
RECOMMENDATION

3. RECOMMEND 2. CLASSIFY
SUITABLE MUSIC EMOTIONS

6
3. METHODOLOGY

7
4. DATABASE
4. 1 PREPROCESSING STEPS
Data Cleaning Integration Transformation Reduction Discretization

Data is cleansed Data with different Data is normalized, This step aims to This involves the
through processes representation are put aggregated and present a reduced reduction of a number
such as filling in together and conflicts generalized. representation of the of values of a
missing values, within the data are data in a continuous
smoothing the noisy resolved. data warehouse. attribute by dividing
data or resolving the the range of attribute
inconsistencies in the intervals.
data.

8
4.2 PREPROCESSING TECHNIQUES
● Uniform aspect ratio

● Image scaling

● Normalizing image inputs

● Dimensionality reduction

9
4.3.1 DATABASE - 1 (Frontal Face images from CK+)

10
4.3.2 DATABASE - 2 (Preprocessed from CK+)

11
4.3.3 DATABASE - 3 (Collected from ‘--update’ mode)

12
4.3.4 DATABASE- 4 (Music)

13
5. IMPLEMENTATION
- Software used : Python 3.7
- Run and test for different emotions
- Output can be from YouTube/device
- Face Detection using Viola Jones
- Emotion Recognition using FisherFaces algorithm

14
 5.1 DETECTION ALGORITHMS

Fig 2. Haar Classifier

Fig 3. Local Binary Pattern Histogram (LBP)

Method
Fig 1. Haar Classifier v/s LBP
15
5.2 FACE DETECTION- VIOLA JONES ALGORITHM (HAAR)

Fig 1. Forming the integral image

Fig 2. Haar Features

Fig 3. Cascading Classifier 16

5.3 CLASSIFICATION : EMOTIONS

HAPPY ● WIDER LIPS INDICATING SMILE

● EYE REGION NARROW TO INDICATE SMILE
● WIDER JAWS

SAD ● NARROW LIPS AND WIDER EYES

● EYEBROWS DROOPING DOWNWARDS

ANGRY ● CONSTRICTED EYEBROWS

● PURSED LIPS TO INDICATE FROWN

NEUTRAL ● NATURAL POSITION OF A HUMAN FACE

17
5.5 EMOTION DETECTION - FISHERFACES RECOGNIZER

- Uses LDA (Linear

Discriminant Analysis)
- Extracts features from
individual images
- Illumination is an important
feature

Prepare dataset ---> Train recognizer ---> Test ----> Update dataset ---> Update Recognizer
18
5.6 FACE RECOGNITION AND EMOTION DETECTION

19
6. RESULTS

20
ANGRY

21
NEUTRAL

22
ERRORS IN DETECTION - NEUTRAL AS ANGRY

23
7. ADDITIONAL WORK

24
7.1 WEATHER BASED MUSIC PLAYER
- Plays music according to real-time weather

- Divided into five categories

- Extracting from an API

- Music extracted from YouTube

25
7.1.1 DATABASE - WEATHER API

26
7.1.2 DATABASE - SONGS

27
7.1.3 RESULTS - WEATHER BASED MUSIC PLAYER

28
29
7.2 MUSE - An Immersive Musical Experience
A sample frontend for a desktop application (Windows/Mac OS) that can implement
both, emotion based and weather based music player.

30
8. FUTURE WORK
Real world applications:
- Web and android
application
3
development
Test the
- Extending to
algorithm to
mental disorders
increase
accuracy as
2
much as possible
and reduce errors

Expand database
from online/user
images

31
9. REFERENCES
1. http://www.paulvangent.com/2016/04/01/emotion-recognition-with-python-opencv-and-a-face-
dataset/ - Emotion Recognition with Python and OpenCV (Paul Van Gent)
2. https://www.superdatascience.com/opencv-face-recognition/ - Face Recognition using Python and
OpenCV (Super Data Science Blogs)
3. https://en.wikipedia.org/wiki/Facial_recognition_system - Facial Recognition System (Wikipedia)
4. https://github.com/opencv/opencv/tree/master/data/haarcascades - Frontal face Haar Cascades
(GitHub)
5. https://towardsdatascience.com/how-to-build-a-simple-song-recommender-296fcbc8c85 - Building a
simple Song Recommendation System (Towards Data Science - Medium)
6. https://dev.to/kev_mcg/introduction-to-the-frontend-and-backendweb-development-fundamentals-
i2e - Introduction to Frontend and Backend Development Systems
7. https://electronjs.org/docs - Documentation for Electron
8. http://www.scholarpedia.org/article/Fisherfaces - Fisherfaces (Scholarpedia)
9. https://www.researchgate.net/publication/323471647_Emotion_Detection_using_Image_Processing
_in_Python - Emotion Detection using Image Processing and Python
32
REFERENCES - II
10. Emotion Based Music Player Using Facial Recognition - Prof. Vijaykumar R. Ghule, Abhijeet B. Benke,
Shubham S. Jadhav, Swapnil A. Joshi (COE,Pune)

11. Emotion Extraction and Recognition from Music - Fan Zhang and Hongyig Meng (Brunel University,
London)

12. Emotion Profile-Based Music Recommendation -Yu-Hao Chin, Szu-Hsien Lin, Chang-Hong Lin, Ernestasia
Siahaan, Aufaclav Frisky, and Jia-Ching Wang (National Central University, Taiwan, R.O.C.)

13. Mood Detector – On Using Machine Learning to Identify moods and Emotions - Alexandra Cernian,
Adriana Olteanu, Dorin Carstoiu, Cristina Mares (University Politehnica of Bucharest, Romania)

33
THANK YOU!

34