SYNOPSIS

MOOD BASED MUSIC RECOMMENDATION SYSTEM

B. E. [Information Technology]
of
Savitribai Phule Pune University

By

Shelke Akshay Sharad

Sinare Rajeshree Tulshiram
Sanap Madhuri Shivaji
Rasal Rutuja Devidas

Under the guidance of

Dr. Rupali .S. Khule
 Matoshri Education Society’s
MATOSHRI COLLEGE OF ENGINEERING AND RESEARCH CENTRE,
Near Odha Village, Nashik – Aurangabad Highway Nashik – 422 105, (M.S.), India

Academic Year- 2024-25

Synopsis of Proposed Project for the degree of B.E. (Information

Technology)

1 Name of course Project Stage-I ( 414448)

2
Project Group ID 6
3 Title of Project
Mood Based Music Recommendation
System

4 Project Domain Artificial Intelligence And Data Science

5 Name of Guide Dr. Rupali S. Khule

Shelke Akshay:
Sinare Rajeshree:
Sanap Madhuri:
Rasal Rutuja:

[Student’s name and sign] [Sign of guide]

Prof. N. L. Bhale
Head, Information Technology Department
 SYNOPSIS OF THE WORK

1. Relevance and Introduction:

This project presents a novel music recommendation system that automatically selects and
plays music based on real-time facial emotion recognition. Utilizing a Convolutional Neural
Network (CNN), the system captures the user's facial expressions through a camera,
processes the images to detect emotions such as happiness, sadness, anger, surprise, and
neutrality, and subsequently generates a playlist tailored to the detected mood. The system
aims to enhance user experience by reducing the manual effort of song selection, offering an
emotionally adaptive music experience. The proposed system is evaluated using the
FER2013 dataset, achieving significant accuracy in emotion detection, and demonstrates a
practical application of artificial intelligence in the entertainment domain. Music has a
profound impact on human emotions, influencing mood and behavior in various contexts. In
recent years, the integration of emotion recognition technologies with music recommendation
systems has gained significant attention, driven by the desire to create more personalized and
emotionally responsive experiences. Traditional music players require manual input for song
selection, which can be time-consuming and may not always match the user's current
emotional state. This project aims to address this gap by developing a music recommendation
system that leverages facial emotion recognition to automatically curate playlists based on
the user's mood. By employing a Convolutional Neural Network (CNN) for accurate emotion
detection, the system captures real-time facial expressions and maps them to specific music
genres that correspond to the detected emotions. The ultimate goal is to provide users with a
seamless and intuitive music experience that adapts to their emotional needs, enhancing their
overall well-being.

2. Literature review:
We propose a new approach for playing music automatically using facial emotion. Most of
the existing approaches involve playing music manually, using wearable computing devices,
or classifying based on audio features. Instead, we propose to change the manual sorting and
playing. We have used a Convolutional Neural Network for emotion detection. For music
recommendations, Pygame & Tkinter are used. Our proposed sys-tem tends to reduce the
computational time involved in obtaining the results and the overall cost of the designed
system, thereby increasing the system’s overall accuracy. Testing of the system is done on the
FER2013 dataset. Facial expressions are captured using an inbuilt camera. Feature extraction
is performed on input face images to detect emotions such as happy, angry, sad, surprise, and
neutral. Automatically music playlist is generated by identifying the current emotion of the
user. It yields better performance in terms of computational time, as compared to the
algorithm in the existing literature[1].
One of the most significant fields in the man–machine interface is emotion recognition using
facial expressions. Some of the challenges in the emotion recognition area are facial
accessories, non-uniform illuminations, pose variations, etc. Emotion detection using
conventional approaches having the drawback of mutual optimization of feature extraction
and classification. To overcome this problem, researchers are showing more attention toward
deep learning techniques. Nowadays, deep-learning approaches are playing a major role in
classification tasks. This paper deals with emotion recognition by using transfer learning
approaches. In this work pre-trained networks of Resnet50, vgg19, Inception V3, and Mobile
Net are used. The fully connected layers of the pre-trained ConvNets are eliminated, and we
add our fully connected layers that are suitable for the number of instructions in our task.
Finally, the newly added layers are only trainable to update the weights. The experiment was
conducted by using the CK + database and achieved an average accuracy of 96% for emotion
detection problems [2].

In the current era of media and technology, music information retrieval techniques have made
progress in the past years but the music recommendation system is still at a rudimentary
stage. However personalized music recommendation is quotidian, but recommending songs
based on emotions is still an uphill battle. Music has greatly influenced the human brain and
helps dispense an exhilarating and frivolous state of mind because it helps us work more
effectively. Recommending songs based on emotions will comfort the listener by suggesting
music in keeping with the listeners' pervading mental and physical state. Hence, Natural
Language Processing and Deep Learning technologies made it possible for machines to read
and interpret emotions through texts by recognizing patterns and finding correlations. In this
paper, various deep learning models such as Long Short-Term Memory (LSTM),
Convolution Neural network (CNN), CNN-LSTM, and LSTM CNN Architectures were
collated for detecting emotions such as angry, happy, love, and sad, the best model was
integrated into the application. To enhance the application, a CNN model was used to detect
emotions through facial expressions. The application takes text input from the user or a facial
expression input. Depending upon the emotion detected, it recommends the user songs and
playlists[3].
It is often confusing for a person to decide which music he/she have to listen from a massive
collection of existing options. There have been several suggestion frameworks available for
issues like music, dining, and shopping depending upon the mood of user. The main objective
of our music recommendation system is to provide suggestions to the users that fit the user's
preferences. The analysis of the facial expression/user emotion may lead to understanding
the current emotional or mental state of the user. Music and videos are one region where there
is a significant chance to prescribe abundant choices to clients in light of their inclinations
and also recorded information. It is well known that humans make use of facial expressions
to express more clearly what they want to say and the context in which they meant their
words. More than 60 percent of the users believe that at a certain point of time the number of
songs present in their songs library is so large that they are unable to figure out the song
which they have to play. By developing a recommendation system, it could assist a user to
make a decision regarding which music one should listen to helping the user to reduce his/her
stress levels. The user would not have to waste any time in searching or to look up for songs
and the best track matching the user's mood is detected, and songs would be shown to the
user according to his/her mood. The image of the user is captured with the help of a webcam.
The user's picture is taken and then as per the mood/emotion of the user an appropriate song
from the playlist of the user is shown matching the user's requirement[4].

Learning effectiveness is normally analyzed by data collection through tests or

questionnaires. However, instant feedback is usually not available. Learners’ facial emotion
and learning motivation has a positive relationship. Therefore, the system identifying
learners’ facial emotions can provide feedback that teachers can understand students’ learning
situation and provide help or improve teaching strategy. Studies have found that
convolutional neural networks provide a good performance in basic facial emotion
recognition. Convolutional neural networks do not require manual design features like
traditional machine learning, they automatically learn the necessary features of the entire
image. This article improves the FaceLiveNet network with low and high accuracy in basic
emotion recognition, and proposes the framework of Dense_FaceLiveNet. We use
Dense_FaceLiveNet for two-phases of transfer learning. First, from the relatively simple data
JAFFE and KDEF basic emotion recognition model transferring to the FER2013 basic
emotion dataset and obtained an accuracy of 70.02%. Secondly, using the FER2013 basic
emotion recognition model transferring to learning emotion recognition model, the test
accuracy rate is as high as 91.93%, which is 12.9% higher than the accuracy rate of 79.03%
without using the transfer learning model, which proves that the use of transfer learning can
effectively improve the recognition accuracy of learning emotion recognition model. In
addition, in order to test the generalization ability of the Learning Emotion Recognition
Model, videos recorded by students from a national university in Taiwan during class learning
were used as test data. The original database of learning emotions did not consider that
students would have exceptions such as over eyebrows, eyes closed and hand hold the chin
etc. To improve this situation, after adding the learning emotion database to the images of the
exceptions mentioned above, the model was rebuilt, and the recognition accuracy rate of the
model was 92.42%. By comparing the output of maps, the rebuilt model does have the
characteristics of success in learning images such as eyebrows, chins, and eyes closed.
Furthermore, after combining all the students’ image data with the original learning emotion
database, the model was rebuilt and obtained the accuracy rate reached 84.59%. The result
proves that the Learning Emotion Recognition Model can achieve high recognition accuracy
by processing the unlearned image through transfer learning. The main contribution is to
design two-phase transfer learning for establishing the learning emotion recognition model
and overcome the problem for small amounts of learning emotion data. Our experiment
results have shown the performance improvement of two-phase transfer learning[5].

3. Motivation :
The motivation for developing a music recommendation system based on mood detection
stems from a few key areas of interest and practical needs:

1.Personalized Listening Experience .

2. Advancement of Artificial Intelligence in Entertainment .

3.Improvement over traditional recommendation system.

4.Mental health and emotional support .

4. Problem statement:
To design a system that can accurately recommend music based on the real-time detection of
a user's mood.

5. Objectives:
• To study the literature to understand the challenges in developing and automatic mood
recommendation system.
• To study various techniques to enhance the performance parameters of an automatic mood
recommendation system.
• To design a system that can accurately recommend music based on the real-time detection
of a user's mood.
• To Achieve High Accuracy in Emotion Detection.
• To validate the results of the proposed system.

6. Scope of Project
The “Mood Based Music Recommendation” introduces an innovative approach to music
recommendation, merging advanced image processing and mood recognition with
personalized music selection. Utilizing a Convolutional Neural Network (CNN), the project
focuses on identifying a listener’s current emotional state — Angry, Happy, Sad, or Calm —
through analysing facial expressions in images or videos. This mood recognition aspect is
critical as it forms the basis for a tailored music recommendation system. The project’s unique
feature lies in its ability to align music recommendations with the detected emotional state,
enhancing the relevance and impact of the suggested songs. In the second phase of the project,
songs are categorized into the emotional states based on their intrinsic features, employing
content-based filtering for recommendations. This method ensures that the music selection is
not only aligned with the user’s preferences but also resonates with their current mood. The
system’s aim is to enrich the music listening experience, offering songs that are not just liked
but are emotionally fitting as well. By integrating emotional intelligence into music
recommendation, the project intends to create responsive and empathetic AI systems, ensuring
a more personalized and emotionally engaging user experience.
7. List of required hardware, software, or other equipment for
executing the project:
Software Requirements:

1. Operating System: Windows 10/11, macOS, or Linux.

2. Programming Languages: Python 3.6 or higher.

3. IDE/Code Editor: PyCharm, Visual Studio Code, or Jupyter Notebook.

4. Libraries and Frameworks:

 ○ TensorFlow or Keras for implementing Convolutional Neural Networks.

○ OpenCV for image processing and face detection.

○ NumPy and Pandas for data manipulation.

○ SciPy for scientific computations.

5. Database: SQLite, MySQL, or PostgreSQL for storing user preferences and music data.

6. API/Services: Spotify API or any music streaming service API for song recommendations.

Hardware Requirements:
1. Processor: Intel Core i5 or equivalent for smooth performance during model training and
testing.
2. RAM: Minimum 8GB RAM (16GB recommended) to handle real-time image processing
and model predictions.
3. Storage: At least 256GB of SSD for fast read/write operations, especially for handling
large datasets.
4. Graphics Card: NVIDIA GPU (e.g., GTX 1060 or higher) if using GPU acceleration for
training deep learning models.

5. Webcam: A high-definition webcam for capturing real-time facial expressions.

6. Camera Stand: For stable and accurate face capturing.

8. Proposed system and expected outcomes:

Proposed System :
The objective of this research is to understand the effectiveness of transfer learning models
on the prediction of emotions from facial expression images and to assess the efficiency and
accuracy of personalized and appropriate music recommendations according to the predicted
emotions. For this, the proposed architecture is broadly segregated into 4 sections. The first
section involves using the publicly available FER2013 dataset (Verma, 2018) and extending
the dataset by adding own images into each of the 7 categories from Google. The second
section involves preprocessing the images by using image augmentation techniques so that
the images are processed as per the model requirement. The third section involves fine-tuning
the pre trained transfer learning models and training on the processed images and predicting
emotions. The fourth section involves using the model with the highest accuracy for emotion
prediction and building a music recommendation system that would suggest appropriate
 music according to the predicted emotions. The proposed system architecture is shown in
Fig. 1. The idea is to use “deep neural networks” to select the optimum feature abstraction.

Fig 1: proposed system architecture.

Expected Outcomes :
Accurate Mood Detection:
The system should be able to accurately detect the user’s mood using inputs such as facial
expressions, voice, or text. The detection system can analyze emotions like happiness, sadness,
anger, or relaxation and identify the corresponding mood.
Personalized Music Recommendations:
Based on the detected mood, the system will suggest music that fits the emotional state of the
user. For instance, calming music for relaxation, upbeat tracks for happiness, or slower tempos
for sad moods.
Improved User Experience:
The system aims to enhance the music streaming experience by offering emotionally intelligent
recommendations. Users would likely find this more engaging compared to traditional
recommendation systems based on genres or playlists.
Dynamic Music Adjustments:
The system could adapt in real-time as the user’s mood shifts, continuously updating the music
recommendations, ensuring that the music matches their mood throughout the session.
Enhanced Mental Well-being:
By offering mood-enhancing or mood-supporting music, the system could help users manage
stress, anxiety, or even improve focus and productivity, contributing to overall mental well-being.
Data-Driven Insights:
 The system can provide data on how music preferences correlate with emotions, which can be
useful for further studies or improvements in recommendation algorithms.
Integration with Music Streaming Platforms:
The system could be integrated with popular platforms like Spotify or Apple Music, allowing
users to receive recommendations seamlessly within their existing music libraries.
Scalability and Customization:
The final system should be scalable, allowing for customization based on user preferences,
cultural background, or even activity (e.g., workout, studying).

9. Architecture and initial phase of design (DFD)

Figure 2: Architecture initial phase of design

10. References:

1. Aggarwal, K., Mijwil, M. M., Al-Mistarehi, A. H., Alomari, S., Gök, M., Alaabdin, A. M. Z.,
& Abdulrhman, S. H. (2022). Has the future started? The current growth of artificial
intelligence, machine learning and deep learning. Iraqi Journal for Computer Science and
Mathematics, 3(1), 115-123. https://doi.org/10.52866/ijcsm.2022.01.01.013
2. Athavle, M., Mudale, D., Shrivastav, U., & Gupta, M. (2021). Music Recommendation Based
on Face Emotion Recognition. Journal of Informatics Electrical and Electronics Engineering
(JIEEE), 2(2), 1-11. https://doi.org/10.54060/JIEEE/002.02.018
3. Bhattarai, B., & Lee, J. (2019). Automatic music mood detection using transfer learning and
multilayer perceptron. International Journal of Fuzzy Logic and Intelligent Systems, 19(2),
88-96. https://doi.org/10.5391/IJFIS.2019.19.2.88
4. Chowdary, M. K., Nguyen, T. N., & Hemanth, D. J. (2021). Deep learning-based facial
emotion recognition for human computer interaction applications. Neural Computing and
Applications, 1-18. https://doi.org/10.1007/s00521-021-06012-8
5. Chung, T. S., Rust, R. T., & Wedel, M. (2009). My mobile music: An adaptive personalization
system for digital audio players. Marketing Science, 28(1), 52-68.
https://doi.org/10.1287/mksc.1080.0371
6. Florence, S. M., & Uma, M. (2020, August). Emotional detection and music recommendation
system based on user facial expression. In IOP Conference Series: Materials Science and
Engineering (Vol. 912, No. 6, p. 062007). IOP Publishing.
https://doi.org/10.1088/1757899X/912/6/062007
7. Hasib, K. M., Tanzim, A., Shin, J., Faruk, K. O., Al Mahmud, J., & Mridha, M. F. (2022).
BMNet-5: A novel approach of neural network to classify the genre of Bengali music based
on audio features. IEEE Access, 10, 108545-108563.
https://ieeexplore.ieee.org/abstract/document/9916245/
8. Hung, J. C., Lin, K. C., & Lai, N. X. (2019). Recognizing learning emotion based on
convolutional neural networks and transfer learning. Applied Soft Computing, 84, 105724.
https://doi.org/10.1016/j.asoc.2019.105724
9. IFPI. (2018). Global Music Report - Annual state of the industry. Retrieved from IFPI.
https://www.ifpi.org/ifpi-global-music-report-2018/ [Accessed 15 April 2023].
10. James, H. I., Arnold, J. J. A., Ruban, J. M. M., Tamilarasan, M., & Saranya, R. (2019). Emotion
based music recommendation system. Emotion, 6(03).
11. Joshi, S., Jain, T., & Nair, N. (2021, July). Emotion based music recommendation system using
LSTM-CNN architecture. In 2021 12th International Conference on Computing
Communication and Networking Technologies (ICCCNT) (pp. 01-06). IEEE.
https://ieeexplore.ieee.org/abstract/document/9579813
12. Khanzada, A., Bai, C., & Celepcikay, F. T. (2020). Facial expression recognition with deep
learning. Ar Xiv preprint arXiv:2004.11823. https://doi.org/10.48550/arXiv.2004.11823
11.Base papers
1.Tital: A Study on Emotion Analysis and Music Recommendation Using Transfer Learning.
Authors: Krishna Kumar Singh and Payal Dembla.

2. Tital: Music recommendation system based on facial emotion gestures.

Authors: Ankit Maithani, Department of CSE, DBGI Dehradun, India