Fashion Recommender System Using CNN and
ResNet50
April 29, 2025
Sevan Kumar
Indian Institute of Technology, Kharagpur
Table of Contents
1. Introduction
2. Literature Review
3. Methodology
4. Results
5. Challenges Faced
6. Future Work
7. Conclusion
1.1 Background, and Motivation
• Traditional fashion recommendation relies on human stylists.
• E-commerce growth demands automated, intelligent systems.
• Visual similarity is essential in online shopping.
• CNNs enable hierarchical visual feature learning.
• Transfer learning (e.g., ResNet50) boosts efficiency.
1.2 Project Objective
Goal
Allow users to upload an image of a fashion item—such as a shirt,
jeans, saree, wallet, or shoes—and receive five visually similar rec-
ommendations based on deep learning features.
Approach:
1. Use ResNet50 for feature extraction.
2. Use KNN to retrieve similar items.
3. Deploy via Streamlit for user interaction.
2.1 Fashion Recommender Systems
• Collaborative Filtering: Based on user or item similarity.
• Content-Based Filtering: Uses product features for
recommendations.
• Hand-Crafted Feature Limits: Miss high-level visual
semantics.
• Deep Learning: CNNs learn rich visual representations.
• Visual Recommenders: Combine CNNs with ranking or metric
learning.
2.2 CNNs in Fashion
• CNNs excel in image recognition tasks.
• ResNet is a deep CNN capable of high-level abstraction.
• Suitable for similarity-based fashion matching.
2.3 Transfer Learning
• Pre-trained Models: Use of models like ResNet50 trained on
ImageNet.
• Knowledge Transfer: Generic features transfer well to fashion
domain.
• Fine-Tuning Benefits: Improves task-specific performance.
• Data Efficiency: Reduces need for large labeled datasets.
• Rapid Prototyping: Speeds up development using ready
models.
2.4 Nearest Neighbor Search
• Feature vectors compared using Euclidean distance.
• KNN finds the most similar items efficiently.
• Scales with indexing techniques.
2.5 System Limitations and Solutions
• Scalability: Use Approximate Nearest Neighbors (ANN).
• Diversity: Recommend varied styles using filters.
• Personalization: Integrate user behavior data.
3. Methodology
Steps:
1. Import ResNet50 model.
2. Preprocess and extract features.
3. Save feature embeddings.
4. Use KNN to recommend top 5 items.
Tools: ResNet50, KNN, Streamlit
3.1 Dataset Overview
• The dataset is sourced from a large fashion e-commerce
platform.
• Includes both image files and a metadata file styles.csv.
• Contains approximately 44,441 high-resolution labeled
images.
• Each product has a unique Product ID mapping images to
metadata.
3.2 Feature Extraction
• Use ResNet50 (pre-trained on ImageNet) as a feature extractor.
• Apply Global Max Pooling to the final convolutional maps.
• Convert each image to a compact feature vector.
• Normalize vectors and save for similarity comparisons.
3.3 Recommendation Generation
• KNN used with Euclidean distance.
• Compare input vector with dataset vectors.
• Top 5 closest vectors returned as recommendations.
3.4 Streamlit Web Interface
• Interactive UI for image upload and results.
• Real-time recommendation rendering.
• User-friendly, minimal code via Streamlit.
4.1 Evaluation Strategy
• Visual inspection of recommended items.
• Evaluation based on similarity in:
◦ Color
◦ Texture
◦ Style
4.2 Sample Input and Recommendations
Input Image: Sample watch
Recommended items
5.1 Technical Hurdles
• Image resizing and preprocessing.
• KNN search inefficiencies with larger datasets.
• Lack of diversity in visual-based recommendations.
6.1 Enhancements
• Incorporate user preferences for personalization.
• Use ANN for faster similarity search.
• Train on diverse fashion datasets.
• Categorize recommendations (e.g., casual, formal).
7.1 Summary
• Successfully built an image-based recommender system.
• Leveraged ResNet50 and KNN.
• Delivered real-time results via Streamlit.
• Robust foundation for scalable, personalized
recommendations.
7.2 Takeaways
Key Insights
• Deep learning simplifies feature extraction.
• Visual similarity is essential in fashion matching.
• Recommender systems benefit from hybrid models.
