Personalizing the listening experience
9 likes•2,333 views
The document outlines Spotify's mission to personalize the listening experience by effectively matching fans with artists through tailored playlists, podcasts, and album recommendations. It discusses the use of machine learning algorithms, such as the BART model, to optimize user interaction based on various intents and success metrics, while also balancing relevance and diversity in recommendations. Key elements include co-clustering user and playlist data to enhance personalized experiences and improve user satisfaction across different listening intents.
Personalizing the listening experience
- 1. Personalizing the listening experience Mounia Lalmas Netflix HQ, Los Gatos, 10 May 2019
- 2. What we do at Spotify
- 3. Spotify’s mission is to unlock the potential of human creativity — by giving a million creative artists the opportunity to live off their art and billions of fans the opportunity to enjoy and be inspired by it.
- 4. Our team mission: Match fans and artists in a personal and relevant way. ARTISTS FANS
- 5. What does it mean to match fans and artists in a personal and relevant way?
- 6. songs playlists podcasts ... catalog Home users What does it mean to match fans and artists in a personal and relevant way?Artists Fans
- 7. Home Home is the default screen of the mobile app for all Spotify users worldwide. It surfaces the best of what Spotify has to offer, for every situation, personalized playlists, new releases, old favorites, and undiscovered gems. Help users find something they are going to enjoy listening to, quickly.
- 8. Qualitative&quantitativeresearch KPIs&businessmetrics Algorithms Training & datasets Optimizationmetrics Evaluation offline & online Measurement & signals Features (item) Features (user) Features (context) Bias Making machine learning work for Spotify Home
- 9. Qualitative&quantitativeresearch KPIs&businessmetrics Algorithms Training & datasets Optimizationmetrics Evaluation offline & online Measurement & signals Features (item) Features (user) Features (context) Bias Making machine learning work for Spotify Home … personalizing Home with respect to success, intent & diversity 3. intent 1. Home 2.success 4. diversity
- 11. Streaming UserBaRT Explore, Exploit, Explain: Personalizing Explainable Recommendations with Bandits. J McInerney, B Lacker, S Hansen, K Higley, H.Bouchard, A Gruson & R Mehrotra. RecSys 2018. BaRT: Machine learning algorithm for Spotify Home
- 12. BaRT (Bandits for Recommendations as Treatments) how to rank playlists (cards) in each shelf first, and then how to rank the shelves?
- 13. Explore vs Exploit Flip a coin with given probability of tail If head, pick best card in M according to predicted reward r → EXPLOIT If tail, pick card from M at random → EXPLORE BaRT: Multi-armed bandit algorithm for Home https://hackernoon.com/reinforcement-learning-part-2-152fb510cc54
- 14. Success is captured by the reward function Reward Binarised Streaming Time BaRT UserStreaming success is when user streams the playlist for at least 30s
- 15. Personalizing with respect to success
- 16. consumption time of a sleep playlist is longer than average playlist consumption time jazz listeners consume Jazz and other playlists for longer period than average users Is success the same for all playlists and users?
- 17. one reward function for all users and all playlists success independent of user and playlist one reward function per user x playlist success depends on user and playlist too granular, sparse, noisy, costly to generate & maintain one reward function per group of users x playlists success depends on group of users listening to group of playlists Personalizing the reward function
- 18. Co-clustering using streaming time users playlists user groups playlist groups Dhillon, Mallela & Modha, "Information-theoretic co-clustering”, KDD 2003. group = cluster group of user x playlist = co-cluster
- 19. user type playlist type
- 20. Reward function (success) per co-cluster continuousadditivemean
- 21. co-clustering performs better than baseline, user-only & playlist-only clustering global distribution-aware reward (no cluster) better than baseline mean better than additive itself better than continuous Experiments … expected stream rate using distribution-aware reward functions
- 22. Towards personalizing with respect to success Deriving User- and Content-specific Rewards for Contextual Bandits. P Dragone, R Mehrotra & M Lalmas. WWW 2019. co-clustering and distribution-aware reward functions highlight importance of affinity type features (type of content x type of user) over generic features (age/day)
- 23. Personalizing with respect to intent
- 24. User intents on Home Passively Listening - quickly access playlists or saved music (2) - play music matching mood or activity (4) - find music to play in background (6) Actively Engaging - discover new music to listen to now (3) - to find X (5) - save new music or follow new playlists for later (7) - explore artists or albums more deeply (8) Other Home is default screen (1) Mixed-methods approach: user interview & in-app survey & Bayesian non-parametric clustering intent important to interpret user interaction Jointly Leveraging Intent and Interaction Signals to Predict User Satisfaction with Slate Recommendations. R Mehrotra, M Lalmas, D Kenney, T Lim-Meng & G Hashemian. WWW 2019.
- 25. Varying degree of satisfaction across intents Passively Listening (2) quickly access playlists or saved music (4) play music matching mood or activity (6) play music to play in background Actively Engaging (3) discover new music to listen to now (5) to find X (7) save new music or follow new playlists for later (8) explore artists or albums more deeply Other (1) Home is default screen some intents easier to satisfy intent 6 → lean-back mood intents 3 & 7 → discovery mood
- 26. Modeling user intents for Home Global Model: single model across all-intents ● ignores intent-level variations in interaction data ● inadvertently suppresses important variations Per-intent Model: separate for each intent ● using local intent specific information only ● ignores information & insights from other intent Multi-level model: shared learning across intents ● account for intent level grouping ● allows for variability in interaction behavior across intents ● facilitate information sharing across different intents 20% improvement in satisfaction prediction over global model shared learning across intents beneficial
- 27. different interaction important for different intents (2) quickly access playlists or saved music → time to success and dwell time (8) explore artists or albums more deeply → building relationships (save or download tracks) Towards personalizing with respect to user intents
- 28. Personalizing with respect to diversity
- 29. Relevance playlist is relevant to user if closely resembles user interest profile (embedding based representation of users & tracks) Satisfaction playlist satisfies user if user interacts with it (stream for more than 30s) Diversity playlist is diverse if contains tracks from artists belonging to different groups (popularity) Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems. R Mehrotra, J McInerney, H Bouchard, M Lalmas & F Diaz. CIKM 2018. One user intent on Home is “discover new music” Spotify’s mission is to unlock the potential of human creativity — by giving a million creative artists the opportunity to live off their art and billions of fans the opportunity to enjoy and be inspired by it.
- 30. very few playlists have both high relevance & high diversity diversity relevance The interplay between relevance, diversity & satisfaction optimizing for relevance vs optimizing for diversity and its impact on satisfaction … but … but … but Normalized Diversity Estimate Match fans and artists in a personal and relevant way.
- 31. % loss in diversity % loss in relevance % gain in satisfaction optimizing relevance (β=1) 69.1 0 0 Optimizing diversity (β=0) 0 57.7 -32.2 trade-off relevance & diversity (β =0.7) 42.7 9.8 -10.2 guaranteed relevance (rel >= 0.7) 51.7 7.8 4.4 diversity affinity aware 15 21.2 12.1 simple interpolation from β = 0 to 1 leads to high satisfaction loss or high diversity loss guaranteed relevance hurts diversity diversity affinity aware provides the best overall trade-off → personalizing diversity Towards personalizing with respect to diversity
- 32. Some final words
- 33. Qualitative&quantitativeresearch KPIs&businessmetrics Algorithms Training & datasets Optimizationmetrics Evaluation offline & online Measurement & signals Features (item) Features (user) Features (context) Bias Personalizing the listening experience … making machine learning work 3. intent 1. Home 2.success 4. diversity
- 34. Qualitative&quantitativeresearch KPIs&businessmetrics Algorithms Training & datasets Optimizationmetrics Evaluation offline & online Measurement & signals Features (item) Features (user) Features (context) Bias Personalizing the listening experience … making machine learning work 3. type of moment 1. Home 2.typeofcontentxuser 4. discovery
- 35. Thank you!