BigML Fall 2016 Release
1 like558 views
The BigML Fall 2016 release webinar introduces topic modeling, an unsupervised learning method for discovering hidden structures in unstructured text using Latent Dirichlet Allocation (LDA). It highlights the differences between text analysis and topic modeling, and demonstrates how to utilize topic models for various applications such as classification and recommendation. The webinar also discusses best practices for setting parameters and tuning models for effective topic discovery.
BigML Fall 2016 Release
- 1. BigML Fall 2016 Release Introducing Topic Models
- 2. BigML, Inc 2Fall Release Webinar - November 2016 Fall 2016 Release CHARLES PARKER, (VP Algorithms) Enter questions into chat box – we’ll answer some via chat; others at the end of the session https://bigml.com/releases ATAKAN CETINSOY, (VP Predictive Applications) Resources Moderator Speaker Contact [email protected] Twitter @bigmlcom Questions
- 4. BigML, Inc 4Fall Release Webinar - November 2016 Topic Modeling • Method for discovering structure in "unstructured" text. • Based on LDA, introduced by David Blei, Andrew Ng, and Michael I. Jordan in 2003. • Now "BigML Easy"
- 5. BigML, Inc 5Fall Release Webinar - November 2016 BigML Resources SOURCE DATASET CORRELATION STATISTICAL TEST MODEL ENSEMBLE LOGISTIC REGRESSION EVALUATION ANOMALY DETECTOR ASSOCIATION DISCOVERY SINGLE/BATCH PREDICTION SCRIPT LIBRARY EXECUTION Data Exploration Supervised Learning Unsupervised Learning Automation CLUSTER Scoring TOPIC MODEL
- 6. BigML, Inc 6Fall Release Webinar - November 2016 Unsupervised Learning Features Instances • Learn from instances • Each instance has features • There is no label Clustering Find similar instances Anomaly Detection Find unusual instances Association Discovery Find feature rules
- 7. BigML, Inc 7Fall Release Webinar - November 2016 Topic Model Text Fields • Unsupervised algorithm • Learns only from text fields • Finds hidden topics that model the text • How is this different from the Text Analysis that BigML already offers? • What does it output and how do we use it • Unsupervised… model? Questions:
- 8. BigML, Inc 8Fall Release Webinar - November 2016 Text Analysis Be not afraid of greatness: some are born great, some achieve greatness, and some have greatness thrust upon 'em. great: appears 4 times Bag of Words
- 9. BigML, Inc 9Fall Release Webinar - November 2016 Text Analysis … great afraid born achieve … … … 4 1 1 1 … … … … … … … … … Be not afraid of greatness: some are born great, some achieve greatness, and some have greatness thrust upon ‘em. Model The token “great” occurs more than 3 times The token “afraid” occurs no more than once
- 10. Text Analysis Demo #1
- 11. BigML, Inc 11Fall Release Webinar - November 2016 Text Analysis vs Topic Models Text Topic Model Creates thousands of hidden token counts Token counts are independently uninteresting No semantic importance No measure of co- occurrence Creates tens of topics that model the text Topics are independently interesting Semantic meaning extracted Support for bigrams
- 12. BigML, Inc 12Fall Release Webinar - November 2016 Generative Modeling • Decision trees are discriminative models • Aggressively model the classification boundary • Parsimonious: Don’t consider anything you don’t have to • Topic Models are generative models • Come up with a theory of how the data is generated • Tweak the theory to fit your data
- 13. BigML, Inc 13Fall Release Webinar - November 2016 Generating Documents cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… shoe asteroid flashlight pizza… plate giraffe purple jump… Be not afraid of greatness: some are born great, some achieve greatness… • "Machine" that generates a random word with equal probability with each pull. • Pull random number of times to generate a document. • All documents can be generated, but most are nonsense. word probability shoe ϵ asteroid ϵ flashlight ϵ pizza ϵ … ϵ
- 14. BigML, Inc 14Fall Release Webinar - November 2016 Topic Model • Written documents have meaning - one way to describe meaning is to assign a topic. • For our random machine, the topic can be thought of as increasing the probability of certain words. Intuition: Topic: travel cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… airplane passport pizza … word probability travel 23,55 % airplane 2,33 % mars 0,003 % mantle ϵ … ϵ Topic: space cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… mars quasar lightyear soda word probability space 38,94 % airplane ϵ mars 13,43 % mantle 0,05 % … ϵ
- 15. BigML, Inc 15Fall Release Webinar - November 2016 Topic Model plate giraffe purple jump… airplane passport pizza … Topic: "1" cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… word probability travel 23,55 % airplane 2,33 % mars 0,003 % mantle ϵ … ϵ Topic: "k" cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… word probability shoe 12,12 % coffee 3,39 % telephone 13,43 % paper 4,11 % … ϵ …Topic: "2" cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… word probability space 38,94 % airplane ϵ mars 13,43 % mantle 0,05 % … ϵ plate giraffe purple jump… • Each text field in a row is concatenated into a document • The documents are analyzed to generate "k" related topics • Each topic is represented by a distribution of term probabilities
- 16. Topic Model Demo #1
- 17. BigML, Inc 17Fall Release Webinar - November 2016 Uses • As a preprocessor for other techniques • Bootstrapping categories for classification • Recommendation • Discovery in large, heterogeneous text datasets
- 18. BigML, Inc 18Fall Release Webinar - November 2016 Topic Distribution • Any given document is likely a mixture of the modeled topics… • This can be represented as a distribution of topic probabilities Intuition: Will 2020 be the year that humans will embrace space exploration and finally travel to Mars? Topic: travel cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… word probability travel 23,55 % airplane 2,33 % mars 0,003 % mantle ϵ … ϵ 11% Topic: space cat shoe zebra ball tree jump pen asteroid cable box step cabinet yellow plate flashlight… word probability space 38,94 % airplane ϵ mars 13,43 % mantle 0,05 % … ϵ 89%
- 19. Topic Model Demo #2
- 20. BigML, Inc 20Fall Release Webinar - November 2016 Clustering? Unlabelled Data Centroid Label Unlabelled Data topic 1 prob topic 3 prob topic k prob Clustering Batch Centroid Topic Model Text Fields Batch Topic Distribution …
- 21. Topic Model Demo #3
- 22. BigML, Inc 22Fall Release Webinar - November 2016 Some Tips • Setting k • Much like k-means, the best value is data specific • Too few will agglomerate unrelated topics, too many will partition highly related topics • I tend to find the latter more annoying than the former • Tuning the Model • Remove common, useless terms • Set term limit higher, use bigrams
- 23. Questions? Twitter: @bigmlcom Mail: [email protected] Docs: https://bigml.com/releases