BigML Summer 2016 Release
1 like782 views
The BigML Summer 2016 release introduced Logistic Regression, expanding its capabilities as a classification algorithm that models the probability of an output class. The release includes details on handling multiple classes, categorical inputs through various encoding methods, and comparisons to decision tree algorithms. Additionally, BigML promotes its educational program, offering free resources and support to students and institutions interested in machine learning.
![BigML, Inc 22Summer Release Webinar - September 2016
LR Multi-Class
• Instead of a binary class ex: [ true, false ], we have multi-
class ex: [ red, green, blue, … ]
• consider “k” classes
• solve “k” one-vs-rest LRs
• Result: coefficients βᵢ for
each of the “k” classes](https://image.slidesharecdn.com/bigml-summer-2016-release-160927185548/85/BigML-Summer-2016-Release-22-320.jpg)
BigML Summer 2016 Release
- 1. BigML Summer 2016 Release Introducing Logistic Regression
- 2. BigML, Inc 2Summer Release Webinar - September 2016 Summer 2016 Release POUL PETERSEN (CIO) Enter questions into chat box – we’ll answer some via chat; others at the end of the session https://bigml.com/releases ATAKAN CETINSOY, (VP Predictive Applications) Resources Moderator Speaker Contact [email protected] Twitter @bigmlcom Questions
- 4. BigML, Inc 4Summer Release Webinar - September 2016 Logistic Regression • Introduced by David Cox in 1958 • BigML API since 2015 • Now Fully "BigML"
- 5. BigML, Inc 5Summer Release Webinar - September 2016 BigML Resources SOURCE DATASET CORRELATION STATISTICAL TEST MODEL ENSEMBLE LOGISTIC REGRESSION EVALUATION ANOMALY DETECTOR ASSOCIATION DISCOVERY PREDICTION BATCH PREDICTIONSCRIPT LIBRARY EXECUTION Data Exploration Supervised Learning Unsupervised Learning Automation CLUSTER Scoring
- 6. BigML, Inc 6Summer Release Webinar - September 2016 Supervised Learning LabelFeatures Instances • Learn from instances • Each instance has features • And a known label Label is a categorical • Will this customer churn? • What item should I recommend? • Does this patient have diabetes? Label is a numeric • How many customers will churn? • How much will they spend? • What is your life expectancy? Classification Regression
- 7. BigML, Inc 7Summer Release Webinar - September 2016 Logistic Regression • Classification implies a discrete objective. How can this be a regression? • Why do we need another classification algorithm? • more questions…. Logistic Regression is a classification algorithm
- 8. BigML, Inc 8Summer Release Webinar - September 2016 Linear Regression
- 9. BigML, Inc 9Summer Release Webinar - September 2016 Linear Regression
- 10. BigML, Inc 10Summer Release Webinar - September 2016 Polynomial Regression
- 11. BigML, Inc 11Summer Release Webinar - September 2016 Regression • What function can we fit to discrete data? Key Take-Away: Fitting a function to the data
- 12. BigML, Inc 12Summer Release Webinar - September 2016 Discrete Data Function?
- 13. BigML, Inc 13Summer Release Webinar - September 2016 Discrete Data Function? ????
- 14. BigML, Inc 14Summer Release Webinar - September 2016 Logistic Function •x→-∞ : f(x)→0 •x→∞ : f(x)→1 •Looks promising, but still not "discrete"
- 15. BigML, Inc 15Summer Release Webinar - September 2016 Probabilities P≈0 P≈10<P<1
- 16. BigML, Inc 16Summer Release Webinar - September 2016 Logistic Regression • Assumes that output is linearly related to "predictors" … but we can "fix" this with feature engineering • How do we "fit" the logistic function to real data? LR is a classification algorithm … that models the probability of the output class.
- 17. BigML, Inc 17Summer Release Webinar - September 2016 Logistic Regression β₀ is the "intercept" β₁ is the "coefficient" The inverse of the logistic function is called the "logit": In which case solving is now a linear regression
- 18. BigML, Inc 18Summer Release Webinar - September 2016 Logistic Regression If we have multiple dimensions, add more coefficients:
- 19. Logistic Regression Demo #1
- 20. BigML, Inc 20Summer Release Webinar - September 2016 LR Parameters 1. Bias: Allows an intercept term. Important if P(x=0) != 0 2. Regularization: • L1: prefers zeroing individual coefficients • L2: prefers pushing all coefficients towards zero 3. EPS: The minimum error between steps to stop. 4. Auto-scaling: Ensures that all features contribute equally. • Unless there is a specific need to not auto-scale, it is recommended.
- 21. BigML, Inc 21Summer Release Webinar - September 2016 Logistic Regression • How do we handle multiple classes? • What about non-numeric inputs?
- 22. BigML, Inc 22Summer Release Webinar - September 2016 LR Multi-Class • Instead of a binary class ex: [ true, false ], we have multi- class ex: [ red, green, blue, … ] • consider “k” classes • solve “k” one-vs-rest LRs • Result: coefficients βᵢ for each of the “k” classes
- 23. BigML, Inc 23Summer Release Webinar - September 2016 LR Field Codings • LR is expecting numeric values to perform regression. • How do we handle categorical values, or text? Class color=red color=blue color=green color=NULL red 1 0 0 0 blue 0 1 0 0 green 0 0 1 0 NULL 0 0 0 1 One-hot encoding Only one feature is "hot" for each class
- 24. BigML, Inc 24Summer Release Webinar - September 2016 LR Field Codings Dummy Encoding Chooses a *reference class* requires one less degree of freedom Class color_1 color_2 color_3 *red* 0 0 0 blue 1 0 0 green 0 1 0 NULL 0 0 1
- 25. BigML, Inc 25Summer Release Webinar - September 2016 LR Field Codings Contrast Encoding Field values must sum to zero Allows comparison between classes …. so which one? Class field red 0,5 blue -0,25 green -0,25 NULL 0 influence positive negative negative excluded
- 26. BigML, Inc 26Summer Release Webinar - September 2016 LR Field Codings • The "text" type gives us new features that have counts of the number of times each token occurs in the text field. "Items" can be treated the same way. token "hippo" "safari" "zebra" instance_1 3 0 1 instance_2 0 11 4 instance_3 0 0 0 instance_4 1 0 3 Text / Items ?
- 27. Logistic Regression Demo #2
- 28. BigML, Inc 28Summer Release Webinar - September 2016 Curvilinear LR Instead of We could add a feature Where ???? Possible to add any higher order terms or other functions to match shape of data
- 29. Logistic Regression Demo #3
- 30. BigML, Inc 30Summer Release Webinar - September 2016 LR versus DT • Expects a "smooth" linear relationship with predictors. • LR is concerned with probability of a binary outcome. • Lots of parameters to get wrong: regularization, scaling, codings • Slightly less prone to over-fitting • Because fits a shape, might work better when less data available. • Adapts well to ragged non-linear relationships • No concern: classification, regression, multi-class all fine. • Virtually parameter free • Slightly more prone to over-fitting • Prefers surfaces parallel to parameter axes, but given enough data will discover any shape. Logistic Regression Decision Tree
- 31. BigML, Inc 31Summer Release Webinar - September 2016 DT Boundaries Splits x <= 0.5 y > -0.29 x < -0.18 z=1
- 33. BigML, Inc 33Summer Release Webinar - September 2016 BigML Education • 78 BigML ambassadors and increasing everyday…
- 34. BigML, Inc 34Summer Release Webinar - September 2016 BigML Education • Many students from over 620 universities are learning with the education program.
- 35. BigML, Inc 35Summer Release Webinar - September 2016 BigML Education • Enjoy the BigML PRO subscription plan, worth $300 per month, free of charge for a full year. • Promote BigML in your campus and spread the word. • We help you organize Machine Learning events, workshops, meetups, etc., and provide you with learning material. We are open to new ideas. • Get a BigML t-shirt and other merchandising material. • Be part of the BigML community!
- 36. Questions? Twitter: @bigmlcom Mail: [email protected] Docs: https://bigml.com/releases