Machine Learning Prediction vs Classification

• “Machine Learning at its most basic is the practice of using algorithms to parse data, learn from it, • Prediction: Prediction is used to determine the numeric-valued outcome. Ex:
and then make a determination or prediction about something in the world.” – Nvidia Predicting how much customer will spend during a sale
• Supervised learning technique analyzes the labelled data and it maps the input to output based
example data points.
• Classification: Model is constructed and used to predict categorical labels. Ex: To
• Unsupervised learning works with unlabeled data in order to find previously undetected patterns predict if loan applications from a customer is safe or risky
or insights from the data set.
• Classification and regression are the types of problems solved using the supervised learning
method. Clustering and associative rule mining techniques are example of unsupervised machine
learning.
https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-
learning-ai/
Machine Learning vs Statistics
Machine Learning Statistics
Ultimate objective is to attain higher predictive power Statistical model’s primary objective is to make
inference about data , finding and understanding
Predictive Techniques relationship among variables.
Sacrifice interpretability over predictive power Models are interpretable
Linear Regression
Assess the model performance on unseen data/test Model is evaluated on the basis of significance and
data and it’s ability to make future predictions robustness of model parameters
Model Performance of Linear Regression Controlling for overfitting
Using Following metrics on the test data: • Overfitting happens when a model is able to give accurate outcome on train data, but
fails to perform well on test data.
• MAE: Mean Absolute Error • Penalizing model for complexity can reduce the overfitting
• Penalizes the extreme observations from the train data , as these might be observed by
• MSE : Mean Square Error chance and less likely to be reflected in model.
• L1 and L2 regularization (Lasso and Ridge) are used for controlling the overfitting in the
• RMSE : Root Mean Square Error model.
• MAPE : Mean Absolute Percentage Error • Regularization technique shrinks the values of beta coefficients in linear regression
model, thus restricting the model learning from train data.
Steps for building a predictive model
Define Problem and Gather Train a Supervised ML model
Linear Regression Data using Train Data
• A linear regression is presented as following:
Data Cleaning and Predict for X_test using the
processing trained Model
Partition Data into X and y Compute error or accuracy
X: Input variables of model using predicted
Y: Target or output and actual y values for test
data
• Finding the model parameters values for which following Mean Square error is minimized.
Partition Data into Train and
Test
N is number of observations in train data, fi is the value predicted by the model for data point i, and yi is the actual value of data point i. (X_Train, X_test) (y_train,
y_test)
Bias vs Variance Overfitting Vs Underfitting
• Bias = E (y_predicted) – y • Overfitting is reflected by a High Variance model or estimator.
• Variance= E (y_predicted – E(y_predicted))^2 • To control overfitting: Reducing the Variance of the estimator, Ex: increase
regularization, obtain larger data set, decrease number of features etc.
• High bias model will have high difference between actual and predicted
values. It means a simplistic model that is build using only few beliefs or • Underfitting relates to having a High Bias model. To fight underfitting, we can use
assumptions from data. High bias lead to underfitting. less regularization, use more features, use complex models etc.
• High variance model means that model is too sensitive to training data and
it’s output varies a lot with change in train data. It also learns from the
noise in data. High variance may also lead to overfitting.
L1 and L2 Regularization
• Regression model using L2 regularization is also called Ridge regression. It uses
following cost function.
• Regression model using L1 regularization is also called lasso regression. It shrinks
the values of smaller coefficients to 0. hence, it is also used in feature selection as
well.
Classification Models
Logistic Regression
• Logistic regression is a linear classifier
𝑓(𝐱) = 𝑏₀ + 𝑏₁𝑥₁ + ⋯ + 𝑏ᵣ𝑥ᵣ --Linear Function
Thank You!! 𝑏₀, 𝑏₁, …, 𝑏ᵣ are the coefficients
𝑝(𝐱) = 1 / (1 + exp(−𝑓(𝐱)) ---- Sigmoid Function
y=1, if p(x)>=threshold
y=0, otherwise
• Minimizes the following cost function:
Cost= Σᵢ -(𝑦ᵢ log(𝑝(𝐱ᵢ)) - (1 − 𝑦ᵢ) log(1 − 𝑝(𝐱ᵢ)))
Tuning Logistic Regression model
• Regularizing the model parameters by using L1 and L2 regularization.
• By default, probability threshold for assigning a data point to positive class is 0.5.
• In case of imbalanced datasets, analyst may validate the model for various
threshold values and chose a threshold value which provides the highest f1-score
Metrics for assessing Logistic regression performance AUC-ROC Curve
Predicted
0 1
• Important metrics for checking any classification model’s performance
0 True False
Actual Negative Positive
1 False True • It tells how much model is capable of distinguishing between classes
Negative Positive
• AUC near the 1 means model is good in separating the classes, AUC equal to 0.5
means that model has no class separation capacity whatsoever.