Logistic Regression
Dr.D. Kalpanapriya
SAS
Logistic Regression
Logistic regression is a statistical method used for binary classification,
where the goal is to predict the probability of a binary outcome (usually 0 or
1) based on one or more predictor variables (features). Unlike linear
regression, which predicts continuous values, logistic regression is used
when the dependent variable (response) is categorical.
Key Concepts of Logistic
Regression
Sigmoid Function (Logistic Function)
Logistic regression uses the sigmoid function to model the relationship between the input
features and the probability of the target class. The sigmoid function maps any real-valued number to a
value between 0 and 1, which can be interpreted as a probability.
Where:
is a linear combination of the input features, i.e., +……. +b and is the predicted probability that the
output is 1.
Decision Boundary
Logistic regression predicts class labels by applying a threshold (typically 0.5) to the output of
the sigmoid function:
•If , the output class is 1.
•If , the output class is 0.
Key Concepts of Logistic
Regression
Multinomial Logistic Regression (Softmax Regression)
For multiclass classification problems, logistic regression can be extended to
multinomial logistic regression, where the softmax function is used to predict probabilities
across multiple classes.
Optimization (Gradient Descent)
The model parameters (weights and bias) are typically optimized using gradient descent.
Gradient descent adjusts the weights to minimize the cost function iteratively.
Applications of Logistic
Regression
• Binary classification tasks such as spam detection, disease diagnosis (yes/no), and churn prediction.
• Logistic regression is also used as the foundation for more complex algorithms, such as neural networks.
Assumptions in Logistic
Regression
• Linearity in the log-odds: The log-odds (logarithm of the odds of the outcome) is assumed to
be a linear combination of the input features.
• Independence of observations: Each observation should be independent of others.
• No multicollinearity: The independent variables should not be too highly correlated with each other.
Example 1
The data of pass or fail in an exam of 5 students is given below
Hours 29 15 33 28 39
spend
Pass or Fail 0 0 1 1 1
Use Logistic regression as classifier to answer the following questions
(i) Calculate the probability of pass for the student who studied 33 hours
(ii)At least how any hours student should study that makes he will pass the course with the
probability of more than 95%.
Assume that the model suggested by the optimizer for odds of passing the course is
