MACHINE LEARNING

CLASSIFICATION OF MACHINE
LEARNING
Types

There are perhaps four main types of classification tasks that you may
encounter; they are:

• Binary Classification

• Multi-Class Classification

• Multi-Label Classification

• Imbalanced Classification
BINARY CLASSIFICATION

Binary classification refers to those classification tasks that have two

class labels.
Examples include:
• Email spam detection (spam or not).

• Churn prediction (churn or not).

• Conversion prediction (buy or not).

BINARY CLASSIFICATION

Binary classification refers to those classification tasks that have two

class labels.
Examples include:
• Email spam detection (spam or not).

• Churn prediction (churn or not).

• Conversion prediction (buy or not).

Binary Classification
Typically, binary The class for the normal state is assigned the
classification tasks involve
one class that is the normal class label 0 and the class with the abnormal
state and another class that is state is assigned the class label 1
the abnormal state.
Popular algorithms that can be used for binary
For example “not spam” is the classification include:
normal state and “spam” is the
abnormal state. Another • Logistic Regression
example is “cancer not • k-Nearest Neighbors
detected” is the normal state • Decision Trees
of a task that involves a
medical test and “cancer • Support Vector Machine
detected” is the abnormal • Naive Bayes
state.
TYPES OF MACHINE LEARNING

1. Supervised Machine Learning

2. Unsupervised Machine Learning
3. Semi-Supervised Machine Learning
4. Reinforcement Learning
.
1. Supervised Machine Learning

Supervised learning is defined

as when a model gets trained
on a “Labelled Dataset”.
Labelled datasets have both
input and output parameters.
In Supervised
Learning algorithms learn to
map points between inputs
and correct outputs. It has
both training and validation
datasets labelled.
 Example:
Consider a scenario where you have to build an image classifier to
differentiate between cats and dogs. If you feed the datasets of dogs and
cats labelled images to the algorithm, the machine will learn to classify
between a dog or a cat from these labeled images. When we input new
dog or cat images that it has never seen before, it will use the learned
algorithms and predict whether it is a dog or a cat. This is
how supervised learning works, and this is particularly an image
classification.
There are two main categories of supervised learning that are mentioned below:
Classification
Regression
Classification
• Here are some classification
algorithms:
Classification deals with
predicting categorical target • Logistic Regression
variables, which represent
discrete classes or labels. For • Support Vector Machine
instance, classifying emails as • Random Forest
spam or not spam, or
predicting whether a patient • Decision Tree
has a high risk of heart
disease. Classification • K-Nearest Neighbors (KNN)
algorithms learn to map the
input features to one of the
• Naive Bayes
predefined classes.
Regression • Here are some regression
algorithms:
Regression, on the other • Linear Regression
hand, deals with
predicting continuous target • Polynomial Regression
variables, which represent • Ridge Regression
numerical values. For
example, predicting the price • Lasso Regression
of a house based on its size,
location, and amenities, or • Decision tree
forecasting the sales of a
product. Regression • Random Forest
algorithms learn to map the
input features to a continuous
numerical value.
Advantages

• Supervised Learning models can have high accuracy as they

are trained on labelled data.
• The process of decision-making in supervised learning
models is often interpretable.
• It can often be used in pre-trained models which saves time
and resources when developing new models from scratch.
Disadvantages
• It has limitations in knowing patterns and may struggle
with unseen or unexpected patterns that are not present
in the training data.
• It can be time-consuming and costly as it relies
on labeled data only.
• It may lead to poor generalizations based on new data.
Applications
• Clustering: Group similar data points into clusters.
• Anomaly detection: Identify outliers or anomalies in data.
• Dimensionality reduction: Reduce the dimensionality of data while preserving
its essential information.
• Recommendation systems: Suggest products, movies, or content to users based
on their historical behavior or preferences.
• Topic modeling: Discover latent topics within a collection of documents.
• Density estimation: Estimate the probability density function of data.
• Image and video compression: Reduce the amount of storage required for
multimedia content.
• Data preprocessing: Help with data preprocessing tasks such as data cleaning,
imputation of missing values, and data scaling.
Contd..
• Market basket analysis: Discover associations between products.
• Genomic data analysis: Identify patterns or group genes with similar expression
profiles.
• Image segmentation: Segment images into meaningful regions.
• Community detection in social networks: Identify communities or groups of
individuals with similar interests or connections.
• Customer behavior analysis: Uncover patterns and insights for better marketing
and product recommendations.
• Content recommendation: Classify and tag content to make it easier to
recommend similar items to users.
• Exploratory data analysis (EDA): Explore data and gain insights before defining
specific tasks.
Semi-Supervised Learning

• Semi-Supervised learning is a machine learning algorithm

that works between the supervised and unsupervised learning
so it uses both labelled and unlabelled data. It’s particularly
useful when obtaining labeled data is costly, time-consuming,
or resource-intensive. This approach is useful when the
dataset is expensive and time-consuming. Semi-supervised
learning is chosen when labeled data requires skills and
relevant resources in order to train or learn from it.
Example:
• Consider that we are building a language translation model,
having labeled translations for every sentence pair can be
resources intensive. It allows the models to learn from
labeled and unlabeled sentence pairs, making them more
accurate. This technique has led to significant improvements
in the quality of machine translation services.
Types of Semi-Supervised Learning Methods
Graph-based semi-supervised Label propagation:
learning:

• This approach uses a graph to • This approach iteratively

represent the relationships propagates labels from the
between the data points. The labeled data points to the
graph is then used to propagate unlabeled data points, based on
labels from the labeled data the similarities between the data
points to the unlabeled data points.
points.
Types of Semi-Supervised Learning Methods
Co-training: Self-training:
• This approach trains two • This approach trains a machine
different machine learning learning model on the labeled
models on different subsets of data and then uses the model to
the unlabeled data. The two predict labels for the unlabeled
models are then used to label data. The model is then retrained
each other’s predictions. on the labeled data and the
predicted labels for the unlabeled
data.
Advantages
 It leads to better generalization as compared to supervised learning, as
it takes both labeled and unlabeled data.
 Can be applied to a wide range of data.

Disadvantages
 Semi-supervised methods can be more complex to implement
compared to other approaches.
 It still requires some labeled data that might not always be available or
easy to obtain.
 The unlabeled data can impact the model performance accordingly.
Applications
•Image Classification and Object Recognition: Improve the accuracy of
models by combining a small set of labeled images with a larger set of
unlabeled images.
•Natural Language Processing (NLP): Enhance the performance of
language models and classifiers by combining a small set of labeled text
data with a vast amount of unlabeled text.
•Speech Recognition: Improve the accuracy of speech recognition by
leveraging a limited amount of transcribed speech data and a more
extensive set of unlabeled audio.
•Recommendation Systems: Improve the accuracy of personalized
recommendations by supplementing a sparse set of user-item interactions
(labeled data) with a wealth of unlabeled user behavior data.
•Healthcare and Medical Imaging: Enhance medical image analysis by
utilizing a small set of labeled medical images alongside a larger set of unlabeled images.
 Unsupervised learning
Unsupervised learning is a type of machine learning in which
models are trained using unlabeled dataset and are allowed to act on
that data without any supervision.

The goal of unsupervised learning is to find the underlying structure

of dataset, group that data according to similarities, and represent
that dataset in a compressed format.
Why use Unsupervised Learning?
•Unsupervised learning is helpful for finding useful insights from the
data.
•Unsupervised learning is much similar as a human learns to think by
their own experiences, which makes it closer to the real AI.
•Unsupervised learning works on unlabeled and uncategorized data
which make unsupervised learning more important.
•In real-world, we do not always have input data with the
corresponding output so to solve such cases, we need unsupervised
learning.
Working of Unsupervised Learning

The unlabeled input data, which means it is not categorized and corresponding outputs are also not
given.
This unlabeled input data is fed to the machine learning model in order to train it. Firstly, it will
interpret the raw data to find the hidden patterns from the data and then will apply suitable algorithms
such as k-means clustering, Decision tree, etc.
Once it applies the suitable algorithm, the algorithm divides the data objects into groups according to
the similarities and difference between the objects.
Types of Unsupervised Learning Algorithm:
The unsupervised learning algorithm can be further categorized into two types of
problems:
clustering: Clustering is a method of grouping the objects into clusters
such that objects with most similarities remains into a group and has less
or no similarities with the objects of another group. Cluster analysis finds
the commonalities between the data objects and categorizes them as per
the presence and absence of those commonalities.

Association: An association rule is an unsupervised learning method

which is used for finding the relationships between variables in the large
database. It determines the set of items that occurs together in the dataset.
Association rule makes marketing strategy more effective.

Eg: people who buy X item (suppose a bread) are also tend to purchase Y
(Butter/Jam) item.
A typical example of Association rule is Market Basket Analysis.
Unsupervised Learning algorithms:
•K-means clustering
•KNN (k-nearest neighbors)
•Hierarchal clustering
•Anomaly detection
•Neural Networks
•Principle Component Analysis
•Independent Component Analysis
•Apriori algorithm
•Singular value decomposition
Advantages of Unsupervised Learning
Unsupervised learning is used for more complex tasks as compared to
supervised learning because, in unsupervised learning, we don't have
labeled input data.
Unsupervised learning is preferable as it is easy to get unlabeled data
in comparison to labeled data.

Disadvantages of Unsupervised Learning

Unsupervised learning is intrinsically more difficult than supervised
learning as it does not have corresponding output.
The result of the unsupervised learning algorithm might be less
accurate as input data is not labeled, and algorithms do not know the
exact output in advance.
Reinforcement Machine Learning
• Reinforcement machine learning algorithm is a learning
method that interacts with the environment by producing
actions and discovering errors. Trial, error, and delay are
the most relevant characteristics of reinforcement learning. In
this technique, the model keeps on increasing its performance
using Reward Feedback to learn the behavior or pattern.
These algorithms are specific to a particular problem e.g.
Google Self Driving car, AlphaGo where a bot competes with
humans and even itself to get better and better performers in
Go Game. Each time we feed in data, they learn and add the
data to their knowledge which is training data. So, the more it
learns the better it gets trained and hence experienced.
Reinforcement learning algorithms:
•Q-learning: Q-learning is a model-free RL algorithm that learns a Q-
function, which maps states to actions. The Q-function estimates the
expected reward of taking a particular action in a given state.
•SARSA (State-Action-Reward-State-Action): SARSA is another model-
free RL algorithm that learns a Q-function. However, unlike Q-learning,
SARSA updates the Q-function for the action that was actually taken, rather
than the optimal action.
•Deep Q-learning: Deep Q-learning is a combination of Q-learning and
deep learning. Deep Q-learning uses a neural network to represent the Q-
function, which allows it to learn complex relationships between states and
actions.
Example:
Consider that you are training an AI agent to
play a game like chess. The agent explores different
moves and receives positive or negative feedback
based on the outcome. Reinforcement Learning also
finds applications in which they learn to perform
tasks by interacting with their surroundings.
Types of Reinforcement Machine
Learning

There are two main types of reinforcement learning:

• Positive reinforcement
• Negative reinforcement
Positive reinforcement
• Rewards the agent for taking a desired action.
• Encourages the agent to repeat the behavior.
• Examples: Giving a treat to a dog for sitting, providing a
point in a game for a correct answer.
Negative reinforcement
• Removes an undesirable stimulus to encourage a desired
behavior.
• Discourages the agent from repeating the behavior.
• Examples: Turning off a loud buzzer when a lever is
pressed, avoiding a penalty by completing a task.
Advantages
•It has autonomous decision-making that is well-suited for tasks and that can
learn to make a sequence of decisions, like robotics and game-playing.
•This technique is preferred to achieve long-term results that are very
difficult to achieve.
•It is used to solve a complex problems that cannot be solved by
conventional techniques.
Disadvantages
•Training Reinforcement Learning agents can be computationally expensive
and time-consuming.
•Reinforcement learning is not preferable to solving simple problems.
•It needs a lot of data and a lot of computation, which makes it impractical
and costly.
Applications
• Game Playing: RL can teach agents to play games, even complex ones.
• Robotics: RL can teach robots to perform tasks autonomously.
• Autonomous Vehicles: RL can help self-driving cars navigate and make
decisions.
• Recommendation Systems: RL can enhance recommendation algorithms
by learning user preferences.
• Healthcare: RL can be used to optimize treatment plans and drug discovery.
• Natural Language Processing (NLP): RL can be used in dialogue systems
and chatbots.
• Finance and Trading: RL can be used for algorithmic trading.
•Supply Chain and Inventory Management: RL can be used to
optimize supply chain operations.
•Energy Management: RL can be used to optimize energy consumption.
•Game AI: RL can be used to create more intelligent and adaptive NPCs
in video games.
•Adaptive Personal Assistants: RL can be used to improve personal
assistants.
•Virtual Reality (VR) and Augmented Reality (AR): RL can be used to
create immersive and interactive experiences.
•Industrial Control: RL can be used to optimize industrial processes.
•Education: RL can be used to create adaptive learning systems.
•Agriculture: RL can be used to optimize agricultural operations.
 INDUCTIVE BIAS
Inductive Bias refers to the set of assumptions a learning algorithm makes to generalize beyond the training
data. It is a fundamental concept in machine learning and artificial intelligence, as it helps determine how
effectively a model can learn patterns and make predictions on unseen data.
Key Concepts
1.Why Inductive Bias is Necessary:
1. Machine learning problems often involve infinite possible hypotheses to explain a dataset.
2. Without a bias, a learning algorithm cannot choose among these hypotheses or generalize effectively.
3. Inductive bias guides the algorithm to favor certain solutions over others, typically those that align with
prior knowledge or constraints.
2. Examples of Inductive Bias:
• Linear Regression: Assumes that the relationship between variables can be modeled as a straight line.
Decision Trees: Prefers simpler, more compact trees over complex ones (Occam's Razor).
• Neural Networks: Assumes that data can be represented and understood in terms of hierarchical
feature transformations.
3. Types of Inductive Bias:
Language or Representation Bias: Constraints on the types of hypotheses the model can express (e.g.,
linear models for linear regression).
Search Bias: Preference for specific ways of exploring the hypothesis space (e.g., greedy algorithms in
decision tree construction).
 Overfitting vs Underfitting
Definition:
•Overfitting occurs when a model learns the training data too well, including its noise and outliers, to the extent that it fails to
generalize to unseen data.
Characteristics:
•High accuracy on training data.
•Poor performance on validation/test data.
•The model is too complex relative to the amount or quality of data (e.g., a high-degree polynomial for a simple trend).
Causes:
1.Excessive model complexity (e.g., too many features or parameters).
2.Small training dataset with many features.
3.Presence of noise or irrelevant features in the data.
Example:
•In a classification problem, a decision tree that splits on every minor detail in the training data may overfit.
Solutions:
1.Simplify the Model:
1. Reduce the number of features.
2. Use a less complex model (e.g., reducing the depth of a decision tree).
2.Regularization:
1. Apply L1​(Lasso) or L2 ​(Ridge) regularization to penalize large coefficients.
3.Increase Training Data:
1. Gather more data to help the model generalize better.
4.Cross-Validation:
1. Use techniques like k-fold cross-validation to evaluate model performance on unseen data.
 Overfitting vs Underfitting
Underfitting Definition:
•Underfitting occurs when a model is too simple to capture the underlying structure of the data, resulting in poor performance on
both training and test data.
Characteristics:
•Low accuracy on training data.
•Poor generalization to new data.
•The model fails to learn meaningful patterns.
Causes:
1.Model complexity is too low (e.g., a linear model for non-linear data).
2.Insufficient training (e.g., too few iterations in gradient descent).
3.Data preprocessing issues (e.g., missing feature scaling).
Example:
•Fitting a straight line to data with a quadratic relationship results in underfitting.
Solutions:
1.Increase Model Complexity:
1. Use a more flexible model (e.g., polynomial regression for non-linear data).
2.Improve Features:
1. Add or engineer new features that better represent the data.
3.Tune Hyperparameters:
1. Optimize parameters like learning rate, number of layers, or units in neural networks.
4.Ensure Proper Training:
1. Train for a sufficient number of iterations while monitoring performance to avoid overfitting.
Overfitting vs Underfitting
 Regression Model Selection and Generalization
Selecting an appropriate regression model is critical for building robust machine
learning solutions that generalize well to unseen data.
Model Selection:
•Definition: Choosing the best model from a set of candidate models based on
performance metrics and generalization ability.
•Objective: Strike a balance between underfitting and overfitting by selecting a
model with appropriate complexity for the dataset.
Generalization:
•Definition: The ability of a model to perform well on unseen data (e.g., test or
validation sets).
•Goal: Ensure that the model captures the true patterns in the data without being
overly tailored to the training dataset.
 Steps in Regression Model Selection
1. Define the Problem:
•Identify the target variable (dependent variable) and predictor variables (independent variables).
•Understand the relationships (linear or non-linear) in the data.
2. Choose Candidate Models:
•Linear Models:
• Simple models assuming a linear relationship.
• Example: Simple Linear Regression, Multiple Linear Regression.
•Non-Linear Models:
• Suitable for datasets with non-linear relationships.
• Example: Polynomial Regression, Decision Trees, Neural Networks.
•Regularized Models:
• Handle overfitting by penalizing large coefficients.
• Example: Ridge Regression, Lasso Regression, Elastic Net.
•Advanced Models:
• For complex datasets or high-dimensional data.
• Example: Support Vector Regression (SVR), Gradient Boosting Machines (GBM).
 Steps in Regression Model Selection
3. Train and Evaluate Models:
•Data Splitting:
• Split the dataset into training, validation, and test sets.
•Training:
• Fit candidate models on the training dataset.
•Evaluation:
• Use validation metrics (e.g., Mean Squared Error, R-squared) to compare models.
4. Hyperparameter Tuning:
•Optimize parameters like the degree of polynomial in Polynomial Regression or the regularization coefficient
(α\alphaα) in Ridge/Lasso Regression.
•Methods:
• Grid Search
• Random Search
• Bayesian Optimization
5. Select the Best Model:
•Choose the model with the best validation performance and a good balance of bias and variance.
 Ensuring Generalization
To build a regression model that generalizes well:
1.Avoid Overfitting:
1. Use regularization techniques (Ridge, Lasso).
2. Restrict model complexity (e.g., limit polynomial degree).
3. Use cross-validation to ensure consistent performance across folds.
2.Prevent Underfitting:
1. Increase model complexity if the model is too simple.
2. Include relevant features or interactions between variables.
3.Feature Engineering:
1. Scale numerical features.
2. Encode categorical variables effectively.
3. Remove irrelevant or redundant features.
4.Cross-Validation:
1. Employ techniques like kkk-fold cross-validation to estimate model performance on unseen data.
5.Evaluation Metrics:
1. Use appropriate metrics for the problem:
1. Mean Absolute Error (MAE): Measures average error magnitude.
2. Mean Squared Error (MSE): Penalizes large errors.
3. R-squared: Explains the proportion of variance captured by the model.