Support Vector Machine Algorithm

Support Vector Machine or SVM is one of the most popular Supervised Loarning algorithms, which
is used for Classification as well as Regresslon problems. However, primarily, it is used for
Classification problems in Machine Learning.

The goal of the SVM algorithm is to create the best line or declsion boundary that can segregate n-
dimensional space into classes so that we can easily put the new data point in the correct category
in the future. This best decision boundary is called a hyperplane.

SVM chooses the extreme points/vectors that help in creating the hyperplane. These extreme
cases are called as support vectors, and hence algorithm is termed as Sup ort Vector Machine.
Consider the below diagram in which there are two diferent categories that are classified using a
decision boundary or hyperplane:

AZ Maxinmum
Margin Positive
Hyperplane

Maximumh
Margin
Hyperplane

Support
Vectors
Negative Hyperplane

Example: SVM can be understood with the example that we have used in the KNN classifier.
Suppose we see a strange cat that also has some features of dogs, so if we want a model that can
accurately identify whether it is a cat or dog, so such a model can be created by using the SVM
algorithm. We will first train our model with lots of images of cats and dogs so that it can learn
about different features of cats and dogs, and then we test it with this strange creature. So as
support vector creates a decision boundary between these two data (cat and dog) and choose
extreme cases (support vectors), it will see thé extreme case of cat and dog. On the basis of the
support vectors, it will classify it as a cat. Consider the below diagram:

SVM algorithm can be used for Face detection, image classification, text categorization, etc.

Types of SVM

SVM can be of two fypes:

o Linear SVM: Linear SVM is used for linearly separable data, which means if a
dataset
can be classified into two classes by using a single straight line, then such data is termed
as linearly separable data, and classifier is used called as Linear SVM classifier.
o Non-linear SVM: Non-Linear SVM is used for non-linearly separated data, which means if
a dataset cannot be classified by using a straight line, then such data is termed as non-
linear data and classifier used is called as Non-inear SVM clasifier.

Hyperplane and Support Vectors in the SVM algorithm:

Hyperplane: There can be multiple lines/decision boundaries to segregate the classes in n
dimensional space, but we need to find out the best decision boundary that helps to classity the
data points. This best boundary is known as the hyperplane of SVM.

The dimensions of the hyperplane depend on the features present in the dataset, which meansi
there are 2 features (as shown in image), then hyperplane will be a straight line. And if there are 3
features, then hyperplane will be a 2-dimension plane.

We always create a hyperplane that has a maximum margin, which means the maximum distance
between the data points.

Support Vectors:

The data points or vectors that are the closest to the hyperplane and which affect the position of
the hyperplane are termed as Support Vector. Since these vectors support the hyperplane, hence
called a Support vector.

How does SVM works?

Linear SVM:

The working of the SVM algorithm can be understood by using an example. Suppose we have a
dataset that has two tags (green and blue), and the dataset has two features x1 and x2. We want a
classifier that can classify the pair(x1, x2) of coordinates in either green or blue. Consider the
below image:

So as it is 2-d space so by just using a straight line, we can easily separate these two classes. But
there can be multiple lines that can separate these classes. Consider the below image:

Hence, the SVM algorithm helps to find the best line or decision boundary: this best boundary or
region is called as a hyperplane. SVM algorithm finds the closest point of the lines from both the
classes. These points are called support vectors. Th listance between the vectors and thee
hyperplane is called as margin. And the goal of SVM IS to maximize this margin.
The hyperplane with maximum margin is called the optimal hyperplane.
 Support vector, Optimal Hyperplane

Maximisej margin Support vector

Non-Linear SVM:

If data is linearly
arranged, then we can separate it by using a straight line, but for non-linear data,
we cannot draw a single straight line. Consider the
below image:

So to separate these data

points, we need to add one more
dimension. For linear data, we have used two dimensions x and so for
y, non-linear data, we will
add a third dimension z. It can be calculated as:

z=x ty2

By adding the third dimension, the sample space will become as below image

So now, SVM will divide the datasets into classes in the following way. Consider the below image:
 Best Hyperplane

SVM classifier does

representation of various classes in a hyperplane in n-dimensional space;
n is the number of features. Then the data items are plotted and finally we draW a
line/optimum hyperplane that separates the 2 classes
The hyperplane will be generated in an iterative manner by SVM so that the error can be
minimized. The goal of SVM is to divide the datasets into classes to find maximum
a marginal
hyperplane (MMH) and it can be done in the following two steps
First, SVM will generate hyperplanes iteratively that segregates the classes in best way.

Then, it will choose the hyperplane that separates the classes correctly.
For example, attributes be age and height in a 2 dimensional
space. Now, every data point
will have 2 coordinates(support vectors) that are: age and height. Support vectors as shown
in Figure are data points that are closer to the hyperplane and influence the position and
orientation of the hyperplane. Using these support vectors, we maximize the margin of the
classifier.

arg
---s--
OGpardungnyperplaneP