Scribd
Upload
2 views

SVM_Experiment_Extended

This document outlines the implementation of Support Vector Machine (SVM) for classification, detailing its theory, mathematical foundations, advantages, and limitations. It includes a Python code example using the scikit-learn library to classify the Iris dataset, demonstrating SVM's effectiveness in identifying hyperplanes for data separation. The conclusion emphasizes the role of support vectors and kernel functions in enhancing SVM's versatility for various data patterns.

Uploaded by

pranayaws15
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
2 views

SVM_Experiment_Extended

This document outlines the implementation of Support Vector Machine (SVM) for classification, detailing its theory, mathematical foundations, advantages, and limitations. It includes a Python code example using the scikit-learn library to classify the Iris dataset, demonstrating SVM's effectiveness in identifying hyperplanes for data separation. The conclusion emphasizes the role of support vectors and kernel functions in enhancing SVM's versatility for various data patterns.

Uploaded by

pranayaws15
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 3

Experiment No: 06

Aim:
To implement Support Vector Machine (SVM) for classification.

Theory:
Support vector machines (SVMs) are a powerful set of supervised learning methods used
for various tasks such as classification, regression, and outlier detection. The primary
objective of the support vector machine algorithm is to identify a hyperplane in an N-
dimensional space (where N is the number of features) that distinctly classifies data points
into different categories.

Hyperplanes are essentially decision boundaries that separate data into different classes.
The placement of the hyperplane depends on the features of the input data. For example, if
there are two input features, the hyperplane is a line; with three input features, it becomes a
two-dimensional plane. In higher-dimensional space, the hyperplane generalizes
accordingly.

The classification accuracy of SVMs largely depends on finding the optimal hyperplane that
maximizes the margin between different classes. The support vectors are the data points
that lie closest to the hyperplane. These vectors influence the position and orientation of the
hyperplane and help build the SVM model.

SVMs are known for their versatility because they can work with both linear and non-linear
data. In cases where the data cannot be separated by a straight line or plane, kernel
functions are used. A kernel function transforms the data into a higher-dimensional space
where a linear hyperplane can be created.

There are various kernels that can be used with SVMs, including linear, polynomial, radial
basis function (RBF), and sigmoid. Each kernel has its own use case, with the linear kernel
being efficient for linearly separable data and the RBF kernel for non-linear data.

Mathematical Foundations of SVM:


SVM optimization aims to solve the following objective:
minimize (1/2) ||w||²
Subject to:
y_i (w^T x_i + b) >= 1, for all i = 1, ..., n

Where:
w is the weight vector,
b is the bias term,
y_i is the class label,
x_i is the feature vector for the i-th instance.

Advantages of SVM:
1. Effective in high-dimensional spaces.
2. Works well with a clear margin of separation.
3. Efficient for both linear and non-linear data through the use of kernel functions.
4. Robust against overfitting, especially in high-dimensional spaces.

Limitations of SVM:
1. Not suitable for large datasets as the training time can be significantly high.
2. Less effective when the data is overlapping or noisy.
3. Choice of kernel functions can significantly affect performance.

Code Implementation:
The following Python code implements SVM using the scikit-learn library to classify the
famous Iris dataset.

```python
from sklearn.utils import shuffle
from sklearn.model_selection import train_test_split
import numpy as np
from sklearn.metrics import accuracy_score
from sklearn.svm import SVC
# Load and preprocess the data
df = df.drop(['SepalWidthCm', 'PetalWidthCm'], axis=1)
X = df[['SepalLengthCm', 'PetalLengthCm']].values
Y = df['Species'].apply(lambda x: -1 if x == 'Iris-setosa' else 1).values
# Shuffle and split the data
X, Y = shuffle(X, Y)
x_train, x_test, y_train, y_test = train_test_split(X, Y, train_size=0.9)
# Train the model using SVM
clf = SVC(kernel='linear')
clf.fit(x_train, y_train)
# Predict and calculate accuracy
y_pred = clf.predict(x_test)
accuracy = accuracy_score(y_test, y_pred)
print('Accuracy of the SVM model:', accuracy)
```
Conclusion:
In this experiment, we successfully implemented a Support Vector Machine (SVM) to
classify the Iris dataset. The model was able to find the hyperplane that best separates the
classes. The results demonstrate that SVM is a powerful and flexible algorithm for
classification tasks.

The experiment also highlighted the importance of support vectors in determining the
optimal hyperplane. These support vectors directly influence the model's decision
boundary, which makes SVM particularly effective for both linearly separable and non-
linearly separable data.

Additionally, kernel functions provide SVM with the ability to handle complex data patterns,
making it a versatile tool in the field of machine learning. Further exploration could include
experimenting with different kernel functions or applying SVM to more complex datasets
for comparison.

You might also like