Faculty of Computing

CS-405 Deep Learning

BS-EE

Lab 2
CLO 3, CLO 4

Date: 5-02-2025

Name: Muhammad Zohaib Irfan

Lab Engineer: Ms. Areeba Rameen

Instructor: Ms. Huma Ameer
Objective
In this lab, you will:

● Implement the classic perceptron learning algorithm using PyTorch.

● Train the perceptron on a linearly separable dataset.
● Design perceptrons for an XOR dataset.
● Explore the applicability of perceptrons on a dataset of your choice.
● Answer key conceptual questions about perceptrons.

Task 1: Implement a Classic Perceptron in PyTorch

(from scratch)
Instructions

● Create a Perceptron class in PyTorch using torch.nn.Module.

● Implement the step function as the activation function (no sigmoid, just
thresholding).
● Train the perceptron using weight updates based on the Perceptron Learning
Rule:

● Do not forget the bias term!

● Train the perceptron on the AND function.
● Print weight updates after each training sample.
● After training, visualize the decision boundary to confirm learning.
Code and output:
import torch
import torch.nn as nn
import torch.optim as optim
import numpy as np
import matplotlib.pyplot as plt

class Perceptron(nn.Module):
def __init__(self, input_size):
super(Perceptron, self).__init__()
self.weights = nn.Parameter(torch.randn(input_size + 1) * 0.01)

def forward(self, x):

x = torch.cat((torch.ones(x.shape[0], 1), x), dim=1)
return torch.where(torch.matmul(x, self.weights) >= 0, 1, 0)

#and gate input data would look something like this

X = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)
y = torch.tensor([0, 0, 0, 1], dtype=torch.float32)

# Training the perceptron

perceptron = Perceptron(input_size=2)
learning_rate = 0.1

for epoch in range(8): # We can write epochs here

total_loss = 0
for i in range(len(X)):
x_sample = X[i].unsqueeze(0)
y_sample = y[i]

prediction = perceptron(x_sample).squeeze()
error = y_sample - prediction

with torch.no_grad():
perceptron.weights += learning_rate * error *
torch.cat((torch.tensor([1.0]), x_sample.squeeze()))

print(f"Sample {X[i].tolist()}, Prediction: {prediction.item()},

Error: {error.item()}, Weights: {perceptron.weights.tolist()}")

# now to plot decision boundary

def plot_decision_boundary(perceptron, X, y):
x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))

grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)

 Z = perceptron(grid).reshape(xx.shape)

plt.contourf(xx, yy, Z.detach().numpy(), levels=[-1, 0.5, 1], alpha=0.3,

colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary for AND Gate')
plt.show()

plot_decision_boundary(perceptron, X, y)
Task 2: Implement XOR through Perceptrons

🛠 Instructions

● Modify your perceptron code to train on the XOR dataset:

● Train the perceptron using the same Perceptron Learning Rule.

● Observe the final predictions and weight updates.
● Plot the decision boundary and analyze whether it correctly classifies XOR.

Code:

import torch
import numpy as np
import matplotlib.pyplot as plt

# Perceptron for XOR

class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
self.weights = torch.randn(input_size + 1) * 0.01
self.learning_rate = learning_rate

def step_function(self, x):

return 1 if x >= 0 else 0

def predict(self, x):

x = torch.cat((torch.tensor([1.0]), x))
return self.step_function(torch.dot(self.weights, x))

def train(self, X, y, epochs=6):

for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]

prediction = self.predict(x_sample)
 error = y_sample - prediction

self.weights += self.learning_rate * error *

torch.cat((torch.tensor([1.0]), x_sample))

print(f"Sample {X[i].tolist()}, Prediction: {prediction},

Error: {error}, Weights: {self.weights.tolist()}")

X = torch.tensor([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=torch.float32)

y = torch.tensor([0, 1, 1, 0], dtype=torch.float32) # XOR function labels

perceptron = Perceptron(input_size=2)
perceptron.train(X, y)

def plot_decision_boundary(perceptron, X, y):

x_min, x_max = -0.5, 1.5
y_min, y_max = -0.5, 1.5
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))

grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)

Z = torch.tensor([perceptron.predict(point) for point in
grid]).reshape(xx.shape)

plt.contourf(xx, yy, Z.numpy(), levels=[-1, 0.5, 1], alpha=0.3,

colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary for XOR Gate')
plt.show()

plot_decision_boundary(perceptron, X, y)

Output:
Results from different epochs:
Task 3: Apply Perceptron to a Dataset of Your Choice
Instructions

● Think of a use-case you want to solve.

● Generate a small synthetic dataset using a Large Language Model (LLM) or
choose a simple dataset that interests you.
● Check if a perceptron can solve the problem:
○ Train a single-layer perceptron on the dataset.
○ Evaluate its performance.
○ If it fails, analyze why.
● Show your working in code, including dataset generation, training, and evaluation.
● Discuss whether the problem is linearly separable and explain your findings.

Code:
Ok so I am taking a random dataset but its arranged in a circle, again it will not be a linear
data and we will see what happens, obviously a simple line wont be able to do it

import torch
import numpy as np
import matplotlib.pyplot as plt

# Perceptron Implementation for Custom Dataset

class Perceptron:
def __init__(self, input_size, learning_rate=0.1):
self.weights = torch.randn(input_size + 1) * 0.01 # Including bias
self.learning_rate = learning_rate

def step_function(self, x):

return 1 if x >= 0 else 0 # Simple threshold activation

def predict(self, x):

x = torch.cat((torch.tensor([1.0]), x)) # Add bias term
return self.step_function(torch.dot(self.weights, x))

def train(self, X, y, epochs=6):

for epoch in range(epochs):
for i in range(len(X)):
x_sample = X[i]
y_sample = y[i]

prediction = self.predict(x_sample)
error = y_sample - prediction

self.weights += self.learning_rate * error *

torch.cat((torch.tensor([1.0]), x_sample))
 print(f"Sample {X[i].tolist()}, Prediction: {prediction},
Error: {error}, Weights: {self.weights.tolist()}")

def generate_circle_data(n_samples=100):
X = torch.rand((n_samples, 2)) * 2 - 1
y = torch.tensor([(1 if x[0]**2 + x[1]**2 < 0.5**2 else 0) for x in X],
dtype=torch.float32)
return X, y

X, y = generate_circle_data(100)

perceptron = Perceptron(input_size=2)
perceptron.train(X, y)

def plot_decision_boundary(perceptron, X, y):

x_min, x_max = -1.1, 1.1
y_min, y_max = -1.1, 1.1
xx, yy = np.meshgrid(np.linspace(x_min, x_max, 100), np.linspace(y_min,
y_max, 100))

grid = torch.tensor(np.c_[xx.ravel(), yy.ravel()], dtype=torch.float32)

Z = torch.tensor([perceptron.predict(point) for point in
grid]).reshape(xx.shape)

plt.contourf(xx, yy, Z.numpy(), levels=[-1, 0.5, 1], alpha=0.3,

colors=['red', 'blue'])
plt.scatter(X[:, 0], X[:, 1], c=y, edgecolors='k', cmap=plt.cm.Paired)
plt.xlabel('X1')
plt.ylabel('X2')
plt.title('Perceptron Decision Boundary (Inside vs. Outside Circle)')
plt.show()

plot_decision_boundary(perceptron, X, y)
Changing epochs size:
We tried to train a single-layer perceptron to classify points as inside or outside a circle. The
perceptron learned something, but it failed to correctly separate the data. The perceptron failed
because it can only learn linear decision boundaries, but our dataset requires a nonlinear one
(a circle).

Task 4: Answer These Conceptual Questions

Write a short reflection answering these questions:

1. Does the perceptron successfully classify the AND function? Why or why not?

Yes. The AND function is linearly separable, so a perceptron can find a straight-line
decision boundary to classify it correctly.

2. Why does a single-layer perceptron fail to learn XOR?

Because XOR is not linearly separable. A single-layer perceptron can only learn straight-
line boundaries, but XOR requires a more complex separation.

3. What type of problems can a perceptron solve?

Linearly separable problems like AND, OR, and simple classification tasks where a
single straight line can split the classes
 4. What modifications would allow a model to solve XOR?

Use a Multi-Layer Perceptron. Adding a hidden layer allows the model to learn complex,
nonlinear decision boundaries.

Following is output after adding multi layers and finally solving it for XOR too.

Submission Guidelines
● Your code should be written in PyTorch.
● Ensure that you print weight updates at each step.
● Include a decision boundary plot for both AND and XOR.
● Submit a brief explanation answering the conceptual questions.

Hints

● Remember to add a bias term to your input features!

● Use a step function (thresholding) for activation, not sigmoid.
● The perceptron should update weights after every training sample, not after the
entire dataset.
● Try visualizing the decision boundary to understand model behavior.

Important Note:
 ● Copied labs will be marked zero.
○ All submissions must reflect your own understanding and effort.

Deadline:

● Submission Deadline for this task is 11:00 am Thursday-(6-Feb-2025)

If you have any questions or need assistance, please reach out to the lab engineer.