Page MARKS

EX.NO DATE NAME OF THE REMARKS

No AWARDE
EXPERIMENT D

1 A 20-01-23 03
Implementation of Uninformed
search algorithms – BFS
20-01-23
B Implementation of Uninformed 05
search algorithms – DFS
2 03-02-23
A Implementation of Informed 08
search algorithms - A*
03-02-23
B Implementation of Informed 12
search algorithms - memory-
bounded A*
3 10-02-23 Implement naïve Bayes models 16

4 17-02-23 Implement Bayesian Networks 19

5 24-02-23 Build Regression models 22

6 03-03-23 Build decision trees

A 26

03-03-23 Build random forests

B 29

7 17-03-23
Build SVM models 33

8 24-03-23 Implement ensembling 35

techniques
9 24-03-23 Implement clustering algorithms 41

1
10 31-03-23 Implement EM for Bayesian 47
networks
11 31-03-23 Build simple NN models 51

12 21-04-23 55
Build deep learning NN
models

2
Ex.No: 1A Implementation of Uninformed search algorithms (BFS)

Date:20-1-23

Aim :

To implement the BFS algorithm using python.

Algorithm:
Step i:To begin, place any one of the vertices of our graph at the lower extreme
of the queue
Step ii:Add the very first element in the created queue to the list of objects that
have already been checked out.
Step iii:Create a list of all the nodes that seem to be near that vertex. Individual
nodes which are not in the visited list should be moved to the rear of the queue.

Step iv:Repeat the above two steps, i.e., steps 2 and 3, till our queue is reduced
to 0.

Program:
graph = {

'5' : ['3','7'],

'3' : ['2', '4'],

'7' : ['8'],

'2' : [],

3
 '4' : ['8'],

'8' : []

visited = [] # List for visited nodes.

queue = [] #Initialize a queue

def bfs(visited, graph, node): #function for BFS

visited.append(node)

queue.append(node)

while queue:

# Creating loop to visit each node

m = queue.pop(0)

print (m, end = " ")

for neighbour in graph[m]:

if neighbour not in visited:

visited.append(neighbour)

queue.append(neighbour)

# Driver Code

print("Following is the Breadth-First Search")

bfs(visited, graph, '5') # function calling

Output:

4
Breadth-First Search is
537248

Result:
Thus, the program for the BFS algorithm using python was implemented
successfully.

Ex.No: 1B Implementation of Uninformed search algorithms (DFS)

5
Date:20-1-23

Aim :

To implement the DFS algorithm using python.

Algorithm:
Step 1:Start by putting any one of the graph's vertices on top of a stack.

Step 2:Take the top item of the stack and add it to the visited list.
Step 3:Create a list of that vertex's adjacent nodes. Add the ones which aren't in
the visited list to the top of the stack.

Step 4:Keep repeating steps 2 and 3 until the stack is empty.

Program:
graph = {

'5' : ['3','7'],

'3' : ['2', '4'],

'7' : ['8'],

'2' : [],

'4' : ['8'],

'8' : []

visited = set() # Set to keep track of visited nodes of graph.

def dfs(visited, graph, node): #function for dfs

if node not in visited:

print (node)

visited.add(node)

6
 for neighbour in graph[node]:

dfs(visited, graph, neighbour)

# Driver Code

print("Following is the Depth-First Search")

dfs(visited, graph, '5')

Output:
Depth-First Search
532487

7
PSNA CET - CSE

8
 PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:
Thus, the program for the BFS algorithm using python was implemented
successfully.

Ex.No: 2A Implement A* Algorithm

Date:3-2-23

Aim :

To Implement A* Algorithm in python using search strategy

Algorithm:
Step 1: Place the starting node into the OPEN list.
Step 2: If the OPEN list is empty, Stop and return failure.
Step 3: Remove the node n, from the OPEN list which has the lowest value of
h(n), and places it
in the CLOSED list.
Step 4: Expand the node n, and generate the successors of node n.
Step 5: Check each successor of node n, and find whether any node is a goal
node not.If any
successor node is goal node, then return success and terminate the search, else
proceed to Step 6.
Step 6: For each successor node, algorithm checks for evaluation function f(n),
and then check

9
if the node has been in either OPEN or CLOSED list. If the node has not been in
both
lists, then add it to the OPEN list.
Step 7: Return to Step 2.
Program:
from collections import deque
class Graph:
def init (self, adjacency_list):
self.adjacency_list = adjacency_list
def get_neighbors(self, v):
return self.adjacency_list[v]
def h(self, n):
H={
'A': 1,
'B': 1,
'C': 1,
'D': 1
}
return H[n]
def a_star_algorithm(self, start_node, stop_node):
open_list = set([start_node])
closed_list = set([]) g = {}
g[start_node] = 0
parents = {}
parents[start_node] = start_node
while len(open_list) > 0:
n = None
for v in open_list:
if n == None or g[v] + self.h(v) < g[n] + self.h(n):
n = v;

10
if n == None:
print('Path does not exist!')
return None
if n == stop_node:
reconst_path = []
while parents[n] != n:
reconst_path.append(n)
n = parents[n]
reconst_path.append(start_node)
reconst_path.reverse()
print('Path found:
{}'.format(reconst_path))
return reconst_path
for (m, weight) in self.get_neighbors(n):
if m not in open_list and m not in closed_list:
open_list.add(m)
parents[m] = n
g[m] = g[n] + weight
else:
if g[m] > g[n] + weight:
g[m] = g[n] + weight
parents[m] = n
if m in closed_list:
closed_list.remove(m)
open_list.add(m)
open_list.remove(n)
closed_list.add(n)
print('Path does not exist!')
return None
adjacency_list = {

11
'A': [('B', 1), ('C', 3), ('D', 7)],
'B': [('D', 5)],
'C': [('D', 12)]
}
graph1 = Graph(adjacency_list)
graph1.a_star_algorithm('A', 'D')

Output:

Path found: ['A', 'B', 'D']

['A', 'B', 'D']

12
Result:
Thus the python program for A* has been implemented successfully

Ex.No:2B Implement Memory-boundedA* Algorithm

Date:3-2-23

Aim :

To Implement memory-bounded A* algorithm in python using search

strategy.
Algorithm:
Step 1: It expands the best leaf until memory is full.
Step 2: At this point it cannot add a new node to the search tree without
dropping an old one.
Step 3: It always drops the node with the highest f-value.
Step 4: If the goal is not reached then it backtracks and goes to the alternative
path.
Step 5: While selecting the node for expansion it may happen that two nodes are
with the same
f-value.
Step 6: SMA* generates new best node and new worst node for expansion and
deletion respectively.
Program:

13
def iterative_deepening_a_star(tree, heuristic, start, goal):
threshold = heuristic[start][goal]
while True:
print("Iteration with threshold: " + str(threshold))
distance = iterative_deepening_a_star_rec(tree, heuristic, start, goal, 0,
threshold)
if distance == float("inf"):
return -1
elif distance < 0:
print("Found the node we're looking for!")
return -distance
else:
threshold = distance
def iterative_deepening_a_star_rec(tree, heuristic, node, goal, distance,
threshold):
print("Visiting Node " + str(node))
if node == goal:
return -distance
estimate = distance + heuristic[node][goal]
if estimate > threshold:
print("Breached threshold with heuristic: " + str(estimate))
return estimate
min = float("inf")
for i in range(len(tree[node])):
if tree[node][i] != 0:
t = iterative_deepening_a_star_rec(tree, heuristic, i, goal, distance +
tree[node][i], threshold)
if t < 0:
return t
elif t < min:

14
min = t
return min

Output:

15
16
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:
Thus, the python program for Memory Bounded A * Algorithm was
implemented successfully.
Ex.No:3 Implement naïve Bayes models

Date:10-2-23

Aim :

To build a naive bayes model using gaussian naive bayes formula in python.
Algorithm:
Step 1: Convert the given dataset into frequency tables.
Step 2: Generate Likelihood table by finding the probabilities of given features.
Step 3: Now, use Bayes theorem to calculate the posterior probability.
Program:
from math import sqrt
from math import pi
from math import exp
def separate_by_class(dataset):
separated = dict()

17
for i in range(len(dataset)):
vector = dataset[i]
class_value = vector[-1]
if (class_value not in separated):
separated[class_value] = list()
separated[class_value].append(vector)
return separated
def mean(numbers):
return sum(numbers)/float(len(numbers))
def stdev(numbers):
avg = mean(numbers)
variance = sum([(x-avg)**2 for x in numbers]) / float(len(numbers)-1)
return sqrt(variance)
def summarize_dataset(dataset):
summaries = [(mean(column), stdev(column), len(column)) for column in
zip(*dataset)]
del(summaries[-1])
return summaries
def summarize_by_class(dataset):
separated = separate_by_class(dataset)
summaries = dict()
for class_value, rows in separated.items():
summaries[class_value] = summarize_dataset(rows)
return summaries
def calculate_probability(x, mean, stdev):
exponent = exp(-((x-mean)**2 / (2 * stdev**2 )))
return (1 / (sqrt(2 * pi) * stdev)) * exponent
def calculate_class_probabilities(summaries, row):
total_rows = sum([summaries[label][0][2] for label in summaries])
probabilities = dict()

18
for class_value, class_summaries in summaries.items():
probabilities[class_value] =
summaries[class_value][0][2]/float(total_rows)
for i in range(len(class_summaries)):
mean, stdev, _ = class_summaries[i]
probabilities[class_value] *= calculate_probability(row[i], mean,
stdev)
return probabilities
dataset = [[3.393533211,2.331273381,0],
[3.110073483,1.781539638,0],
[1.343808831,3.368360954,0],
[3.582294042,4.67917911,0],
[2.280362439,2.866990263,0],
[7.423436942,4.696522875,1],
[5.745051997,3.533989803,1],
[9.172168622,2.511101045,1],
[7.792783481,3.424088941,1],
[7.939820817,0.791637231,1]]
summaries = summarize_by_class(dataset)
probabilities = calculate_class_probabilities(summaries, dataset[0])
print(probabilities)

Output:
{0: 0.05032427673372076, 1: 0.000115577183799457

19
20
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:
Thus,build a naive bayes model using gaussian naive bayes formula in
python were executed successfully.

Ex.No:4 Implement Bayesian Networks

Date:17-2-23

Aim :

To implement python code for Bayes theorem network.

Algorithm:
Step 1: Identify which are the main variables in the problem to solve.
Step 2:Each variable corresponds to a node of the network.
Step 3:It is important to choose the number states for each variable, for instance,
there are
usually two states (true or false).

21
Step 4:Define structure of the network, that is, the causal relationships between
all the variables.
Step 5:Define the probability rules governing the relationships between the
variables.
Program:
pip install pgmpy
from pgmpy.models import BayesianModel
from pgmpy.factors.discrete import TabularCPD
from pgmpy.inference import VariableElimination
import numpy as np
bayesNet = BayesianModel()
bayesNet.add_node("M")
bayesNet.add_node("U")
bayesNet.add_node("R")
bayesNet.add_node("B")
bayesNet.add_node("S")
bayesNet.add_edge("M", "R")
bayesNet.add_edge("U", "R")
bayesNet.add_edge("B", "R")
bayesNet.add_edge("B", "S")
bayesNet.add_edge("R", "S")
cpd_A = TabularCPD('M', 2, values=[[.95], [.05]])
cpd_U = TabularCPD('U', 2, values=[[.85], [.15]])
cpd_H = TabularCPD('B', 2, values=[[.90], [.10]])
cpd_S = TabularCPD('S', 2, values=[[0.98, .88, .95, .6], [.02, .12, .05, .40]],
evidence=['R', 'B'], evidence_card=[2, 2])
cpd_R = TabularCPD('R', 2,
values=[[0.96, .86, .94, .82, .24, .15, .10, .05], [.04, .14, .06, .18, .76, .85,
.90, .95]],
evidence=['M', 'B', 'U'], evidence_card=[2, 2,2])

22
bayesNet.add_cpds(cpd_A, cpd_U, cpd_H, cpd_S, cpd_R)
bayesNet.check_model()
print("Model is correct.")

Output:
Model is correct.

PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:
The program for bayes theorem network is implemented successfully.

Ex.No:5 Build Regression models

Date:24-2-23

Aim :

To implement the univariate linear regression model and perceptron using

python for the set of data points.
Algorithm:

23
Step 1: Initialize the parameters.
Step 2: Predict the value of a dependent variable by giving an independent
variable.
Step 3: Calculate the error in prediction for all data points.
Step 4: Calculate partial derivative w.r.t a0 and a1.
Step 5: Calculate the cost for each number and add them.
Step 6: Update the values of a0 and a1.
Program:

Univariate Regression model:

x=[14,16,27,42,39,50,83]

y=[2,5,7,9,10,13,20]

xy=0

x2=0

xy_all=0

x2_all=0

x_all=0

y_all=0

for i in range(7):

xy_all=xy_all+x[i]*y[i]

x2_all=x2_all+x[i]*x[i]

x_all=x_all+x[i]

y_all=y_all+y[i]

x_bar=x_all/7

y_bar=y_all/7

24
b=(xy_all-7*x_bar*y_bar)/(x2_all-7*x_bar*x_bar)

a=(y_bar-b*x_bar)

print(b)

print(a)

print("Y=",b,"(X)+",a)

Perceptron:

x1=[1,1,-1,-1]

x2=[1,-1,1,-1]

t=[1,-1,-1,-1]

w1=0

w2=0

b=0

yin=0

del_w1=0

del_w2=0

del_b=0

y=0

breakpt=1

yin=0

alpha=1

epoch=1

cnt=0

25
while(breakpt):

cnt=0

for i in range(4):

yin=b+(w1*x1[i])+(w2*x2[i])

if yin==0:

y=0

elif yin>0:

y=1

else:

y=-1

if y !=t[i]:

print("Not equal")

del_b=alpha*t[i]

del_w1=del_b*x1[i]

del_w2=del_b*x2[i]

b=del_b+b

w1=del_w1+w1

w2=del_w2+w2

else:

cnt=cnt+1

if cnt==4:

breakpt=0

26
 epoch=epoch+1

print("the weight and bias value of the perceptron")

print("b=",b,"w1=",w1,"w2=",w2)

Output:

Univariate Regression model:

27
Perceptron:

28
Result:

Thus the calculation of univariate linear regression model and perceptron

has been implemented and verified.

Ex.No: 6A Build decision tree

Date:3-3-23

Aim:

To implement the tree based models decision tree using python for the set
of data points.

Algorithm:

Step 1: It begins with the original set S as the root node.

Step 2: On each iteration of the algorithm, it iterates through the very unused
attribute of the

set S and calculates Entropy(H) and Information gain(IG) of this attribute.

Step 3: It then selects the attribute which has the smallest Entropy or Largest
Information gain.

Step 4: The set S is then split by the selected attribute to produce a subset of the
data.before.

Step 5: The algorithm continues to recur on each subset, considering only

29
attributes never selected

Program:

import math

length={3:[2,0],4:[1,3],5:[2,2]}

gills={'yes':[0,4],'no':[5,1]}

beak={'yes':[5,3],'no':[0,2]}

teeth={'many':[3,4],'few':[2,1]}

features=["length","gills","beak","teeth"]

entrophy=[]

te=[]

n=10;

val=0

for i in length.keys():

p=length[i][0]/(length[i][0]+length[i][1])

if(p== 0 or (1-p)== 0):

e=0;

else:

e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))

entrophy=entrophy+[e]

j=0

for i in length.keys():

t=length[i][0]+length[i][1]

30
val+=(t/n)*entrophy[j]

j=j+1

print("Total Entrophy for Length",val)

te=te+[val]

val=0

entrophy=[]

for i in gills.keys():

p=gills[i][0]/(gills[i][0]+gills[i][1])

if(p== 0 or (1-p)== 0):

e=0;

else:

e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))

entrophy=entrophy+[e]

j=0

for i in gills.keys():

t=gills[i][0]+gills[i][1]

val+=(t/n)*entrophy[j]

j=j+1

print("Total Entrophy for Gills",val)

te=te+[val]

val=0

entrophy=[]

31
for i in beak.keys():

p=beak[i][0]/(beak[i][0]+beak[i][1])

if(p== 0 or (1-p)== 0):

e=0;

else:

e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))

entrophy=entrophy+[e]

j=0

for i in beak.keys():

t=beak[i][0]+beak[i][1]

val+=(t/n)*entrophy[j]

j=j+1

print("Total Entrophy for Beak",val)

te=te+[val]

val=0

entrophy=[]

for i in teeth.keys():

p=teeth[i][0]/(teeth[i][0]+teeth[i][1])

if(p== 0 or (1-p)== 0):

e=0;

else:

e = -p*math.log(p,2)-((1-p)* math.log((1-p),2))

32
entrophy=entrophy+[e]

j=0

for i in teeth.keys():

t=teeth[i][0]+teeth[i][1]

val+=(t/n)*entrophy[j]

j=j+1

print("Total Entrophy for Teeth",val)

te=te+[val]

j=0;

minval=te[0];

for i in range(1,4):

if(te[i]<minval):

minval=te[i]

print(minval)

for i in range(4):

if(minval==te[i]):

break

print("The feature to put at the top is ",features[i])

Output:

33
Total Entrophy for Length 0.7245112497836532

Total Entrophy for Gills 0.39001345298901247

Total Entrophy for Beak 0.763547202339972

Total Entrophy for Teeth 0.965148445440323

Minimum value is 0.39001345298901247

The feature to put at the top is gills

Result:

34
 Thus, the program for implement the decision tree algorithm using python
executed successfully.
Ex.No: 6B Build random forest

Date:3-3-23

Aim:
To implement the random forest algorithm using python.

Algorithm:
Step 1:First, start with the selection of random samples from a given dataset.
Step 2 − Next, this algorithm will construct a decision tree for every sample.
Then it will get the prediction result from every decision tree.
Step 3:In this step, voting will be performed for every predicted result.
Step 4:At last, select the most voted prediction result as the final prediction
result.

Program:
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
path = https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data

headernames = ['sepal-length', 'sepal-width', 'petal-length', 'petal-width

dataset = pd.read_csv(path, names = headernames)

dataset.head()

sepal- sepal- petal- petal- Class

length width length width

35
 0 5.1 3.5 1.4 0.2 Iris-setosa

1 4.9 3.0 1.4 0.2 Iris-setosa

2 4.7 3.2 1.3 0.2 Iris-setosa

3 4.6 3.1 1.5 0.2 Iris-setosa

4 5.0 3.6 1.4 0.2 Iris-setosa

X = dataset.iloc[:, :-1].values

y = dataset.iloc[:, 4].values

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.30)

from sklearn.ensemble import RandomForestClassifier

classifier = RandomForestClassifier(n_estimators = 50)

classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

from sklearn.metrics import classification_report, confusion_matrix,

accuracy_score

result = confusion_matrix(y_test, y_pred)

print("Confusion Matrix:")

36
print(result)

result1 = classification_report(y_test, y_pred)

print("Classification Report:",)

print (result1)

result2 = accuracy_score(y_test,y_pred)

print("Accuracy:",result2)

Output:

Confusion Matrix:

[[14 0 0]

[ 0 18 1]

[ 0 0 12]]

Classification Report:

precision recall f1-score support

Iris-setosa 1.00 1.00 1.00 14

Iris-versicolor 1.00 0.95 0.97 19

Iris-virginica 0.92 1.00 0.96 12

micro avg 0.98 0.98 0.98 45

macro avg 0.97 0.98 0.98 45

weighted avg 0.98 0.98 0.98 45

37
Accuracy: 0.9777777777777777

38
PSNA CET - CSE

39
 PREPARATION 30
PERFORMANCE 30
RECORD 40
TOTAL 100

Result:

Thus the tree based model (Decision tree and Random Forest) were
successfully implemented using python.

Ex.No: 7 Build SVM Models

Date:17-03-23

Aim :

To Build and implement SVM Models using Python .

Algorithm:

Step1: Import the Libraries

Step 2: Load the Dataset

Step 3: Split Dataset into X and Y

Step 4: Split the X and Y Dataset into the Training set and Test set

Step 5: Perform Feature Scaling

Step 6: Fit SVM to the Training set

40
Step 7: Predict the Test Set Results

Step 8: Make the Confusion Matrix

Step 9: Visualise the Test set results

Program:

import numpy as np

import matplotlib.pyplot as plt

import pandas as pd

dataset = pd.read_csv('Social_Network_Ads.csv')

X = dataset.iloc[:, [2, 3]].values

y = dataset.iloc[:, 4].values

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.25,

random_state = 0)

#print(dataset)

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()

X_train = sc.fit_transform(X_train)

X_test = sc.transform(X_test)

from sklearn.svm import SVC

classifier = SVC(kernel = 'rbf', random_state = 0)

classifier.fit(X_train, y_train)

y_pred = classifier.predict(X_test)

from sklearn.metrics import confusion_matrix, accuracy_score

41
cm = confusion_matrix(y_test, y_pred)

print(cm)

accuracy_score(y_test,y_pred)

from matplotlib.colors import ListedColormap

X_set, y_set = X_test, y_test

X1, X2 = np.meshgrid(np.arange(start = X_set[:, 0].min() - 1, stop = X_set[:,

0].max() + 1, step = 0.01),

np.arange(start = X_set[:, 1].min() - 1, stop = X_set[:, 1].max() + 1,

step = 0.01))

plt.contourf(X1, X2, classifier.predict(np.array([X1.ravel(),

X2.ravel()]).T).reshape(X1.shape),

alpha = 0.75, cmap = ListedColormap(('red', 'green')))

plt.xlim(X1.min(), X1.max())

plt.ylim(X2.min(), X2.max())

for i, j in enumerate(np.unique(y_set)):

plt.scatter(X_set[y_set == j, 0], X_set[y_set == j, 1],

c = ListedColormap(('red', 'green'))(i), label = j)

plt.title('SVM (Test set)')

plt.xlabel('Age')

plt.ylabel('Estimated Salary')

plt.legend()

plt.show()

42
Output:

Result:

We have successfully studied and implemented SVM Python From

Scratch

Ex.No: 08 Implement ensembling techniques

Date:24-3-23

Aim:
To implement the ensembling techniques like Bagging, Boosting and
Stacking using Python.

Algorithm:

Step 1:Create multiple datasets from the train dataset by selecting observations
with replacements

Step 2:Run a base model on each of the created datasets independently

43
Step 3:Combine the predictions of all the base models to each the final output

Step 4:Bagging normally uses only one base model (XGBoost Regressor used in
the code below).

Program:

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error

import xgboost as xgb

from sklearn.ensemble import BaggingRegressor

df = pd.read_csv("train_data.csv")

target = df["target"]

train = df.drop("target")

X_train, X_test, y_train, y_test = train_test_split(

train, target, test_size=0.20)

model = BaggingRegressor(base_estimator=xgb.XGBRegressor())

model.fit(X_train, y_train)

pred = model.predict(X_test)

print(mean_squared_error(y_test, pred_final))

Output:

4666

44
ii)Boosting:

Algorithm:

Step i:Take a subset of the train dataset.

Step ii:Train a base model on that dataset.

Step iii:Use third model to make predictions on the whole dataset.

Step iv:Calculate errors using the predicted values and actual values.

Step v:Initialize all data points with same weight.

Step vi:Assign higher weight to incorrectly predicted data points.

Step vii:Make another model, make predictions using the new model in such a
way that errors made by the previous model are mitigated/corrected.

Step viii:Similarly, create multiple models–each successive model correcting the

errors of the previous model.

Step ix:The final model (strong learner) is the weighted mean of all the previous
models (weak learners).

Program:

import pandas as pd

from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error

from sklearn.ensemble import GradientBoostingRegressor

df = pd.read_csv("train_data.csv")

target = df["target"]

train = df.drop("target")

45
X_train, X_test, y_train, y_test = train_test_split(

train, target, test_size=0.20)

model = GradientBoostingRegressor()

model.fit(X_train, y_train)

pred_final = model.predict(X_test)

print(mean_squared_error(y_test, pred_final))

Output:

4789

iii)Stacking:

Algorithm:

Step i:Split the train dataset into n parts

Step ii:A base model (say linear regression) is fitted on n-1 parts and predictions
are made for the nth part. This is done for each one of the n part of the train set.

Step iii:The base model is then fitted on the whole train dataset.

Step iv:This model is used to predict the test dataset.

Step v:The Steps 2 to 4 are repeated for another base model which results in
another set of predictions for the train and test dataset.

Step vi:The predictions on train data set are used as a feature to build the new
model.

Step vii:This final model is used to make the predictions on test dataset

Program:

import pandas as pd

46
from sklearn.model_selection import train_test_split

from sklearn.metrics import mean_squared_error

from sklearn.ensemble import RandomForestRegressor

import xgboost as xgb

from sklearn.linear_model import LinearRegression

from vecstack import stacking

df = pd.read_csv("train_data.csv")

target = df["target"]

train = df.drop("target")

X_train, X_test, y_train, y_test = train_test_split(

train, target, test_size=0.20)

model_1 = LinearRegression()

model_2 = xgb.XGBRegressor()

model_3 = RandomForestRegressor()

all_models = [model_1, model_2, model_3]

s_train, s_test = stacking(all_models, X_train, X_test,

y_train, regression=True, n_folds=4)

final_model = model_1

final_model = final_model.fit(s_train, y_train)

pred_final = final_model.predict(X_test)

47
print(mean_squared_error(y_test, pred_final))

Output:

4510

48
 PSNA CET - CSE
PREPARATION 30
PERFORMANCE 30
RECORD 40
TOTAL 100

Result:

49
 Thus the above programs for ensembling techniques were successfully
implemented using python.

Ex.No: 09 Implementation of clustering algorithms

Date:24-3-23

Aim :

To implement the clustering algorithms like K-Means and Partitioning

Around Medoids (PAM) using python for the set of data points.

Algorithm:

Step 1: Initialize k means with random values

Step 2: For a given number of iterations

Step 3: Iterate through items

Step 4: Find the mean closest to the item

Step 5: Assign item to mean

Step 6: Update mean

Program:

K means clustering:

import math

x = [1, 1, 2, 2, 3, 5]

y = [1.5, 4.5, 1.5, 3.5, 2.5, 6]

c1 = [1, 1.5]

c2 = [2, 1.5]

dist_c1 = []

50
dist_c2 = []

clust_1 = []

clust_2 = []

k=0

while k<2:

clust_1 = clust_2 = []

dist_c1 = dist_c2 = []

cx = cy = 0

for i in range(6):

dist_c1 = dist_c1 + [math.sqrt(((x[i] - c1[0])**2)+((y[i] - c1[1])**2))]

dist_c2 = dist_c2 + [math.sqrt(((x[i] - c2[0])**2)+((y[i] - c2[1])**2))]

if(dist_c1[i] < dist_c2[i]):

clust_1 = clust_1 + [i]

else:

clust_2 = clust_2 + [i]

print(" Cluster_1: ",end = "")

for i in range(len(clust_1)):

print(clust_1[i],end="")

cx = cx + x[clust_1[i]]

cy = cy + y[clust_1[i]]

c1 = [cx/len(clust_1), cy/len(clust_1)]

print()

51
cx = cy = 0

print(" Cluster_2: ",end="")

for i in range(len(clust_2)):

print(clust_2[i],end="")

cx = cx + x[clust_2[i]]

cy = cy + y[clust_2[i]]

c2 = [cx/len(clust_2), cy/len(clust_2)]

print()

print("Centroid C1= (",round(c1[0],3),", ",round(c1[1],3),")")

print("Centroid C2= (",round(c2[0],3),", ",round(c2[1],3),")")

k=k+1

Partition Around Medoids:

Algorithm:

Step 1: Initialize: select k of the n data points as the medoids

Step 2: Associate each data point to the closest medoid.

Step 3: While the cost of the configuration decreases

Step 4: For each medoid m, for each non-medoid data point o

Step 5: Swap m and o, associate each data point to the closest medoid,
recompute the cost (sum of distances of points to their medoid)

Step 6: If the total cost of the configuration increased in the previous step, undo
the swap.

Program:

x = [2, 3, 3, 4, 6, 6, 7, 7, 8, 7]

52
y = [6, 4, 8, 7, 2, 4, 3, 4, 5, 6]

print("Enter the representative objects for 1st iteration:")

i = int(input())

j = int(input())

dist_c1 = []

dist_c2 = []

clust_c1 = []

clust_c2 = []

for k in range(10):

dist_c1 = dist_c1 + [abs(x[k]-x[i]) + abs(y[k]-y[i])]

dist_c2 = dist_c2 + [abs(x[k]-x[j]) + abs(y[k]-y[j])]

if(dist_c1[k] < dist_c2[k]):

clust_c1 = clust_c1 + [k]

else:

clust_c2 = clust_c2 + [k]

q=0

print("Cluster 1: ",end="")

for i in range(len(clust_c1)):

print(clust_c1[i]," ",end="")

q = q + dist_c1[clust_c1[i]]

print()

print("Cluster 2: ",end="")

for i in range(len(clust_c2)):

53
print(clust_c2[i]," ",end="")

q = q + dist_c2[clust_c2[i]]

print()

print("Q = ",q)

print("Enter the second representative points for second iteration:")

i = int(input())

j = int(input())

dist_c1 = []

dist_c2 = []

clust_c1 = []

clust_c2 = []

for k in range(10):

dist_c1 = dist_c1 + [abs(x[k]-x[i]) + abs(y[k]-y[i])]

dist_c2 = dist_c2 + [abs(x[k]-x[j]) + abs(y[k]-y[j])]

if(dist_c1[k] < dist_c2[k]):

clust_c1 = clust_c1 + [k]

else:

clust_c2 = clust_c2 + [k]

q=0

print("Cluster 1: ",end="")

for i in range(len(clust_c1)):

print(clust_c1[i]," ",end="")

54
q = q + dist_c1[clust_c1[i]]

print()

print("Cluster 2: ",end="")

for i in range(len(clust_c2)):

print(clust_c2[i]," ",end="")

q = q + dist_c2[clust_c2[i]]

print()

print("Q = ",q)

Output:

K means:

55
PAM:

56
57
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

58
 TOTAL 100

Result:

Thus the clustering algorithms like K-Means and Partitioning Around

Medoids(PAM) was implemented using python

Ex.No: 10 Implement EM for Bayesian networks

Date:31-3-23

Aim :

To implement Expectation_Maximization(EM) algorithm for Bayesian

networks using python

Algorithm:
Step 1:Create a Bayesian network object with the given structure and initial
values.

Step 2:Call the learn_parameters() method of the BayesianNetwork object with a

dataset of observed data.

Step 3:For each node in the network, if it has no parents, use maximum
likelihood estimation to estimate its parameters.

Step 4:For each node with parents, initialize the parameter values using the MLE
estimates for each node’s conditional probability distribution.

Step 5:.Run the EM algorithm to estimate the parameters by iteratively

computing the excepted sufficient statistics and updating the parameters values
until convergence is reached.

Step 6:.Update the parameters values for each node in the network.

Step 7:.Return the BayesianNetwork object with the learned parameters.

Program:

59
import numpy as np

class BayesianNetwork:

def _init_(self, structure, values):

self.structure = structure

self.values = values

def infer(self, evidence):

pass # TODO: implement inference

def learn_parameters(self, data):

for node in range(len(self.structure)):

parents = self.structure[node]

if parents == []:

self.values[node] = np.mean(data[:, node])

else:

n_parents = len(parents)

n_values = len(self.values[node])

n_data = data.shape[0]

theta = np.zeros((n_values, n_parents + 1))

theta[:, 0] = np.sum(data[:, node]) / n_data

for i in range(n_parents):

parent = parents[i]

for j in range(n_values):

mask = (data[:, parent] == j)

60
theta[j, i+1] = np.sum(data[mask, node]) / np.sum(mask)

while True:

ss = np.zeros((n_values, n_parents + 1))

for i in range(n_data):

x = data[i, node]

parents_x = tuple(data[i, parents])

p_x_given_parents = np.prod(theta[x, 1:] ** parents_x * (1 - theta[x, 1:]) ** (1 -

parents_x))

p_parents = np.prod(theta[:, 1:] ** parents_x * (1 - theta[:, 1:]) ** (1 -

parents_x))

ss[x, 0] += p_x_given_parents / p_parents

for j in range(n_parents):

parent_j = parents[j]

ss[x, j+1] += parents_x[j] * (p_x_given_parents / p_parents)

# M-step: update parameters

old_theta = np.copy(theta)

for j in range(n_values):

theta[j, 0] = ss[j, 0] / n_data

for k in range(n_parents):

theta[j, k+1] = ss[j, k+1] / ss[j, 0]

if np.allclose(theta, old_theta):

break

self.values[node] = theta[:, 0]

61
Output:

62
63
64
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:

65
 Thus the Bayesian networks for EM algorithm was successfully implemented.

Ex.No: 11 Implement NN for Bayesian networks

Date:31-3-23

Aim :

To build and implement Simple NN Model using python

Algorithm:

Step 1: Take the inputs from the training dataset, performed some adjustments
based on their weights, and siphoned them via a method that computed the
output of the ANN.

Step 2: Compute the back-propagated error rate. In this case, it is the difference
between neuron’s predicted output and the expected output of the training
dataset.

Step 3: Based on the extent of the error got, we performed some minor weight
adjustments using the Error Weighted Derivative formula.

Step 4: We iterated this process an arbitrary number of 15,000 times. In every

iteration, the whole training set is processed simultaneously.

Program:

import numpy as np

class NeuralNetwork():

def __init__(self):

np.random.seed(1)

66
 self.synaptic_weights = 2 * np.random.random((3, 1)) - 1

def sigmoid(self, x):

return 1 / (1 + np.exp(-x))

def sigmoid_derivative(self, x):

return x * (1 - x)

def train(self, training_inputs, training_outputs, training_iterations):

#training the model to make accurate predictions while adjusting weights

continually

for iteration in range(training_iterations):

output = self.think(training_inputs)

#computing error rate for back-propagation

error = training_outputs - output

#performing weight adjustments

adjustments = np.dot(training_inputs.T, error *

self.sigmoid_derivative(output))

self.synaptic_weights += adjustments

67
 def think(self, inputs):

inputs = inputs.astype(float)

output = self.sigmoid(np.dot(inputs, self.synaptic_weights))

return output

if __name__ == "__main__":

neural_network = NeuralNetwork()

print("Beginning Randomly Generated Weights: ")

print(neural_network.synaptic_weights)

training_inputs = np.array([[0,0,1],

[1,1,1],

[1,0,1],

[0,1,1]])

training_outputs = np.array([[0,1,1,0]]).T

neural_network.train(training_inputs, training_outputs, 15000)

print("Ending Weights After Training: ")

print(neural_network.synaptic_weights)

user_input_one = str(input("User Input One: "))

user_input_two = str(input("User Input Two: "))

68
 user_input_three = str(input("User Input Three: "))

print("Considering New Situation: ", user_input_one, user_input_two,

user_input_three)

print("New Output data: ")

print(neural_network.think(np.array([user_input_one, user_input_two,
user_input_three])))

print("Success")

Output:

69
70
71
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

Result:

Thus we have successfully built and implemented Simple NN Model

using python.

Ex.No: 12 Build deep learning NN models

Date:21-4-23

Aim :

To build Deep Learning NN Model using Keras model(tensorflow

72
framework).

Algorithm:

Step 1: Load data.

Step 2: define keras model.

Step 3: compile the keras model.

Step 4: start training(fit the model).

Step 5: evaluate the model.

Step 6: making predictions.

Program:

import pandas as pd

data = pd.read_csv('diabetes.csv')

x = data.drop("Outcome", axis=1)

y = data["Outcome"]

from keras.models import Sequential

from keras.layers import Dense

model = Sequential()

model.add(Dense(12, input_dim=8, activation="relu"))

model.add(Dense(12, activation="relu"))

model.add(Dense(1, activation="sigmoid"))

model.compile(loss="binary_crossentropy", optimizer="adam",
metrics=["accuracy"])

model.fit(x,y, epochs=5, batch_size=10)

_, accuracy = model.evaluate(x, y)

73
print("Model accuracy: %.2f"% (accuracy*100))

predictions = model.predict(x)

print([round(x[0]) for x in predictions])

model = Sequential() #define model

model.add(Dense(12, input_dim=8, activation="relu"))

model.add(Dense(8, activation="relu"))

model.add(Dense(1, activation="sigmoid"))

model.compile(loss="binary_crossentropy", optimizer="adam",
metrics=["accuracy"]) #compile model

model.fit(x,y, epochs=5, batch_size=10) #training

_, accuracy = model.evaluate(x,y) #testing

print("Model accuracy: %.2f"% (accuracy*100))

predictions = model.predict(x) #make predictions

#round the prediction

rounded = [round(x[0]) for x in predictions]

OUTPUT:

Epoch 1/5

77/77 [==============================] - 2s 2ms/step - loss: 4.6499 -

accuracy: 0.5977

Epoch 2/5

77/77 [==============================] - 0s 3ms/step - loss: 1.4152 -

accuracy: 0.5339

74
Epoch 3/5

77/77 [==============================] - 0s 3ms/step - loss: 0.9386 -

accuracy: 0.5599

Epoch 4/5

77/77 [==============================] - 0s 3ms/step - loss: 0.8027 -

accuracy: 0.5951

Epoch 5/5

77/77 [==============================] - 0s 3ms/step - loss: 0.7429 -

accuracy: 0.6276

24/24 [==============================] - 0s 3ms/step - loss: 0.7817 -

accuracy: 0.5964

Model accuracy: 59.64

24/24 [==============================] - 0s 1ms/step

[0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1,
1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1,
1, 0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 1,
0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0,
1, 1, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1, 1, 0, 1,
0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0,
0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 0, 1, 1, 1, 1, 1,
1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0,
0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0,
0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 1,
1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0,
0, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 1,
1, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1,

0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0,
0, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0,
1, 0, 0, 0, 1, 0, 1, 1, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 1, 1, 1, 0, 1,

75
1, 0, 1, 1, 1, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0,
0, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 1, 1, 0,
1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 1, 1, 0, 1, 0,
0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,
1, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0,
0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 1, 1, 1, 0, 1, 0, 0, 1,
1, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0,
0, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 0]

Epoch 1/5

77/77 [==============================] - 2s 3ms/step - loss: 8.1264 -

accuracy: 0.6510

Epoch 2/5

77/77 [==============================] - 0s 3ms/step - loss: 1.6608 -

accuracy: 0.6328

Epoch 3/5

77/77 [==============================] - 0s 3ms/step - loss: 1.0499 -

accuracy: 0.6133

Epoch 4/5

77/77 [==============================] - 0s 3ms/step - loss: 0.9159 -

accuracy: 0.6367

Epoch 5/5

77/77 [==============================] - 0s 3ms/step - loss: 0.8502 -

accuracy: 0.6237

24/24 [==============================] - 0s 2ms/step - loss: 0.8137 -

accuracy: 0.6615

Model accuracy: 66.15

24/24 [==============================] - 0s 3ms/step

76
77
 PSNA CET - CSE

PREPARATION 30

PERFORMANCE 30

RECORD 40

TOTAL 100

78
Result:

Thus we have successfully build Deep Learning NN Model using Keras

model(tensorflow framework)

79