JIMS Engineering Management Technical Campus

Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Probabilistic reasoning in Artificial intelligence

Uncertainty:

Till now, we have learned knowledge representation using first-order logic and propositional
logic with certainty, which means we were sure about the predicates. With this knowledge
representation, we might write A→B, which means if A is true then B is true, but consider a
situation where we are not sure about whether A is true or not then we cannot express this
statement, this situation is called uncertainty.

So to represent uncertain knowledge, where we are not sure about the predicates, we need
uncertain reasoning or probabilistic reasoning.

Causes of uncertainty:

Following are some leading causes of uncertainty to occur in the real world.

1. Information occurred from unreliable sources.

2. Experimental Errors
3. Equipment fault
4. Temperature variation
5. Climate change.

Probabilistic reasoning:

Probabilistic reasoning is a way of knowledge representation where we apply the concept of

probability to indicate the uncertainty in knowledge. In probabilistic reasoning, we combine
probability theory with logic to handle the uncertainty.

We use probability in probabilistic reasoning because it provides a way to handle the

uncertainty that is the result of someone's laziness and ignorance.

In the real world, there are lots of scenarios, where the certainty of something is not
confirmed, such as "It will rain today," "behavior of someone for some situations," "A match
between two teams or two players." These are probable sentences for which we can assume
that it will happen but not sure about it, so here we use probabilistic reasoning.

Need of probabilistic reasoning in AI:

o When there are unpredictable outcomes.

o When specifications or possibilities of predicates becomes too large to handle.
o When an unknown error occurs during an experiment.
In probabilistic reasoning, there are two ways to solve problems with uncertain knowledge:

o Bayes' rule
o Bayesian Statistics
As probabilistic reasoning uses probability and related terms, so before understanding
probabilistic reasoning, let's understand some common terms:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Probability: Probability can be defined as a chance that an uncertain event will occur. It is
the numerical measure of the likelihood that an event will occur. The value of probability
always remains between 0 and 1 that represent ideal uncertainties.

. 0 ≤ P(A) ≤ 1, where P(A) is the probability of an event A.

. P(A) = 0, indicates total uncertainty in an event A.

. P(A) =1, indicates total certainty in an event A.

We can find the probability of an uncertain event by using the below formula.

o P(¬A) = probability of a not happening event.

o P(¬A) + P(A) = 1.
Event: Each possible outcome of a variable is called an event.

Sample space: The collection of all possible events is called sample space.

Random variables: Random variables are used to represent the events and objects in the real
world.

Prior probability: The prior probability of an event is probability computed before observing
new information.

Posterior Probability: The probability that is calculated after all evidence or information has
taken into account. It is a combination of prior probability and new information.

Conditional probability:

Conditional probability is a probability of occurring an event when another event has already
happened.

Let's suppose, we want to calculate the event A when event B has already occurred, "the
probability of A under the conditions of B", it can be written as:

Where P(A⋀B)= Joint probability of a and B

P(B)= Marginal probability of B.

Advertisement
If the probability of A is given and we need to find the probability of B, then it will be given
as:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)

It can be explained by using the below Venn diagram, where B is occurred event, so sample
space will be reduced to set B, and now we can only calculate event A when event B is
already occurred by dividing the probability of P(A⋀B) by P( B ).

already occurred by dividing the probability of P(A⋀B) by P( B ).

Example:

In a class, there are 70% of the students who like English and 40% of the students who likes
English and mathematics, and then what is the percent of students those who like English also
like mathematics?

Solution:

Let, A is an event that a student likes Mathematics

B is an event that a student likes English.

Hence, 57% are the students who like English also like Mathematics.

Bayes' theorem:

Bayes' theorem is also known as Bayes' rule, Bayes' law, or Bayesian reasoning, which
determines the probability of an event with uncertain knowledge.

In probability theory, it relates the conditional probability and marginal probabilities of two
random events.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Bayes' theorem was named after the British mathematician Thomas Bayes. The Bayesian
inference is an application of Bayes' theorem, which is fundamental to Bayesian statistics.

It is a way to calculate the value of P(B|A) with the knowledge of P(A|B).

Bayes' theorem allows updating the probability prediction of an event by observing new
information of the real world.

Example: If cancer corresponds to one's age then by using Bayes' theorem, we can determine
the probability of cancer more accurately with the help of age.

Bayes' theorem can be derived using product rule and conditional probability of event A with
known event B:

As from product rule we can write:

. P(A ⋀ B)= P(A|B) P(B) or

Similarly, the probability of event B with known event A:

. P(A ⋀ B)= P(B|A) P(A)

Equating right hand side of both the equations, we will get:

The above equation (a) is called as Bayes' rule or Bayes' theorem. This equation is basic of
most modern AI systems for probabilistic inference.

It shows the simple relationship between joint and conditional probabilities. Here,

P(A|B) is known as posterior, which we need to calculate, and it will be read as Probability
of hypothesis A when we have occurred an evidence B.

P(B|A) is called the likelihood, in which we consider that hypothesis is true, then we calculate
the probability of evidence.

P(A) is called the prior probability, probability of hypothesis before considering the
evidence

P(B) is called marginal probability, pure probability of an evidence.

In the equation (a), in general, we can write P (B) = P(A)*P(B|Ai), hence the Bayes' rule can
be written as:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)

Where A1, A2, A3,........, An is a set of mutually exclusive and exhaustive events.

Applying Bayes' rule:

Bayes' rule allows us to compute the single term P(B|A) in terms of P(A|B), P(B), and P(A).
This is very useful in cases where we have a good probability of these three terms and want to
determine the fourth one. Suppose we want to perceive the effect of some unknown cause,
and want to compute that cause, then the Bayes' rule becomes:

Example-1:

Question: what is the probability that a patient has diseases meningitis with a stiff neck?

Given Data:

A doctor is aware that disease meningitis causes a patient to have a stiff neck, and it occurs
80% of the time. He is also aware of some more facts, which are given as follows:

o The Known probability that a patient has meningitis disease is 1/30,000.

o The Known probability that a patient has a stiff neck is 2%.
Let a be the proposition that patient has stiff neck and b be the proposition that patient has
meningitis. , so we can calculate the following as:

P(a|b) = 0.8

P(b) = 1/30000

P(a)= .02

Hence, we can assume that 1 patient out of 750 patients has meningitis disease with a stiff
neck.

Example-2:

Question: From a standard deck of playing cards, a single card is drawn. The
probability that the card is king is 4/52, then calculate posterior probability
P(King|Face), which means the drawn face card is a king card.

Solution:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)

P(king): probability that the card is King= 4/52= 1/13

P(face): probability that a card is a face card= 3/13

P(Face|King): probability of face card when we assume it is a king = 1

Putting all values in equation (i) we will get:

Application of Bayes' theorem :

Following are some applications of Bayes' theorem:

o It is used to calculate the next step of the robot when the already executed step is
given.
o Bayes' theorem is helpful in weather forecasting.
o It can solve the Monty Hall problem.
Learning is the process of acquiring new knowledge, skills, behaviors, values, attitudes, and
preferences. It can involve studying, instruction, or experience.

Here are some things to know about learning:

• Learning can be immediate: Some learning can happen after a single event, like
being burned by a hot stove.
• Learning can be cumulative: Much learning comes from repeated experiences.
• Learning can be long-lasting: The changes that learning causes can last a lifetime.
• Learning is influenced by environment: Learning is often influenced by the
environments we live in and the relationships we form.
• Learning can be observed: In one experiment, children watched video clips of
adults interacting with a doll. The children learned from observing the adults' actions.

Decision Tree Classification Algorithm

o Decision Tree is a Supervised learning technique that can be used for both
classification and Regression problems, but mostly it is preferred for solving
Classification problems. It is a tree-structured classifier, where internal nodes
represent the features of a dataset, branches represent the decision
rules and each leaf node represents the outcome.
o In a Decision tree, there are two nodes, which are the Decision Node and Leaf
Node. Decision nodes are used to make any decision and have multiple branches,
whereas Leaf nodes are the output of those decisions and do not contain any further
branches.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
o The decisions or the test are performed on the basis of features of the given dataset.
o It is a graphical representation for getting all the possible solutions to a
problem/decision based on given conditions.
o It is called a decision tree because, similar to a tree, it starts with the root node, which
expands on further branches and constructs a tree-like structure.
o In order to build a tree, we use the CART algorithm, which stands for Classification
and Regression Tree algorithm.
o A decision tree simply asks a question, and based on the answer (Yes/No), it further
split the tree into subtrees.
o Below diagram explains the general structure of a decision tree:

Note: A decision tree can contain categorical data (YES/NO) as well as numeric data.

Why use Decision Trees?

There are various algorithms in Machine learning, so choosing the best algorithm for the
given dataset and problem is the main point to remember while creating a machine learning
model. Below are the two reasons for using the Decision tree:

o Decision Trees usually mimic human thinking ability while making a decision, so it is
easy to understand.
o The logic behind the decision tree can be easily understood because it shows a tree-
like structure.

Decision Tree Terminologies

· Root Node: Root node is from where the decision tree starts. It represents the entire dataset,
which further gets divided into two or more homogeneous sets.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
· Leaf Node: Leaf nodes are the final output node, and the tree cannot be segregated further
after getting a leaf node.
· Splitting: Splitting is the process of dividing the decision node/root node into sub-nodes
according to the given conditions.
· Branch/Sub Tree: A tree formed by splitting the tree.
· Pruning: Pruning is the process of removing the unwanted branches from the tree.
· Parent/Child node: The root node of the tree is called the parent node, and other nodes are
called the child nodes.

How does the Decision Tree algorithm Work?

In a decision tree, for predicting the class of the given dataset, the algorithm starts from the
root node of the tree. This algorithm compares the values of root attribute with the record
(real dataset) attribute and, based on the comparison, follows the branch and jumps to the next
node.

For the next node, the algorithm again compares the attribute value with the other sub-nodes
and move further. It continues the process until it reaches the leaf node of the tree. The
complete process can be better understood using the below algorithm:

o Step-1: Begin the tree with the root node, says S, which contains the complete dataset.
o Step-2: Find the best attribute in the dataset using Attribute Selection Measure
(ASM).
o Step-3: Divide the S into subsets that contains possible values for the best attributes.
o Step-4: Generate the decision tree node, which contains the best attribute.
o Step-5: Recursively make new decision trees using the subsets of the dataset created
in step -3. Continue this process until a stage is reached where you cannot further
classify the nodes and called the final node as a leaf node.

Example: Suppose there is a candidate who has a job offer and wants to decide whether he
should accept the offer or Not. So, to solve this problem, the decision tree starts with the root
node (Salary attribute by ASM). The root node splits further into the next decision node
(distance from the office) and one leaf node based on the corresponding labels. The next
decision node further gets split into one decision node (Cab facility) and one leaf node.
Finally, the decision node splits into two leaf nodes (Accepted offers and Declined offer).
Consider the below diagram:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)

Attribute Selection Measures

While implementing a Decision tree, the main issue arises that how to select the best attribute
for the root node and for sub-nodes. So, to solve such problems there is a technique which is
called as Attribute selection measure or ASM. By this measurement, we can easily select
the best attribute for the nodes of the tree. There are two popular techniques for ASM, which
are:

o Information Gain
o Gini Index

1. Information Gain:

o Information gain is the measurement of changes in entropy after the segmentation of

a dataset based on an attribute.
o It calculates how much information a feature provides us about a class.
o According to the value of information gain, we split the node and build the decision
tree.
o A decision tree algorithm always tries to maximize the value of information gain, and
a node/attribute having the highest information gain is split first. It can be calculated
using the below formula:

. Information Gain= Entropy(S)- [(Weighted Avg) *Entropy(each feature)

Entropy: Entropy is a metric to measure the impurity in a given attribute. It specifies
randomness in data. Entropy can be calculated as:

Entropy(s)= -P(yes)log2 P(yes)- P(no) log2 P(no)

Where,

o S= Total number of samples

o P(yes)= probability of yes
o P(no)= probability of no
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
2. Gini Index:

o Gini index is a measure of impurity or purity used while creating a decision tree in
the CART(Classification and Regression Tree) algorithm.
o An attribute with the low Gini index should be preferred as compared to the high Gini
index.
o It only creates binary splits, and the CART algorithm uses the Gini index to create
binary splits.
o Gini index can be calculated using the below formula:

Gini Index= 1- ∑jPj2

Pruning: Getting an Optimal Decision tree

Pruning is a process of deleting the unnecessary nodes from a tree in order to get the optimal
decision tree.

A too-large tree increases the risk of overfitting, and a small tree may not capture all the
important features of the dataset. Therefore, a technique that decreases the size of the learning
tree without reducing accuracy is known as Pruning. There are mainly two types of
tree pruning technology used:

o Cost Complexity Pruning

o Reduced Error Pruning.

Advantages of the Decision Tree

o It is simple to understand as it follows the same process which a human follow while
making any decision in real-life.
o It can be very useful for solving decision-related problems.
o It helps to think about all the possible outcomes for a problem.
o There is less requirement of data cleaning compared to other algorithms.

Disadvantages of the Decision Tree

o The decision tree contains lots of layers, which makes it complex.

o It may have an overfitting issue, which can be resolved using the Random Forest
algorithm.
o For more class labels, the computational complexity of the decision tree may increase.

Python Implementation of Decision Tree

Now we will implement the Decision tree using Python. For this, we will use the dataset
"user_data.csv," which we have used in previous classification models. By using the same
dataset, we can compare the Decision tree classifier with other classification models such
as KNN SVM, LogisticRegression, etc.

Steps will also remain the same, which are given below:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
o Data Pre-processing step
o Fitting a Decision-Tree algorithm to the Training set
o Predicting the test result
o Test accuracy of the result(Creation of Confusion matrix)
o Visualizing the test set result.

1. Data Pre-Processing Step:

Below is the code for the pre-processing step:

. # importing libraries
. import numpy as nm
. import matplotlib.pyplot as mtp
. import pandas as pd
.
. #importing datasets
. data_set= pd.read_csv('user_data.csv')
.
. #Extracting Independent and dependent Variable
. x= data_set.iloc[:, [2,3]].values
. y= data_set.iloc[:, 4].values
.
. # Splitting the dataset into training and test set.
. from sklearn.model_selection import train_test_split
. x_train, x_test, y_train, y_test= train_test_split(x, y, test_size= 0.25, random_state=0)

.
. #feature Scaling
. from sklearn.preprocessing import StandardScaler
. st_x= StandardScaler()
. x_train= st_x.fit_transform(x_train)
. x_test= st_x.transform(x_test)
In the above code, we have pre-processed the data. Where we have loaded the dataset, which
is given as:
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
2. Fitting a Decision-Tree algorithm to the Training set

Now we will fit the model to the training set. For this, we will import
the DecisionTreeClassifier class from sklearn.tree library. Below is the code for it:

. #Fitting Decision Tree classifier to the training set

. From sklearn.tree import DecisionTreeClassifier
. classifier= DecisionTreeClassifier(criterion='entropy', random_state=0)
. classifier.fit(x_train, y_train)
In the above code, we have created a classifier object, in which we have passed two main
parameters;

o "criterion='entropy': Criterion is used to measure the quality of split, which is

calculated by information gain given by entropy.
o random_state=0": For generating the random states.
Below is the output for this:

Out[8]:
DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,
max_features=None, max_leaf_nodes=None,
min_impurity_decrease=0.0, min_impurity_split=None,
min_samples_leaf=1, min_samples_split=2,
min_weight_fraction_leaf=0.0, presort=False,
random_state=0, splitter='best')

3. Predicting the test result

Now we will predict the test set result. We will create a new prediction vector y_pred. Below
is the code for it:

. #Predicting the test set result

. y_pred= classifier.predict(x_test)
Output:

In the below output image, the predicted output and real test output are given. We can clearly
see that there are some values in the prediction vector, which are different from the real vector
values. These are prediction errors.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)

4. Test accuracy of the result (Creation of Confusion matrix)

In the above output, we have seen that there were some incorrect predictions, so if we want to
know the number of correct and incorrect predictions, we need to use the confusion matrix.
Below is the code for it:

. #Creating the Confusion matrix

. from sklearn.metrics import confusion_matrix
. cm= confusion_matrix(y_test, y_pred)
Output:

In the above output image, we can see the confusion matrix, which has 6+3= 9 incorrect
predictions and62+29=91 correct predictions. Therefore, we can say that compared to
other classification models, the Decision Tree classifier made a good prediction.

5. Visualizing the training set result:

Here we will visualize the training set result. To visualize the training set result we will plot a
graph for the decision tree classifier. The classifier will predict yes or No for the users who
have either Purchased or Not purchased the SUV car as we did in Logistic Regression. Below
is the code for it:

. #Visulaizing the trianing set result

. from matplotlib.colors import ListedColormap
. x_set, y_set = x_train, y_train
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
. x1, x2 = nm.meshgrid(nm.arange(start = x_set[:, 0].min() -
1, stop = x_set[:, 0].max() + 1, step =0.01),
. nm.arange(start = x_set[:, 1].min() - 1, stop = x_set[:, 1].max() + 1, step = 0.01))
. mtp.contourf(x1, x2, classifier.predict(nm.array([x1.ravel(), x2.ravel()]).T).reshape(x
1.shape),
. alpha = 0.75, cmap = ListedColormap(('purple','green' )))
. mtp.xlim(x1.min(), x1.max())
. mtp.ylim(x2.min(), x2.max())
. fori, j in enumerate(nm.unique(y_set)):
. mtp.scatter(x_set[y_set == j, 0], x_set[y_set == j, 1],
. c = ListedColormap(('purple', 'green'))(i), label = j)
. mtp.title('Decision Tree Algorithm (Training set)')
. mtp.xlabel('Age')
. mtp.ylabel('Estimated Salary')
. mtp.legend()
. mtp.show()
Output:

The above output is completely different from the rest classification models. It has both
vertical and horizontal lines that are splitting the dataset according to the age and estimated
salary variable.

As we can see, the tree is trying to capture each dataset, which is the case of overfitting.

6. Visualizing the test set result:

Visualization of test set result will be similar to the visualization of the training set except that
the training set will be replaced with the test set.

. #Visulaizing the test set result

. from matplotlib.colors import ListedColormap
. x_set, y_set = x_test, y_test
. x1, x2 = nm.meshgrid(nm.arange(start = x_set[:, 0].min() -
1, stop = x_set[:, 0].max() + 1, step =0.01),
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
. nm.arange(start = x_set[:, 1].min() - 1, stop = x_set[:, 1].max() + 1, step = 0.01))
. mtp.contourf(x1, x2, classifier.predict(nm.array([x1.ravel(), x2.ravel()]).T).reshape(x
1.shape),
. alpha = 0.75, cmap = ListedColormap(('purple','green' )))
. mtp.xlim(x1.min(), x1.max())
. mtp.ylim(x2.min(), x2.max())
. fori, j in enumerate(nm.unique(y_set)):
. mtp.scatter(x_set[y_set == j, 0], x_set[y_set == j, 1],
. c = ListedColormap(('purple', 'green'))(i), label = j)
. mtp.title('Decision Tree Algorithm(Test set)')
. mtp.xlabel('Age')
. mtp.ylabel('Estimated Salary')
. mtp.legend()
. mtp.show()

Output:

As we can see in the above image that there are some green data points within the purple
region and vice versa. So, these are the incorrect predictions which we have discussed in the
confusion matrix.

What is a neural network?

A neural network is a method in artificial intelligence (AI) that teaches computers to process
data in a way that is inspired by the human brain. It is a type of machine learning
(ML) process, called deep learning, that uses interconnected nodes or neurons in a layered
structure that resembles the human brain. It creates an adaptive system that computers use to
learn from their mistakes and improve continuously. Thus, artificial neural networks attempt
to solve complicated problems, like summarizing documents or recognizing faces, with
greater accuracy.
Why are neural networks important?
Neural networks can help computers make intelligent decisions with limited human assistance.
This is because they can learn and model the relationships between input and output data that
are nonlinear and complex. For instance, they can do the following tasks.

Make generalizations and inferences

 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Neural networks can comprehend unstructured data and make general observations without
explicit training. For instance, they can recognize that two different input sentences have a
similar meaning:

• Can you tell me how to make the payment?

• How do I transfer money?

A neural network would know that both sentences mean the same thing. Or it would be able
to broadly recognize that Baxter Road is a place, but Baxter Smith is a person’s name.
What are neural networks used for?
Neural networks have several use cases across many industries, such as the following:

• Medical diagnosis by medical image classification

• Targeted marketing by social network filtering and behavioral data analysis
• Financial predictions by processing historical data of financial instruments
• Electrical load and energy demand forecasting
• Process and quality control
• Chemical compound identification

We give four of the important applications of neural networks below.

Computer vision

Computer vision is the ability of computers to extract information and insights from images
and videos. With neural networks, computers can distinguish and recognize images similar to
humans. Computer vision has several applications, such as the following:

• Visual recognition in self-driving cars so they can recognize road signs and other road
users
• Content moderation to automatically remove unsafe or inappropriate content from
image and video archives
• Facial recognition to identify faces and recognize attributes like open eyes, glasses,
and facial hair
• Image labeling to identify brand logos, clothing, safety gear, and other image details
Speech recognition

Neural networks can analyze human speech despite varying speech patterns, pitch, tone,
language, and accent. Virtual assistants like Amazon Alexa and automatic transcription
software use speech recognition to do tasks like these:

• Assist call center agents and automatically classify calls

• Convert clinical conversations into documentation in real time
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
• Accurately subtitle videos and meeting recordings for wider content reach
Natural language processing

Natural language processing (NLP) is the ability to process natural, human-created text.
Neural networks help computers gather insights and meaning from text data and documents.
NLP has several use cases, including in these functions:

• Automated virtual agents and chatbots

• Automatic organization and classification of written data
• Business intelligence analysis of long-form documents like emails and forms
• Indexing of key phrases that indicate sentiment, like positive and negative comments
on social media
• Document summarization and article generation for a given topic
Recommendation engines

Neural networks can track user activity to develop personalized recommendations. They can
also analyze all user behavior and discover new products or services that interest a specific
user. For example, Curalate, a Philadelphia-based startup, helps brands convert social media
posts into sales. Brands use Curalate’s intelligent product tagging (IPT) service to automate
the collection and curation of user-generated social content. IPT uses neural networks to
automatically find and recommend products relevant to the user’s social media activity.
Consumers don't have to hunt through online catalogs to find a specific product from a social
media image. Instead, they can use Curalate’s auto product tagging to purchase the product
with ease.
How do neural networks work?
The human brain is the inspiration behind neural network architecture. Human brain cells,
called neurons, form a complex, highly interconnected network and send electrical signals to
each other to help humans process information. Similarly, an artificial neural network is made
of artificial neurons that work together to solve a problem. Artificial neurons are software
modules, called nodes, and artificial neural networks are software programs or algorithms that,
at their core, use computing systems to solve mathematical calculations.

Simple neural network architecture

A basic neural network has interconnected artificial neurons in three layers:

Input Layer

Information from the outside world enters the artificial neural network from the input layer.
Input nodes process the data, analyze or categorize it, and pass it on to the next layer.

Hidden Layer

Hidden layers take their input from the input layer or other hidden layers. Artificial neural
networks can have a large number of hidden layers. Each hidden layer analyzes the output
from the previous layer, processes it further, and passes it on to the next layer.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Output Layer

The output layer gives the final result of all the data processing by the artificial neural
network. It can have single or multiple nodes. For instance, if we have a binary (yes/no)
classification problem, the output layer will have one output node, which will give the result
as 1 or 0. However, if we have a multi-class classification problem, the output layer might
consist of more than one output node.

Deep neural network architecture

Deep neural networks, or deep learning networks, have several hidden layers with millions of
artificial neurons linked together. A number, called weight, represents the connections
between one node and another. The weight is a positive number if one node excites another,
or negative if one node suppresses the other. Nodes with higher weight values have more
influence on the other nodes.
Theoretically, deep neural networks can map any input type to any output type. However,
they also need much more training as compared to other machine learning methods. They
need millions of examples of training data rather than perhaps the hundreds or thousands that
a simpler network might need.
What are the types of neural networks?
Artificial neural networks can be categorized by how the data flows from the input node to
the output node. Below are some examples:

Feedforward neural networks

Feedforward neural networks process data in one direction, from the input node to the output
node. Every node in one layer is connected to every node in the next layer. A feedforward
network uses a feedback process to improve predictions over time.

Backpropagation algorithm

Artificial neural networks learn continuously by using corrective feedback loops to improve
their predictive analytics. In simple terms, you can think of the data flowing from the input
node to the output node through many different paths in the neural network. Only one path is
the correct one that maps the input node to the correct output node. To find this path, the
neural network uses a feedback loop, which works as follows:

1. Each node makes a guess about the next node in the path.
2. It checks if the guess was correct. Nodes assign higher weight values to paths that
lead to more correct guesses and lower weight values to node paths that lead to incorrect
guesses.
3. For the next data point, the nodes make a new prediction using the higher weight
paths and then repeat Step 1.
4.
Convolutional neural networks
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
The hidden layers in convolutional neural networks perform specific mathematical functions,
like summarizing or filtering, called convolutions. They are very useful for image
classification because they can extract relevant features from images that are useful for image
recognition and classification. The new form is easier to process without losing features that
are critical for making a good prediction. Each hidden layer extracts and processes different
image features, like edges, color, and depth.
How to train neural networks?
Neural network training is the process of teaching a neural network to perform a task. Neural
networks learn by initially processing several large sets of labeled or unlabeled data. By using
these examples, they can then process unknown inputs more accurately.

Supervised learning

In supervised learning, data scientists give artificial neural networks labeled datasets that
provide the right answer in advance. For example, a deep learning network training in facial
recognition initially processes hundreds of thousands of images of human faces, with various
terms related to ethnic origin, country, or emotion describing each image.

The neural network slowly builds knowledge from these datasets, which provide the right
answer in advance. After the network has been trained, it starts making guesses about the
ethnic origin or emotion of a new image of a human face that it has never processed before.

What is Reinforcement Learning?

o Reinforcement Learning is a feedback-based Machine learning technique in which an

agent learns to behave in an environment by performing the actions and seeing the
results of actions. For each good action, the agent gets positive feedback, and for each
bad action, the agent gets negative feedback or penalty.
o In Reinforcement Learning, the agent learns automatically using feedbacks without
any labeled data, unlike supervised learning.
o Since there is no labeled data, so the agent is bound to learn by its experience only.
o RL solves a specific type of problem where decision making is sequential, and the
goal is long-term, such as game-playing, robotics, etc.
o The agent interacts with the environment and explores it by itself. The primary goal
of an agent in reinforcement learning is to improve the performance by getting the
maximum positive rewards.
o The agent learns with the process of hit and trial, and based on the experience, it
learns to perform the task in a better way. Hence, we can say that "Reinforcement
learning is a type of machine learning method where an intelligent agent
(computer program) interacts with the environment and learns to act within
that." How a Robotic dog learns the movement of his arms is an example of
Reinforcement learning.
o It is a core part of Artificial intelligence, and all AI agent works on the concept of
reinforcement learning. Here we do not need to pre-program the agent, as it learns
from its own experience without any human intervention.
o Example: Suppose there is an AI agent present within a maze environment, and his
goal is to find the diamond. The agent interacts with the environment by performing
some actions, and based on those actions, the state of the agent gets changed, and it
also receives a reward or penalty as feedback.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
o The agent continues doing these three things (take action, change state/remain in
the same state, and get feedback), and by doing these actions, he learns and
explores the environment.
o The agent learns that what actions lead to positive feedback or rewards and what
actions lead to negative feedback penalty. As a positive reward, the agent gets a
positive point, and as a penalty, it gets a negative point.

Terms used in Reinforcement Learning

o Agent(): An entity that can perceive/explore the environment and act upon it.
o Environment(): A situation in which an agent is present or surrounded by. In RL, we
assume the stochastic environment, which means it is random in nature.
o Action(): Actions are the moves taken by an agent within the environment.
o State(): State is a situation returned by the environment after each action taken by the
agent.
o Reward(): A feedback returned to the agent from the environment to evaluate the
action of the agent.
o Policy(): Policy is a strategy applied by the agent for the next action based on the
current state.
o Value(): It is expected long-term retuned with the discount factor and opposite to the
short-term reward.
o Q-value(): It is mostly similar to the value, but it takes one additional parameter as a
current action (a).

Key Features of Reinforcement Learning

o In RL, the agent is not instructed about the environment and what actions need to be
taken.
o It is based on the hit and trial process.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
o The agent takes the next action and changes states according to the feedback of the
previous action.
o The agent may get a delayed reward.
o The environment is stochastic, and the agent needs to explore it to reach to get the
maximum positive rewards.

Approaches to implement Reinforcement Learning

There are mainly three ways to implement reinforcement-learning in ML, which are:

1. Value-based:
The value-based approach is about to find the optimal value function, which is the
maximum value at a state under any policy. Therefore, the agent expects the long-
term return at any state(s) under policy π.
2. Policy-based:
Policy-based approach is to find the optimal policy for the maximum future rewards
without using the value function. In this approach, the agent tries to apply such a
policy that the action performed in each step helps to maximize the future reward.
The policy-based approach has mainly two types of policy:

o Deterministic: The same action is produced by the policy (π) at any state.
o Stochastic: In this policy, probability determines the produced action.

3. Model-based: In the model-based approach, a virtual model is created for the

environment, and the agent explores that environment to learn it. There is no
particular solution or algorithm for this approach because the model representation is
different for each environment.

Elements of Reinforcement Learning

There are four main elements of Reinforcement Learning, which are given below:

1. Policy
2. Reward Signal
3. Value Function
4. Model of the environment

1) Policy: A policy can be defined as a way how an agent behaves at a given time. It maps the
perceived states of the environment to the actions taken on those states. A policy is the core
element of the RL as it alone can define the behavior of the agent. In some cases, it may be a
simple function or a lookup table, whereas, for other cases, it may involve general
computation as a search process. It could be deterministic or a stochastic policy:

For deterministic policy: a = π(s)

For stochastic policy: π(a | s) = P[At =a | St = s]

2) Reward Signal: The goal of reinforcement learning is defined by the reward signal. At
each state, the environment sends an immediate signal to the learning agent, and this signal is
known as a reward signal. These rewards are given according to the good and bad actions
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
taken by the agent. The agent's main objective is to maximize the total number of rewards for
good actions. The reward signal can change the policy, such as if an action selected by the
agent leads to low reward, then the policy may change to select other actions in the future.

3) Value Function: The value function gives information about how good the situation and
action are and how much reward an agent can expect. A reward indicates the immediate
signal for each good and bad action, whereas a value function specifies the good state and
action for the future. The value function depends on the reward as, without reward, there
could be no value. The goal of estimating values is to achieve more rewards.

4) Model: The last element of reinforcement learning is the model, which mimics the
behavior of the environment. With the help of the model, one can make inferences about how
the environment will behave. Such as, if a state and an action are given, then a model can
predict the next state and reward.

The model is used for planning, which means it provides a way to take a course of action by
considering all future situations before actually experiencing those situations. The approaches
for solving the RL problems with the help of the model are termed as the model-based
approach. Comparatively, an approach without using a model is called a model-free
approach.
Passive and Active learning in Machine Learning

Machine learning is a subfield of artificial intelligence that deals with the creation of
algorithms that can learn and improve themselves without explicit programming. One of the
most critical factors that contribute to the success of a machine learning model is the quality
and quantity of data used to train it. Passive learning and active learning are two approaches
used in machine learning to acquire data.

Passive Learning:

Passive learning, also known as batch learning, is a method of acquiring data by processing a
large set of pre-labeled data. In passive learning, the algorithm uses all the available data to
learn and improve its performance. The algorithm does not interact with the user or request
additional data to improve its accuracy.
Example:- An example of passive learning is training a machine learning model to classify
emails as spam or not spam. The algorithm is fed a large dataset of labeled emails and uses it
to learn how to identify spam emails. Once the training is complete, the algorithm can
accurately classify new emails without any further input from the user.

Active Learning:

Active learning is a method of acquiring data where the algorithm interacts with the user to
acquire additional data to improve its accuracy. In active learning, the algorithm starts with a
small set of labeled data and requests the user to label additional data. The algorithm uses the
newly labeled data to improve its performance and may continue to request additional data
until a satisfactory level of accuracy is achieved.
 JIMS Engineering Management Technical Campus
Unit 3 Notes
Intelligent and Expert Sytem(AI-405T)
Example:- An example of active learning is training a machine learning model to recognize
handwritten digits. The algorithm may start with a small set of labeled data and ask the user to
label additional data that the algorithm is uncertain about. The algorithm uses the newly
labeled data to improve its accuracy, and the process repeats until the algorithm can
accurately recognize most handwritten digits.

Passive learning and Active learning

Difference Between Passive Learning and Active Learning:

The following table summarizes the differences between passive learning and active learning:

Passive Learning Active Learning

Uses a large set of pre-labeled data to train the Starts with a small set of labeled data and
algorithm requests additional data from the user

The algorithm interacts with the user to

The algorithm does not interact with the user
acquire additional data

It does not require user input after training is May continue to request additional data until
complete a satisfactory level of accuracy is achieved

Suitable for applications where a large dataset Suitable for applications where labeled data
is available is scarce or expensive to acquire