Yuba Raj Devkota

Kathmandu Shiksha Campus

 What is Neural Network?
• A neural network is a computational model inspired by the way biological neural
networks, such as the human brain, work. It consists of interconnected processing
units, or "neurons," organized into layers. Each neuron processes information and
passes it on to the next layer. Neural networks are designed to recognize patterns,
solve complex problems, and make decisions by learning from data.
• Here are the key components and concepts related to neural networks:
1.Neurons: Neurons are the basic processing units that receive inputs, apply weights to them, and
produce an output. The output is often transformed using an activation function.
2.Layers: Neurons are organized into layers within a neural network. Common layer types include
input layers, hidden layers, and output layers.
3.Weights and Biases: The connections between neurons have associated weights and biases,
which are adjusted during training to learn from the data. These weights determine the strength of
the connections between neurons.
4.Activation Functions: Activation functions introduce non-linearity to the model. Common
activation functions include sigmoid, ReLU (Rectified Linear Unit), and tanh (hyperbolic
tangent).
5.Feedforward and Backpropagation: In a feedforward neural network, information flows from
the input layer to the output layer. Backpropagation is used to calculate and adjust the gradients
of the loss function with respect to the network's weights and biases during training.
 Applications of Neural Network?
1. Image Recognition: Convolutional Neural Networks (CNNs) are widely used for tasks like image classification, object
detection, and image segmentation.
2. Natural Language Processing (NLP): Recurrent Neural Networks (RNNs) and more advanced models like Transformers are
used for tasks like language translation, sentiment analysis, and text generation.
3. Speech Recognition: Neural networks can be used for automatic speech recognition systems, enabling applications like voice
assistants.
4. Recommendation Systems: Neural networks can power recommendation engines in e-commerce, entertainment, and content
platforms to suggest products or content to users.
5. Autonomous Vehicles: Deep learning models are used in self-driving cars for tasks like object detection, path planning, and
control.
6. Healthcare: Neural networks are used in medical imaging for tasks like diagnosing diseases from X-rays, MRIs, and CT scans.
7. Finance: They are used for fraud detection, algorithmic trading, and credit scoring.
8. Gaming: Neural networks can be employed to create intelligent non-player characters (NPCs) and enhance game realism.
9. Robotics: In robotics, neural networks can be used for tasks like robot control and object recognition.
10.Anomaly Detection: Neural networks are used to detect anomalies in various data, such as network traffic, system logs, and
manufacturing quality control.
11.Language Generation: GPT-3 and similar models are capable of generating human-like text, making them useful for content
generation and chatbots.
 Artificial Neural Network (ANN)
• The term "Artificial Neural Network" is derived from Biological neural networks that develop the
structure of a human brain. Similar to the human brain that has neurons interconnected to one
another, artificial neural networks also have neurons that are interconnected to one another in
various layers of the networks. These neurons are known as nodes.

Biological Neural Network. Artificial Neural Network

 • For example, In OR Gate, the output depends upon input. Our brain does not perform the same
task. The outputs to inputs relationship keep changing because of the neurons in our brain, which
are "learning.“
The architecture of an artificial neural network:

Input Layer: As the name suggests, it accepts inputs in

several different formats provided by the programmer.

Hidden Layer: The hidden layer presents in-between

input and output layers. It performs all the
calculations to find hidden features and patterns.

Output Layer: The input goes through a series of

transformations using the hidden layer, which finally
results in output that is conveyed using this layer.

It determines weighted total is passed as an input to an activation function to produce

the output. Activation functions choose whether a node should fire or not. Only those
who are fired make it to the output layer.
 Activation Functions in Neural Networks
• It’s just a thing function that you use to get the output of node. It is also known as Transfer
Function.
• It is used to determine the output of neural network like yes or no. It maps the resulting values in
between 0 to 1 or -1 to 1 etc. (depending upon the function).
• The Activation Functions can be basically divided into 2 types-
1. Linear Activation Function
2. Non-linear Activation Functions
 Linear or Identity Activation Function

• The function is a line or linear.

Therefore, the output of the
functions will not be confined
between any range.
• Equation : f(x) = x
• Range : (-infinity to infinity)
• It doesn’t help with the
complexity or various
parameters of usual data that is
fed to the neural networks.
 Non-Linear Activation Function
• The Nonlinear Activation Functions
are the most used activation functions.
Nonlinearity helps to makes the graph
look something like this
• It makes it easy for the model to
generalize or adapt with variety of
data and to differentiate between the
output.
• The main terminologies needed to
understand for nonlinear functions
are:
• Derivative or Differential: Change in
y-axis w.r.t. change in x-axis. It is also
known as slope.
• Monotonic function: A function
which is either entirely non-increasing The Nonlinear Activation Functions are mainly divided on the
or non-decreasing. basis of their range or curves-
 1. Sigmoid or Logistic Activation Function
• The Sigmoid Function curve looks
like a S-shape.
• The main reason why we use
sigmoid function is because it exists
between (0 to 1). Therefore, it is
especially used for models where we
have to predict the probability as
an output. Since probability of
anything exists only between the
range of 0 and 1, sigmoid is the right
choice.
• The function is differentiable. That
means, we can find the slope of the
sigmoid curve at any two points. • The logistic sigmoid function can cause a neural network to get
stuck at the training time.
• The function is monotonic but • The SoftMax function is a more generalized logistic activation
function’s derivative is not. function which is used for multiclass classification.
 2. Tanh or hyperbolic tangent Activation Function
• tanh is also like logistic sigmoid but better.
The range of the tanh function is from (-1
to 1). tanh is also sigmoidal (s - shaped).
• The advantage is that the negative inputs
will be mapped strongly negative and the
zero inputs will be mapped near zero in the
tanh graph.
• The function is differentiable.
• The function is monotonic while
its derivative is not monotonic.
• The tanh function is mainly used
classification between two classes.
• Both tanh and logistic sigmoid activation
functions are used in feed-forward nets.
 Training and Testing – Basic Concept

• The main difference between training data and testing

data is that training data is the subset of original data
that is used to train the machine learning model,
whereas testing data is used to check the accuracy of
the model.
• The training dataset is generally larger in size
compared to the testing dataset. The general ratios of
splitting train and test datasets are 80:20, 70:30, or
90:10.
• Training data is well known to the model as it is used
to train the model, whereas testing data is like
unseen/new data to the model.
• Machine Learning algorithms enable the machines to
make predictions and solve problems on the basis of
past observations or experiences. These experiences or
observations an algorithm can take from the training
data, which is fed to it. Further, one of the great things
about ML algorithms is that they can learn and
improve over time on their own, as they are trained
with the relevant training data.
 Example/Task: Classify whether an email is spam (1) or not spam (0) based on its content.
Steps for testing a neural network:
• Data Preparation:
• You need a dataset of labeled examples, where each email is labeled as spam or not spam. Split your dataset into a
training set and a test set. The training set is used to train the neural network, and the test set is used to evaluate its
performance.
• Load the Pre-trained Model:
• If you have a pre-trained neural network (e.g., a trained model saved on disk), load it into your program. If not, you'll
need to train a neural network using the training set first.
• Data Preprocessing:
• Preprocess the test data in the same way you preprocessed the training data. This typically involves tasks like
tokenization, converting text data into numerical form (e.g., word embeddings), and normalizing numerical features.
• Inference:
• Use the pre-trained neural network to make predictions on the test data. For binary classification, the output will be a
probability score, which can be thresholder (e.g., using 0.5 as a threshold) to classify as spam or not spam.
 Performance Evaluation:
To assess the performance of your
neural network, you can use various
evaluation metrics such as accuracy,
precision, recall, F1-score, and ROC-
AUC. You can also create a confusion
matrix to see the number of true
positives, true negatives, false
positives, and false negatives.

Visualize Results (Optional):

You can create visualizations, such as ROC curves or precision-recall curves, to get a better
understanding of your model's performance.

Iterate and Improve:

Based on the results of testing, you may want to make adjustments to your neural network,
such as changing hyperparameters, modifying the architecture, or collecting more data, to
improve its performance.
 McCulloch-Pitts
Model of Neuron

• The McCulloch-Pitts neural model, which was the earliest ANN model, has only two types of
inputs — Excitatory and Inhibitory.
• The excitatory inputs have weights of positive magnitude and the inhibitory weights have
weights of negative magnitude. The inputs of the McCulloch-Pitts neuron could be either 0 or 1.
• It has a threshold function as an activation function. So, the output signal yout is 1 if the
input ysum is greater than or equal to a given threshold value, else 0.
• Simple McCulloch-Pitts neurons can be used to design logical operations. For that purpose, the
connection weights need to be correctly decided along with the threshold function (rather than the
threshold value of the activation function). For better understanding purpose, let us consider an
example:
• John carries an umbrella if it is sunny or if it is raining. There are four given situations. we need to
decide when John will carry the umbrella. The situations are as follows:
• First scenario: It is not raining, nor it is sunny
• Second scenario: It is not raining, but it is sunny
• Third scenario: It is raining, and it is not sunny
• Fourth scenario: It is raining as well as it is sunny
• To analyse the situations using the McCulloch-Pitts neural model, we can consider the input
signals as follows:
• X1: Is it raining?
• X2 : Is it sunny?
• So, the value of both scenarios can be either 0 or 1. We can use the value of both weights X1 and
X2 as 1 and a threshold function as 1. So, the neural network model will look like:
 So, we can say that,

• The truth table built with respect to the problem is depicted

above.
• From the truth table, we can conclude that in the situations
where the value of yout is 1, John needs to carry an umbrella.
• Hence, he will need to carry an umbrella in scenarios 2, 3
and 4.
Logic Gates with MP Neurons

• We can use McCulloch-Pitts neurons to implement the basic logic

gates.

• All we need to do is find the appropriate connection weights and

neuron thresholds to produce the right outputs for each set of inputs.
Realization of AND gate
• Looking at the logic table for the A^B, we can see that we only
want the neuron to output a 1 when both inputs are activated.

Presented By: Tekendra Nath Yogi 18

Contd..
• To do this, we want the sum of both inputs to be greater than the threshold,

but each input alone must be lower than the threshold.

• Let's use a threshold of 1 (simple and convenient!). So, now we need to

choose the weights according to the constraints :

• E.g., 0.6 and 0.6

• With these weights, individual activation of either input A or B will not

exceed the threshold, while the sum of the two will be 1.2, which exceeds

the threshold and causes the neuron to fire. Here is a diagram:

Presented By: Tekendra Nath Yogi 19

Contd…
• Here, letter theta denote the threshold.

• A and B are the two inputs. They will each be set to 1 or 0

depending upon the truth of their proposition.

• The red neuron is our decision neuron. If the sum of the

weighted inputs is greater than the threshold, this will
output a 1, otherwise it will output 0 .

• So, to test it by hand, we can try setting A and B to the

different values in the truth table and seeing if the decision
neuron's output matches the A^B column:
Presented By: Tekendra Nath Yogi 20
Contd..
• If A=0 & B=0 --> 0*0.6 + 0*0.6 = 0. This is not greater
than the threshold of 1, so the output = 0. Good!

• If A=0 & B=1 --> 0*0.6 + 1*0.6 = 0.6. This is not greater
than the threshold, so the output = 0. Good.

• If A=1 & B=0 --> exactly the same as above. Good.

• If A=1 & B=1 --> 1*0.6 + 1*0.6 = 1.2. This exceeds the
threshold, so the output = 1. Good.

Presented By: Tekendra Nath Yogi 21

Realization of OR Gate
• Logic table for the OR GATE is as shown in below:

• In this case, we want the output to be 1 when either or both of

the inputs, A and B, are active, but 0 when both of the inputs
are 0.

Presented By: Tekendra Nath Yogi 22

Contd..
• This is simple enough. If we make each synapse greater than the threshold,
then it'll fire whenever there is any activity in either or both of A and B.
This is shown in figure below. Synaptic values of 1.1 are sufficient to
surpass the threshold of 1 whenever their respective input is active.

Presented By: Tekendra Nath Yogi 23

Realization of NOT Gate
• Logic table for the NOT gate is as shown in table below:

A ¬A

0 1

1 0

• The NOT operator simply negates the input, When A is true

(value 1), ¬A is false (value 0) and vice-versa.

Presented By: Tekendra Nath Yogi 24

Contd...
• This is a little tricky. We have to change a 1 to a 0 - this is easy, just
make sure that the input doesn't exceed the threshold.

• However, we also have to change a 0 to a 1 - how can we do this?

• The answer is to think of our decision neuron as a tonic neuron - one

whose natural state is active.

• To make this, all we do is set the threshold lower than 0, so even

when it receives no input, it still exceeds the threshold.

• In fact, we have to set the synapse to a negative number (here use -

1) and the threshold to some number between that and 0 ( use -0.5).

Presented By: Tekendra Nath Yogi 25

Contd..
• The decision neuron shown below, with threshold -0.5 and
synapse weight -1, will reverse the input:
A=1 --> output = 0
A=0 --> output = 1

Presented By: Tekendra Nath Yogi 26

Realization of XOR Gate
• XOR (exclusive OR) operator actually put a spanner in the
works of neural network research for a long time because it is
not possible to create an XOR gate with a single neuron, or
even a single layer of neurons - we need to have two layers.

• The truth table for the XOR gate is as shown below:

Presented By: Tekendra Nath Yogi 27

Contd..
• Exclusive OR means that we want a truth value of 1 either when A is 1, or
when B is 1(i.e. A and B have different truth value), but not when both A and
B are 1 or 0(i.e., both have same truth value).

• The truth table on the above shows this.

• To use a real world example, in the question "would you like tea or coffee?",
the "or" is actually and exclusive or, because the person is offering you one
or the other, but not both. Contrast this with "would you like milk or
sugar?". In this case you can have milk, or sugar, or both.

Presented By: Tekendra Nath Yogi 28

Contd..
• The only way to solve this problem is to have a bunch of neurons working
together. start by breaking down the XOR operation into a number of
simpler logical functions:

• A xor B = (AvB) ^ ¬(A^B)

• This line of logic contains three important operations: an OR operator in the

brackets on the left, an AND operator in the brackets on the right, and
another AND operator in the middle.

• We can create a neuron for each of these operations, and stick them together
like this:

Presented By: Tekendra Nath Yogi 29

Contd..
• The upper of the two red neurons in the first layer has two inputs with synaptic
weights of 0.6 each and a threshold of 1, exactly the same as the AND gate we made
earlier. This is the AND function in the brackets on the right of the formula wrote
earlier.

• Notice that it is connected to the output neuron with a negative synaptic weight (-
2). This accounts for the NOT operator that precedes the brackets on the right hand
side.

• The lower of the two red neurons in the first layer has two synaptic weights of 1.1
and a threshold of 1, just like the OR gate we made earlier. This neuron is doing the
job of the OR operator in the brackets on the left of the formula.

• The output neuron is performing another AND operation - the one in the middle of
the formula. Practically, this output neuron is active whenever one of the inputs, A or
B is on, but it is overpowered by the inhibition of the upper neuron in cases when
Presented By: Tekendra Nath Yogi 30
both A and B are on.
 1. Single-layer feed-forward network
Neural Network 2. Multilayer feed-forward network
3. Adaline and Madaline Network
Architectures
4. Perceptron Neural Network

1. Single-layer feed-forward network

• In this type of network, we have only two layers

input layer and the output layer but the input layer
does not count because no computation is
performed in this layer.
• The output layer is formed when different weights
are applied to input nodes and the cumulative effect
per node is taken.
• After this, the neurons collectively give the output
layer to compute the output signals.
 2. Multilayer feed-forward network

• This layer also has a hidden layer that is internal to the network and has no direct contact with the external layer.
• The existence of one or more hidden layers enables the network to be computationally stronger, a feed-forward network
because of information flow through the input function, and the intermediate computations used to determine the
output Z.
• There are no feedback connections in which outputs of the model are fed back into itself.
 Adaline and Madaline Network
1. Adaline (Adaptive Linear Neural) :
• Adaline is a type of neural network that works with supervised learning, making binary decisions,
and performing regression tasks. It's designed as a single−layer model. Although Adaline has
similarities to the Perceptron, it also showcases some crucial differences.

• A network with a single linear unit is called Adaline (Adaptive Linear Neural). A unit with a linear
activation function is called a linear unit. In Adaline, there is only one output unit and output values
are bipolar (+1,-1). Weights between the input unit and output unit are adjustable.

• The learning rule is found to minimize the mean square error between activation and target values.
Adaline consists of trainable weights, it compares actual output with calculated output, and based
on error training algorithm is applied.
First, calculate the net input to your Adaline network then apply the activation function
to its output then compare it with the original output if both the equal, then give the
output else send an error back to the network and update the weight according to the
error which is calculated by the delta learning rule.
Architecture:

In Adaline, all the input neuron is directly connected to the

output neuron with the weighted connected path. There is a
bias b of activation function 1 is present.
 2. Madaline (Multiple Adaptive Linear Neuron) :
• The Madaline(supervised Learning) model consists of many Adaline in parallel with a single
output unit. The Adaline layer is present between the input layer and the Madaline layer hence
Adaline layer is a hidden layer. The weights between the input layer and the hidden layer are
adjusted, and the weight between the hidden layer and the output layer is fixed.
• It may use the majority vote rule, the output would have an answer either true or false. Adaline
and Madaline layer neurons have a bias of ‘1’ connected to them. use of multiple Adaline helps
counter the problem of non-linear separability.

Architecture:

There are three types of a layer present in Madaline First input

layer contains all the input neurons, the Second hidden layer
consists of an adaline layer, and weights between the input and
hidden layers are adjustable and the third layer is the output
layer the weights between hidden and output layer is fixed
they are not adjustable.
 Perceptron (The Simplest Artificial neural network)
• Perceptron is one of the first and
most straightforward models of
artificial neural networks. Despite
being a straightforward model, the
perceptron has been proven to be
successful in solving specific
categorization issues.
• A single-layer feedforward neural
network was introduced in the late
1950s by Frank Rosenblatt. It was
the starting phase of Deep
Learning and Artificial neural
networks.

It is the simplest type of feedforward neural network, consisting of a single layer of input nodes that are fully
connected to a layer of output nodes. It can learn the linearly separable patterns. it uses slightly different types of
artificial neurons known as threshold logic units (TLU). it was first introduced by McCulloch and Walter Pitts in
the 1940s.
 Natural Language Processing (NLP)

• NLP stands for Natural Language

Processing, which is a part of Computer
Science, Human language, and Artificial
Intelligence.
• It is the technology that is used by machines
to understand, analyze, manipulate, and
interpret human's languages.
• It helps developers to organize knowledge
for performing tasks such as translation,
automatic summarization, Named Entity
Recognition (NER), speech recognition,
relationship extraction, and topic
segmentation.
 • NLP helps users to ask questions about any subject and get a direct response
within seconds.
• NLP offers exact answers to the question means it does not offer unnecessary
and unwanted information.
Advantages of NLP • NLP helps computers to communicate with humans in their languages.
• It is very time efficient.
• Most of the companies use NLP to improve the efficiency of documentation
processes, accuracy of documentation, and identify the information from large
databases.

• NLP may not show context.

• NLP is unpredictable
Disadvantages of NLP • NLP may require more keystrokes.
• NLP is unable to adapt to the new domain, and it has a limited function
that's why NLP is built for a single and specific task only.
 Components of NLP
1. Natural Language Understanding (NLU)
• Natural Language Understanding (NLU) helps the machine to understand and analyze human
language by extracting the metadata from content such as concepts, entities, keywords, emotion,
relations, and semantic roles.
• NLU mainly used in Business applications to understand the customer's problem in both spoken
and written language.
• NLU involves the following tasks -
• It is used to map the given input into useful representation.
• It is used to analyze different aspects of the language.

2. Natural Language Generation (NLG)

• Natural Language Generation (NLG) acts as a translator that converts the computerized data into
natural language representation. It mainly involves Text planning, Sentence planning, and Text
Realization.
• The NLU is difficult than NLG.
 Applications of NLP
Question Answering focuses on building systems
1. Question Answering that automatically answer the questions asked by
humans in a natural language.

2. Spam Detection Spam detection is used to detect unwanted e-mails getting to a user's inbox.
 3. Sentiment Analysis 4. Machine Translation

Sentiment Analysis is also known as opinion mining. It is Machine translation is used to translate text or
used on the web to analyse the attitude, behaviour, and speech from one natural language to another
emotional state of the sender. This application is natural language.
implemented through a combination of NLP (Natural
Language Processing) and statistics by assigning the
values to the text (positive, negative, or natural), identify
the mood of the context (happy, sad, angry, etc.)

5. Spelling correction

Microsoft Corporation provides word processor

software like MS-word, PowerPoint for the
spelling correction.
 6. Speech Recognition
7. Chatbot
Speech recognition is used for converting spoken words
Implementing the Chatbot is one of the
into text. It is used in applications, such as mobile,
important applications of NLP. It is used by many
home automation, video recovery, dictating to
companies to provide the customer's chat
Microsoft Word, voice biometrics, voice user interface,
services.
and so on.

8. Information extraction

Information extraction is one of the most important

applications of NLP. It is used for extracting structured
information from unstructured or semi-structured
machine-readable documents.

9. Natural Language Understanding (NLU)

It converts a large set of text into more formal

representations such as first-order logic structures that
are easier for the computer programs to manipulate
notations of the natural language processing.
 How to build an NLP pipeline / Steps for NLP / Machine
Translation Steps
Step1: Sentence Segmentation
• Sentence Segment is the first step for building the NLP pipeline. It breaks the paragraph into separate sentences.
• Example: Independence Day is one of the important festivals for every Indian citizen. It is celebrated on the
15th of August each year ever since India got independence from the British rule. The day celebrates
independence in the true sense.
• Sentence Segment produces the following result:
1. "Independence Day is one of the important festivals for every Indian citizen."
2. "It is celebrated on the 15th of August each year ever since India got independence from the British rule."
3. "This day celebrates independence in the true sense.“

Step2: Word Tokenization

• Word Tokenizer is used to break the sentence into separate words or tokens.
• Example: JavaTpoint offers Corporate Training, Summer Training, Online Training, and Winter Training.
• Word Tokenizer generates the following result:
• "JavaTpoint", "offers", "Corporate", "Training", "Summer", "Training", "Online", "Training", "and",
"Winter", "Training", "."
Step3: Stemming
• Stemming is used to normalize words into its base form or root form. For example, celebrates,
celebrated and celebrating, all these words are originated with a single root word "celebrate." The
big problem with stemming is that sometimes it produces the root word which may not have any
meaning.
• For Example, intelligence, intelligent, and intelligently, all these words are originated with a single root
word "intelligen." In English, the word "intelligen" do not have any meaning.
Step 4: Lemmatization
• Lemmatization is quite similar to the Stemming. It is used to group different inflected forms of the
word, called Lemma. The main difference between Stemming and lemmatization is that it produces
the root word, which has a meaning.
• For example: In lemmatization, the words intelligence, intelligent, and intelligently has a root word
intelligent, which has a meaning.
Step 5: Identifying Stop Words
• In English, there are a lot of words that appear very frequently like "is", "and", "the", and "a". NLP
pipelines will flag these words as stop words. Stop words might be filtered out before doing any
statistical analysis.
• Example: He is a good boy.
Step 6: Dependency Parsing
• Dependency Parsing is used to find that how all the words in the sentence are related to each other.
Step 7: POS tags
• POS stands for parts of speech, which includes Noun, verb, adverb, and Adjective. It indicates that
how a word functions with its meaning as well as grammatically within the sentences. A word has
one or more parts of speech based on the context in which it is used.
• Example: "Google" something on the Internet.
• In the above example, Google is used as a verb, although it is a proper noun.
Step 8: Named Entity Recognition (NER)
• Named Entity Recognition (NER) is the process of detecting the named entity such as person name,
movie name, organization name, or location.
• Example: Steve Jobs introduced iPhone at the Macworld Conference in San Francisco, California.
• Step 9: Chunking
• Chunking is used to collect the individual piece of information and grouping them into bigger
pieces of sentences.
 Phases of NLP

1. Lexical Analysis and Morphological

• The first phase of NLP is the Lexical Analysis. This phase
scans the source code as a stream of characters and converts it
into meaningful lexemes. It divides the whole text into
paragraphs, sentences, and words.

2. Syntactic Analysis (Parsing)

• Syntactic Analysis is used to check grammar, word
arrangements, and shows the relationship among the words.
• Example: Agra goes to the Poonam
• In the real world, Agra goes to the Poonam, does not make any
sense, so this sentence is rejected by the Syntactic analyzer.
3. Semantic Analysis
• Semantic analysis is concerned with the meaning representation. It mainly focuses
on the literal meaning of words, phrases, and sentences.

4. Discourse Integration
• Discourse Integration depends upon the sentences that proceeds it and also
invokes the meaning of the sentences that follow it.

5. Pragmatic Analysis
• Pragmatic is the fifth and last phase of NLP. It helps you to discover the intended
effect by applying a set of rules that characterize cooperative dialogues.
• For Example: "Open the door" is interpreted as a request instead of an order.
Semantic Analysis
Semantic analysis in NLP focuses on understanding the meanings of individual words, phrases, sentences, and entire texts,
independent of context. It involves deciphering the literal meaning or dictionary definitions of the words and phrases. This
process often involves tasks like word sense disambiguation, semantic role labeling, and named entity recognition.
Example of Semantic Analysis:
Consider the sentence: "The bank will close at 5 PM."
Semantic analysis helps the machine understand that "bank" refers to a financial institution (as opposed to the side of a river),
"close" means to cease operations (rather than to move physically closer to something), and "5 PM" indicates a specific time of
day. The analysis focuses on decoding the literal meanings of these terms to grasp the sentence's overall meaning.
Pragmatic Analysis
Pragmatic analysis goes beyond the literal meanings to understand the intended message in its specific context, including the
speaker's or writer's intentions, the relationship between the speaker and the listener, and the circumstances surrounding the
communication. It involves interpreting implied meanings, understanding idiomatic expressions, and recognizing the effect of
context on meaning.
Example of Pragmatic Analysis:
Consider the same sentence but in a different scenario: "The bank will close at 5 PM, but you can use the ATM outside."
In this case, pragmatic analysis helps the machine understand that although the physical location closes at 5 PM, the speaker
implies that banking services are still accessible through the ATM. Furthermore, it might infer that this information is being
provided as a helpful suggestion to someone who may need banking services after hours.
Pragmatic analysis would consider the context (e.g., the conversation taking place at 4:45 PM) and understand the urgency or the
convenience being highlighted by mentioning the ATM's availability.
 Why NLP is difficult? / Challenges of NLP
NLP is difficult because Ambiguity and Uncertainty exist in the language.
Ambiguity: There are the following three ambiguity -
1. Lexical Ambiguity: Lexical Ambiguity exists in the presence of two or more possible meanings
of the sentence within a single word. Example:
• Manya is looking for a match.
• In the above example, the word match refers to that either Manya is looking for a partner or
Manya is looking for a match. (Cricket or other match)
2. Syntactic Ambiguity: Syntactic Ambiguity exists in the presence of two or more possible
meanings within the sentence. Example:
• I saw the girl with the binocular.
• In the above example, did I have the binoculars? Or did the girl have the binoculars?
3. Referential Ambiguity
• Referential Ambiguity exists when you are referring to something using the pronoun.
• Example: Kiran went to Sunita. She said, "I am hungry."
• In the above sentence, you do not know that who is hungry, either Kiran or Sunita.
 Parameters used in NLP

• Hyperparameters:
• Learning Rate, Batch Size, Dropout rate, Hidden Layer size etc.
• Architecture-Specific Parameters:
• No of transformers layers, Hidden size, Convolution Kernel Size etc.
• Tokenization and Text Processing Parameters:
• Vocabulary size, Min Token Frequency, Max Sequence Length etc.
• Evaluation Parameters:
• Evaluation metrics (Accuracy, score), Validation Split etc.
• Hardware and Runtime Parameters:
• CPU configuration metrics, Parallelism parameters etc.