Artificial Neural
Network
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
What are artificial neural networks
(ANN)?
Neural networks were first developed in the 1950s to address this issue.
An artificial neural network is an attempt to simulate the network of neurons that
make up a human brain so that the computer will be able to learn things and
make decisions in a human like manner.
ANNs are created by programming regular computers to behave as though they
are interconnected brain cells.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Neuron versus Artificial Neuron
Neural networks reflect the
behavior of the human
brain, allowing computer
programs to recognize
patterns and solve
common problems in the
fields of AI, machine
learning, and deep
learning.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
How do artificial neural networks
work?
The input layer receives various forms of information from the outside world. This is
the data that the network aims to process or learn about. From the input unit, the
data goes through one or more hidden units. The hidden unit’s job is to transform
the input into something the output unit can use.
The majority of neural networks are fully connected from one layer to another.
These connexions are weighted; the higher the number the greater influence
one unit has on another, similar to a human brain. As the data goes through
each unit the network is learning more about the data.
In order for ANNs to learn, they need to have a tremendous amount of
information thrown at them called a training set. When you are trying to teach
an ANN how to differentiate a cat from dog, the training set would provide
thousands of images tagged as a dog so the network would begin to learn.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Why use Neural networks?
A trained neural network can be thought of as an “expert” in the category of information it has
been given to analyse. This expert can then be used to provide projections given new situations
of interest and answer “what if” questions.
Other advantages include:
1.Adaptive learning: An ability to learn how to do tasks based on the data given for training or
initial experience.
2.Self-Organisation: An ANN can create its own organisation or representation of the information
it receives during learning time.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
What are artificial neural networks
used for?
Google uses a 30-layered neural network to power Google Photos as well as to
power its “watch next” recommendations for YouTube videos.
Facebook uses artificial neural networks for its DeepFace algorithm, which can
recognise specific faces with 97% accuracy.
It’s also an ANN that powers Skype’s ability to do translations in real-time.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Network Layers
• Input units:- The activity of the input units
represents the raw information that is fed into
the network. this also called input layer.
• Hidden units:- The activity of each hidden unit is
determined by the activities of the input units
and the weights on the connections between the
input and the hidden units. this also called
hidden layer.
• Output units:- The behaviour of the output units
depends on the activity of the hidden units and
the weights between the hidden and output
units. this also called output layer.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Procedures of a Perceptron
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
A Single Neuron
The basic unit of computation in a neural network is the neuron, often called as a node or unit.
It receives input from some other nodes, or from an external source and computes an output.
Each input has an associated weight (w), which is assigned on the basis of its relative
importance to other inputs.
The network takes numerical inputs X1 and X2 and has
weights w1 and w2 associated with those inputs.
Additionally, there is another input 1 with weight b (called
the Bias) associated with it.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Activation Function
The output Y from the neuron is computed as shown in the
Figure. The function f is non-linear and is called
the Activation Function. The purpose of the activation
function is to introduce non-linearity into the output of a
neuron. This is important because most real world data is
non linear and we want neurons to learn these non linear
representations.
Every activation function (or non-linearity) takes a single
number and performs a certain fixed mathematical
operation on it.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Activation Functions
Sigmoid: takes a real-valued input and squashes it to range between 0 and 1.
◦ σ(x) = 1 / (1 + exp(−x))
Softmax function: In classification tasks, we generally use a Softmax function as the Activation
Function in the Output layer of the Multi Layer Perceptron to ensure that the outputs are
probabilities and they add up to 1.
◦ Probability (Pass) + Probability (Fail) = 1
tanh: takes a real-valued input and squashes it to the range [-1, 1]
ReLU: ReLU stands for Rectified Linear Unit. It takes a real-valued input and thresholds it at zero
◦ f(x) = max(0, x)
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Feed Forward Propogation
In order to generate some output, the input data should be fed in the forward direction only.
The data should not flow in reverse direction during output generation otherwise it would form
a cycle and the output could never be generated.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-1 (Feed Forward Propogation)
To build a model that predicts height of tree using moisture and grain size
◦ 58% moisture and 1.3mm grain size.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-1 (Feed Forward Propogation)
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Back propagation Neural Network
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-2
Neural network training is about finding weights that minimize prediction error. We usually start
our training with a set of randomly generated weights. Then, backpropagation is used to update
the weights in an attempt to correctly map arbitrary inputs to outputs.
Our initial weights will be as following: w1 = 0.11, w2 = 0.21, w3 = 0.12, w4 = 0.08, w5 = 0.14 and
w6 = 0.15
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-2
Our single sample is as following
inputs=[2, 3] and output=[1].
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-2
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-2
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-2
We can notice that the prediction 0.26 is a little
bit closer to actual output than the previously
predicted one 0.191. We can repeat the same
process of backward and forward pass
until error is close or equal to zero.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-3
We’ll choose random weights with which to begin the
perceptron. Say our perceptron begins with the weights:
When it is excited, it outputs 1 , and at other times it outputs -1
The perceptron predicts that this example’s label will be 1
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-3
A perceptron normally starts with random weights. But each
time it makes a mistake by predicting 1 when the correct
answer is -1 , we change all weights as follows:
Here r represents the learning rate, which should be chosen to
be some small number like 0.05.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-3
To compute the prediction for Iris B, we determine
exceeds 0. Since it does not, the perceptron predicts that Iris B
is labeled -1.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Example-3
How the weights change while going through all the examples three times
(iterations) over.
On the first pass through the examples, the perceptron classified only 1 of the
5 flowers correctly. On the second pass, it got 2 correct. And on the third pass,
it got 3 correct.
It finally gets all 5 flowers right on the 1358 th iteration through the
example.
After going through all these iterations using r= 0.05, the final weights would
be:
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Neural Networks in Weka
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Introducing Deep Learning
Deep learning is a subset of machine
learning, which is essentially a neural
network with three or more layers.
These neural networks attempt to
simulate the behavior of the human
brain allowing it to “learn” from large
amounts of data.
While a neural network with a single
layer can still make approximate
predictions, additional hidden layers can
help to optimize and refine for accuracy.
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
ANN trying to predict the image(data)
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Deep Learning for Image Recognition
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Convolutional Neural Network
Convolutional neural networks are distinguished from other neural networks by their
superior performance with image, speech, or audio signal inputs. They have three main
types of layers, which are:
•Convolutional layer
•Pooling layer
•Fully-connected (FC) layer
•Flatten Layer
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Convolutional Layer
The filter size is typically a 3x3 matrix
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Convolutional Layer
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Pooling Layer
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Flatten Layer
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Fully Connected Layer
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Convolutional Neural Network
Architecture
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Image to CNN
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
How CNN recognizes images?
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
How CNN recognizes images?
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Digit Image Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Digit Image Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Digit Image Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Digit Image Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Digit Image Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Koala Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Koala Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
Koala Detection
BİL477-2021-2022-FALL- INTRODUCTION TO DATA MINING DR. GÖKHAN MEMIŞ
GENEL- PUBLIC
