Scribd
Upload
61 views

Lecture Notes 3

This document provides an overview of machine learning and deep learning concepts. It discusses how deep learning can model complex input-output relationships better than traditional machine learning algorithms. It then compares human brain neurons to artificial neurons in neural networks, describing the key components of each. Artificial neurons perform summation and activation, similarly to biological neurons. Common activation functions like sigmoid, tanh, and ReLU are also introduced.

Uploaded by

fgsfgs
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
61 views

Lecture Notes 3

This document provides an overview of machine learning and deep learning concepts. It discusses how deep learning can model complex input-output relationships better than traditional machine learning algorithms. It then compares human brain neurons to artificial neurons in neural networks, describing the key components of each. Artificial neurons perform summation and activation, similarly to biological neurons. Common activation functions like sigmoid, tanh, and ReLU are also introduced.

Uploaded by

fgsfgs
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Chapter 1 Introduction to Machine Learning

The output needs to be either turn, drive fast or slowly, apply brakes,
etc. As you might think, the relationship between input variables and
output variables is pretty complex in nature. Hence, the traditional
machine learning algorithm finds it hard to map this kind of relationship.
Deep learning outperforms machine learning algorithms in such
situations as it is able to learn those nonlinear features as well.

Human Brain Neuron vs. Artificial Neuron


As mentioned, deep learning is extension of neural networks only and
also known as deep neural networks. Neural networks are a little different
compared to other machine learning algorithms in terms of learning.
Neural networks are loosely inspired by neurons in the human brain.
Neural networks are made up of artificial neurons. Although I don’t claim
to be an expert of neuroscience or functioning of the brain, let me try to
give you a high-level overview of “how the human brain functions.” As you
might be already aware, the human brain is made up of billions of neurons
and an incredible number of connections between them. Each neuron
is connected to multiple other neurons, and they repeatedly exchange
information (signal). Each activity that we do physically or mentally fires
up a certain set of neurons in our brains. Now, every single neuron consists
of three basic components.

• Dendrites

• Cell body

• Terminals

23
Chapter 1 Introduction to Machine Learning

As we can see in Figure 1-10, the dendrites are responsible for


receiving the signal from other neurons. A dendrite act as a receiver to
the particular neuron and passes information to the cell body where this
specific information is processed. Now, based on the level of information,
it either activates (fires up) or doesn’t trigger. This activity depends on a
particular threshold value of the neuron. If the incoming signal value is
below that threshold, it would not fire; otherwise, it activates. Finally, the
third component are the terminals that are connected with dendrites of
other neurons. Terminals are responsible for passing on the output of the
particular neuron to other relevant connections.

Figure 1-10.  Neuron

Now, we come to the artificial neuron, which is the basic building


block of a neural network. A single artificial neuron consists of two
parts mainly; one is the summation, and other is activation, as shown
in Figure 1-11. This is also known as a perceptron. Summation refers to
adding all the input signals, and activation refers to deciding whether the
neuron would trigger or not based on the threshold value.

24
Chapter 1 Introduction to Machine Learning

Figure 1-11.  Artificial neuron

Let’s say we have two binary inputs (X1, X2) and the weights of their
respective connections (W1, W2). The weights can be considered similar
to the coefficients of input variables in traditional machine learning.
These weights indicate how important the particular input feature is in the
model. The summation function calculates the total sum of the input. The
activation function then uses this total summated value and gives a certain
output, as shown in Figure 1-12. Activation is sort of a decision-making
function. Based on the type of activation function used, it gives an output
accordingly. There are different types of activation functions that can be
used in a neural network layer.

Figure 1-12.  Neuron calculation

25
Chapter 1 Introduction to Machine Learning

A
 ctivation Functions
Activation functions play a critical role in neural networks as the output
varies based on the type of activation function used. There are typically
four main activation functions that are widely used. We will briefly cover
these in this section.

Sigmoid Activation Function


The first type of activation function is a sigmoid function. This activation
function ensures the output is always between 0 and 1 irrespective of
the input, as shown in Figure 1-13. That’s why it is also used in logistic
regression to predict the probability of the event.

1
f ( x) =
1 + e- x

Figure 1-13.  Sigmoid

H
 yperbolic Tangent
The other activation function is known as the hyperbolic tangent activation
function, or tanh. This function ensures the value remains between -1 to 1
irrespective of the output, as shown in Figure 1-14. The formula of the tanh
activation function is as follows:

26
Chapter 1 Introduction to Machine Learning

e2 x - 1
f ( x) =
e2 x + 1

Figure 1-14.  Tanh

Rectified Linear Unit


The rectified linear unit (relu) has been really successful over the past
couple of years and has become the default choice for the activation
function. It is powerful as it produces a value between 0 and ∞. If the input
is 0 or less than 0, then the output is always going to be 0, but for anything
more than 0, the output is similar to the input, as shown in Figure 1-15.
The formula for relu is as follows:
f(x)= max⁡(0,x)

Figure 1-15.  Relu

27

You might also like