AI2111 Smart Science AI-Powered Discoveries for Academic Minds 1

Deep Learning Basics

Module 2 – Deep Learning Basics

At the end of this module, you are expected to:

1. Identify the basic knowledge in deep learning;
2. Understand how neural networks work; and
3. Recognize the convolutional layer.

INTRODUCTION

Deep learning is fundamentally a machine learning method. It teaches a computer how to anticipate
and categorize data by filtering inputs through layers. Observations can be expressed verbally, visually, or
sonically. The way the human brain processes information is the source of inspiration for deep learning.
Its goal is to imitate brain function in order to produce some genuine magic.
There are around 100 billion neurons in the human brain. Approximately 100,000 of a neuron's neighbors
are connected to it. That is our goal, but we want to do it at a level and in a manner that is machine-friendly. In
terms of neurons, axons, dendrites, and so on, what does this mean? An axon, dendrites, and body make up a
neuron. The axon carries the signal from one neuron to the dendrites of the subsequent cell. A synapse is the point
in the link where the signal passes. Neurons are not very useful on their own. However, when you have a large
number of them, they combine to produce some quite powerful magic. That's how a deep learning algorithm
works! Observation provides information, which you then incorporate into a single layer. The output produced
by that layer is used as the input for the following layer, and so on.

Thus, a signal or signals (input values) are received by the neuron (or node), and they flow through the
neuron. The output signal is sent by that neuron. Consider your senses, such as sight, smell, and touch, to be the
input layer. These variables stand alone for a solitary observation. This data is divided into numerical values and
computer-useable binary data bits. (To ensure that these variables are inside the same range, you must either
normalize or standardize them.) What role do synapses play? Artificial Neural Networks require weights, which
are assigned to individual synapses (ANNs). Weights are used in ANN training. By adjusting the weights, the
ANN controls how far signals are transmitted forward. Throughout your network's training process, you alter the
weights.

ARTIFICIAL NEURAL NETWORKS

Neural networks, a kind of artificial intelligence, teach computers to process information similarly to the
human brain. Deep learning is a subset of machine learning that uses networked nodes or neurons stacked to
resemble the structure of the human brain. This adaptive strategy may be used by computers to learn from their
mistakes and continue improving. Consequently, artificial neural networks strive to provide more precise
responses to difficult problems like facial recognition and document summarization. Neural networks enable
computers to make intelligent decisions with little assistance from humans. Their ability to understand and
simulate complex, nonlinear relationships between input and output data is the reason behind this.
Course Module
 AI2111 Smart Science AI-Powered Discoveries for Academic Minds 2
Deep Learning Basics

Neural networks come in several varieties. They are often divided into feedback and feedforward
networks.
A feedforward network is a network with hidden layers, inputs, and outputs. There is just one direction in
which the signals may go: forward. A layer where computations are done receives input data. Every processing
component calculates using the total of its weighted inputs. The updated values feed the subsequent layer as the
new input values (feed-forward). This defines the output and is repeated through all the levels. For instance,
feedforward networks are frequently utilized in data mining.
A feedback network, like a recurrent neural network, has feedback channels. This suggests that they can
have signals traveling in both directions by using loops. Neuronal connections of any kind are allowed. Because
of its loops, this type of network is a non-linear dynamic system that changes continuously until it reaches
equilibrium. Feedback networks are often used in optimization problems to determine the best arrangement of
connected components.
Numerous sectors have found applications for neural networks, including the following:

• Medical picture categorization for medical diagnosis

• Behavioral data analysis and social network filtering for targeted marketing
• Financial forecasts using financial instrument historical data processing
• Forecasting electrical load and energy consumption
• Procedure and quality assurance
• Identification of a chemical compound

Below, we list four significant uses for neural networks.

Computer Vision
The capacity of computers to derive knowledge and insights from pictures and movies is known as computer
vision. Computers can differentiate and recognize pictures that resemble people thanks to neural networks. There
are several uses for computer vision, including the following:
• Self-driving automobiles that employ visual recognition to identify other drivers and road signs
• Using content moderation, picture and video archives may automatically filter out offensive or dangerous
information.
• Using facial recognition to detect faces and distinguish features like facial hair, spectacles, and open eyes
• Labeling images to recognize apparel, safety gear, company logos, and other details
Recognition of Speech
Despite differences in language, accent, pitch, tone, and speech patterns, neural networks are capable of analyzing
human voice. Speech recognition is used by virtual assistants such as Amazon Alexa and automatic transcription
software to do duties such as these:
• Help contact center representatives and categorize calls automatically
• Real-time conversion of clinical talks into documentation
• Subtitle meetings and videos accurately to increase the reach of your work.
Natural Language Interpretation
Text that has been naturally written by humans may be processed using natural language processing, or NLP.
Computers can extract meaning and insights from textual data and texts with the use of neural networks. NLP is
useful in a variety of contexts, such as the following:
Course Module
 AI2111 Smart Science AI-Powered Discoveries for Academic Minds 3
Deep Learning Basics

• Chatbots and automated virtual agents

• Automated categorization and arrangement of textual data
• Analysis of business intelligence for lengthy documents, such as forms and emails
• Indexing of sentiment-indicating keywords, such as favorable and unfavorable social media remarks
• Summarizing documents and creating articles on a certain subject

CONVOLUTIONAL NEURAL NETWORKS

Neural networks are the foundation of deep learning techniques, which are a subset of machine learning.
They consist of node layers with one or more hidden levels, an input layer, and an output layer. Each node has a
weight and a threshold that are related to each other. If a node's output surpasses a predetermined threshold value,
it becomes active and transmits data to the network's next tier. If not, nothing is forwarded to the next tier of the
network. There are several types of neural networks, and they are used for a wide range of data types and
application situations. For example, convolutional neural networks, or CNNs or ConvNets, are more commonly
utilized in computer vision and classification applications than recurrent neural networks, which are more
commonly used in speech recognition and natural language processing applications. Before CNNs were invented,
object recognition in images needed time-consuming, manual feature extraction methods. Conversely,
convolutional neural networks now provide a more scalable way to identify objects in photos and categorize
images by employing the ideas of linear algebra and matrix multiplication to identify patterns in images. However,
they can be computationally demanding, requiring GPUs in order to train models.

CONVOLUTIONAL LAYER

The convolutional layer is the first layer of a convolutional network. Even though pooling layers or
additional convolutional layers may come after convolutional layers, the fully-connected layer is always the last
layer. With every layer, the CNN gets more intricate and can recognize a wider portion of the image. Prior layers
highlight fundamental components like colors and boundaries. As the image input passes through the layers of
the CNN, it starts to detect larger parts or features of the item, and finally it recognizes the desired object. The
convolutional layer, a CNN's primary structural element, is where most of the processing occurs. A feature map,
a filter, and input data are required. For the time being, let's assume that the input will be a color image made up
of a three-dimensional matrix of pixels. This means that the RGB values of an image will match the three
dimensions of the input—height, width, and depth. Furthermore, we have a feature detector, which is also called
a kernel or filter, which will search the receptive fields of the picture to see whether the feature is there. This
procedure is known as a convolution.

A two-dimensional (2-D) array of weights that represents a portion of the image serves as the feature
detector. The filter size is usually a 3x3 matrix, however they can vary in size; this also establishes the size of the
receptive field. After applying the filter to a portion of the picture, the dot product between the input pixels and
the filter is computed. An output array is then supplied with this dot product. The filter then moves one step
forward and backward until the kernel has scanned the entire picture. A feature map, activation map, or convolved
feature is the result of the series of dot products created from the input and filter.

It should be noted that parameter sharing, or the feature detector moving over the picture with fixed
weights, occurs. Gradient descent and backpropagation are used in training to modify certain parameters, such as
the weight values. However, before the neural network starts training, three hyperparameters that impact the
output's volume size must be established. Among them are:
Course Module
 AI2111 Smart Science AI-Powered Discoveries for Academic Minds 4
Deep Learning Basics

1. The number of filters affects the depth of the output. For example, getting three distinct feature maps with three
distinct filters would provide three depths.
2. It is known how many pixels, or strides, the kernel traverses over the input matrix. Although stride numbers of
two or higher are unusual, a larger stride corresponds to a lower output.
3. In most cases, zero-padding is employed when the filters are too large for the input picture. This results in an
output that is larger or equal in size by setting any elements that are outside of the input matrix to zero. Three
different kinds of padding exist:
• No padding is sometimes referred to as valid padding. In this instance, if dimensions are not aligned, the
last convolution is omitted.
• The same padding makes sure that the size of the input layer and the output layer are the same.
• Complete padding: By adding zeros to the input's border, this kind of padding makes the output larger.

Course Module
 AI2111 Smart Science AI-Powered Discoveries for Academic Minds 5
Deep Learning Basics

References and Supplementary Materials

1. Buduma, N. & Lacascio, N. (2017). Fundamentals of deep learning. O’Reilly Media.

2. [Link]
3. [Link]
network/#:~:text=A%20neural%20network%20is%20a,that%20resembles%20the%20h
uman%20brain.
4. [Link]
45591a1c4abb#:~:text=At%20a%20very%20basic%20level,the%20human%20brain%20
filters%20information.

Course Module