Name：Abid Ali

Submitted to : Dr Rawaba Arif

University of Jhang
Neural Networks and a Deep Dive Into Artificial Neural Networks

Introduction

1. Biological Neurons
Biological neurons are specialized cells in the brain responsible for transmitting information
throughout the body. Structurally, each neuron has dendrites that receive signals, a cell body that
processes these signals, and an axon that carries the output signal to other neurons. Information
travels through neurons via electrical impulses and across neurons via chemical
neurotransmitters. From the axon terminals, information "jump" to the next neuron . (Fausett,
2016) Each time we learn, new neuronal connections are formed, and frequently used pathways
are strengthened over time. This dynamic structure allows the brain to adapt, store memories,
and process complex information. The nervous system in vertebrates (e.g., human beings) is
where intelligence resides. It is composed of the brain, the spinal cord, and peripheral nerves all
over the body. (D. Purves, 2017) (E. R. Kandel, 2000).

2. Biological Neural Network

A neural network is a computational system modeled on the architecture of biological neural
network. In this network, computations that support intelligent functions such as memorizing and
forgetting, learning, and decision making tasks occur in various neural connections in the brain.
Human brain have these three basic levels of organization. On the top level, the human nervous
system is a vast mesh of different types of neural networks organized hierarchically to support
different computational functions such as vision, audition, emotion, etc. On the medium level,
the basic unit is a neuron, which is composed of a soma, with many dendrites to receive inputs,
and a single axon (usually with many branches) to send out outputs. These neurons connect to
one another via synapses. On the bottom level, different types of ion channels form the
molecular basis for electrical activities in neurons and support the transmission and processing of
information. The number and properties of ion channels in a neuron are regulated by other
cellular signaling machinery. Neural networks are capable of solving challenging real-world
problems by learning from examples. They are highly useful in tasks like classification,
prediction, filtering, optimization, and pattern recognition, where conventional algorithms often
struggle.

Figure A The human nervous system (left panel) has different types of neural networks whose basic functional elements
are neurons and synapses, in which different types of ion channels underlie electrical neuronal activity.
Characteristics Of Biological Neural Network
 Data is processed at local level
 Distributed memory
 Strength of a synapse can be changed by experience
 Synaptic neurotransmitters can be excitatory or inhibitory

3. Artificial Neuron
An artificial neuron simulates the operations of a biological neuron. It consists of three main
components:

 Weights associated with each input,(Interconnection between ANs)

 Summation of weighted input(by training,algorithum and learning)
 Activation function to determine the output.

One early model, the Perceptron, uses multiple binary inputs to compute a single binary output.
Later improvements, like the Sigmoid neuron and Rectified Linear Unit (ReLU), allow smoother
learning and better performance by enabling minor changes in weights to cause proportional
output changes crucial for deep learning.

Working of Perceptron
In above figure the perceptron have three inputs which might be greater or fewer than this
number in general .Rosenblatt developed a simple rule i.e he created the weights and by
multiplying the all inputs with weights and if the weighted sum inputs are equal or greater than
threshold number the neuron output would be 1 means neuron is activated and if weighted
summed inputs are less than threshold number then neuron output would be 0 means neuron is
inactivated (Nielsen, 2019).
 This is how perceptron works on basic mathematical model and acts as device that makes judgments by balancing
facts

4. Artificial Neural Network (ANNs)

Artificial neural networks ANNs are artificially designed data processing system that
works in same way as human neural network do execute tasks with same or more
performance. (Hassoun, 1995) .These ANNs are actually extensions of mathematicals
models of human cognition or neuro biology and following assumptions are taken for
ANNs creation
 Large number of neurons for processing information
 Connections between neurons for signal smooth transfer
 Connection strength between neurons is termed as weight
 Output neurons generate by applying activation function

Artificial Neural Networks (ANNs) are composed of layers of artificial neurons. Nodes in
hidden layers are connected to nodes in input layes and nodes of output layer are connected
to nodes in hidden layer .Link act as synapses in human brain to transfer signal .Input
Layers receive raw data and hidden layers processe data and transferred to output layer.
the They perform computations by transmitting information from the input layer through
hidden layers to the output layer. Each neuron passes information using activation
functions like Sigmoid, Tanh, or ReLU. The connections between neurons have adjustable
weights, and through learning algorithms, these weights are tuned to optimize performance.
ANNs can learn complex patterns and make intelligent decisions without needing to be
explicitly programmed for specific tasks. (Chiu, 2018)

5. Characteristics of Biological Neural Network and Artificial Neural

Network
1. Learning:

BNNs adjust synaptic strengths ANNs adjust connection weights.

2. Memory:

BNNs store long-term information in synapses ANNs encode knowledge in weights.

3. Signal Processing:

BNNs use electrochemical signal ANNs use mathematical functions.

4. Processing

BNNs have massive parallel processing ANNS have serial processing

5.Speed

BNNs have slow processing speed ANNs have faster speed

6.Storage

BNNs have 1015 interconnections and neurons ANNs have Replaceable information
storage

on order of 1011
1. History of Artificial Neural Network
In 1940 Warren McCulloch and Walter Pitts introduced the first mathematical model of a
neuron which is basic algorithm based system to mimc activities of human brain. In 1957 Frank
Rosenblatt developed the Perceptron, pioneering machine learning concepts . This was a
revolutionary algorithm based model which had ability to solve complicated recognition based
problems .In 1970s-80s Research slowed due to limited computing resources and criticism from
scholars.In 2000 the computer science experts got what they actually need that is increasing
processing power and hardware. With the advent of powerful GPUs, large datasets, and better
algorithms (like Backpropagation and ReLU activation), neural networks resurged dramatically,
enabling breakthroughs in fields like computer vision and natural language processing.
Today, neural networks form the backbone of modern Artificial Intelligence (AI)
systems.

2. Analysis of neural network

Architecture of Artificial Neural Network

A typical ANN consists of:

1. Input Layer: Receives raw data.

2. Hidden Layers: Process inputs through weighted transformations.

3. Output Layer: Produces the final decision or prediction.

Architectures can be:

Feedforward Networks:
Data flows only forward; commonly used for pattern recognition.This is only one way signal
transferring model and with zero or many hidden layers but with an input layer and a single
output layer .

Feedback/Recurrent Networks:

Data can cycle backward, enabling memory, important for sequence-based tasks like speech
recognition.Double way signal transferring capacity due to network loop of hidden layers is
present and they are mostly used in job where series of event have to do in specific sequence .

Additional architectural components:

Weights: Strength of link or connection between neurons are measured in term of mathematical
number is called weight and weight between neurons changes throughout learning of ANNs and
are adjusted during training.

Activation Functions: These are mathematical equations that calculate the neural network
output ared called activation functions Like Sigmoid, Tanh, and ReLU, introducing non-
linearity.

Cost Functions: Measure the error between predicted and actual outputs, guiding optimization.

Characteristics of ANNs
Due to following characteristics the ANNs have capacity to solve problems like human ;

 ADAPTIVE LEARNING

ANNs have ability to train themselves like humans which represent non- linear and complicated
interactions through adaptive learning . (Mehrotra, 1997).

 FAULT TOLERANCE

Whenever any part or important portion of network is lost or missed ANNs are able to the gap
produced due to missing that portion and this skill is very useful in space travels . (Cheng, 1994)

 PROGONSIS

ANNs present in forcast based models have capacity to predict the weather and transportation.

 REAL TIME RESPONSE

ANNs have real time response which lead us to self- driving cars and drones navigation

 SELF ORGANIZATION
Neural network are especially suited for organising the complex visual issues created by
 medical imaging analysis due to their capacity to cluster and categorise large volumes of
data. (M. C. Soriano, 2015)

3. Learning
Learning in neural networks is the process of adjusting weights to minimize prediction error.

Machine Learning (ML):

Employs simpler neural networks for tasks like classification and regression.

Neural network for machine learning

 Back propagation network

 Deep neural network
 Hopfield network
 Multilayer perceptron

Machine learning is major field where multiple tyes of neural networks are used for learning this
learning may be deep learning for which deep neural network which is multilayered neural
network is used. (W. Zhang, 2019)

Deep Learning (DL):

Deep is subtype of machine learning which uses multilayered neural network which help to
learn complex patterns (A. Chattopadhyay, 2017)

Uses networks with many hidden layers, enabling the system to automatically learn hierarchical
representations (like detecting edges, textures, and objects in an image).
Types of Machine learning
Supervised Learning:

Input data is paired with the correct output.

Unsupervised Learning:

The model discovers patterns without labeled outputs.

Reinforcement Learning:

Learning by receiving rewards or penalties (Kalchbrenner, 2014).

Techniques such as Gradient Descent, Backpropagation, and Optimization Algorithms like

Adam are critical to efficient learning. (P. A. Merolla, 2014)

8. Artificial Neural Network Algorithm

Important ANN algorithms include:

Perceptron Learning Rule: Adjusts weights based on classification errors.

Hebbian Learning Rule: Strengthens connections that are simultaneously active.

BFS nets : Calculates the gradient of error and propagates it backward to update weights.

Gradient Descent: Moves weights in the direction that minimizes error.

ReLU Activation: Solves vanishing gradient problems and accelerates training.

Stochastic Gradient Descent (SGD) and Adam: Faster and more stable optimization methods.

Newer improvements include dropout (to prevent overfitting) and batch normalization (to
stabilize and speed up learning). (Cheng, "Neural networks: A review from a statistical
perspective., 1994)

9. Types of Artificial Neural Network

Different ANNs serve specialized tasks:

1.Feedforward Neural Network (FNN): Simple data flows forward only. (Fine, 2016)

2.Convolutional Neural Network (CNN): Specialized for image recognition; uses

convolutional layers to capture spatial hierarchies. (Thakur, 2017)

3.Recurrent Neural Network (RNN): Designed for sequential data, remembers previous
inputs. (Sherstinsky, 2020)

4.Long Short-Term Memory (LSTM): Advanced RNN capable of remembering information

over long time steps. (Vt, 1994)

5.Radial Basis Function Network (RBFN): Good for pattern classification based on proximity.
(Fine, Feedforward neural network methodology, 2006)
6.Modular neural network: This type of network consist of several networks that work
independently and execute sub tasks . Due to individual working of each network the
complicated computing process is divided into separate components and task is accomplished
more quickly. (Gardner, 1998)

7.Autoencoders: Unsupervised networks used for feature learning and dimensionality reduction.

10. Real Life Applications

Neural Networks are now embedded in everyday technologies:

Handwriting Recognition: Used in digital notepads and postal automation. (Widrow, 1996)

Image Compression: Reduces image size while maintaining quality for websites and storage.

Stock Market Prediction: Analyzes patterns in stock price movements for forecasts.

Medical Diagnostics: Assists in early disease detection from imaging and genetic data.

Driverless Cars: Helps in real-time object detection and decision-making.

Face Recognition: Powers phone unlocking, surveillance, and biometric verification.

Natural Language Processing (NLP): Underlies voice assistants, translation apps, and chatbots.

Robotics: Enhances perception, decision-making, and interaction abilities.

11.Neural network and Artificial Intelligence

Artificial intelligence (AI) is defined as simulation of intelligence in machines.Intelligence is
complex and multifaceted concept that have several abilities which are indeed the capacity to
learn ,understand ,reason,make decision,and adaptation to new situation (Cross SS, 1995).

AI Tools working
AI tool like ChatGPT works on large language based model LLM ,how this models train and
learn.The LLM learn itself from large and vast amount of data and generate this data in text
form by using complex neural network of transformer architectures which process input and
generate output .

AI Working
Neural Network Organization

First, ANNs need to have the structure. For simple tasks, ANNs can work well with just a dozen
neurons in a single hidden layer. Adding more neurons and layers allow ANNs to tackle more
complex problems. Deep learning refers to the use of big ANNs, featuring at least two hidden
layers, each containing many neurons.These layers allow the ANN to develop more abstract
conceptualisations of problems by splitting them into smaller sub-problems, and to deliver more
nuanced responses. It has been suggested that three hidden layers are enough to solve any kind of
problem although, in practice, many ANNs include millions of neurons organised in dozens of
hidden layers. By way of comparison, human brains contain ~100 billion neurons, cockroach
brains ~1 million and snail brains ~10 thousand.

Training of structured Neural network

Once the correct structure of the ANN is in place, it needs to be trained. While in theory this can
be done by hand, it would require a human expert to painstakingly adjust neurons to reflect their
own expertise of how to play a good game. Instead, a ML algorithm is applied to automate the
process. The training process can be an intensive and complicated process, and often never really
ends, as constant updates respond to new data availability and changes in the problem faced.
Once a well-trained ANN is in place, it can be applied to new data very quickly and efficiently.

Data mining, big data and data in the wild

Data is so central to contemporary AI development, several data-related concepts are frequently

raised during debates about AI. AI engineers spend as much time thinking about data as
algorithms. They need lots of good quality data to perform effective ML, and even more to test
the results. 'Data mining', is a field of computation focused on the automated identification of
patterns and anomalies in datasets. The dataset could be anything from text posted on social
media to precise measurements of underground geological formations, and the mining process
could deploy ANNs, statistics and modelling to identify useful features. 'Big data' refers to
datasets that are so large and complex – including content from different sources, in different
formats, and with different degrees of authenticity and accuracy – that they cannot be stored or
processed in the same way as smaller datasets. This brings us to 'data in the wild', which usually
refers to data that was produced for one purpose but remains somehow accessible and can be
used for other purposes. (V. Milo, 2016)

For the backgammon playing ANN, the engineer needs to express the game board and dice as a
signal to be sent to the input layer. They also need to find a way of interpreting the output as a
legitimate move – an instruction about which counters to move, and where to move them. They
also need to work out a strategy for dealing with illegitimate moves, e.g. by designingthe output
layer so that its signal can always be interpreted as a legitimate move.
.

AI WORKING
REFERENCE :

A. Chattopadhyay, C. H. (2017). Emerging Technologyand Architecture for Big-Data Analytics. Springer

International Publishing, Cham.

Cheng, B. a. (1994). "Neural networks: A review from a statistical perspective. Statistical science , 2-30.

Cheng, B. a. (1994). Neural networks: A review from a statistical perspective. Statistical science, 2-30.

Chiu, C.-C. T. (2018). State-of-the-art speech tate-of-the-art speech. In 2018 IEEE International
Conference on Acoustics, Speech and Signal Processing (ICASSP),.

Cross SS, H. R. (1995). Introduction to neural networks. Lancet, 1075-1079.

D. Purves, G. J. (2017). Neuroscience. Oxford University Press.

E. R. Kandel, J. H. (2000). Principles of Neural Science. McGraw-Hill Education .

Fausett, L. V. (2016). Fundamentals of neural networks: architectures, algorithms and applications.

Pearson Education India.

Fine, T. L. (2006). Feedforward neural network methodology. Springer Science & Business Media,.

Fine, T. L. (2016). Feedforward neural network methodology. Springer Science & Business Media,.

Gardner, M. W. (1998). Artificial neural networks (the multilayer perceptron)—a review of applications
in the atmospheric sciences.". Artificial neural networks (the multilayer perceptron)—a review of
applications in the atmospheric sciences.", 2627-2636.

Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press.

Kalchbrenner, N. E. (2014). "A convolutional neural network for modelling sentences. arXiv preprint
arXiv:.

M. C. Soriano, S. O. (2015). Neural Networks Learn. Syst.

Mehrotra, K. C. (1997). Elements of artificial neural networks. MIT press.

Nielsen, M. (2019). Neural Networks and Deep Learning Perceptron, Architecture, Algorithms.

P. A. Merolla, J. V.-I. (2014). Science, 345, 668.

Sherstinsky, A. (2020). Fundamentals of recurrent neural network (RNN) and long short-term memory
(LSTM) network. physica D: Nonlinear Phenomena 404.

Thakur, A. H. (2017). "White-Box Cartoonization Using An Extended GAN Framework.". arXiv preprint
arXiv:.

V. Milo, G. P. (2016). IEEE Int. Electron Devices Meet. IEEE,Piscataway, NJ.

Vt, S. E. (1994). Radial basis function neural network for approximation and estimation of nonlinear
stochastic dynamic system. IEEE transactions on neural networks 5, no. 4, 594-603.

W. Zhang, B. G. (2019). physics status solid. RRL.

Widrow, B. D. (1996). neural networks: applications in industry, business and science. " Communications
of the ACM, 93-106.