Scribd
Upload
139 views48 pages

Foundations of Deep Learning

The document provides an overview of the foundations of deep learning. It discusses what deep learning is, what enabled its recent revolution including large amounts of data and computational power. It introduces key deep learning concepts like neural networks, activation functions, universal approximation theorem, and how neural networks are trained via backpropagation. Applications discussed include classification, regression, clustering, reinforcement learning, and self-driving cars.

Uploaded by

takunda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
139 views48 pages

Foundations of Deep Learning

The document provides an overview of the foundations of deep learning. It discusses what deep learning is, what enabled its recent revolution including large amounts of data and computational power. It introduces key deep learning concepts like neural networks, activation functions, universal approximation theorem, and how neural networks are trained via backpropagation. Applications discussed include classification, regression, clustering, reinforcement learning, and self-driving cars.

Uploaded by

takunda
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 48

Foundations of Deep Learning

Dr. Panashe Chiurunge


Agenda
Session 1: Foundations of Deep Learning
 What is Deep Learning?
 What is behind the Deep Learning revolution?
 Deep Learning terminology
 Deep Neural Networks
 How Deep Neural Networks Learn
 The power of Deep Neural Networks
Agenda
Session 2: Deep learning coding labs
 Introduction Deep Learning with TensorFlow
 Regression with Deep Learning
 Deep Learning for Classification
 TensorFlow Keras API
What is AI?
What is AI?

• Intelligence:
– “the capacity to learn and solve problems” (Websters dictionary)
– in particular,
• the ability to solve novel problems
• the ability to act rationally
• the ability to act like humans

• Artificial Intelligence
– build and understand intelligent entities or agents
– 2 main approaches: “engineering” versus “cognitive modeling”
What is AI?

• What is artificial intelligence?


It is the science and engineering of making intelligent machines, especially intelligent
computer programs. It is related to the similar task of using computers to understand
human intelligence, but AI does not have to confine itself to methods that are
biologically observable.

• Yes, but what is intelligence?


Intelligence is the computational part of the ability to achieve goals in the world.
Varying kinds and degrees of intelligence occur in people, many animals and some
machines.

• Isn't there a solid definition of intelligence that doesn't depend on relating it


to human intelligence?
Not yet. The problem is that we cannot yet characterize in general what kinds of
computational procedures we want to call intelligent. We understand some of the
mechanisms of intelligence and not others.

• More in: http://www-formal.stanford.edu/jmc/whatisai/node1.html


What is AI?
What is Deep Learning ?

Deep learning and AI used interchangeably


What is behind the DL Revolution?
 Deep learning requires lots of data
What is behind the DL Revolution?
How happens online in 60 secs
How happens online in 60 secs
How much data is generated per day ?

 According to IBM 25 Zettabytes of data are generated every


day
How much data is generated per day ?
DL requires computational power
Deep Learning foundations
 Biological inspiration from neurons
Deep Learning foundations
Deep Learning foundations
Deep Learning foundations
 Artificial Neuron (Perceptron)
Deep Learning foundations
 Artificial Neuronal Networks

Deep Neural Network has at least 2 hidden layers


Deep Learning foundations
 Feed Forward Multilayer Perceptron (MLP)

Feed Forward Neural Network


Deep Learning foundations
Deep Learning foundations
Sigmoid
Sigmoid Activation function
 Squashes the neurons pre-activation between 0 and 1
 Always positive
 Bounded
 Strictly increasing
Tanh
Hyperbolic tangent (tanh) Activation function
 Squashes the neurons pre-activation between -1 and 1
 Can be positive or negative
 Bounded
 Strictly increasing
ReLu
Rectified Linear Unit (ReLu) Activation function
 Bounded below by 0 (always non-negative)
 Can be positive or negative
 Not upper Bounded
 Strictly increasing
Deep Learning foundations
Inputs x1, x2, ..Xn are known as Features
Universal Approximation theorem

Universality: For any arbitrary function f(x), there exists a neural network tha
closely approximate it for any input x

Universality is an incredible property!* And it holds for just 1


hidden layer
Universal Approximation theorem

Universality: For any arbitrary functions f1(x) and f2(x), there exists a neural
network that closely approximate it for many inputs x1, x2, x3

Universality also holds for multiple inputs, multiple outputs and


multiple hidden layers
This is the incredible power of
deep learning
Deep Learning foundations
Deep Learning foundations
Deep Learning foundations
Supervised learning: Classification
Deep Learning foundations
Deep Learning foundations
Clustering
Deep Learning foundations
Reinforcement Learning
Self Driving Cars
Deep Learning foundations
Reinforcement Learning
Deep Learning foundations
Reinforcement Learning
Deep Learning foundations
Training and Testing
Deep Learning foundations
How Neural Networks learn: Backpropagation
Deep Learning foundations
Backpropagation
Deep Learning foundations
Backpropagation
Backpropagation
Deep Learning foundations
Deep Learning foundations
Deep Learning foundations
Deep Learning foundations
Learning is an optimization problem
Deep Learning foundations
Neural Network Playground
Deep learning foundations

Q&A

You might also like