Course Code Course Type LP
Deep Learning
CSA4020 Credits 3
Course Objectives:
Gain knowledge in Machine Learning Basics
Understand and apply Optimization on Deep Models and Networks
Understand and analyze Recurrent and Recursive Networks
Understand the representation of neural networks in machine learning.
Course Outcomes:
Explore the fundamentals of Machine Learning Algorithms
Analyze Deep learning Mathematical Models
Elucidate the Deep Feedforward Networks
Apply knowledge for Optimization on Deep Models and Convolutional Networks
Elucidate the Recurrent and Recursive Networks and Natural language Processing
Student Outcomes (SO): a, b, j, k
Module Hrs
Module Description SO
No. .
1 Introduction: Historical Trends in Deep Learning - Linear Algebra: Scalars - 8 a, b
Vectors - Matrices - Tensors - Matrices - Norms – Eigen decomposition -
Probability and Information Theory: Random variable and distributed Probability
- Bayes Rule -Information Theory and structured probabilistic models.
Machine Learning: Numerical Computation: Overflow and Underflow –
Gradient-based Optimization – Constrained Optimization - Learning Algorithms:
Capacity - Overfitting – Underfitting - Bayesian Classification - Supervised -
unsupervised algorithms - Building machine learning algorithm
2 Fundamental Deep Learning Methods: Artificial Neural Networks (ANN): 8 a, k
Perceptron, learning laws, layers, back propagation - The scope of learning,
popular architectures, An overview of Parallel and Distributed Processes (PDP),
linear associative models, and stochastic networks; - Convolutional Neural
Networks (CNN): convolution, filters, pooling, stride, drop out, layers and
applications; - Recurrent Neural Networks (RNN): unfolding, Backpropagation
Through Time (BPTT), LSTM models, bidirectional networks, encoder, decoder
and attention models.
3 Advanced Deep Learning Methods: Variational Autoencoder (VAE), 9 a, k
Deep Autoencoder (DAE). Generative Adversarial Network (GAN), Deep
Boltzmann Machines, Deep Neural Network applications for multimedia,
Sequence and Streaming data, Deep Belief Network.
4 OPTIMIZATION ON DEEP MODELS: Optimization for Training Deep 10 a, b, k
Models: Challenges in Neural Networks optimization - Basic Algorithms -
Algorithms Adaptive learning Rates - Approximate Second Order Methods -
Optimization Strategies and Meta Algorithms -Convolutional Networks:
Motivation - Structured Output - Unsupervised features -Neuroscientific basics
for Convolutional Networks.
5 RECURRENT AND RECURSIVE NETWORKS: Computational graphs - 8 a, b
Recurrent Neural networks - Bidirectional RNN - Deep Recurrent Networks -
Echo State Networks - Practical Methodology Applications: Large Scale Deep
Learning, Case studies in classification, Regression and deep networks.
6. Guest Lecture on Contemporary Topics 2 j
Total 45
Mode of Teaching and Learning:
Flipped Class Room, Activity Based Teaching/Learning, Digital/Computer based models, wherever possible
to augment lecture for practice/tutorial and minimum 2 hours lectures by industry experts on contemporary
topics
Mode of Evaluation:
The assessment and evaluation components may consist of unannounced open book examinations, quizzes,
student’s portfolio generation and assessment, and any other innovative assessment practices followed by
faculty, in addition to the Continuous Assessment Tests and Term End Examination.
An indicative list of Experiments: The following software experiments can be performed. (Using Python
Platform)
Based on Artificial Neural Network
Based on Convolution Neural Network
Based on Recurrent Neural Network
Based on Autoencoder
Based on Generative Adversarial Network
Based on Deep Belief Network
Text Book(s):
1. Ian Goodfellow, Yoshua Bengio, Aaron Courville, Deep Learning, MIT Press, 2016.
2. Michael Nielsen, Neural Networks and Deep Learning, Determination Press, 2015.
Reference Book(s):
1. Deng & Yu, Deep Learning: Methods and Applications, Now Publishers, 2013. 2. Russell,
2. S. and Norvig, N. Artificial Intelligence: A Modern Approach. Prentice-Hall Series in Artificial
Intelligence. 2003.
3. 3. Bishop, C. M. Neural Networks for Pattern Recognition. Oxford University Press. 1995.
Recommendation by the Board of Studies on 11.05.2022
Approval by Academic council on
Compiled by
