Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

Reinforcement Learning and Deep Learning L P C

3 3

Discipline(s) / EAE / OAE Semester Group Sub‐group Paper Code

CSE‐AIML 7 PC PC ML‐409T
EAE 7 AIML‐EAE AIML‐EAE‐4 ML‐409T

Marking Scheme:
1. Teachers Continuous Evaluation: 25 marks
2. Term end Theory Examinations: 75 marks
Instructions for paper setter:
1. There should be 9 questions in the term end examinations question paper.
2. The first (1st) question should be compulsory and cover the entire syllabus. This question should be
objective, single line answers or short answer type question of total 15 marks.
3. Apart from question 1 which is compulsory, rest of the paper shall consist of 4 units as per the syllabus.
Every unit shall have two questions covering the corresponding unit of the syllabus. However, the student
shall be asked to attempt only one of the two questions in the unit. Individual questions may contain upto 5
sub‐parts / sub‐questions. Each Unit shall have a marks weightage of 15.
4. The questions are to be framed keeping in view the learning outcomes of the course / paper. The standard
/ level of the questions to be asked should be at the level of the prescribed textbook.
5. The requirement of (scientific) calculators / log‐tables / data – tables may be specified if required.
Course Objectives :
1. To introduce the foundation of Reinforcement learning foundation and Q Network algorithm)
2. To understand policy optimization ,recent advanced techniques and applications of Reinforcement
learning
3. To introduce the concept of deep learning and neural network
4. To understand the concept of NLP and computer vision in deep learning
Course Outcomes (CO)
CO 1 Learn how to define RL tasks and the core principals behind the RL, including policies, value functions,
deriving Bellman equations and underst and work with approximate solution(deep Q Network based
algorithms)
CO 2 Learn the policy gradient methods from vanilla to more complex cases and learn application and
advanced techniques in Reinforcement Learning
CO 3 Apply neural networks for problem solving
CO 4 Able to Analyse images and have basic understanding of NLP in deep learning
Course Outcomes (CO) to Programme Outcomes (PO) mapping (scale 1: low, 2: Medium, 3: High)
PO01 PO02 PO03 PO04 PO05 PO06 PO07 PO08 PO09 PO10 PO11 PO12
CO 1 3 2 3 3 3 2 2 ‐ ‐ ‐ ‐ 2
CO 2 3 2 3 3 3 2 2 ‐ ‐ ‐ ‐ 2
CO 3 3 2 3 3 3 2 2 ‐ ‐ ‐ ‐ 2
CO 4 3 2 3 3 3 2 2 ‐ ‐ ‐ ‐ 2

UNIT‐I

Reinforcement Learning Foundation: Introduction to Reinforcement learning and its terms,Features and
elements of RL, Defining RL Framework and Markov Decision Process , Polices, Value Functions and Bellman
Equations, Exploration vs. Exploitation, Code Standards and Libraries used in RL (Python/Keras/Tensorflow)
Tabular Methods and Q‐networks: Planning through the use of Dynamic Programming and Monte Carlo,
Temporal‐Difference learning methods (TD(0), SARSA, Q‐Learning), Deep Q‐networks (DQN, DDQN, Dueling
DQN, Prioritised Experience Replay)

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 1306
 Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

UNIT‐II

Policy Optimization: Introduction to policy‐based methods, Vanilla Policy Gradient, REINFORCE algorithm and
stochastic policy search, Actor‐critic methods (A2C, A3C) ,Advanced policy gradient (PPO, TRPO, DDPG),
Model‐Based RL: Model‐based RL approach
Recent Advances and Applications: Meta‐learning. Multi‐Agent Reinforcement Learning, Partially Observable
Markov Decision Process, Applying RL for real‐world problems

UNIT‐III

Introduction to Deep learning: Introduction to deep learning and its application,Examples of deep learning
Introduction to Neural Network: Introduction to Neural Network its types and application,Introduction to
keras,Introduction to ANN Perceptron and its uses,Multilayer perceptron and deep neural network,Activation
function and its working TanH function,sigma ,relu etc ,Feed forward network, Cost function, Backpropagation,
Gradient Descent, Regulariztion and dropout technique, Batch normalization.
Types of Neural Network: Convolutional Neural network,CNN Pooling,CNN Layers,Flattening and Full
connection,Preparing a fully connected neural network,Introduction to RNN, Deep RNN, Long Short Term
Memory, GRU, Transfer Learning,

UNIT – IV

Deep Learning for Natural Language Processing: Introduction to NLP and Vector Space Model of Semantics
Word Vector Representations: Continuous Skip‐Gram Model, Continuous Bag‐of‐Words model (CBOW), Glove,
Evaluations and Applications in word similarity, analogy reasoning
Deep Learning for Computer Vision: Image segmentation, object detection, automatic image captioning, Image
generation with Generative adversarial networks, video to text with LSTM models. Attention models for
computer vision tasks.

Textbook(s):
1.Richard S. Sutton and Andrew G. Barto, "Reinforcement learning: An introduction", 2nd Edition, MIT Press,
2019
2. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. "Deep learning." MIT press, 2016.
3. Antonio Gulli and Sujit Pal,”Deep learning with Keras”

References:
1. Wiering, Marco, and Martijn Van Otterlo. "Reinforcement learning: Adaptation, Learning, and Optimization”
(2012)
2. Daniel Slater, Gianmario Spacagna and Peter Roelants, “Python Deep Learning”, Packt Publication.

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 1307
 Handbook of B.Tech. Programmes offered by USICT at Affiliated Institutions of the University.

Reinforcement Learning and Deep Learning Lab L P C

2 1

Discipline(s) / EAE / OAE Semester Group Sub‐group Paper Code

CSE‐AIML 7 PC PC ML‐409P
EAE 7 AIML‐EAE AIML‐EAE‐4 ML‐409P

Marking Scheme:
1. Teachers Continuous Evaluation: 40 marks
2. Term end Theory Examinations: 60 marks
Instructions:
1. The course objectives and course outcomes are identical to that of (Reinforcement Learning and Deep
Learning) as this is the practical component of the corresponding theory paper.
2. The practical list shall be notified by the teacher in the first week of the class commencement under
intimation to the office of the Head of Department / Institution in which the paper is being offered from
the list of practicals below. Atleast 10 experiments must be performed by the students, they may be asked
to do more. Atleast 5 experiments must be from the given list.

1. Setting up the Spyder IDE Environment and Executing a Python Program

2. Installing Keras, Tensorflow and Pytorch libraries and making use of them
3. Implement Q‐learning with pure Python to play a game
 Environment set up and intro to OpenAI Gym
 Write Q‐learning algorithm and train agent to play game
 Watch trained agent play game
4. Implement deep Q‐network with PyTorch
5. Python implementation of the iterative policy evaluation and update.
6. Chatbot using bi‐directional LSTMs
7. Image classification on MNIST dataset (CNN model with fully connected layer)
8. Train a sentiment analysis model on IMDB dataset, use RNN layers with LSTM/GRU
9. Applying the Deep Learning Models in the field of Natural Language Processing
10. Applying the Convolution Neural Network on computer vision problems

Applicable from Batch Admitted in Academic Session 2021-22 Onwards Page 1308