JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY KAKINADA
KAKINADA – 533 003, Andhra Pradesh, India
CSE (DS) (R23-IInd YEAR COURSE STRUCTURE & SYLLABUS)
DATA ENGINEERING
Course Objectives:
Explain basic concepts of Data Engineering
Discuss bout Data Engineering Life Cycle
How to design Good Data Architecture
UNIT-I: Introduction to Data Engineering: Definition, Data Engineering Life Cycle,
Evolution of Data Engineer, Data Engineering Versus Data Science, Data Engineering Skills
and Activities,
Data Maturity, Data Maturity Model, Skills of a Data Engineer, Business Responsibilities,
Technical Responsibilities, Data Engineers and Other Technical Roles.
UNIT-II: Data Engineering Life Cycle: Data Life Cycle Versus Data Engineering Life
Cycle, Generation: Source System, Storage, Ingestion, Transformation, Serving Data.
Major undercurrents across the Data Engineering Life Cycle: Security, Data
Management, DataOps, Data Architecture, Orchestration, Software Engineering.
UNIT-III: Designing Good Data Architecture: Enterprise Architecture, Data Architecture,
Principles of Good Data Architecture, Major Architecture Concepts.
Data Generation in Source Systems: Sources of Data, Files and Unstructured Data, APIs,
Application Databases (OLTP), OLAP, Change Data Capture, Logs, Database Logs, CRUD,
Source System Practical Details.
UNIT-IV: Storage: Raw Ingredients of Data Storage, Data Storage Systems, Data
Engineering Storage Abstractions, Data warehouse, Data Lake, Data Lakehouse.
Ingestion: Data Ingestion, Key Engineering considerations for the Ingestion Phase, Batch
Ingestion Considerations, Message and Stream Ingestion Considerations, Ways to Ingest Data
UNIT-V: Queries, Modeling and Transformation: Queries, Life of a Query, Query
Optimizer, Queries on Streaming Data, Data Modelling, Modeling Streaming Data,
Transformations, Streaming Transformations and Processing.
30
JAWAHARLAL NEHRU TECHNOLOGICAL UNIVERSITY KAKINADA
KAKINADA – 533 003, Andhra Pradesh, India
CSE (DS) (R23-IInd YEAR COURSE STRUCTURE & SYLLABUS)
Serving Data for Analytics, Machine Learning and Reverse ETL: General Considerations
for serving Data, Business Analytics, Operational Analytics, Embedded Analytics, Ways to
serve data for analytics and ML, Reverse ETL.
Text Books:
1. Joe Reis, Matt Housley, Fundamentals of Data Engineering, O'Reilly Media, Inc.,June
2022,ISBN: 9781098108304
Reference Books:
1. Paul Crickard , Data Engineering with Python,Packt Publishing, October 2020.
2. Ralph Kimball, Margy Ross, The Data Warehouse Toolkit: The Definitive Guide to
Dimensional Modeling, Wiley, 3rd Edition, 2013
3. James Densmore, Data Pipelines Pocket Reference: Moving and Processing Data for
Analytics, O'Reilly Media, 1st Edition, 2021
31
