CSE601 DS Mathematics for Data Science Spring 2023
Instructor Dr. Nasir Touheed
Office (Room No.) Faculty Office 14, Tabba Building, South Wing, 2nd Floor, Main Campus
Office Hours After the class (or by appointment)
Email
[email protected] Telephone 0308 2227089
About this Course
For a lot of higher-level courses in Machine Learning and Data Science, one needs to freshen up on the
basics in mathematics - stuff you may have studied before in school or university, but which was taught
in another context, or not very intuitively, such that you struggle to relate it to how it’s used in Computer
Science. This specialization aims to bridge that gap, getting you up to speed in the underlying
mathematics, building an intuitive understanding, and relating it to Machine Learning and Data Science.
In this course, we introduce the mathematical concepts underlying data science algorithms. This
includes the Probability, Statistics, Linear Algebra, and Multivariate Calculus / Vector Calculus required
to understand the machine learning / data science algorithms covered in subsequent courses.
Probability and Statistics form the basis of Data Science. The probability theory is very much helpful for
making the prediction. Estimates and predictions form an important part of Data science. With the help
of statistical methods, we make estimates for the further analysis. Thus, statistical methods are largely
dependent on the theory of probability. And all of probability and statistics is dependent on Data.
In Linear Algebra, we look at what linear algebra is and how it relates to data. Then we look through
what vectors and matrices are and how to work with them.
In Multivariate Calculus / Vector Calculus, we discuss ways to optimize fitting functions to get good fits
to data. It starts from introductory calculus and then uses the vectors and matrices from the Linear
Algebra course to look at data fitting.
At the end we will look at the Dimensionality Reduction with Principal Component Analysis, which
uses the mathematics from Linear Algebra, and Multivariate Calculus / Vector Calculus to compress
high-dimensional data. This part of the course is of intermediate difficulty and will require Python and
numpy knowledge.
SKILLS YOU WILL GAIN AFTER SUCCESSFUL COMPLETION OF
THIS COURSE
Probability and Statistics
Demonstrate knowledge and properties of statistical models and construct tests
Be familiar with some statistical distribution and their applications in inferential statistics.
Linear Regression
Linear Algebra
Basis (Linear Algebra)
Eigenvalues And Eigenvectors
Multivariable Calculus
Transformation Matrix
Vector Calculus
Gradient Descent
Principal Component Analysis (PCA)
Dimensionality Reduction
Some experience in using computer software to manipulate vectors and matrices
Course Grading:
The course grade will be based on the following components and their weights:
Assignments 20%
Quizzes 20%
Mid Term Exam 20%
Final Exam 40%
Course Contents:
Weekly Plan (Tentative)
Lecture Topics
I. Probability and Statistics
1 Probability
Basic definitions, types of events, Conditional probability and Baye’s theorem.
2-3 Random variables and their probability distributions
Discrete and continuous random variables, pdf, cdf, joint distributions, marginal distributions,
conditional distributions, independence, the expected value of a random variable, expectations in
joint distributions, the variance and standard deviation, the median, the mode, the covariance and
correlation, conditional expectation, conditional variance.
4-5 Some basic distributions
Discrete distributions: The Bernoulli, the binomial and the Poison distributions.
Continuous distributions: Uniform, Gaussian (normal), T, Chi-square and F distributions, Bivariate
normal distribution.
6-7 Sampling Distributions:
Population and sample, Some basic types of sampling, sampling distributions for sample mean,
Variance and proportion, Convergence (law of large numbers, central limit theorem).
8-9 Estimation and Inferences
Parameter and statistics, characteristics of estimators, unbiasedness, efficiency, sufficiency,
consistency. BLUE. Point and interval estimations of population parameter. Confidence intervals.
10 Statistical Hypothesis testing
Testing mean(s), proportions(s), variance(s), Testing normality techniques, etc.
II. Linear Algebra
11-12 Linear Equations in Linear Algebra
Systems of Linear Equations
Row Reduction and Echelon Forms
Vector Equations
The Matrix Equation Ax = b
Solution Sets of Linear Systems
Linear Independence
Introduction to Linear Transformations
The Matrix of a Linear Transformation
13 Matrix Algebra
Matrix Operations
The Inverse of a Matrix
Characterizations of Invertible Matrices
14 Determinants
Introduction to Determinants
Properties of Determinants
15-16 Eigenvalues and Eigenvectors
Eigenvectors and Eigenvalues,
The Characteristic Equation
Diagonalization
17-19 Orthogonality and Least Squares
Inner Product, Length, and Orthogonality
Orthogonal Sets
Orthogonal Projections
The Gram-Schmidt Process
Least-Squares Problem
20-21 Linear Regression
Dependent, independent variables
Multiple and polynomial regression
Overfitting and cross validation
22-24 Gradient Descent
Functions, convexity, gradients
Fitting models, stochastic gradient descent
25-28 Principle Component Analysis
SVD, lower rank approximations, eigenvalues
Power method, MDS, dimensionality reduction
Textbooks
The course does not follow a particular textbook, but you may find the following textbooks useful to dig
deeper into material discussed in class.
Reference Books
Foundations of Data Science by Blum, Hopcroft, Kannan (advanced material).
Pattern Recognition and Machine Learning by Chris Bishop.
An Introduction to Statistical Learning by James, Witten, Hastie, Tibshirani.
Applied Statistics and Probability for Engineers by Montgomery, D. C. and G. C. Runger (2011).
5th Edition.
John E. Fruend, Mathematical Statistics, Prentice Hall.
Walpole Myres, Myres and Ye, Probability and Statistics for Engineering and Science.
Introductory Statistics, Neil A. Weiss, 9th Edition, Boston: Pearson, 2011.
Michael Baron - Probability and statistics for computer scientists (2019)
Introduction to Probability, Statistics, and Random Processes by Hossein Pishro-Nik (z-lib.org)
Irwin Miller, Marylees Miller - John E. Freund_s Mathematical Statistics with Applications-
Pearson _2014_
John Schiller, R. Alu Srinivasan, Murray Spiegel - Schaum's Outline of Probability and
Statistics, 3rd Ed. (Schaum's Outline Series)-McGraw-Hill (2008)
Moss, Charles B. 2015. Mathematical Statistics for Applied Econometrics
An Introduction to Statistical Learning by James, Witten, Hastie, Tibshirani.
Applied Statistics and Probability for Engineers by Montgomery, D. C. and G. C. Runger.
Introduction to Probability (2nd ed.). Dimitri P. Bertsekas, John N. Tsitsiklis. Athena Scientific.
David C. Lay, Linear Algebra and its Applications, Pearson Education.
Introduction to Linear Algebra by G. Strang
Linear Algebra and Its Applications (4ed) by Gilbert Strang (z-lib.org)
Howard Anton and Chris Rorres, Elementary Linear Algebra, John Wiley and Sons.
Matrix Analysis (2nd ed.). Roger A. Horn, Charles R. Johnson. Cambridge University Press.
No bullshit guide to linear algebra by Ivan Savov.
The following are also extra references.
- Foundations of Data Science by Blum, Hopcroft, Kannan (advanced material).
- [Bis] Bishop, Pattern Recognition and Machine Learning, Springer, 2006 (Chaps 2, 8, 9, 13)
- Justin Solomon - Numerical Algorithms_ Methods for Computer Vision, Machine Learning, and
Graphics-CRC Press (2015)
- [Sol] Solomon, Numerical algorithms, CRC Press, 2015 (Chaps 4-7)
Web Resources
Gilbert Strang lectures on Linear Algebra (MIT)
35 videos - Lecture videos from Gilbert Strang's course on Linear Algebra at MIT.
https://www.youtube.com/playlist?list=PL49CF3715CB9EF31D
Essence of linear algebra
16 videos - A geometric understanding of matrices, determinants, eigen-stuffs and more.
https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
