Department of M.A.C.

S
NIT-K

MACHINE LEARNING

Lecture 1 by Nandagopal S A
Assistant Lecturer
Department of M.A.C.S
NIT-K

January 22, 2025

M. FOUNDATIONS OF DATA SCIENCE

1 Machine Learning
Introduction
OVERVIEW

■ Why Machine Learning?

■ Pattern recognition.. heuristics.. ML
■ Consider the example of recognizing handwritten digits. Each digit
corresponds to a 2828 pixel image and so can be represented by a vector
xcomprising 784 real numbers. The goal is to build a machine that will take
such a vector x as input and that will produce the identity of the digit 0, ..., 9 as
the output.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 2/ 9

OVERVIEW

■ Why Machine Learning?

■ Pattern recognition.. heuristics.. ML
■ Consider the example of recognizing handwritten digits. Each digit
corresponds to a 2828 pixel image and so can be represented by a vector
xcomprising 784 real numbers. The goal is to build a machine that will take
such a vector x as input and that will produce the identity of the digit 0, ..., 9 as
the output.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 2/ 9

OVERVIEW

■ Why Machine Learning?

■ Pattern recognition.. heuristics.. ML
■ Consider the example of recognizing handwritten digits. Each digit
corresponds to a 2828 pixel image and so can be represented by a vector
xcomprising 784 real numbers. The goal is to build a machine that will take
such a vector x as input and that will produce the identity of the digit 0, ..., 9 as
the output.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 2/ 9

CONTD..

▶ The machine learning approach will be to train a large set of N digits x1 , ..., xN
called a training set to tune the parameters of an adaptive model.
▶ The categories of the digits in the training set are known in advance, typically
by inspecting them individually and hand-labelling them. We can express the
category of a digit using target vector t, which represents the identity of the
corresponding digit.
▶ Note that there is one such target vector t for each digit image x.
■ The result of running the machine learning algorithm can be expressed as a
function y(x) which takes a new digit image x as input and that generates an
output vector y, encoded in the same way as the target vectors.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 3/ 9

CONTD..

▶ The machine learning approach will be to train a large set of N digits x1 , ..., xN
called a training set to tune the parameters of an adaptive model.
▶ The categories of the digits in the training set are known in advance, typically
by inspecting them individually and hand-labelling them. We can express the
category of a digit using target vector t, which represents the identity of the
corresponding digit.
▶ Note that there is one such target vector t for each digit image x.
■ The result of running the machine learning algorithm can be expressed as a
function y(x) which takes a new digit image x as input and that generates an
output vector y, encoded in the same way as the target vectors.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 3/ 9

CONTD..

▶ The machine learning approach will be to train a large set of N digits x1 , ..., xN
called a training set to tune the parameters of an adaptive model.
▶ The categories of the digits in the training set are known in advance, typically
by inspecting them individually and hand-labelling them. We can express the
category of a digit using target vector t, which represents the identity of the
corresponding digit.
▶ Note that there is one such target vector t for each digit image x.
■ The result of running the machine learning algorithm can be expressed as a
function y(x) which takes a new digit image x as input and that generates an
output vector y, encoded in the same way as the target vectors.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 3/ 9

CONTD..

▶ The machine learning approach will be to train a large set of N digits x1 , ..., xN
called a training set to tune the parameters of an adaptive model.
▶ The categories of the digits in the training set are known in advance, typically
by inspecting them individually and hand-labelling them. We can express the
category of a digit using target vector t, which represents the identity of the
corresponding digit.
▶ Note that there is one such target vector t for each digit image x.
■ The result of running the machine learning algorithm can be expressed as a
function y(x) which takes a new digit image x as input and that generates an
output vector y, encoded in the same way as the target vectors.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 3/ 9

CONTD..

▶ The machine learning approach will be to train a large set of N digits x1 , ..., xN
called a training set to tune the parameters of an adaptive model.
▶ The categories of the digits in the training set are known in advance, typically
by inspecting them individually and hand-labelling them. We can express the
category of a digit using target vector t, which represents the identity of the
corresponding digit.
▶ Note that there is one such target vector t for each digit image x.
■ The result of running the machine learning algorithm can be expressed as a
function y(x) which takes a new digit image x as input and that generates an
output vector y, encoded in the same way as the target vectors.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 3/ 9

CONTD..

■ precise form of the function y(x) is determined during the training phase, also
known as the learning phase, on the basis of the training data.
■ Once the model is trained it can then determine the identity of new digit
images, which are said to comprise a test set.
■ The ability to categorize correctly new examples that differ from those used
for training is known as generalization.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 4/ 9

CONTD..

■ precise form of the function y(x) is determined during the training phase, also
known as the learning phase, on the basis of the training data.
■ Once the model is trained it can then determine the identity of new digit
images, which are said to comprise a test set.
■ The ability to categorize correctly new examples that differ from those used
for training is known as generalization.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 4/ 9

CONTD..

■ precise form of the function y(x) is determined during the training phase, also
known as the learning phase, on the basis of the training data.
■ Once the model is trained it can then determine the identity of new digit
images, which are said to comprise a test set.
■ The ability to categorize correctly new examples that differ from those used
for training is known as generalization.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 4/ 9

CONTD..

■ For most practical applications, the original input variables are typically
preprocessed to transform them into some new space of variables where, it is
hoped, the pattern recognition problem will be easier to solve.
■ For instance, in the digit recognition problem, the images of the digits are
typically translated and scaled so that each digit is contained within a box of a
fixed size.
■ This greatly reduces the variability within each digit class, because the
location and scale of all the digits are now the same, which makes it much
easier for a subsequent pattern recognition algorithm to distinguish between
the different classes.
■ This pre-processing stage is sometimes also called feature extraction.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 5/ 9

CONTD..

■ For most practical applications, the original input variables are typically
preprocessed to transform them into some new space of variables where, it is
hoped, the pattern recognition problem will be easier to solve.
■ For instance, in the digit recognition problem, the images of the digits are
typically translated and scaled so that each digit is contained within a box of a
fixed size.
■ This greatly reduces the variability within each digit class, because the
location and scale of all the digits are now the same, which makes it much
easier for a subsequent pattern recognition algorithm to distinguish between
the different classes.
■ This pre-processing stage is sometimes also called feature extraction.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 5/ 9

CONTD..

■ For most practical applications, the original input variables are typically
preprocessed to transform them into some new space of variables where, it is
hoped, the pattern recognition problem will be easier to solve.
■ For instance, in the digit recognition problem, the images of the digits are
typically translated and scaled so that each digit is contained within a box of a
fixed size.
■ This greatly reduces the variability within each digit class, because the
location and scale of all the digits are now the same, which makes it much
easier for a subsequent pattern recognition algorithm to distinguish between
the different classes.
■ This pre-processing stage is sometimes also called feature extraction.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 5/ 9

CONTD..

■ For most practical applications, the original input variables are typically
preprocessed to transform them into some new space of variables where, it is
hoped, the pattern recognition problem will be easier to solve.
■ For instance, in the digit recognition problem, the images of the digits are
typically translated and scaled so that each digit is contained within a box of a
fixed size.
■ This greatly reduces the variability within each digit class, because the
location and scale of all the digits are now the same, which makes it much
easier for a subsequent pattern recognition algorithm to distinguish between
the different classes.
■ This pre-processing stage is sometimes also called feature extraction.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 5/ 9

CLASSES OF PROBLEMS

■ Applications in which the training data comprises examples of the input

vectors along with their corresponding target vectors are known as supervised
learning problems.
■ Cases such as the digit recognition example, in which the aim is to assign
each input vector to one of a finite number of discrete categories, are called
classification problems.
■ If the desired output consists of one or more continuous variables, then the
task is called regression.
■ An example of a regression problem would be the prediction of the yield in a
chemical manufacturing process in which the inputs consist of the
concentrations of reactants, the temperature, and the pressure.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 6/ 9

CLASSES OF PROBLEMS

■ Applications in which the training data comprises examples of the input

vectors along with their corresponding target vectors are known as supervised
learning problems.
■ Cases such as the digit recognition example, in which the aim is to assign
each input vector to one of a finite number of discrete categories, are called
classification problems.
■ If the desired output consists of one or more continuous variables, then the
task is called regression.
■ An example of a regression problem would be the prediction of the yield in a
chemical manufacturing process in which the inputs consist of the
concentrations of reactants, the temperature, and the pressure.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 6/ 9

CLASSES OF PROBLEMS

■ Applications in which the training data comprises examples of the input

vectors along with their corresponding target vectors are known as supervised
learning problems.
■ Cases such as the digit recognition example, in which the aim is to assign
each input vector to one of a finite number of discrete categories, are called
classification problems.
■ If the desired output consists of one or more continuous variables, then the
task is called regression.
■ An example of a regression problem would be the prediction of the yield in a
chemical manufacturing process in which the inputs consist of the
concentrations of reactants, the temperature, and the pressure.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 6/ 9

CLASSES OF PROBLEMS

■ Applications in which the training data comprises examples of the input

vectors along with their corresponding target vectors are known as supervised
learning problems.
■ Cases such as the digit recognition example, in which the aim is to assign
each input vector to one of a finite number of discrete categories, are called
classification problems.
■ If the desired output consists of one or more continuous variables, then the
task is called regression.
■ An example of a regression problem would be the prediction of the yield in a
chemical manufacturing process in which the inputs consist of the
concentrations of reactants, the temperature, and the pressure.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 6/ 9

CONTD..

■ In other pattern recognition problems, the training data consists of a set of

input vectors x without any corresponding target values.
■ The goal in such unsupervised learning problems may be to discover groups
of similar examples within the data, where it is called clustering, or to
determine the distribution of data within the input space, known as density
estimation, or to project the data from a high-dimensional space down to two
or three dimensions for the purpose of visualization.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 7/ 9

CONTD..

■ In other pattern recognition problems, the training data consists of a set of

input vectors x without any corresponding target values.
■ The goal in such unsupervised learning problems may be to discover groups
of similar examples within the data, where it is called clustering, or to
determine the distribution of data within the input space, known as density
estimation, or to project the data from a high-dimensional space down to two
or three dimensions for the purpose of visualization.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 7/ 9

CONTD..

■ Finally, the technique of reinforcement learning (Sutton and Barto, 1998) is

concerned with the problem of finding suitable actions to take in a given
situation in order to maximize a reward.
■ Here the learning algorithm is not given examples of optimal outputs, in
contrast to supervised learning, but must instead discover them by a process
of trial and error.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 8/ 9

CONTD..

■ Finally, the technique of reinforcement learning (Sutton and Barto, 1998) is

concerned with the problem of finding suitable actions to take in a given
situation in order to maximize a reward.
■ Here the learning algorithm is not given examples of optimal outputs, in
contrast to supervised learning, but must instead discover them by a process
of trial and error.

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 8/ 9

CURVE FITTING

January 22, 2025 Nandagopal S A, Department of M.A.C.S, NIT-K Surathkal 9/ 9