Traditional Programming vs Machine learning and Models in Machine Learning

Unit-I: Introduction to Machine Learning
Introduction to Machine Learning, Comparison of Machine
Learning with traditional programming, ML Vs AI Vs Data
Science.
Types of Learning: Supervised, Unsupervised, Semi-
Supervised, Reinforcement Learning.
Models of Machine Learning: Geometric Model, Probabilistic
Models of Machine Learning: Geometric Model, Probabilistic
Models, Logical Models, Grouping and Grading Models,
Parametric and Non-Parametric Models.
Important Elements of ML- Data Formats, Learnability,
Statistical Learning Approaches.

Machine Learning
● Learning: The ability to improve behaviour based on experience is called learning
● Machine: A mechanically, electrically or electronically operated device for performing a task is
machine
● Machine Learning: Machine learning explores algorithm learn/ build model from data and that
model is used for prediction, decision making, and for solving tasks.
model is used for prediction, decision making, and for solving tasks.
Definition: A computer program is said to learn from experience E (data) with respect to some class
of task T (prediction, classification etc..) and performance measure P if its performance on task in T
as measured by P improves with experience E.
● Machine Learning is a subset of artificial intelligence which focuses mainly on machine
learning from their experience and making predictions based on its experience.
Department of Computer Engineering

Machine Learning
● It enables the computers or the machines
to make data-driven decisions rather than
being explicitly programmed for carrying
being explicitly programmed for carrying
out a certain task.
● These programs or algorithms are
designed in a way that they learn and
improve over time when are exposed to
new data.

Machine Learning
Flow of Machine Learning

Machine Learning
Flow of Machine Learning
● Machine Learning algorithm is trained using a training data set to create a model.
● When new input data is introduced to the ML algorithm, it makes a prediction on
the basis of the model.
the basis of the model.
● The prediction is evaluated for accuracy and if the accuracy is acceptable, the
Machine Learning algorithm is deployed.
● If the accuracy is not acceptable, the Machine Learning algorithm is trained again
and again with an augmented training data set.

Machine Learning
Comparison of Machine Learning with Traditional Programming

Machine Learning
● Traditional programming is a manual process — meaning a person (programmer) creates the
program.
● But without anyone programming the logic, one has to manually formulate or code rules.
● We have the input data, and someone (programmer) coded a program that uses that data and runs
on a computer to produce the desired output.
● Machine Learning, on the other hand, the input data and output are fed to an algorithm to create a
program.
● In Traditional programming one has to manually formulate/code rules while in Machine Learning
the algorithms automatically formulate the rules from the data, which is very powerful.
● If the Traditional Programming is automation, Then machine learning is automating the process
of automation.

Machine Learning
● For any solution, the first task is the creation of the most suitable algorithm and writing the
code.
● Thereafter, it is mandatory to set the input parameters and, in fact, if an implemented algorithm
is ok it will produce the expected result.
● However, when we need to predict something, we need to use an algorithm with a variety of
input parameters.
● To solve the same problem using ML-methods, data engineers use a totally different procedure.
Instead of developing an algorithm on its own, they need to collect an array of historical data
that will be used for semi-automatic model building.
● Following managing a satisfactory set of data, the data engineer loads it into already tailored
ML-algorithms. The result is a model that can predict a new result, receiving new data as input.

Machine Learning
● A distinctive feature of ML is there is no need to build a model.
● This complicated yet meaningful responsibility is executed by ML-algorithms.
● Another significant difference between ML and Programming is determined by the
● Another significant difference between ML and Programming is determined by the
number of input parameters that the model is capable of processing.
● For an accurate prediction, you have to add thousands of parameters and do it with
high accuracy, as every bit will affect the final result.
● A human being a priori cannot build an algorithm that will use all of those details
in a reasonable way.

Machine Learning
ML vs AI vs Data Science
Data Science:
● Based on strict analytical
evidence
● Deal with structured and
Artificial Intelligence
● Deal with structured and
unstructured data
● Includes various data
operations
● Imparts human intellects to
machine
● Uses logic and decision
trees
● Includes Machine Learning
Machine Learning
● Subset of AI
● Uses Statistical models
● Machines improved
with experience

Machine Learning
Data Science vs. Artificial Intelligence
● Data science deals with pre-processing, analyzing, visualizing, and predicting the data. Whereas,
AI implements a predictive model used for forecasting future events.
● Data science banks on statistical techniques while AI leverages computer algorithms.
● Data science banks on statistical techniques while AI leverages computer algorithms.
● The tools used in data science are much more in quantity than the ones used in AI.
● The reason for this is – there are multiple steps for analyzing data and extracting insights from it.
● In data science, the focus remains on building models that use statistical insights, whereas, for AI,
the aim is to build models that can emulate human intelligence.
● Data science strives to find hidden patterns in the raw and unstructured data while AI is about
assigning autonomy to data models.

Machine Learning
Data Science vs. Machine Learning
● To be precise, Machine Learning fits within the purview of data science.
● The main difference between data science and machine learning lies in the fact that
data science is much broader in its scope and while focusing on algorithms and
statistics (like machine learning) also deals with entire data processing.
● Data science is essentially used to extract insights from data while Machine
learning is about techniques that data scientists use so that machines learn from
data.
● Data Science actually banks on tools such as machine learning and data analytics.

Machine Learning
Artificial Intelligence vs. Machine Learning
● Artificial intelligence essentially makes machines simulate human intelligence while ML deals with
learning from past data without being explicitly programmed.
● AI focuses on making systems that can solve complex problems while ML aims to make machines
learn from available data and generate accurate outputs.
learn from available data and generate accurate outputs.
● AI works towards maximizing the chances of success while ML is concerned with understanding
patterns and giving accurate results.
● AI involves the process of learning, reasoning, and self-correction while ML deals with learning and
self-correction only when introduced to new data.
● Artificial Intelligence deals with structured, unstructured, and semi-structured data while Machine
learning deals only with structured and semi-structured data.

Machine Learning
Types of Learning
● As with any method, there are different ways to train machine learning algorithms, each with
their own advantages and disadvantages.
● In ML, there are two kinds of data — labeled data and unlabeled data.
● Labeled data has both the input and output parameters in a completely machine-readable
● Labeled data has both the input and output parameters in a completely machine-readable
pattern, but requires a lot of human labour to label the data, to begin with.
● Unlabeled data only has one or none of the parameters in a machine-readable form.
● This negates the need for human labour but requires more complex solutions.
● There are also some types of machine learning algorithms that are used in very specific use-
cases, but three main methods are used today.

Machine Learning
Types of Learning
1.Supervised Machine Learning
● Supervised learning is one of the most basic types of machine learning.
● In this type, the machine learning algorithm is trained on labeled data.
● Even though the data needs to be labeled accurately for this method to work, supervised learning
is extremely powerful when used in the right circumstances.

Machine Learning
Types of Learning
● In supervised learning, the ML algorithm is given a small training dataset to work with.
● This training dataset is a smaller part of the bigger dataset and serves to give the algorithm a
basic idea of the problem, solution, and data points to be dealt with.
basic idea of the problem, solution, and data points to be dealt with.
● The training dataset is also very similar to the final dataset in its characteristics and provides the
algorithm with the labeled parameters required for the problem.
● The algorithm then finds relationships between the parameters given, essentially establishing a
cause and effect relationship between the variables in the dataset.
● At the end of the training, the algorithm has an idea of how the data works and the relationship
between the input and the output.

Machine Learning
Types of Learning
● In supervised learning, learning data comes with description, labels, targets or desired outputs and
the objective is to find a general rule that maps inputs to outputs.
● This kind of learning data is called labeled data.
● This kind of learning data is called labeled data.
● The learned rule is then used to label new data with unknown outputs.
● Supervised learning involves building a machine learning model that is based on labeled samples.
● For example, if we build a system to estimate the price of a plot of land or a house based on
various features, such as size, location, and so on, we first need to create a database and label it.
We need to teach the algorithm what features correspond to what prices. Based on this data, the
algorithm will learn how to calculate the price of real estate using the values of the input features.

Machine Learning
Types of Learning
● Supervised learning is commonly used in real world applications, such as face and speech recognition,
products or movie recommendations, and sales forecasting.
● Supervised learning deals with learning a function from available training data. Here, a learning
algorithm analyzes the training data and produces a derived function that can be used for mapping new
examples.
● Supervised learning can be further classified into two types - Regression and Classification.
a) Regression
● Regression trains on and predicts a continuous-valued response, for example predicting real estate
prices.
● When output Y is discrete valued, it is classification and when Y is continuous, then it is Regression.

Machine Learning
Types of Learning
● Regression algorithms are used if there is a relationship between the input variable and the output
variable.
● It is used for the prediction of continuous variables, such as Weather forecasting, Market Trends, etc.
● It is used for the prediction of continuous variables, such as Weather forecasting, Market Trends, etc.
i) Linear Regression
ii) Regression Trees
iii) Non-Linear Regression
iv) Bayesian Linear Regression
v)Polynomial Regression
vi)Logistic Regression

Machine Learning
Types of Learning
b) Classification
● Classification attempts to find the appropriate class label, such as analyzing positive/negative sentiment,
male and female persons, benign and malignant tumors, secure and unsecure loans etc.
male and female persons, benign and malignant tumors, secure and unsecure loans etc.
● Classification algorithms are used when the output variable is categorical, which means there are two
classes such as Yes-No, Male-Female, True-false, etc.
i) Decision Trees
ii) Random Forest
iii) Support vector Machines
iv) Neural network
Common examples of supervised learning
include
classifying emails into spam and not-spam
categories, labeling web pages based on
their content, and voice recognition.

Machine Learning
Types of Learning
1.Unsupervised Machine Learning
● Unsupervised machine learning holds the advantage of being able to work with unlabeled data.
● This means that human labor is not required to make the dataset machine-readable, allowing much
larger datasets to be worked on by the program.
larger datasets to be worked on by the program.
● In supervised learning, the labels allow the algorithm to find the exact nature of the relationship
between any two data points.
● However, unsupervised learning does not have labels to work off of, resulting in the creation of
hidden structures.
● Relationships between data points are perceived by the algorithm in an abstract manner, with no
input required from human beings.

Machine Learning
Types of Learning
● The creation of these hidden structures is what makes unsupervised learning algorithms versatile.
● Instead of a defined and set problem statement, unsupervised learning algorithms can adapt to the data
by dynamically changing hidden structures.
by dynamically changing hidden structures.
● This offers more post-deployment development than supervised learning algorithms.
● Unsupervised learning is used to detect anomalies, outliers, such as fraud or defective equipment, or to
group customers with similar behaviours for a sales campaign.
● It is the opposite of supervised learning. There is no labeled data here.
● When learning data contains only some indications without any description or labels, it is up to the coder
or to the algorithm to find the structure of the underlying data, to discover hidden patterns, or to
determine how to describe the data. This kind of learning data is called unlabeled data.

Machine Learning
Types of Learning
● Suppose that we have a number of data points, and we want to classify them into several
groups. We may not exactly know what the criteria of classification would be.
● So, an unsupervised learning algorithm tries to classify the given dataset into a certain
number of groups in an optimum way.
● Unsupervised learning algorithms are extremely powerful tools for analyzing data and for
identifying patterns and trends.
● They are most commonly used for clustering similar input into logical groups.
● It has two types clustering and Association

Machine Learning
Types of Learning
a. Clustering:
● Clustering is a method of grouping the objects into clusters such that objects with most similarities
remains into a group and has less or no similarities with the objects of another group.
● Cluster analysis finds the commonalities between the data objects and categorizes them as per the
presence and absence of those commonalities.
b. Association:
● An association rule is an unsupervised learning method which is used for finding the relationships
between variables in the large database.
● It determines the set of items that occurs together in the dataset.
● Association rule makes marketing strategy more effective. Such as people who buy X item (suppose
a bread) are also tend to purchase Y (Butter/Jam) item.
● A typical example of Association rule is Market Basket Analysis.

Machine Learning
The list of some popular unsupervised learning algorithms:
● K-means clustering
● KNN (K-Nearest Neighbors)
● Hierarchical clustering
● Hierarchical clustering
● Anomaly detection
● Neural Networks
● Principle Component Analysis
● Independent Component Analysis
● Apriori algorithm
● Singular value decomposition

Machine Learning
Sr.No. Supervised Unsupervised
1 Supervised learning algorithms are trained
using labeled data.
Unsupervised learning algorithms are trained
using unlabeled data.
2 Supervised learning model takes direct
feedback to check if it is predicting correct
output or not.
Unsupervised learning model does not take any
feedback.
3 Supervised learning model predicts the
output.
Unsupervised learning model finds the hidden
patterns in data.
4 In supervised learning, input data is
provided to the model along with the output.
In unsupervised learning, only input data is
provided to the model.
5 The goal of supervised learning is to train
the model so that it can predict the output
when it is given new data.
The goal of unsupervised learning is to find the
hidden patterns and useful insights from the
unknown dataset.

Machine Learning
Sr.No. Supervised Unsupervised
6 Supervised learning needs supervision to train
the model.
Unsupervised learning does not need
supervision to train the model.
any
7 Supervised learning can be categorized in
Classification and Regression problems.
Unsupervised Learning can be classified
Clustering and Associations problems.
in
8 Supervised learning can be used for those Unsupervised learning can be used for those cases
Supervised learning can be used for those
cases where we know the input as well as
corresponding outputs.
Unsupervised learning can be used for those cases
where we have only input data and no
corresponding output data.
9 Supervised learning
accurate result.
model produces an Unsupervised learning model may give less
accurate result as compared to supervised learning.
10 Supervised learning is not close to true
Artificial intelligence as in this, we first train
the model for each data, and then only it can
predict the correct output.
Unsupervised learning is more close to the true
Artificial Intelligence as it learns similarly as a
child learns daily routine things by his experiences.

Machine Learning
Types of Learning
3.Semi Supervised Machine Learning
● The most basic disadvantage of any Supervised Learning algorithm is that the dataset has to
be hand-labeled either by a Machine Learning Engineer or a Data Scientist. This is a very
costly process, especially when dealing with large volumes of data. The most basic
disadvantage of any Unsupervised Learning is that its application spectrum is limited.
● To counter these disadvantages, the concept of Semi-Supervised Learning was introduced.
● It is partly supervised and partly unsupervised .
● If some learning samples are labeled, but some other are not labeled, then it is semi-
supervised learning.
● It makes use of a large amount of unlabeled data for training and a small amount of labeled
data for testing.

Machine Learning
Types of Learning
3.Semi Supervised Machine Learning
● Semi-supervised learning is applied in cases where it
is expensive to acquire a fully labeled dataset while
more practical to label a small subset.
● Supervised learning: where a student is under the
supervision of a teacher at both home and school,
Unsupervised learning: where a student has to figure
out a concept himself and Semi-Supervised learning:
where a teacher teaches a few concepts in class and
gives questions as homework which are based on
similar concepts.

Machine Learning
Types of Learning
4.Reinforcement Learning
● Reinforcement learning directly takes inspiration from how human beings learn from data in their
lives.
● It features an algorithm that improves upon itself and learns from new situations using a trial-and-error
method.
method.
● Favourable outputs are encouraged or ‘reinforced’, and non-favourable outputs are discouraged or
‘punished’.
● Based on the psychological concept of conditioning, reinforcement learning works by putting the
algorithm in a work environment with an interpreter and a reward system.
● In every iteration of the algorithm, the output result is given to the interpreter, which decides whether
the outcome is favourable or not.

Machine Learning
Types of Learning
● In case of the program finding the correct solution, the interpreter reinforces the solution by providing a
reward to the algorithm.
● If the outcome is not favourable, the algorithm is forced to reiterate until it finds a better result.
● If the outcome is not favourable, the algorithm is forced to reiterate until it finds a better result.
● In most cases, the reward system is directly tied to the effectiveness of the result.

Machine Learning
Types of Learning
● In typical reinforcement learning use-cases, such as finding the shortest route between two points on a map,
the solution is not an absolute value.
● Instead, it takes on a score of effectiveness, expressed in a percentage value.
● The higher this percentage value is, the more reward is given to the algorithm.
● Thus, the program is trained to give the best possible solution for the best possible reward.
● Here learning data gives feedback so that the system adjusts to dynamic conditions in order to achieve a
certain objective.
● The system evaluates its performance based on the feedback responses and reacts accordingly.
● The best known instances include self-driving cars and chess master algorithm AlphaGo.
● There are two important learning models in reinforcement learning: Markov Decision Process & Q learning

Machine Learning
Types of Learning

Models of Machine Learning
1.Geometric Models
● In Geometric models, features could be described as points in two dimensions (x- and y-axis) or
a three-dimensional space (x, y, and z).
● Even when features are not intrinsically geometric, they could be modeled in a geometric
manner (for example, temperature as a function of time can be modeled in two axes).
● In geometric models, there are two ways we could impose similarity.
● We could use geometric concepts like lines or planes to segment (classify) the instance space.
These are called Linear models .
● Alternatively, we can use the geometric notion of distance to represent similarity. In this case, if
two points are close together, they have similar values for features and thus can be classed as
similar. We call such models as Distance-based models.

1.Geometric Models
a. Linear Model
● Linear models are relatively simple. In this case, the function is
represented as a linear combination of its inputs.
● Thus, if x1 and x2 are two scalars or vectors of the same dimension
and a and b are arbitrary scalars, then ax1 + bx2 represents a linear
combination of x1 and x2.
● In the simplest case where f(x) represents a straight line, we have
an equation of the form f (x) = mx + c where c represents the
intercept and m represents the slope.

1.Geometric Models
a. Linear Model
● Linear models are parametric, which means that they have a ﬁxed form with a small number of numeric
parameters that need to be learned from data.
● For example, in f (x) = mx + c, m and c are the parameters that we are trying to learn from the data.
● This technique is different from tree or rule models, where the structure of the model (e.g., which
● This technique is different from tree or rule models, where the structure of the model (e.g., which
features to use in the tree, and where) is not fixed in advance.
● Linear models are stable, i.e., small variations in the training data have only a limited impact on the
learned model.
● In contrast, tree models tend to vary more with the training data, as the choice of a different split at the
root of the tree typically means that the rest of the tree is different as well.
● As a result of having relatively few parameters, Linear models have low variance and high bias.

1.Geometric Models
a. Linear Model
● This implies that Linear models are less likely to overfit the training data than some other
models. However, they are more likely to underfit.
● For example, if we want to learn the boundaries between countries based on labeled data, then
● For example, if we want to learn the boundaries between countries based on labeled data, then
linear models are not likely to give a good approximation.
a. Distance Model
● Distance-based models are the second class of Geometric models.
● Like Linear models, distance-based models are based on the geometry of data.
● As the name implies, distance-based models work on the concept of distance.

1.Geometric Models
b. Distance Model
● In the context of Machine learning, the concept of distance is not based on merely the physical
distance between two points.
● Instead, we could think of the distance between two points considering the mode of transport
● Instead, we could think of the distance between two points considering the mode of transport
between two points.
● Travelling between two cities by plane covers less distance physically than by train because as
the plane is unrestricted.
● Similarly, in chess, the concept of distance depends on the piece used – for example, a Bishop
can move diagonally.

1.Geometric Models
b. Distance Model
● Thus, depending on the entity and the mode of travel, the concept of distance can be experienced
differently.
● The distance metrics commonly used are Euclidean, Minkowski, Manhattan, and Mahalanobis.
● Distance is applied through the concept of neighbors and exemplars.
● Neighbors are points in proximity with respect to the distance measure expressed through exemplars.
● Exemplars are either centroids that find a centre of mass according to a chosen distance metric or
medoids that find the most centrally located data point.
● The most commonly used centroid is the arithmetic mean, which minimizes squared Euclidean distance
to all other points.
● The algorithms under Geometric Model: KNN, Linear Regression, SVM, Logistic Regression etc

2.Probabilistic Models
● The third family of machine learning algorithms is the probabilistic models.
● The k-nearest neighbour algorithm uses the idea of distance (e.g., Euclidean distance) to classify
entities, and logical models use a logical expression to partition the instance space.
● Here the probabilistic models use the idea of probability to classify new entities.
● Here the probabilistic models use the idea of probability to classify new entities.
● Probabilistic models see features and target variables as random variables.
● The process of modeling represents and manipulates the level of uncertainty with respect to these
variables.
● There are two types of probabilistic models: Predictive and Generative.
● Predictive probability models use the idea of a conditional probability distribution P (Y |X) from
which Y can be predicted from X.

● Generative models estimate the joint distribution P (Y, X). Once we know the joint
distribution for the generative models, we can derive any conditional or marginal
distribution involving the same variables.
● Thus, the generative model is capable of creating new data points and their labels, knowing
● Thus, the generative model is capable of creating new data points and their labels, knowing
the joint probability distribution.
● The joint distribution looks for a relationship between two variables.
● Once this relationship is inferred, it is possible to infer new data points.
● The algorithms under Probabilistic Models: Naïve Bayes , Gaussian Process Regression etc

Naïve Bayes is an example of a probabilistic classifier.
● The goal of any probabilistic classifier is given a set of features (x_0 through x_n) and a set of
classes (c_0 through c_k), we aim to determine the probability of the features occurring in each
class, and to return the most likely class.
● Therefore, for each class, we need to calculate P(c_i | x_0, …, x_n).
● We can do this using the Bayes rule defined as
● The Naïve Bayes algorithm is based on the idea of Conditional Probability.
● Conditional probability is based on finding the probability that something will happen, given that
something else has already happened.
● The task of the algorithm then is to look at the evidence and to determine the likelihood of a
specific class and assign a label accordingly to each entity.

3.Logical Models
● Logical models use a logical expression to divide the instance space into segments and hence
construct grouping models.
● A logical expression is an expression that returns a Boolean value, i.e., a True or False outcome.
● Once the data is grouped using a logical expression, the data is divided into homogeneous
● Once the data is grouped using a logical expression, the data is divided into homogeneous
groupings for the problem we are trying to solve.
● For example, for a classification problem, all the instances in the group belong to one class.
● There are mainly two kinds of logical models: Tree models and Rule models.
● Rule models consist of a collection of implications or IF-THEN rules. For tree-based models, the
‘if-part’ defines a segment and the ‘then-part’ defines the behaviour of the model for this
segment. Rule models follow the same reasoning.

3.Logical Models
● Tree models can be seen as a particular type of rule model where the if-parts of the rules are
organized in a tree structure.
● Both Tree models and Rule models use the same approach to supervised learning. The
approach can be summarized in two strategies:
approach can be summarized in two strategies:
a) we could first find the body of the rule (the concept) that covers a sufficiently homogeneous
set of examples and then find a label to represent the body.
b) Alternately, we could approach it from the other direction, i.e., first select a class we want
to learn and then find rules that cover examples of the class.

3.Logical Models
● A simple tree-based model is shown below.
● The tree shows survival numbers of passengers on the
Titanic ("sibsp" is the number of spouses or siblings
aboard).
aboard).
● The values under the leaves show the probability of
survival and the percentage of observations in the leaf.
● The model can be summarized as: Your chances of
survival were good if you were (i) a female or (ii) a
male younger than 9.5 years with less than 2.5 siblings.

3.Logical Models
● To understand logical models further, we need to understand the idea of Concept Learning. Concept
Learning involves learning logical expressions or concepts from examples.
● The idea of Concept Learning fits in well with the idea of Machine learning, i.e., inferring a general
function from specific training examples.
● Concept learning forms the basis of both tree-based and rule-based models.
● Concept learning forms the basis of both tree-based and rule-based models.
● More formally, Concept Learning involves acquiring the definition of a general category from a
given set of positive and negative training examples of the category.
● A Formal Definition for Concept Learning is “The inferring of a Boolean-valued function from
training examples of its input and output.”
● In concept learning, we only learn a description for the positive class and label everything that
doesn’t satisfy that description as negative.
● The algorithms under Logical Models: Decision Tree, Random Forest etc.

4.Grouping and Grading Models
The key difference between Grouping and Grading is the way they handle the instance space.
a) Grouping Model:
● Grouping models breaks ups the instance space into groups or segments , the number of
which is determined at training time.
● They have fixed resolution that is they cannot distinguish instances beyond resolution.
● They have fixed resolution that is they cannot distinguish instances beyond resolution.
● At the finest resolution grouping models assign the majority class to all instances that fall into
the segment.
● Determine the right segments and label all the objects in that segment.
● Example the tree model split the instance space into smaller subsets. Trees are usually of
limited depth and don't contain all the available features. The subset at the leaves of the tree
partition , the instance space with some finite resolution. Instances filtered into the same leaf
of the tree are treated the same regardless of any features not in the tree that might be able to
distinguish them.

4.Grouping and Grading Models
b) Grading Model:
● They don't use the notion of segment.
● Forms one global model over instance space.
● Grading models are usually able to distinguish between arbitrary instances, no matter how similar
● Grading models are usually able to distinguish between arbitrary instances, no matter how similar
they are.
● Resolution in theory , infinite particularly when working in Cartesian instance space
● SVM and other geometric classifiers are the examples of grading models.
● They work in Cartesian instance space. They exploit the minute differences between instances.
● Some models combines features of both grouping and grading models.
● Linear classifiers are the primary example of a grading model. Instances on a line or plane parallel to
the decision boundary can't be distinguished by a liner model. There are infinitely many segments.

5.Parametric and Non Parametric Models
a) Parametric Model:
● Assumptions can greatly simplify the learning process, but can also limit what can be
learned. Algorithms that simplify the function to a known form are called parametric
machine learning algorithms.
● A learning model that summarizes data with a set of parameters of fixed size (independent
of the number of training examples) is called a parametric model.
of the number of training examples) is called a parametric model.
● No matter how much data you throw at a parametric model, it won’t change its mind about
how many parameters it needs.
The algorithms involve two steps:
1. Select a form for the function.
2. Learn the coefficients for the function from the training data.

● An easy to understand functional form for the mapping function is a line, as is used in linear
regression:b0 + b1*x1 + b2*x2 = 0
● Where b0, b1 and b2 are the coefficients of the line that control the intercept and slope, and x1
● Where b0, b1 and b2 are the coefficients of the line that control the intercept and slope, and x1
and x2 are two input variables.
● Assuming the functional form of a line greatly simplifies the learning process.
● Now, all we need to do is estimate the coefficients of the line equation and we have a predictive
model for the problem.
● Often the assumed functional form is a linear combination of the input variables and as such
parametric machine learning algorithms are often also called “linear machine learning
algorithms“.

● The problem is, the actual unknown underlying function may not be a linear function like a line.
● It could be almost a line and require some minor transformation of the input data to work right.
● Or it could be nothing like a line in which case the assumption is wrong and the approach will
● Or it could be nothing like a line in which case the assumption is wrong and the approach will
produce poor results.
● Some more examples of parametric machine learning algorithms include:
1. Logistic Regression
2. Linear DiscriminantAnalysis
3. Perceptron
4. Naive Bayes
5. Simple Neural Networks

Benefits
● Simpler: These methods are easier to understand and interpret results.
● Speed: Parametric models are very fast to learn from data.
● Less Data: They do not require as much training data and can work well even if the fit to the data is
not perfect.
Limitations
● Constrained: By choosing a functional form these methods are highly constrained to the specified
form.
● Limited Complexity: The methods are more suited to simpler problems.
● Poor Fit: In practice the methods are unlikely to match the underlying mapping function.

b) Non Parametric Model:
● Algorithms that do not make strong assumptions about the form of the mapping function are
called nonparametric machine learning algorithms.
● By not making assumptions, they are free to learn any functional form from the training data.
● Nonparametric methods are good when you have a lot of data and no prior knowledge, and
when you don’t want to worry too much about choosing just the right features.
● Nonparametric methods seek to best fit the training data in constructing the mapping function,
whilst maintaining some ability to generalize to unseen data.
● As such, they are able to fit a large number of functional forms.

● An easy to understand nonparametric model is the k-nearest neighbors algorithm that makes
predictions based on the k most similar training patterns for a new data instance.
● The method does not assume anything about the form of the mapping function other than
patterns that are close are likely to have a similar output variable.
Some more examples of popular nonparametric machine learning algorithms are:
1. k-Nearest Neighbors
2. Decision Trees like CART and C4.5
3. Support Vector Machines

Benefits of Nonparametric Machine Learning Algorithms:
● Flexibility: Capable of fitting a large number of functional forms.
● Power: No assumptions (or weak assumptions) about the underlying function.
● Power: No assumptions (or weak assumptions) about the underlying function.
● Performance: Can result in higher performance models for prediction.
Limitations of Nonparametric Machine Learning Algorithms:
● More data: Require a lot more training data to estimate the mapping function.
● Slower: A lot slower to train as they often have far more parameters to train.
● Overfitting: More of a risk to overfit the training data and it is harder to explain why specific
predictions are made.

Data Formats in Machine Learning
● Each data format represents how the input data is represented in memory.
● This is important as each machine learning application performs well for a particular data
format and worse for others.
● Interchanging between various data formats and choosing the correct format is a major
● Interchanging between various data formats and choosing the correct format is a major
optimization technique.
There are four types of data formats:
1. NHWC
2. NCHW
3. NCDHW
4. NDHWC

Important Elements of Machine Learning
Each letter in the formats denotes a particular aspect/ dimension of the data:
● N: Batch size : is the number of images passed together as a group for inference
● C: Channel : is the number of data components that make a data point for the input data. It
● C: Channel : is the number of data components that make a data point for the input data. It
is 3 for opaque images and 4 for transparent images.
● H: Height : is the height/ measurement in y axis of the input data
● W: Width : is the width/ measurement in x axis of the input data
● D: Depth : is the depth of the input data

1) NHWC
NHWC denotes (Batch size, Height, Width, Channel). This means there is a 4D array where the first
dimension represents batch size and accordingly. This 4D array is laid out in memory in row major
order. Hence, you can visualize the memory layout to imagine which operations will access
order. Hence, you can visualize the memory layout to imagine which operations will access
consecutive memory (fast) or memory separated by other data (slow).
2) NCHW
NCHW denotes (Batch size, Channel, Height, Width). This means there is a 4D array where the first
dimension represents batch size and accordingly. This 4D array is laid out in memory in row major
order.

3) NCDHW
NCHW denotes (Batch size, Channel, Depth, Height, Width). This means there is a 5D array
where the first dimension represents batch size and accordingly. This 5D array is laid out in
memory in row major order.
4) NDHWC
NCHW denotes (Batch size, Depth, Height, Width, Channel). This means there is a 5D array
where the first dimension represents batch size and accordingly. This 5D array is laid out in
memory in row major order.

Learnability in Machine Learning
● Learn ability is a quality of products and interfaces that allows users to quickly become familiar with
them and able to make good use of all their features and capabilities.
● Learn ability is one component of usability and is often heard in the context of user interface or user
experience design, as well as usability and user acceptance testing.
experience design, as well as usability and user acceptance testing.
● A very learnable interface or product is sometimes said to be intuitive because the user can immediately
grasp how to interact with the system.
● First-time learn ability refers to the degree of ease with which a user can learn a newly-encountered
system without referring to documentation, such as manuals, user guides or FAQ (frequently-asked
questions) lists.
● One element of first-time learn ability is discoverability, which is the degree of ease with which the user
can find all the elements and features of a new system when they first encounter it.

● Learn ability over time, on the other hand, is the capacity of a user to gain expertise in working with a
given system through repeated interaction.
● There are three different aspects of Learn ability
1.First use learn ability
1.First use learn ability
2.Steepness of learning curve
3.Efficiency of ultimate plateau
● Relatively simple systems with good learn ability are said to have short or steep learning curves,
meaning that most learning associated with the system happens very quickly, after which the rate of
learning levels off or plateaus.

● More complex systems typically involve a longer (shallower) learning curve.
● Within any system that applies standards to measurement, a steep learning curve
refers to something easily learned.
refers to something easily learned.
● As displayed in a graph, for example, the steepness indicates that the degree of
learning obtained rises quickly.
● Contrary to the term's actual definition, however, most people use the term steep
learning curve to indicate difficulty, similarly to the way that a steep hill is difficult to
climb.

Measuring Linear ability
● High learnability contributes to usability.
● High learnability contributes to usability.
● It results in quick system onboarding which translates to low training costs.
● Additionally, good learnability can result in high satisfaction because users will feel confident in their
abilities.
● If the system and corresponding tasks are complex and ones that users access frequently, your
product may be a good case for a learnability study

Statistical Learning
● Statistics is a collection of tools that you can use to get answers to important questions about
data.
● You can use descriptive statistical methods to transform raw observations into information that
you can understand and share.
● You can use inferential statistical methods to reason from small samples of data to whole
● You can use inferential statistical methods to reason from small samples of data to whole
domains.
● Statistical learning theory is a framework for machine learning that draws from statistics and
functional analysis.
● It deals with finding a predictive function based on the data presented.
● The main idea in statistical learning theory is to build a model that can draw conclusions from
data and make predictions.

Statistical Learning Approaches
1. Statistics in Data Preparation
Statistical methods are required in the preparation of train and test data for your machine learning
model.
This includes techniques for:
● Outlier detection.
● Missing value imputation.
● Data sampling.
● Data scaling.
● Variable encoding and much more.
A basic understanding of data distributions, descriptive statistics, and data visualization is required to
help you identify the methods to choose when performing these tasks.

2. Statistics in Model Evaluation
Statistical methods are required when evaluating the skill of a machine learning model on data not
seen during training.
● Data sampling
● Data Resampling
● Experimental design
Re-sampling techniques such as k-fold cross-validation are often well understood by machine learning
practitioners, but the rationale for why this method is required is not.

3. Statistics in Model Selection
Statistical methods are required when selecting a final model or model configuration to use for a
predictive modeling problem.
These include techniques for:
● Checking for a significant difference between results.
● Quantifying the size of the difference between results.
This might include the use of statistical hypothesis tests.

4. Statistics in Model Presentation
Statistical methods are required when presenting the skill of a final model to stakeholders.
● Summarizing the expected skill of the model on average.
●Quantifying the expected variability of the skill of the model in practice.
This might include estimation statistics such as confidence intervals.

5. Statistics in Prediction
Statistical methods are required when making a prediction with a finalized model on new
data.
● Quantifying the expected variability for the prediction.
● This might include estimation statistics such as prediction intervals.

6.Problem Framing:
● Requires the use of exploratory data analysis and data mining.
7.Data Cleaning.
●Requires the use of outlier detection, imputation and more.
8.Data Selection:
● Requires the use of data sampling and feature selection methods.
9. Model Configuration:
● Requires the use of statistical hypothesis tests and estimation statistics.

Traditional Programming vs Machine learning and Models in Machine Learning

Traditional Programming vs Machine learning and Models in Machine Learning

More Related Content

Similar to Traditional Programming vs Machine learning and Models in Machine Learning(20)

Recently uploaded(20)

Traditional Programming vs Machine learning and Models in Machine Learning