Machine learning-in-details-with-out-python-code

Supervised learning
'-^
ML/DL
s_J
Unsupervised learning
Classification
Regression
ML
DL
image classification Machine translation Stock prediction Image masking
PCA
Dimension reduction
t-SNE(more for visualization)
K-mean
Clustering GMMs
HMMs
Mutual information MINE/D1M
Representation learning Disentanglement VAE
Information bottleneck
GANs
Generative models
VAE
Machine learning (ML) is rapidly changing the world, from diverse types of
applications and research pursued in industry and academia. Machine learning is
affecting every part of our daily lives. From voice assistants using NLP and
machine learning to make appointments, check our calendar and play music, to
programmatic advertisements — that are so accurate that they can predict what
we will need before we even think of it.
More often than not, the complexity of the scientific field of machine learning can
be overwhelming, making keeping up with “what is important” a very challenging
task. However, to make sure that we provide a learning path to those who seek to
learn machine learning, but are new to these concepts. In this article, we look at
the most critical basic algorithms that hopefully make your machine learning
journey less challenging.
Any suggestions or feedback is crucial to continue to improve. Please
let us know in the comments if you have any.

Index
. Introduction to Machine Learning.
• Major Machine Learning Algorithms.
• Supervised vs. Unsupervised Learning.
• Linear Regression.
• Multivariable Linear Regression.
• Polynomial Regression.
• Exponential Regression.
• Sinusoidal Regression.
• Logarithmic Regression.
What is machine learning?
A computer program is said to
learn from experience E with
respect to some class of tasks T
and performance measure P, if
its performance at tasks in T, as
measured by P, improves with
experience E.
~ Tom Mitchell
(on Machine Learning's Operational Definition)
Carnegie Mellon University
Machine Learning
Source: Machine Learning Department at Carnegie Mellon

A computer program is said to learn from experience E
with respect to some class of tasks T and performance
measure P, if its performance at tasks in T, as measured by
P, improves with experience E. ~ Tom M. Mitchell [1]
Machine learning behaves similarly to the growth of a child. As a child grows, her
experience E in performing task T increases, which results in higher performance
measure(P).
For instance, we give a “shape sorting block” toy to a child. (Now we all know that
in this toy, we have different shapes and shape holes). In this case, our task T is to
find an appropriate shape hole for a shape. Afterward, the child observes the
shape and tries to fit it in a shaped hole. Let us say that this toy has three shapes:
a circle, a triangle, and a square. In her first attempt at finding a shaped hole, her
performance measure(P) is 1/3, which means that the child found 1 out of 3
correct shape holes.
Second, the child tries it another time and notices that she is a little experienced
in this task. Considering the experience gained (E), the child tries this task
another time, and when measuring the performance(P), it turns out to be 2/3.
After repeating this task (T) 100 times, the baby now figured out which shape
goes into which shape hole.
So her experience (E) increased, her performance(P) also increased, and then we
notice that as the number of attempts at this toy increases. The performance also
increases, which results in higher accuracy.

Such execution is similar to machine learning. What a machine does is, it takes
a task (T), executes it, and measures its performance (P). Now a machine has a
large number of data, so as it processes that data, its experience (E) increases
over time, resulting in a higher performance measure (P). So after going
through all the data, our machine learning model’s accuracy increases, which
means that the predictions made by our model will be very accurate.
Another definition of machine learning by Arthur Samuel:
Machine Learning is the subfield of computer science that
gives “computers the ability to learn without being
explicitly programmed.” ~ Arthur Samuel [2]
Let us try to understand this definition: It states “learn without being explicitly
programmed” — which means that we are not going to teach the computer with a
specific set of rules, but instead, what we are going to do is feed the computer with
enough data and give it time to learn from it, by making its own mistakes and
improve upon those. For example, We did not teach the child how to fit in the
shapes, but by performing the same task several times, the child learned to fit the
shapes in the toy by herself.
Therefore, we can say that we did not explicitly teach the child how to fit the
shapes. We do the same thing with machines. We give it enough data to work on
and feed it with the information we want from it. So it processes the data and
predicts the data accurately.
Why do we need machine learning?

What machine learning does is process the data with different kinds of algorithms
and tells us which feature is more important to determine whether it is a cat or a
dog. So instead of applying many sets of rules, we can simplify it based on two or
three features, and as a result, it gives us a higher accuracy. The previous method
was not generalized enough to make predictions.
Machine learning models helps us in many tasks, such as:
Object Recognition , Summarization , Prediction , Classification ,
Clustering , Recommender systems
What is a machine learning model?
A machine learning model is a question/answering system that takes care of
processing machine-learning related tasks. Think of it as an algorithm system
that represents data when solving problems. The methods we will tackle below
are beneficial for industry-related purposes to tackle business problems.
For instance, let us imagine that we are working on Google Adwords ’ ML system,
and our task is to implementing an ML algorithm to convey a particular
demographic or area using data. Such a task aims to go from using data to gather
valuable insights to improve business outcomes.
Major Machine Learning Algorithms:
1. Regression (Prediction) (jf£i') j'^Vi

We use regression algorithms for predicting continuous values.
Regression algorithms:
Linear Regression , Polynomial Regression , Exponential
Regression , Logistic Regression , Logarithmic Regression
2. Classification
We use classification algorithms for predicting a set of items’ class or category.
Classification algorithms:
K-Nearest Neighbors , Decision Trees , Random Forest , Support Vector
Machine , Naive Bayes
3. Clustering
We use clustering algorithms for summarization or to structure data. Clustering
algorithms:
K-means , DBSCAN , Mean Shift , Hierarchical , Association
We use association algorithms for associating co-occurring items or events.
Association algorithms: Apriori
5. Anomaly Detection
We use anomaly detection for discovering abnormal activities and unusual
cases like fraud detection ^1' ^&ll.

6. Sequence Pattern Mining
We use sequential pattern mining for predicting the next data events between
data examples in a sequence.
7. Dimensionality Reduction
We use dimensionality reduction for reducing the size of data to extract only
useful features from a dataset.
8. Recommendation Systems
We use recommenders algorithms to build recommendation engines. Examples:
Netflix recommendation system , A book recommendation system ,
A product recommendation system on Amazon ,
Nowadays, we hear many buzz words like artificial intelligence, machine learning,
deep learning,
What are the fundamental differences between
Artificial Intelligence, Machine Learning, and Deep
Learning?
Artificial Intelligence (AI):
Artificial intelligence (AI), as defined by Professor Andrew Moore, is the science
and engineering of making computers behave in ways that, until recently, we
thought required human intelligence [4].

These include:
Computer Vision , Language Processing , Creativity , Summarization
Machine Learning (ML):
As defined by Professor Tom Mitchell, machine learning refers to a scientific
branch of AI, which focuses on the study of computer algorithms that allow
computer programs to automatically improve through experience [3].
These include:
Classification , Neural Network , Clustering
Deep Learning:
Deep learning is a subset of machine learning in which layered neural networks,
combined with high computing power and large datasets, can create powerful
machine learning models. [3]
Neural network abstract representation | Photo by Clink Adair via Unsplash
Why do we prefer Python to implement machine
learning algorithms?
Python is a popular and general-purpose programming language. We can write
machine learning algorithms using Python, and it works well. The reason why
Python is so popular among data scientists is that Python has a diverse variety of
modules and libraries already implemented that make our life more
comfortable.
Let us have a brief look at some exciting Python libraries.

1. Numpy: It is a math library to work with n-dimensional arrays in Python. It
enables us to do computations effectively and efficiently.
2. Scipy: It is a collection of numerical algorithms and domain-specific tool-
box, including signal processing, optimization, statistics, and much more. Scipy is
a functional library for scientific and high-performance computations.
3. Matplotlib: It is a trendy plotting package that provides 2D plotting as well
as 3D plotting.
4. Scikit-learn: It is a free machine learning library for python programming
language. It has most of the classification, regression, and clustering algorithms,
and works with Python numerical libraries such as Numpy, Scipy.
Machine learning algorithms classify into two groups :
• Supervised Learning algorithms
• Unsupervised Learning algorithms
I. Supervised Learning Algorithms:
Goal: Predict class or value label.
Supervised learning is a branch of machine learning(perhaps it is the mainstream
of machine/deep learning for now) related to inferring a function from
labeled training data. Training data consists of a set of *(input, target)* pairs,
where the input could be a vector of features, and the target instructs what we
desire for the function to output. Depending on the type of the *target*, we can
roughly divide supervised learning into two categories: classification and
regression. Classification involves categorical targets; examples ranging from
some simple cases, such as image classification, to some advanced topics, such as
machine translations and image caption.

Regression involves continuous targets. Its applications include stock
prediction, image masking, and others- which all fall in this category.
To illustrate the example of supervised learning below | Source: Photo by Shirota Yuri, Unsplash
To understand what supervised learning is, we will use an example. For instance,
we give a child 100 stuffed animals in which there are ten animals of each kind
like ten lions, ten monkeys, ten elephants, and others. Next, we teach the kid to
recognize the different types of animals based on different characteristics
(features) of an animal. Such as if its color is orange, then it might be a lion. If it is
a big animal with a trunk, then it may be an elephant.
We teach the kid how to differentiate animals, this can be an example of
supervised learning. Now when we give the kid different animals, he should be
able to classify them into an appropriate animal group.
For the sake of this example, we notice that 8/10 of his classifications were
correct. So we can say that the kid has done a pretty good job. The same applies to
computers. We provide them with thousands of data points with its actual labeled
values (Labeled data is classified data into different groups along with its feature
values). Then it learns from its different characteristics in its training period.
After the training period is over, we can use our trained model to make
predictions. Keep in mind that we already fed the machine with labeled data, so
its prediction algorithm is based on supervised learning. In short, we can say that
the predictions by this example are based on labeled data.
Example of supervised learning algorithms :

Linear Regression , Logistic Regression , K-Nearest Neighbors , Decision Tree ,
Random Forest , Support Vector Machine
II. Unsupervised Learning:
Goal: Determine data patterns/groupings.
In contrast to supervised learning. Unsupervised learning infers from unlabeled
data, a function that describes hidden structures in data.
Perhaps the most basic type of unsupervised learning is dimension reduction
methods, such as PCA, t-SNE, while PCA is generally used in data preprocessing,
and t-SNE usually used in data visualization.
A more advanced branch is clustering, which explores the hidden patterns in data
and then makes predictions on them; examples include K-mean clustering,
Gaussian mixture models, hidden Markov models, and others.
Along with the renaissance of deep learning, unsupervised learning gains more
and more attention because it frees us from manually labeling data. In light of
deep learning, we consider two kinds of unsupervised learning: representation
learning and generative models.
Representation learning aims to distill a high-level representative feature that is
useful for some downstream tasks, while generative models intend to reproduce
the input data from some hidden parameters.
To illustrate the example of unsupervised learning below | Source: Photo by Jelleke Vanooteghem, Unsplash

Unsupervised learning works as it sounds. In this type of algorithms, we do not
have labeled data. So the machine has to process the input data and try to make
conclusions about the output. For example, remember the kid whom we gave a
shape toy? In this case, he would learn from its own mistakes to find the perfect
shape hole for different shapes.
But the catch is that we are not feeding the child by teaching the methods to fit the
shapes (for machine learning purposes called labeled data). However, the child
learns from the toy’s different characteristics and tries to make conclusions about
them. In short, the predictions are based on unlabeled data.
Examples of unsupervised learning algorithms:
Dimension Reduction , Density Estimation , Market Basket Analysis , Generative
adversarial networks (GANs) , Clustering

What would a neural network look like in an abstract real-life example? | Source: Timo Volz, Unsplash
For this article, we will use a few types of
regression algorithms with coding samples in
Python.

References:
[1] Mitchell, Tom. (1997). Machine Learning. McGraw Hill. p. 2. ISBN 007-
042807-7
[2] Machine Learning, Arthur Samuel, Carnegie
Mellon, http://www.contrib.andrew.cmu.edu/~mndarwis/ML.html
[3] Machine Learning (ML) vs. AI, Towards AI, https://towardsai.net/ai-vs-ml
[4] Key Machine Learning Definitions, Towards
AI, https://towardsai.net/machine-learning-definitions
Published via Towards AI

Machine learning-in-details-with-out-python-code

Recommended

More Related Content

What's hot (20)

Similar to Machine learning-in-details-with-out-python-code (20)

More from Osama Ghandour Geris (20)

Recently uploaded (20)

Machine learning-in-details-with-out-python-code