Machine Learning Note

01. What is machine learning and how does it differ from traditional
programming?

Machine learning
Machine learning is a subset of AI, which uses algorithms that learn from data to make
predictions. These predictions can be generated through supervised learning, where
algorithms learn patterns from existing data, or unsupervised learning, where they
discover general patterns in data.

Differences between machine learning and traditional programming are given

below:

Machine learning Traditional programming

1. In machine learning, a programmer 1. In traditional programming, a
trains a model using a large dataset. The programmer writes explicit rules or
model learns patterns and relationships instructions for the computer to follow.
from the data, enabling it to make
predictions or decisions
2. Heavily reliant on data. The quality and 2. Relies less on data. The quality of the
quantity of the training data significantly output depends mainly on the logic
impact the performance and accuracy of defined by the programmer.
the model.
3. Better for dealing with complex 3. Best suited for problems with clear,
problems where patterns and deterministic logic.
relationships are not evident, such as
image recognition
4. Offers higher adaptability to new 4. Has limited flexibility. Changes in the
scenarios, especially if the model is problem domain require manual updates
retrained with updated data. to the code.
5. Predictions or decisions made by a 5. The outcome is highly predictable if the
machine learning model can sometimes inputs and the logic are known.
be less interpretable, especially with
complex models like deep neural
networks.

© Research Wing | Complexity IT Job Care Page 1

 Machine Learning Note

02. Describe the bias-variance tradeoff in the context of machine learning

models.

To describe the bias-variance tradeoff in the context of machine learning modes, at first
we have to know about the bias, variance and the relation between them.

Bias:While making predictions, a difference occurs between prediction values made by the
model and actual values/expected values, and this difference is known as bias errors or
Errors due to bias.
Variance:variance tells that how much a random variable is different from its expected
value.
Relation between bias and variance:

• Low-Bias,Low-Variance:The combination of low bias and low variance shows an

ideal machine learning model. However, it is not possible practically.
• Low-Bias, High-Variance: With low bias and high variance, model predictions
are inconsistent and accurate on average. This case occurs when the model learns
with a large number of parameters and hence leads to an overfitting
• High-Bias, Low-Variance: With High bias and low variance, predictions are
consistent but inaccurate on average. This case occurs when a model does not

© Research Wing | Complexity IT Job Care Page 2

 Machine Learning Note

learn well with the training dataset or uses few numbers of the parameter. It leads
to underfitting problems in the model.
• High-Bias,High-Variance:
With high bias and high variance, predictions are inconsistent and also inaccurate
on average.

While building the machine learning model, it is really important to take care of bias and
variance in order to avoid overfitting and underfitting in the model. If the model is very
simple with fewer parameters, it may have low variance and high bias. Whereas, if the
model has a large number of parameters, it will have high variance and low bias. So, it is
required to make a balance between bias and variance errors, and this balance between
the bias error and variance error is known as the Bias-Variance trade-off.

03. What is cross validation and why is it important in machine learning?

Cross Validation:
Cross validation is a technique used in machine learning to evaluate the performance
of a model on unseen data. It involves dividing the available data into multiple folds or
subsets, using one of these folds as a validation set, and training the model on the
remaining folds. This process is repeated multiple times, each time using a different
fold as the validation set.

Importance of cross validation:

The main purpose of cross validation is to prevent overfitting, which occurs when a
model is trained too well on the training data and performs poorly on new, unseen
data. By evaluating the model on multiple validation sets, cross validation provides a
more realistic estimate of the model’s generalization performance, i.e., its ability to
perform well on new, unseen data.

04. Can you explain difference between classification and regression algorithm?

Regression Algorithm Classification Algorithm

In Regression, the output variable must In Classification, the output variable must
be of continuous nature or real value. be a discrete value.
The task of the regression algorithm is to The task of the classification algorithm is

© Research Wing | Complexity IT Job Care Page 3

 Machine Learning Note

map the input value (x) with the to map the input value(x) with the
continuous output variable(y). discrete output variable(y).
Regression Algorithms are used with Classification Algorithms are used with
continuous data. discrete data.
In Regression, we try to find the best fit
In Classification, we try to find the
line, which can predict the output more decision boundary, which can divide the
accurately. dataset into different classes.
Regression algorithms can be used to Classification Algorithms can be used to
solve the regression problems such as solve classification problems such as
Weather Prediction, House price
Identification of spam emails, Speech
prediction, etc. Recognition, Identification of cancer cells,
etc.
The regression Algorithm can be further The Classification algorithms can be
divided into Linear and Non-linear divided into Binary Classifier and Multi-
Regression. class Classifier.

05. What are some common evaluation metrics used for classification task?

There are many ways for measuring classification performance. Accuracy, confusion
matrix, log-loss, and AUC-ROC are some of the most popular metrics. Precision-
recall is a widely used metrics for classification problems.

Confusion metrix:
A confusion matrix, also known as an error matrix. A confusion matrix is a table that is
often used to describe the performance of a classification model.
Actual data

Predicted Data (Positive) (Negative)

(Positive) TP FP

(Negative) FN TN

Confusion matrix

The confusion matrix contains the following four entries:

• TP (true positive): The number of records classified as true while they were actually
true.

© Research Wing | Complexity IT Job Care Page 4

 Machine Learning Note

• FP (false positive): The number of records classified as true while they were actually
false.
• FN (false negative): The number of records classified as false while they were actually
true.
• TN (true negative): The number of records classified as false while they were actually
false.
Accuracy:
Accuracy simply measures how often the classifier correctly predicts. We can define
accuracy as the ratio of the number of correct predictions and the total number of
predictions.
True positive+True Negative
Accuracy= True positive+False positive+False Negative+True Negative

Precision:
It explains how many of the correctly predicted cases actually turned out to be
positive.
True positive
Precision= True positive+False positive

Recall:
It explains how many of the actual positive cases we were able to predict correctly
with our model.
True positive
Recall= True positive+False Negative

F1 Score:
It gives a combined idea about Precision and Recall metrics. It is maximum when
Precision is equal to Recall.
Precision+Recall
F1 Score= 2

06. Explain the concept of feature engineering and its importance in machine
learning?

Feature engineering:
Feature engineering is the process of transforming raw data into features that are
suitable for machine learning models. In other words, it is the process of selecting,
extracting, and transforming the most relevant features from the available data to
build more accurate and efficient machine learning models.

Importance of feature engineering:

© Research Wing | Complexity IT Job Care Page 5

 Machine Learning Note

1.Improved Predictive Performance

Well-engineered features can significantly enhance the performance of machine
learning models. By providing relevant and informative input to the model, we can
uncover hidden patterns, reduce noise, and capture the most important characteristics
of the data.
2.Dimensionality Reduction
Feature engineering helps in reducing the dimensionality of the dataset by eliminating
irrelevant or redundant features.

3.Handling Missing Data

Feature engineering allows us to address missing data issues by employing techniques
like imputation. By filling in missing values with appropriate estimations, we can
preserve the integrity of the dataset and prevent bias in model training.
4.Handling Non-Numerical Data
Most machine learning algorithms work with numerical data. Feature engineering
provides methods to encode categorical variables and convert non-numerical data into
numeric representations, making them compatible with various models.

07. What is the curse of dimentionality and how does it affect machine learning
models?

Curse of dimensionality:
The Curse of Dimensionality refers to the various challenges and complications that
arise when analyzing and organizing data in high-dimensional spaces (often hundreds
or thousands of dimensions).

The curse of dimensionality in machine learning is when we face many problems.

• Exponential Increase in Data Volume: When there are more features, the space
where data exists gets much bigger. This makes data points spread out. Which
can make it tricky to get reliable results and increases the chance of overfitting.
• Increased Computational Complexity: In ML, the curse of dimensionality works
with lots of data takes more computer power and time. Algorithms that handle
big datasets often become slow or too hard.
• Data Sparsity: When there are lots of dimensions, there’s not much data in each
one. This makes it tough to guess probability distributions, find close data points,
or spot patterns reliably. So, it’s harder to make correct predictions or group
things accurately.

© Research Wing | Complexity IT Job Care Page 6

 Machine Learning Note

• Increased Model Complexity and Overfitting: When there are lots of data with
many features, models get more complicated to handle all those features. Which
makes them more likely to memorize random stuff from the training data instead
of real patterns. curse of dimensionality in machine learning leads to problems
when trying to predict new data accurately.

08. Describe the difference between batch gradient descent, stochastic gradient
descent and mini-batch gradient descent.

Gradient descent is an optimization algorithm used to minimize the loss function in

machine learning models. There are three main variants of gradient descent: batch
gradient descent, stochastic gradient descent, and mini-batch gradient descent. Each
variant differs in how it uses the training data to compute the gradient of the loss
function.

Batch Gradient Descent:

• Batch gradient descent computes the gradient of the loss function with respect to
the entire training dataset.
• It updates the model parameters after processing all the training examples.
• Each update step requires calculating the gradient over the entire dataset, which
can be very slow and computationally expensive, especially for large datasets.
• Requires storing the entire dataset in memory, which may not be feasible for very
large datasets.

Stochastic Gradient Descent (SGD):

• Stochastic gradient descent computes the gradient of the loss function with
respect to a single training example.
• It updates the model parameters for each training example.
• Much faster than batch gradient descent because it updates the model
parameters more frequently (after each training example).
• Requires much less memory since it processes one example at a time.

Mini-Batch Gradient Descent:

• Mini-batch gradient descent is a compromise between batch gradient descent
and stochastic gradient descent.
• It computes the gradient of the loss function with respect to a small, random
subset of the training dataset (mini-batch) and updates the model parameters
after processing each mini-batch.

© Research Wing | Complexity IT Job Care Page 7

 Machine Learning Note

• Provides a balance between the faster updates of SGD and the stable
convergence of batch gradient descent. It is generally faster than batch gradient
descent and less noisy than SGD.
• Requires more memory than SGD but less than batch gradient descent since it
processes a small subset of the data at a time.
•
09. What is the purpose of regularization in machine learning, and how does it

work?
Regularization is a fundamental concept in machine learning that addresses the issue

of overfitting. Overfitting occurs when a model becomes too complex and memorizes
the training data rather than learning the underlying patterns. This leads to poor

performance on unseen data.

The main purpose of regularization is to prevent overfitting and improve the

generalizability of a machine learning model. It achieves this by introducing a penalty

term to the model's loss function. This penalty term discourages the model from

becoming overly complex by penalizing large coefficients or weights assigned to

features.

How Regularization works:

1. Loss Function: The loss function measures how well the model's predictions fit the

training data. The goal is to minimize the loss function during training.
2. Regularization Term: A penalty term is added to the loss function. This term is

typically proportional to the magnitude of the model's coefficients.

3. Impact on Training: During training, the model minimizes the combined loss

function, which includes both the original loss and the regularization penalty.
4.Simpler Model: By penalizing large coefficients, regularization discourages the model

from fitting too closely to the training data. This leads to a simpler model that is less
likely to overfit.

There are different techniques for regularization, each with its own way of penalizing
coefficients. Some common examples include:

© Research Wing | Complexity IT Job Care Page 8

 Machine Learning Note

L1 Regularization (Lasso): This technique adds a penalty equal to the absolute value of

the coefficients. It can shrink some coefficients to zero, effectively performing feature
selection.

L2 Regularization (Ridge):This technique shrinks the coefficients towards zero but

doesn't necessarily set them to zero.

By applying regularization, you achieve a balance between fitting the training data well
and keeping the model simple enough to generalize to unseen data. This is crucial for

building robust and effective machine learning models.

10. Explain the difference between L1 and L2 regularization.

Answer:

L1 Regularization L2 Regularization

1.The penalty term is based on the 1.The penalty term is based on the squares

absolute values of the model's parameters. of the model's parameters.

2.Produces sparse solutions (some 2.Produces non-sparse solutions (all

parameters are shrunk towards zero). parameters are used by the model).

3.Sensitive to outliers. 3.Robust to outliers.

4.Selects a subset of the most important 4.All features are used by the model.

features.

5.Optimization is non-convex. 5.Optimization is convex.

6.The penalty term is less sensitive to 6.The penalty term is more sensitive to
correlated features. correlated features.

7. Useful when dealing with high- 7. Useful when dealing with high-

dimensional data with many correlated dimensional data with many correlated
features. features and when the goal is to have a

less complex model.

8. Also known as Lasso regularization. 8. Also known as Ridge regularization.

© Research Wing | Complexity IT Job Care Page 9

 Machine Learning Note

11. Describe the ensemble learning technique, particularly bagging and

boosting.
Ensemble learning is a powerful technique in machine learning that combines multiple

models to improve overall predictive performance. The core idea is that an ensemble of
weaker models can often outperform a single, complex model. There are various

ensemble methods, but two of the most popular are bagging and boosting:
Bagging (Bootstrap Aggregating):

Reduces variance and improves stability.

Process:

Step-1: Creates multiple training datasets by sampling with replacement from the
original data (bootstrapping). This means some data points may appear multiple times

in a single training set, while others might be left out.

Step-2: Trains a separate model (usually the same type of model) on each bootstrapped

dataset.
Step-3: Combines predictions from all the models using averaging (regression) or

majority vote (classification) to make the final prediction.

Boosting:

Reduces bias and improves accuracy on complex problems.

Process:

Step-1: Trains a weak learner on the original data.

Step-2: Identifies errors made by the weak learner and assigns higher weights to data

points that the model struggled with.

Step-3: Trains a new weak learner on the adjusted data, focusing on the previously

challenging points. This process is repeated for multiple iterations.

Step-4: Combines the predictions from all the weak learners using a weighted voting

scheme, with stronger learners having more influence in the final prediction.
Differences between Bagging and Boosting:

© Research Wing | Complexity IT Job Care Page 10

 Machine Learning Note

Bagging Boosting

1. Reduce variance, improve stability. 1. Reduce bias, improve accuracy on

complex problems.

2. Trains models independently. 2. Trains models sequentially, each

focusing on errors of prior models.

3. For data sampling, Uses bootstrapping 3. For data sampling, Uses original data,
with replacement. adjusts weights based on errors.

4. Averages predictions (regression) or 4. Uses weighted voting scheme based

uses majority vote (classification) for on model performance for Model

Model Combination. Combination.

Choosing between bagging and boosting depends on the specific problem and data.

Bagging is generally simpler to implement and works well when the base models have
high variance. Boosting can achieve higher accuracy, especially for complex problems,

but requires careful tuning and can be more susceptible to overfitting.

12. What is the difference between a generative and discriminative model?

GENERATIVE Model DISCRIMINATIVE Model

1. Models the joint probability 1. Modeling the conditional probability

distribution of the input characteristics distribution of labels given input attributes

and labels in a generative manner. is the main focus of discriminative

modeling.

2. Creates new samples by learning the 2. Acquires the threshold of judgment that
distribution of the underlying data. distinguishes various classes or categories.

© Research Wing | Complexity IT Job Care Page 11

 Machine Learning Note

3. For example: Text generation and 3. Used in activities like sentiment analysis

image synthesis . and image categorization.

4. Effective with inadequate or missing 4. Excellent at distinguishing between

data. classes or categories is discrimination.

5. May have trouble distinguishing 5. Less useful when dealing with

different classes in large datasets. incomplete or missing data.

13. What is the role of dimensionality reduction techniques in machine

learning?
Dimensionality reduction techniques play a crucial role in machine learning by

simplifying complex datasets with many features. Here's how they benefit machine
learning models:

1. Reduced Complexity: High-dimensional data can lead to complex models that

are prone to overfitting. Dimensionality reduction reduces the number of

features, making the model simpler and easier to train.

2. Improved Performance:By focusing on the most relevant features,

dimensionality reduction can improve the accuracy and efficiency of machine

learning models. Less computation is needed to train the model on a lower

number of dimensions.
3. Visualization:With fewer dimensions, it becomes easier to visualize the data

graphically, which can be helpful for understanding relationships between

features and identifying patterns.

4. Noise Reduction:Dimensionality reduction techniques can help remove

irrelevant information and noise from the data, leading to a more robust model.

Here are some common applications of dimensionality reduction in machine

learning:

© Research Wing | Complexity IT Job Care Page 12

 Machine Learning Note

1. Feature Selection:Techniques like Principal Component Analysis (PCA) can

identify the most important features for a specific task, allowing you to focus on
the most informative data.

2. Clustering: Dimensionality reduction can help visualize data clusters in high-

dimensional spaces, making it easier to group similar data points.

3. Classification and Regression:By reducing irrelevant features, dimensionality

reduction can improve the performance of classification and regression models,

leading to more accurate predictions.

4. Choosing the Right Technique:

There are various dimensionality reduction techniques, each with its own advantages
and disadvantages. The best choice depends on the specific characteristics of your data

and the machine learning task at hand. Some popular techniques include:
1. Principal Component Analysis (PCA):Identifies a new set of features (principal

components) that capture most of the variance in the data.

2. Linear Discriminant Analysis (LDA):Focuses on maximizing the separation

between different classes in the data, particularly useful for classification tasks.
3. t-Distributed Stochastic Neighbor Embedding (t-SNE): Effective for visualizing

high-dimensional data in lower dimensions while preserving local similarities

between data points.

By incorporating dimensionality reduction techniques into your machine learning

workflow, you can gain valuable insights from your data, build more efficient models,

and achieve better overall performance.

14. Can you describe the working principles of support vector machines

(SVMs)?
Support Vector Machines (SVMs) are a powerful machine learning algorithm known

for their effectiveness in various tasks, especially classification.

Classification with Hyperplanes:

© Research Wing | Complexity IT Job Care Page 13

 Machine Learning Note

SVMs aim to find an optimal hyperplane in high-dimensional space that best separates

data points belonging to different classes. This hyperplane can be thought of as a

decision boundary that maximizes the margin between the classes.

Margins and Support Vectors:

The margin refers to the distance between the hyperplane and the closest data points

from each class, called support vectors. These support vectors play a crucial role in
defining the optimal hyperplane.

SVMs strive to maximize this margin, as a larger margin indicates a clearer separation
between the classes and better generalization to unseen data.

Handling Non-Linear Data:

While SVMs excel at finding linear separation, real-world data is often non-linear. To

address this, SVMs can leverage a technique called the kernel trick.
The kernel trick implicitly maps the data points to a higher-dimensional feature space

where a linear separation might exist. This allows SVMs to effectively handle non-linear
data.

Key Steps in SVM Training:

1. Data Representation:Each data point is represented as a feature vector in an n-

dimensional space (n being the number of features).

2. Hyperplane Definition: The SVM algorithm finds the hyperplane that maximizes

the margin between the classes. This involves calculating the distance between
the hyperplane and the support vectors.

3. Classification: New data points are mapped to the same feature space, and the
SVM model predicts their class labels based on which side of the hyperplane they

fall on.
Advantages of SVMs:

1. Effective in high-dimensional spaces: SVMs perform well even with a large

number of features.

© Research Wing | Complexity IT Job Care Page 14

 Machine Learning Note

2. Robust to noise: The focus on maximizing the margin makes SVMs less

susceptible to noisy data points.

3. Kernel trick for non-linear data: SVMs can handle non-linear data through the

kernel trick.
4. Memory efficiency: The model primarily relies on the support vectors for

prediction, making it memory efficient.

Disadvantages of SVMs:

1. Can be computationally expensive:Training SVMs, especially with large

datasets, can be computationally intensive.

2. Tuning hyperparameters: The effectiveness of SVMs can be sensitive to the

choice of hyperparameters, requiring careful tuning.

3. Interpretability:The inner workings of the model can be less interpretable

compared to simpler models.

Overall, SVMs are a versatile and powerful tool for various machine learning tasks,
particularly classification. Their ability to handle high-dimensional data, robustness to

noise, and effectiveness with non-linear data through the kernel trick make them a
popular choice for many applications.

How does SVM work:

Let’s consider two independent variables x1, x2, and one dependent variable which is

either a blue circle or a red circle.

© Research Wing | Complexity IT Job Care Page 15

 Machine Learning Note

From the figure above we can say that there are multiple lines that segregate our data

points or do a classification between red and blue circles. So how do we choose the best
line or in general the best hyperplane that segregates our data points?

One reasonable choice as the best hyperplane is the one that represents the largest
separation or margin between the two classes.

So we choose the hyperplane whose distance from it to the nearest data point on each

side is maximized. If such a hyperplane exists it is known as the maximum-margin

hyperplane/hard margin. So from the above figure, we choose L2. Let’s consider a

scenario like shown below-

© Research Wing | Complexity IT Job Care Page 16

 Machine Learning Note

Here we have one blue ball in the boundary of the red ball. The blue ball in the

boundary of red ones is an outlier of blue balls. The SVM algorithm has the
characteristics to ignore the outlier and finds the best hyperplane that maximizes the

margin. SVM is robust to outliers.

So in this type of data point what SVM does is, finds the maximum margin as done with

previous data sets along with that it adds a penalty each time a point crosses the
margin. So the margins in these types of cases are called soft margins. When there is a

soft margin to the data set, the SVM tries to minimize. Hinge loss is a commonly used
penalty. If no violations no hinge loss.If violations hinge loss proportional to the distance

of violation.
15. Describe a real-world machine learning project you've worked on, including

the problem statement, data preprocessing steps, choice of algorithm,

model evaluation, and any challenges faced.

1. Problem Statement:Develop a system to predict churn rate (customer

cancellation) for a telecommunications company.

2. Data Preprocessing:This will help the company identify customers at risk of

churning and implement targeted retention strategies.

© Research Wing | Complexity IT Job Care Page 17

 Machine Learning Note

3. Data Collection: Customer data is collected, including demographics, service

plans, call history, and payment information.

4. Data Cleaning:Missing values are addressed, inconsistencies are corrected, and

outliers are identified and handled appropriately.

5. Feature Engineering:New features might be created based on existing ones

(e.g., total monthly call duration). Categorical features might be encoded

numerically.

6. Choice of Algorithm:Considering the classification task and the potentially non-

linear relationship between features and churn, a Random Forest algorithm could

be a good choice. Random Forests are robust to overfitting and can handle a
variety of data types.

7. Model Training and Evaluation:The data is split into training and testing
sets.The Random Forest model is trained on the training data.The model's

performance is evaluated on the testing set using metrics like accuracy, precision,
recall, and F1-score. These metrics provide insights into how well the model

identifies churned and non-churned customers.

Hyperparameter tuning might be performed to optimize the model's performance.

Challenges Faced:
1. Data Quality: Ensuring the accuracy and completeness of customer data is

crucial. Techniques to handle missing values and outliers are necessary.

2. Imbalanced Classes:The churn rate might be a minority class compared to non-

churning customers. This can lead to models biased towards the majority class.
Techniques like oversampling or undersampling the minority class can be

explored.
3. Model Interpretability: While Random Forests are powerful, interpreting their

inner workings can be challenging. Feature importance scores can be used to

understand which features have the most significant impact on churn prediction.

© Research Wing | Complexity IT Job Care Page 18

 Machine Learning Note

Additional Considerations:

1. Model Deployment:Once a satisfactory model is achieved, it can be deployed

into production to score new customer data and identify potential churners.

2. Monitoring and Improvement: The model's performance should be monitored

over time, and retraining might be necessary as customer behavior and market

conditions evolve.
This is a simplified example, but it highlights the key steps involved in a real-world

machine learning project. The specific details will vary depending on the problem and
the chosen approach.

16. Define machine learning and differentiate between supervised,

unsupervised, and reinforcement learning.

Supervised Learning Unsupervised Learning Reinforcement Learning

Input data is labeled. Input data is not labeled. Input data is not predefined.

Learn pattern of inputs and Divide data into classes. Find the best reward

their labels. between a start and an end

state.

Finds a mapping equation Finds similar features in Maximizes reward by

on input data and its input data to classify it into assessing the results of

labels. classes. state-action pairs.

Model is built and trained Model is built and trained The model is trained and
prior to testing. prior to testing. tested simultaneously.

Deal with regression and Deals with clustering and Deals with exploration and
classification problems. associative rule mining exploitation problems.

problems.

© Research Wing | Complexity IT Job Care Page 19

 Machine Learning Note

Decision trees, linear K-means clustering, k- Q-learning, SARSA, Deep Q

regression, K-nearest medoids clustering, Network

neighbors agglomerative clustering

Image detection, Customer segmentation, Drive-less cars, self-

Population growth feature elicitation, targeted navigating vacuum cleaners,

prediction. marketing, etc etc

17. What are the main steps involved in a typical machine learning project
pipeline?
A Machine Learning pipeline is a process of automating the workflow of a complete
machine learning task.

Machine Learning Pipeline Steps

A typical ML pipeline includes the following stages:

1. Data Ingestion

Each ML pipeline starts with the Data ingestion step. In this step, the data is processed
into a well-organized format, which could be suitable to apply for further steps. This
step does not perform any feature engineering; rather, this may perform the versioning
of the input data.

2. Data Validation

© Research Wing | Complexity IT Job Care Page 20

 Machine Learning Note

The next step is data validation, which is required to perform before training a new
model. Data validation focuses on statistics of the new data, e.g., range, number of
categories, distribution of categories, etc. In this step, data scientists can detect if any
anomaly present in the data. There are various data validation tools that enable us to
compare different datasets to detect anomalies.

3. Data Pre-processing

Data pre-processing is one of the most crucial steps for each ML lifecycle as well as the
pipeline. We cannot directly input the collected data to train the model without pre-
processing it, as it may generate an abrupt result.

The pre-processing step involves preparing the raw data and making it suitable for the
ML model. The process includes different sub-steps, such as Data cleaning, feature
scaling, etc. The product or output of the data pre-processing step becomes the final
dataset that can be used for model training and testing. There are different tools in ML
for data pre-processing that can range from simple Python scripts to graph models.

4. Model Training & Tuning

The model training step is the core of each ML pipeline. In this step, the model is trained
to take the input (pre-processed dataset) and predicts an output with the highest
possible accuracy.

However, there could be some difficulties with larger models or with large training data
sets. So, for this, efficient distribution of the model training or model tuning is required.

This issue of the model training stage can be solved with pipelines as they are scalable,
and a large number of models can be processed concurrently.

5. Model Analysis

After model training, we need to determine the optimal set of parameters by using the
loss of accuracy metrics. Apart from this, an in-depth analysis of the model's
performance is crucial for the final version of the model. The in-depth analysis includes
calculating other metrics such as precision, recall, AUC, etc. This will also help us in
determining the dependency of the model on features used in training and explore how
the model's predictions would change if we altered the features of a single training
example.

6. Model Versioning

© Research Wing | Complexity IT Job Care Page 21

 Machine Learning Note

The model versioning step keeps track of which model, set of hyperparameters, and
datasets have been selected as the next version to be deployed. For various situations,
there could occur a significant difference in model performance just by applying
more/better training data and without changing any model parameter. Hence, it is
important to document all inputs into a new model version and track them.

7. Model Deployment

After training and analyzing the model, it's time to deploy the model. An ML model can
be deployed in three ways, which are:

o Using the Model server,

o In a Browser
o On Edge device

However, the common way to deploy the model is using a model server. Modelservers
allow to host multiple versions simultaneously, which helps to run A/B tests on models
and can provide valuable feedback for model improvement.

8. Feedback Loop

Each pipeline forms a closed-loop to provide feedback. With this close loop, data
scientists can determine the effectiveness and performance of the deployed models.
This step could be automated or manual depending on the requirement.

18. Explain the bias-variance tradeoff and its significance in machine learning
model selection.

Bias-Variance Trade-Off

While building the machine learning model, it is really important to take care of bias and
variance in order to avoid overfitting and underfitting in the model. If the model is very
simple with fewer parameters, it may have low variance and high bias. Whereas, if the
model has a large number of parameters, it will have high variance and low bias. So, it is
required to make a balance between bias and variance errors, and this balance between
the bias error and variance error is known as the Bias-Variance trade-off.

© Research Wing | Complexity IT Job Care Page 22

 Machine Learning Note

Total Error = Bias2 + Variance + Irreducible Error

For an accurate prediction of the model, algorithms need a low variance and low bias.
But this is not possible because bias and variance are related to each other:

o If we decrease the variance, it will increase the bias.

o If we decrease the bias, it will increase the variance.

Bias-Variance trade-off is a central issue in supervised learning. Ideally, we need a model

that accurately captures the regularities in training data and simultaneously generalizes
well with the unseen dataset. Unfortunately, doing this is not possible simultaneously.
Because a high variance algorithm may perform well with training data, but it may lead
to overfitting to noisy data. Whereas, high bias algorithm generates a much simple
model that may not even capture important regularities in the data. So, we need to find
a sweet spot between bias and variance to make an optimal model.

19. Describe various methods to handle missing data in a dataset.

Missing values are a common challenge in data analysis, and there are several
strategies for handling them. Here’s an overview of some common approaches:

Removing Rows with Missing Values

• Simple and efficient: Removes data points with missing values altogether.
• Reduces sample size: Can lead to biased results if missingness is not random.
• Not recommended for large datasets: Can discard valuable information.

Imputation Methods
• Replacing missing values with estimated values.

© Research Wing | Complexity IT Job Care Page 23

 Machine Learning Note

• Preserves sample size: Doesn’t reduce data points.

• Can introduce bias: Estimated values might not be accurate.

Here are some common imputation methods:

1- Mean, Median, and Mode Imputation:
• Replace missing values with the mean, median, or mode of the relevant variable.
• Simple and efficient: Easy to implement.
• Can be inaccurate: Doesn’t consider the relationships between variables.

Forward and Backward Fill

• Replace missing values with the previous or next non-missing value in the same
variable.
• Simple and intuitive: Preserves temporal order.
• Can be inaccurate: Assumes missing values are close to observed values
These fill methods are particularly useful when there is a logical sequence or order in
the data, and missing values can be reasonably assumed to follow a pattern.

3. Interpolation Techniques
• Estimate missing values based on surrounding data points using techniques like
linear interpolation or spline interpolation.
• More sophisticated than mean/median imputation: Captures relationships between
variables.
• Requires additional libraries and computational resources.

These interpolation techniques are useful when the relationship between data points
can be reasonably assumed to follow a linear or quadratic pattern.

• Linear interpolation assumes a straight line between two adjacent non-missing

values.
• Quadratic interpolation assumes a quadratic curve that passes through three
adjacent non-missing values.

20. What is feature selection, and how does it contribute to model

performance?

Feature selection is a process that chooses a subset of features from the original
features so that the feature space is optimally reduced according to a certain criterion.
There are three general classes of feature selection algorithms: Filter methods,
wrapper methods and embedded methods.

© Research Wing | Complexity IT Job Care Page 24

 Machine Learning Note

The role of feature selection in machine learning is,

1. To reduce the dimensionality of feature space.
2. To speed up a learning algorithm.
3. To improve the predictive accuracy of a classification algorithm.
4. To improve the comprehensibility of the learning results.

21. Differentiate between parametric and non-parametric machine learning

algorithms.
There are several Difference between Parametric and Non-Parametric Methods are as
follows:
Parametric Methods Non-Parametric Methods

Non-Parametric Methods use the flexible

Parametric Methods uses a fixed number
number of parameters to build the
of parameters to build the model.
model.

Parametric analysis is to test group A non-parametric analysis is to test

means. medians.

It is applicable for both – Variable and

It is applicable only for variables.
Attribute.

It always considers strong assumptions It generally fewer assumptions about

about data. data.

Parametric Methods require lesser data Non-Parametric Methods requires much

than Non-Parametric Methods. more data than Parametric Methods.

Parametric methods assumed to be a There is no assumed distribution in non-

normal distribution. parametric methods.

Parametric data handles – Intervals data But non-parametric methods handle

© Research Wing | Complexity IT Job Care Page 25

 Machine Learning Note

Parametric Methods Non-Parametric Methods

or ratio data. original data.

Here when we use parametric methods When we use non-parametric methods

then the result or outputs generated can then the result or outputs generated
be easily affected by outliers. cannot be seriously affected by outliers.

Parametric Methods can perform well in Similarly, Non-Parametric Methods can

many situations but its performance is at perform well in many situations but its
peak (top) when the spread of each performance is at peak (top) when the
group is different. spread of each group is the same.

Non-parametric methods have less

Parametric methods have more statistical
statistical power than Parametric
power than Non-Parametric methods.
methods.

As far as the computation is considered As far as the computation is considered

these methods are computationally faster these methods are computationally
than the Non-Parametric methods. slower than the Parametric methods.

Examples: Logistic Regression, Naïve

Examples: KNN, Decision Tree Model, etc.
Bayes Model, etc.

22. Explain the concept of cross-validation and why it is essential in model

evaluation.
Cross validation is a technique used in machine learning to evaluate the performance
of a model on unseen data. It involves dividing the available data into multiple folds or
subsets, using one of these folds as a validation set, and training the model on the
remaining folds. This process is repeated multiple times, each time using a different
fold as the validation set. Finally, the results from each validation step are averaged to
produce a more robust estimate of the model’s performance. Cross validation is an
important step in the machine learning process and helps to ensure that the model
selected for deployment is robust and generalizes well to new data.

© Research Wing | Complexity IT Job Care Page 26

 Machine Learning Note

The main purpose of cross validation is to prevent overfitting, which occurs when a
model is trained too well on the training data and performs poorly on new, unseen
data. By evaluating the model on multiple validation sets, cross validation provides a
more realistic estimate of the model’s generalization performance, i.e., its ability to
perform well on new, unseen data.
23. What is regularization, and how does it help prevent overfitting in machine
learning models?
Regularization is a technique in machine learning that helps prevent from overfitting.
Regularization prevents the model from fitting the training data too closely, which is a
common cause of overfitting. Instead, it promotes a balance between model
complexity and performance, leading to better generalization on new, unseen data.

How Regularization used to prevent overfitting

1. By introducing the regularization term in loss function that act like a constrain
function of the model’s parameter. This function penalize certain parameter values
in model, discouraging them from becoming too large or complex.

2. Regularization introduces a trade-off between fitting the training data and keeping
the model’s parameters small. The strength of regularization is controlled by a
hyperparameter, often denoted as lambda (λ). A higher λ value leads to stronger
regularization and a simpler model.

3. Regularization techniques help control the complexity of the model. They make the
model more robust by constraining the parameter space. This results in smoother
decision boundaries in the case of classification and smoother functions in
regression, reducing the potential for overfitting.

4. Regularization oppose overfitting by discouraging the model from fitting the

training data too closely. It prevents parameters from taking extreme values, which
might be necessary to fit the training data.

24. Discuss the differences between batch gradient descent, stochastic gradient
descent, and mini-batch gradient descent optimization algorithms.

Gradient Descent Stochastic Gradient Mini-batch gradient

Descent descent

Gradient Descent Stochastic gradient Mini-batch Gradient Descent

determines the cost descent involves updating updates the model

© Research Wing | Complexity IT Job Care Page 27

 Machine Learning Note

function's gradient the model parameters parameters based on the

throughout the whole and computing the mean gradient of the cost
training dataset and gradient of the cost function with respect to the
updates the model's function for a single model parameters over a
parameters based on the random training example mini-batch, which is a smaller
mean of all training at each iteration. subset of the training dataset
examples across each of equivalent size.
epoch.

As each iteration of the SGD adjusts the model In order to strike a reasonable
approach requires parameters more often balance between speed and
computing the gradient of than GD, which causes it accuracy, the model
the cost function across the to converge more quickly parameters are changed
whole training dataset, GD more frequently than GD but
takes some time to less frequently than SGD.
converge.

Due to the requirement to As just one training Just a percentage of the

retain the whole training sample needs to be training samples had to be
dataset, GD consumes a lot stored for each iteration, retained for each repetition,
of memory. SGD requires less therefore the memory use is
memory. manageable.

GD is computationally As the cost function's As the gradient of the cost

expensive because the gradient only needs to be function must be calculated
gradient of the cost calculated once for each for a portion of the training
function must be repeat of training data, examples for each iteration, it
computed for the whole SGD is computationally is computationally efficient.
training dataset at each efficient.
iteration.

With little error, GD Due to the fact that SGD Mini-batch Gradient Descent
modifies the model's is updated using just one has a significant amount of
parameters based on the training sample, it has a noise because the update is
average of all training lot of noise. based on a small number of
samples. training examples.

© Research Wing | Complexity IT Job Care Page 28

 Machine Learning Note

25. Describe the workings of decision trees and how they handle both
classification and regression tasks.

Decision trees are a type of supervised learning algorithm used for both classification
and regression tasks. They operate by splitting the data into subsets based on the value
of input features.

Tree Structure:

•A decision tree is composed of nodes and branches.

•Each internal node represents a test on an attribute, each branch represents the
outcome of the test.

•Each leaf node represents the final outcome (class label or numerical value).

Handling Classification Tasks

In classification tasks, decision trees aim to partition the data such that each
partition contains instances of a single class as much as possible.

1. Impurity Measures:
• The quality of a split is evaluated using impurity measures such as Gini impurity, entropy
(information gain), or misclassification error.
• Gini Impurity: Measures the probability of incorrectly classifying a randomly chosen
element if it was randomly labeled according to the distribution of labels in the subset.
• Entropy: Measures the amount of disorder or randomness. Information gain is the
reduction in entropy.
2. Classification Decision:
• At each node, the algorithm calculates the impurity for each possible split and selects
the split that minimizes the impurity.
• Once the tree is built, to classify a new instance, the instance is passed through the tree
following the splits corresponding to the values of its features until it reaches a leaf
node. The class label of the leaf node is the predicted class for the instance.

Handling Regression Tasks

© Research Wing | Complexity IT Job Care Page 29

 Machine Learning Note

In regression tasks, decision trees aim to partition the data such that each partition is as
homogeneous as possible with respect to the target variable (i.e., the values in each
partition should be close to each other).

1. Variance Reduction:

• The quality of a split in regression is often measured by variance reduction (minimizing

the mean squared error) or other similar metrics.
• The algorithm looks for splits that minimize the variance of the target variable within
each subset created by the split.

2. Regression Decision:

• At each node, the algorithm evaluates the potential splits and selects the one that
results in the greatest reduction in variance.
• For making predictions, a new instance is passed through the tree down to a leaf node.
The value at the leaf node, often the mean value of the target variable in that leaf, is the
predicted value for the instance.

26. What are ensemble learning methods, and how do bagging and boosting
differ?

Ensemble learning methods are techniques that create multiple models and combine
them to solve a particular computational problem. By aggregating the predictions of
several models, it can create a more accurate and robust model that often achieve
better performance than any single model could. The main idea behind ensemble
learning is to leverage the strengths and compensate for the weaknesses of individual
models.

Differences Between Bagging and Boosting

Bagging Boosting

The simplest way of combining A way of combining predictions that

1. predictions that belong to the same type belong to the different types.

© Research Wing | Complexity IT Job Care Page 30

 Machine Learning Note

Bagging Boosting

2. Reduces variance, not bias. Reduces bias, not variance.

Models are weighted according to their

3. Each model receives equal weight. performance.

Models are trained sequentially; each

model are influenced
Models are trained independently and in by the performance of previously built
4. parallel. models.

Final prediction is made by averaging the Final prediction is typically a weighted

predictions (for regression) or by majority sum of the predictions of all the models,
voting (for classification) of all the where more accurate models have higher
5. models. weights.

Bagging tries to solve the over-fitting

6. problem. Boosting tries to reduce bias.

Generally simpler to implement and can More complex due to the sequential
be easily parallelized since models are nature and the need to adjust weights
7. trained independently. and focus on errors iteratively.

Training time is often longer because

Training time can be shorter due to models are built sequentially, and each
parallelism, especially when using step depends on the results of the
8. complex models like decision trees. previous one.

Example: The Random forest model uses Example: The AdaBoost, Gradient
Bagging. Boostinguses Boosting techniques

27. Describe the K-nearest neighbors (KNN) algorithm and its applications.

The K-nearest neighbors (KNN) algorithm is a simple, instance-based learning algorithm

used for both classification and regression tasks in machine learning. Here's a detailed
overview of the KNN algorithm and its applications:

© Research Wing | Complexity IT Job Care Page 31

 Machine Learning Note

Algorithm Overview:

1. KNN is simple yet powerful.

2. It belongs to the supervised learning domain.
3. Widely used in pattern recognition, data mining and intrusion detection.
4. Non-parametric: Makes no assumptions about data distribution.

Description of KNN Algorithm

1. Instance-Based Learning: KNN is a non-parametric, instance-based learning algorithm,

which means it does not make any assumptions about the underlying data distribution.
Instead, it stores all the training data and makes decisions based on the entire dataset.

2. Basic Principle: The core idea of KNN is to predict the label of a new data point based
on the labels of the 'k' nearest data points in the training set. The 'k' in KNN is a user-
defined constant and determines the number of neighbors to consider.

3. Distance Metric: To find the nearest neighbors, a distance metric is used. The most
common distance metric is Euclidean distance, but others like Manhattan distance,
Minkowski distance, or Hamming distance (for categorical data) can also be used.

4. Classification:

• For a classification task, the new data point is assigned the class that is most common
among its 'k' nearest neighbors. This process is often referred to as "majority voting."
• For example, if k=3 and the three nearest neighbors have labels A, B, and B, the new
data point would be classified as B.

5. Regression:

• For a regression task, the value assigned to the new data point is the average (or
sometimes weighted average) of the values of its 'k' nearest neighbors.
• For example, if k=3 and the three nearest neighbors have values 5, 6, and 7, the
predicted value for the new point would be (5+6+7)/3 = 6.

6. Choosing 'k': The choice of 'k' is crucial:

© Research Wing | Complexity IT Job Care Page 32

 Machine Learning Note

• A small 'k' (like 1 or 3) makes the algorithm sensitive to noise but captures the local
structure well.
• A large 'k' provides a smoother decision boundary but might overlook local structures,
leading to underfitting.

Applications of KNN

1. Pattern Recognition: KNN is widely used in various pattern recognition tasks, including
image recognition and handwriting recognition. For instance, in optical character
recognition (OCR), KNN can classify characters based on pixel intensity.

2. Recommendation Systems: KNN is used in recommendation systems to find similar

users or items. For example, in collaborative filtering for movie recommendations, KNN
can identify users with similar tastes and recommend movies they have liked.

3. Medical Diagnosis: In healthcare, KNN can be used for disease prediction and
diagnosis. For instance, it can help predict whether a patient has a particular disease
based on their symptoms and historical data.

4. Finance: In financial applications, KNN can be used for stock price prediction, customer
segmentation, and credit scoring by analyzing historical financial data and trends.

5. Anomaly Detection: KNN is useful in detecting anomalies in data, such as fraudulent

transactions or network intrusions, by identifying data points that do not conform to the
patterns of their nearest neighbors.

6. Text Categorization: In natural language processing, KNN can be applied to text

categorization tasks, such as classifying documents or emails into predefined categories
based on their content and similarity to other documents.

© Research Wing | Complexity IT Job Care Page 33

 Machine Learning Note

28. What is the difference between precision and recall, and how are they
relevant in classification problems?

Precision Recall

Precision is about how many of the Recall is about how many of the actual
1. predicted positives are actual positives positives were identified by the model.

It focuses on the accuracy of the positive It focuses on the coverage of the positive
2. predictions. instances.

It is affected by false positives. More false It is affected by false negatives. More false
3. positives lead to lower precision. negatives lead to lower recall.

Precision is crucial when the cost of false Recall is crucial when the cost of false
4. positives is high negatives is high

A model with high precision makes fewer A model with high recall captures most
5. false positive errors. positives instances

Formula: Formula:
𝑇𝑃 𝑇𝑃
𝑇𝑃 + 𝐹𝑃 𝑇𝑃 + 𝐹𝑁
6.

Relevance in Classification Problems

1.Imbalanced Datasets:

• In classification problems with imbalanced datasets (e.g., rare disease detection),

precision and recall are more informative than accuracy. Accuracy might be misleading if
the majority class dominates the dataset.

2. Application-Specific Requirements:

• The choice between optimizing for precision or recall depends on the specific
requirements of the application. For example, in fraud detection, you might prioritize
recall to catch as many fraudulent transactions as possible, even if it means more false
positives.
© Research Wing | Complexity IT Job Care Page 34
 Machine Learning Note

3. Trade-Off and Balance:

• There is often a trade-off between precision and recall. Improving one usually leads to a
decrease in the other. The balance between them can be evaluated using the F1 score,
Precesion ∗ Recall
which is the harmonic mean of precision and recall: F1 Score = 2 ∗ Precesion + Recall

• The F1 score provides a single metric that balances both precision and recall, making it
useful for evaluating model performance when both metrics are important.

Understanding and balancing precision and recall are crucial for developing effective
classification models, particularly in applications where the costs of false positives and
false negatives are significant. By focusing on these metrics, practitioners can tailor their
models to meet the specific needs and constraints of their applications.

29. Discuss the concept of clustering and provide examples of clustering

algorithms.

Clustering is a type of unsupervised learning technique used in data analysis and

machine learning. The primary objective of clustering is to group a set of objects in such
a way that objects in the same group (or cluster) are more similar to each other than to
those in other groups. This similarity is often based on a defined metric or distance
measure.

Concept of Clustering

Definition:

Clustering involves partitioning a dataset into distinct groups where the data points
within each group are more similar to each other than to those in other groups.

It is used to discover structure and patterns in data that is not labeled.

Applications:

Customer segmentation in marketing to identify groups of customers with similar

behavior.

Document clustering for organizing large sets of text data.

Image segmentation in computer vision.

© Research Wing | Complexity IT Job Care Page 35

 Machine Learning Note

Anomaly detection by identifying outliers as separate clusters.

Genomic data analysis for grouping genes or proteins with similar expressions.

Examples of Clustering Algorithms

1. K-Means Clustering:

Description:

o Divides data into K clusters based on similarity.

o Iteratively assigns data points to the nearest cluster centroid.
o Centroids are updated based on the mean of points in each cluster.

Use Cases:

Customer segmentation, image compression, recommendation systems.

2. Hierarchical Clustering:

Description:

o Builds a hierarchy of clusters (dendrogram).

o Agglomerative (bottom-up) or divisive (top-down) approach.
o Merges or splits clusters based on similarity.

Use Cases:

Taxonomy creation, gene expression analysis.

30. Explain the importance of feature scaling and normalization in machine

learning.

Feature scaling and normalization are crucial preprocessing techniques in machine

learning. Let’s explore their significance and why they matter:

Feature Scaling:

Definition: Feature scaling transforms the values of features (variables) in a dataset to a

similar scale.

Purpose:

© Research Wing | Complexity IT Job Care Page 36

 Machine Learning Note

• Ensures that all features contribute equally to the model.

• Avoids domination of features with larger values.

Importance:

• Some machine learning algorithms (e.g., gradient descent-based methods) are

sensitive to feature scaling.
• Scaling facilitates meaningful comparisons between features.
• Improves model convergence by ensuring consistent step sizes during
optimization.
• Prevents certain features from overshadowing others based solely on their
magnitude.

Normalization:

Definition: Normalization standardizes feature values to a common scale (often

between 0 and 1).

Process:

• Subtract the minimum value from each feature.

• Divide by the range (maximum - minimum).

Importance:

• Ensures that all features contribute equally to the model.

• Prevents larger-magnitude features from dominating others.

31. What is dimensionality reduction, and how does it help in machine learning
tasks?

Dimensionality reduction is a crucial technique in machine learning that aims to reduce

the number of input features (dimensions) in a dataset while retaining as much relevant
information as possible.The goal is to simplify the dataset's representation, making it
easier to analyze, visualize, and model, while also potentially improving the performance
and efficiency of machine learning algorithms. Dimensionality reduction can be broadly
categorized into two types: feature selection and feature extraction.

© Research Wing | Complexity IT Job Care Page 37

 Machine Learning Note

Benefits of Dimensionality Reduction:

1.Improved Model Performance:

• Simplifies the model by focusing on essential features.

• Reduces overfitting and improves generalization to new data.

2.Computational Efficiency:

•Faster training and prediction due to fewer features.

3.Visualization:

• Projects high-dimensional data into lower dimensions for visualization.

4.Noise Reduction:

• Removes irrelevant or noisy features.

5.Feature Engineering:

• Creates new informative features from existing ones.

32. Describe the working of neural networks and their various architectures.

A neural network is a machine learning model inspired by the function and structure of
the human brain. They consist of interconnected nodes, or neurons, organized into
layers. Each neuron receives input signals, processes them through an activation
function, and passes the result to neurons in the next layer.

Basic Structure:

A neural network consists of interconnected layers of artificial neurons (also called

nodes or units).

Three main types of layers:

Input Layer: Receives input data (features).

Hidden Layer(s): Intermediate layers that process information.

Output Layer: Produces the final prediction or decision.

© Research Wing | Complexity IT Job Care Page 38

 Machine Learning Note

Working of Neural Networks:

Forward Propagation:

• Input data flows through the network layer by layer.

• Each neuron computes a weighted sum of its inputs, applies an activation
function, and passes the result to the next layer.
• Activation functions introduce non-linearity, allowing neural networks to model
complex relationships.

Backpropagation:

• During training, the network adjusts its weights to minimize the prediction error
(loss).
• Backpropagation computes gradients of the loss with respect to each weight.
• Gradient descent updates weights to minimize the loss.

Various Architectures of Neural Networks

Feedforward Neural Networks (FNN):

FNNs are the simplest form of neural networks, where information flows in one
direction, from input to output layers. They are suitable for tasks such as regression and
classification.

Convolutional Neural Networks (CNN):

CNNs are specialized for processing grid-like data, such as images. They consist of
convolutional layers that apply filters to extract spatial hierarchies of features, followed
by pooling layers to reduce dimensionality and fully connected layers for classification.

Recurrent Neural Networks (RNN):

RNNs are designed for sequential data processing, such as time series or natural
language. They have connections that form directed cycles, allowing them to maintain
internal state and capture temporal dependencies.

Generative Adversarial Networks (GAN):

GANs consist of two neural networks, a generator and a discriminator, trained

simultaneously in a game-theoretic framework. The generator learns to generate

© Research Wing | Complexity IT Job Care Page 39

 Machine Learning Note

realistic data samples, while the discriminator learns to distinguish between real and
generated samples.

Long Short-Term Memory (LSTM) Networks:

LSTMs are a variant of RNNs that address the vanishing gradient problem. They use
specialized memory cells with gating mechanisms to selectively retain or forget
information over long sequences.

Autoencoder:

Autoencoders are neural networks trained to reconstruct input data at the output layer,
typically through a bottleneck layer with a lower dimensionality than the input. They are
used for unsupervised feature learning and dimensionality reduction.

© Research Wing | Complexity IT Job Care Page 40