Scribd
Upload
9 views3 pages

Generatuvemodals

Uploaded by

apogeestudycentre
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views3 pages

Generatuvemodals

Uploaded by

apogeestudycentre
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

1.

Generative Models

Definition:

Generative models attempt to model how data is generated by learning the probability distribution of the
input data. After training, these models can generate new data points similar to those in the training set.

Key Concepts:

 Generative vs. Discriminative:


o Generative models learn the joint probability distribution P(x,y)P(x, y)P(x,y), where xxx
is the input (features) and yyy is the label (or outcome).
o Discriminative models (like logistic regression) learn the conditional distribution
P(y∣x)P(y|x)P(y∣x) to predict labels.

How It Works:

 Generative models estimate the probability P(x)P(x)P(x) of the data itself. Once trained, they can
be used to:
o Generate new data (e.g., create new images or text).
o Simulate what unseen data points might look like.

Examples:

1. Naive Bayes Classifier:


o A generative model that assumes features are conditionally independent, given the label. It
is simple but works well for text classification.
2. Gaussian Naive Bayes:
o Assumes that the features follow a normal (Gaussian) distribution. It calculates the
probability of each class given the input data and predicts the class with the highest
likelihood.
3. Generative Adversarial Networks (GANs):
o Two networks: A generator and a discriminator. The generator creates fake data, and the
discriminator tries to distinguish between real and fake data. The two networks "compete"
until the generator becomes skilled at producing realistic data.
o Example: Generating realistic images of people or objects.
4. Variational Autoencoders (VAEs):
o Uses neural networks to learn a compressed, latent representation of the input data, then
generates new data based on this learned representation.
o Example: Generate handwritten digits after being trained on the MNIST dataset.

Applications:

 Image and video generation (GANs).


 Text generation (VAEs, GPT).
 Data augmentation (creating synthetic data for training).

2. Mixture Models

Definition:
A mixture model is a probabilistic model that represents data as being generated from multiple
underlying distributions. Each data point belongs to one of these distributions, but we don't know in
advance which one.

Key Concepts:

 Latent Variables: These variables indicate which distribution (or "component") a data point
came from.
 Soft Clustering: In contrast to hard clustering (like k-means), where each data point belongs to
exactly one cluster, mixture models assign probabilities to data points belonging to different
clusters.

How It Works:

 Mixture models assume the data is generated by a combination of several probability


distributions.
 For a new data point, the model assigns a probability that it belongs to each distribution.
 This probability can be interpreted as a "soft" cluster assignment.

Examples:

1. Gaussian Mixture Model (GMM):


o A GMM assumes that the data is generated from a mixture of several Gaussian
distributions. Each component of the model is a Gaussian distribution with its own mean
and variance.
o Mathematically, the GMM represents the probability of the data as:
P(x)=∑k=1KπkN(x∣μk,Σk)P(x) = \sum_{k=1}^{K} \pi_k \mathcal{N}(x|\mu_k,
\Sigma_k)P(x)=k=1∑KπkN(x∣μk,Σk) Where KKK is the number of Gaussian components,
πk\pi_kπk is the weight of the kkk-th component, and N(x∣μk,Σk)\mathcal{N}(x|\mu_k,
\Sigma_k)N(x∣μk,Σk) is the normal distribution.
2. Expectation-Maximization (EM) Algorithm:
o E-step: Estimate which component each data point likely came from (assign probabilities
to each component).
o M-step: Maximize the likelihood by updating the parameters of each distribution (mean,
variance, and weights).
o EM is often used to fit a GMM to data.

Applications:

 Clustering: GMM can be used to identify clusters in data where each cluster represents a
different Gaussian distribution.
 Anomaly detection: In scenarios like fraud detection, data points that don’t fit well into any
Gaussian component can be flagged as anomalies.

3. Latent Factor Models

Definition:

Latent factor models aim to explain the observed data using hidden variables, called latent factors.
These factors represent the underlying structure or patterns that influence the data, even though they
cannot be directly observed.

Key Concepts:
 Latent Variables: Hidden or unobserved variables that indirectly affect the observed data.
 Dimensionality Reduction: By finding latent factors, these models can reduce the number of
observed features into a smaller, more manageable set.

How It Works:

 Latent factor models decompose observed data into a set of underlying latent factors and their
contributions to each observation.
 These models try to uncover the underlying reasons (latent factors) why the observed data
behaves in a certain way.

Examples:

1. Principal Component Analysis (PCA):


o A method of dimensionality reduction that projects data onto a lower-dimensional space.
o It finds directions (principal components) along which the data varies the most and uses
these components to represent the data in fewer dimensions.
o Example: In image compression, PCA can reduce the number of pixels used to represent
an image by capturing only the most important variation.
2. Matrix Factorization (used in Recommendation Systems):
o Collaborative Filtering: Used to predict user preferences based on latent factors. In a
movie recommendation system, latent factors could represent things like “action vs.
drama” or “complexity of plot”.
o Factorization: The observed matrix (user ratings) is factorized into two lower-rank
matrices: one representing users and one representing items (movies, books, etc.). These
latent factors explain why certain users prefer certain items.
3. Latent Dirichlet Allocation (LDA):
o A generative probabilistic model used in natural language processing (NLP) for topic
modeling. It assumes that documents are mixtures of topics, and topics are distributions
over words.
o Example: Discovering the main themes (topics) in a collection of news articles.

Applications:

 Recommendation systems: Finding latent factors that influence user preferences.


 Natural Language Processing (NLP): Topic modeling with LDA.
 Data Compression: PCA for reducing the size of data while retaining important information.

Summary Table:

Model Type Purpose Examples Applications


Learn to model data
Generative Data generation, text/image
distribution and generate Naive Bayes, GANs, VAEs
Models synthesis
data
Mixture Model data as coming from Gaussian Mixture Models
Clustering, anomaly detection
Models a mixture of distributions (GMM), EM Algorithm
Latent Factor Discover hidden factors that PCA, Matrix Factorization, Dimensionality reduction,
Models explain observed data LDA recommendation systems

You might also like