Unit 4: Cluster Analysis and Clustering Methods

1. Introduc on to Cluster Analysis

What is Cluster Analysis?
Cluster analysis is an unsupervised learning technique that groups similar data points into clusters based on their
characteris cs. Unlike classifica on, clustering does not rely on labeled data. The goal is to maximize intra-cluster
similarity and minimize inter-cluster similarity.
Requirements for Cluster Analysis:
1. Scalability: Ability to handle large datasets efficiently.
2. Ability to Iden fy Arbitrary Shapes: Should detect clusters of arbitrary shapes and sizes.
3. Minimal Domain Knowledge: Should require li le or no prior knowledge.
4. Noise Tolerance: Should be robust against noisy data.
5. Interpretability: Results should be meaningful and easily interpretable.

2. Basic Clustering Methods

k-Means Clustering:
1. Overview:
o Par ons data into k clusters by minimizing the sum of squared distances between data points and
their cluster centroids.
o Requires the number of clusters (k) to be specified in advance.
2. Algorithm Steps:
o Ini alize k centroids randomly.
o Assign each data point to the nearest centroid.
o Recalculate centroids based on the assigned data points.
o Repeat un l convergence.
3. Advantages:
o Simple and efficient.
o Works well for spherical clusters.
4. Disadvantages:
o Sensi ve to ini al centroid posi ons.
o Assumes clusters are of similar sizes.
k-Medoids Clustering:
1. Overview:
o Similar to k-Means but uses actual data points (medoids) as cluster centers.
o More robust to noise and outliers.
2. Algorithm Steps:
o Ini alize k medoids randomly.
o Assign data points to the nearest medoid.
o Swap medoids to minimize the total cost (distance).
o Repeat un l convergence.

3. Density-Based Clustering
DBSCAN (Density-Based Spa al Clustering of Applica ons with Noise):
1. Overview:
o Groups data points into clusters based on regions of high density.
o Iden fies noise as points that do not belong to any cluster.
2. Key Parameters:
o Epsilon (ε): Neighborhood radius.
 o MinPts: Minimum number of points required to form a dense region.
3. Advantages:
o Detects clusters of arbitrary shapes.
o Handles noise effec vely.
4. Disadvantages:
o Sensi ve to the choice of ε and MinPts.
o Not suitable for datasets with varying densi es.
Gaussian Mixture Model (GMM):
1. Overview:
o Represents clusters as a mixture of Gaussian distribu ons.
o Uses the Expecta on-Maximiza on (EM) algorithm to es mate parameters.
2. Advantages:
o Handles overlapping clusters well.
o Probabilis c model provides so clustering.
3. Disadvantages:
o Requires the number of clusters in advance.
o Computa onally expensive for large datasets.

4. Hierarchical Clustering
BIRCH (Balanced Itera ve Reducing and Clustering using Hierarchies):
1. Overview:
o Incrementally builds a hierarchical clustering tree (CF tree).
o Suitable for large datasets.
2. Advantages:
o Scalable and memory-efficient.
o Automa cally determines the number of clusters.
3. Disadvantages:
o Assumes spherical clusters.
Agglomera ve Hierarchical Clustering:
1. Overview:
o Bo om-up approach where each data point starts as a single cluster.
o Merges clusters itera vely based on similarity.
2. Linkage Methods:
o Single Linkage: Minimum distance between points in clusters.
o Complete Linkage: Maximum distance between points in clusters.
o Average Linkage: Average distance between points in clusters.
3. Advantages:
o Does not require the number of clusters in advance.
4. Disadvantages:
o Computa onally expensive.
o Sensi ve to noise and outliers.
Divisive Hierarchical Clustering:
1. Overview:
o Top-down approach where all points start in one cluster.
o Splits clusters itera vely.
2. Advantages:
o Provides a global perspec ve of the dataset.
3. Disadvantages:
 o Computa onally expensive.

5. Other Clustering Algorithms

Affinity Propaga on:
1. Overview:
o Message-passing algorithm that iden fies exemplars (cluster centers).
o Does not require the number of clusters in advance.
2. Advantages:
o Handles non-spherical clusters.
o Flexible and adaptable.
3. Disadvantages:
o High computa onal cost.
Mean-Shi Clustering:
1. Overview:
o Iden fies dense regions in the data.
o Moves cluster centers itera vely towards the mean of the data.
2. Advantages:
o Does not require the number of clusters.
o Detects arbitrarily shaped clusters.
3. Disadvantages:
o Computa onally expensive for large datasets.
OPTICS (Ordering Points to Iden fy the Clustering Structure):
1. Overview:
o Extension of DBSCAN that handles datasets with varying densi es.
o Produces a reachability plot to iden fy clusters.
2. Advantages:
o Detects clusters of varying densi es.
3. Disadvantages:
o Requires post-processing to extract clusters.

6. Measuring Clustering Goodness

Internal Measures:
1. Silhoue e Score:
o Measures how similar a point is to its own cluster compared to other clusters.
o Ranges from -1 to 1.
2. Davies-Bouldin Index:
o Measures the average similarity ra o of intra-cluster to inter-cluster distances.
o Lower values indicate be er clustering.
3. Dunn Index:
o Ra o of minimum inter-cluster distance to maximum intra-cluster distance.
External Measures:
1. Rand Index:
o Measures agreement between predicted and true cluster labels.
2. Adjusted Rand Index (ARI):
o Adjusts for chance grouping.
3. Mutual Informa on:
o Measures shared informa on between clustering and ground truth.
Real-World Considera ons:
 1. Interpretability:
o Results should be meaningful for the given applica on.
2. Scalability:
o Ensure the method can handle large datasets.
3. Flexibility:
o Adapt to different cluster shapes and sizes.

Conclusion
Clustering is a fundamental unsupervised learning technique that helps uncover hidden pa erns in data. By
understanding and applying various clustering algorithms and evalua ng their results, one can effec vely segment
data and gain valuable insights.