Computer Graphics &

Image Processing
(21CS63)
Image Segmentation

Module - 5
 Image Segmentation:
Introduction
Image segmentation is a fundamental task in computer vision that involves
partitioning an image into multiple segments or regions based on certain
characteristics such as color, intensity, texture, or motion.
The goal of segmentation is to simplify and/or change the representation of an
image into something that is more meaningful and easier to analyze.

What is Image Segmentation: Image

segmentation is the process of dividing an
image into multiple parts or regions. Each
part typically represents objects or areas with
similar characteristics, such as color, texture,
or intensity.
Why is Image Segmentation Important
Image segmentation is a crucial step in various computer vision tasks, including
object detection, image recognition, and medical image analysis. By segmenting an
image, we can extract meaningful information from it, enabling further analysis
and understanding.
Types of Image Segmentation
Semantic Segmentation: This type of region based segmentation
assigns a class label to each pixel in the image. It aims to partition the
image into regions that represent different object categories or classes.
Pixel-wise segmentation: Semantic segmentation operates on the pixel
level, aiming to classify each pixel in the input image into one of several
predefined classes or categories.
Instance Segmentation: Instance segmentation goes a step further than
semantic segmentation by not only labeling each pixel with a class but
also distinguishing between different instances of the same class. For
example, in an image with multiple dogs, instance segmentation would
label each dog separately.
 Challenges in Image
Segmentation
Noise: Images may contain noise or artifacts that can affect segmentation accuracy.

Variability: Objects in images can vary greatly in terms of size, shape, color, and
texture, making segmentation challenging.

Complexity: Some images may contain overlapping or occluded objects, making it

difficult to accurately segment them.
 Applications of Image
Segmentation
Medical Imaging: Segmentation is used
to identify and analyze various anatomical
structures in medical images, aiding in
diagnosis and treatment planning.

Autonomous Driving: In self-driving cars,

segmentation helps in identifying
pedestrians, vehicles, and other objects on
the road.

Satellite Imaging: Image segmentation is

used to analyze satellite images for land
cover classification, urban planning, and
environmental monitoring.

Object Detection and Recognition:

Segmentation is a crucial step in detecting
and recognizing objects in images or videos.
 Image classification

Image classification is a fundamental task in computer vision that involves

categorizing an image into predefined classes or categories. The goal is to
train a machine learning model to recognize and assign a label to an input
image based on its visual content.
Image classification
Techniques
 Image classification - Steps
The major steps of image classification may include image
preprocessing, feature extraction, selection of training samples,
selection of suitable classification approaches, post-classification
processing, and accuracy assessment.
 Detection of
Discontinuities in
Images
1. Detecting discontinuities in images is an essential task in image
processing and computer vision.

2. Discontinuities refer to abrupt changes or transitions in pixel intensity,

color, or texture within an image.

3. These changes often represent important visual cues that can be used
for various purposes such as edge detection, boundary extraction, and
segmentation.
 Detection of Discontinuities in
Images
Edge Detection:
Edge detection is one of the most common methods for detecting discontinuities
in images.

Edges represent areas of significant intensity contrast, where pixel values

change rapidly.

Various edge detection algorithms, such as the Sobel operator, Canny edge
detector, and Prewitt operator, aim to locate these abrupt changes in intensity by
applying convolutional kernels to the image.
 Detection of Discontinuities in
Images
Gradient-based Methods:
Gradient-based methods detect edges by calculating the
gradient of the image intensity function. Discontinuities in
the intensity function result in high gradient values,
indicating the presence of edges.
 Detection of Discontinuities in
Images
Zero Crossing Detection
Zero crossing detection methods identify points in the image where the
second derivative changes sign. These points often correspond to locations
where the intensity transitions from dark to light or vice versa, indicating
potential edges or boundaries.
 Corner Detection
Corners represent points in the image where intensity changes occur in
multiple directions. Corner detection algorithms, such as the Harris
corner detector and the Shi-Tomasi corner detector, identify these points
by analyzing local intensity variations and gradients.

 Edge detection aims to

identify significant intensity
changes or transitions in an
image.

 The corner detection

focuses on identifying specific
points where two or more
edges meet or where there is
a significant change in image
gradient in two directions
 Corner Detection

Key Properties of Corners: Applications:

Corners possess certain key Feature detection and matching in image

properties that distinguish them from registration and object recognition tasks.
other image features:
Camera calibration and pose estimation in
High variation in image intensity in robotics and augmented reality.
multiple directions.
Image stitching and panorama creation by
identifying common features across multiple
The presence of two or more images.
dominant edges meeting at a point.
Motion tracking and optical flow estimation
A significant change in gradient by tracking distinctive points over time in
magnitude and direction in the local video sequences.
neighborhood.
 Edge Detection
Edge detection

Edge detection is a fundamental technique in image processing and

computer vision that aims to identify and localize significant intensity
changes or discontinuities in an image.

Edges typically represent boundaries or transitions between different

regions in an image.
 Edge Detection

Edge detection is used for various purposes, including:

Object detection and recognition

Image segmentation

Image registration

Feature extraction

Image enhancement
 Edge Detection
Algorithms
Used for image segmentation and data extraction in areas such as
image processing, computer vision, and machine vision.
Common edge detection algorithms include Sobel, Canny,
Prewitt, Roberts, and fuzzy logic method
 Sobel Operator

The Sobel operator is a widely used edge detection algorithm that applies
convolutional kernels to the image to approximate the gradient of the image
intensity function.

It computes separate gradients in the horizontal and vertical directions and

combines them to estimate the overall edge strength and orientation.

Sobel operator
 Prewitt Operator
Prewitt Operator: Similar to the Sobel operator, the Prewitt operator
computes gradient approximations using convolutional kernels. It's
particularly effective for detecting edges with a 3x3 convolutional kernel.

Prewitt Operator
 Canny Edge Detector
Canny Edge Detector: The Canny edge detector is a multi-stage algorithm
that is considered one of the most effective edge detection methods.

It involves Gaussian smoothing to reduce noise, gradient calculation to find

edge strength and orientation, non-maximum suppression to thin edges, and
hysteresis thresholding to detect and connect edges.
 Canny Edge Detector
The Canny edge detector operates through multiple stages, each designed to
enhance edge detection and suppress noise.
Step -1: Gaussian Smoothing: The first step of the Canny edge detector
involves applying Gaussian smoothing to the input image. Gaussian
smoothing helps to reduce noise and eliminate high-frequency components
that can interfere with edge detection.

Gradient Calculation: After Gaussian smoothing, the gradient of the image

intensity is calculated using derivative operators (such as Sobel or Prewitt
operators) in both the horizontal and vertical directions. These operators
compute the first-order derivatives of the image with respect to the spatial
coordinates (x and y).

Edge Strength and Orientation: From the gradient images obtained in the
previous step, the magnitude (edge strength) and orientation of the
gradient at each pixel are computed. The magnitude represents the
strength of the edge, while the orientation indicates the direction of the
edge
 Canny Edge Detector
Non-Maximum Suppression: Non-maximum suppression is applied to the
gradient magnitude image to thin out the edges and preserve only the local
maxima (or edges).

For each pixel, non-maximum suppression checks if the pixel's gradient

magnitude is the maximum along the direction of its gradient orientation. If
it is, the pixel is preserved as a potential edge pixel; otherwise, it is
suppressed.

Edge Tracking by Hysteresis: The final stage of the Canny edge detector involves edge
tracking by hysteresis thresholding. This stage aims to link adjacent edge pixels into
continuous edge contours.

It utilizes two threshold values: a high threshold (T_high) and a low threshold (T_low). Pixels
with gradient magnitudes above the high threshold are considered strong edge pixels, while
those between the low and high thresholds are considered weak edge pixels.

Starting from strong edge pixels, the algorithm follows edges in both directions (based on
gradient orientation) and includes weak edge pixels that are connected to strong edge
pixels. This process continues until no more weak edge pixels are added.
 Canny Edge Detector

Edge Localization: After edge tracking, additional processing steps may

be applied to refine and localize the detected edges.

These steps may include edge thinning to ensure single-pixel width edges
and edge tracing to connect discontinuous edge segments.

Canny Edge Detector

Canny Edge Detector Algorithm

 End
Module - 4