CSET344/CMCA544

Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 10th Feb to 14th Feb 2025
[email protected]
1
Canny Edge Detector
Requirements of a good edge detector Change Direction

1. Good Detection: Find all the real edges and ignoring noise or artifacts.

2. Good Localization: Detect edges as close as possible to true edges. NMS

3. Single Response: Return one point only for each true edge point.

Steps in Canny Edge Detector.

Edge
1. Smooth the image to reduce the noise. (Gaussian Smoothening)

2. Gradient or Magnitude (M) Calculation. Angle calculation.

3. Apply Non-Maximum Suppression. (NMS)

Gradient Magnitude
High Threshold
(H)
4. Hysteresis Thresholding.
If M > H: Low Threshold
(L)
Edge
elif L< M<H:
Check is it connected with strong or weak edge
else:
No edge 2
Canny Edge Detector Example

Original Image Thresholded Image Original Image Thresholded Image

Laplacian of Gaussian Canny Edge Detector Laplacian of Gaussian Canny Edge Detector
3
Comparison of all Edge Detector

Laplacian of
Feature Prewitt Sobel Gaussian (LOG) Canny

Gradient Operator Simple difference Weighted difference (more Second derivative (finds zero Optimized gradient &
robust) crossings) sophisticated criteria

Noise Sensitivity High Moderate High (but Gaussian blur helps) Low (due to Gaussian blur)

Edge Localization Less precise More precise Moderate Very precise

Spurious Responses More Fewer More (but zero-crossing Fewer

helps)
Edge Thickness Thicker Thinner Thinner Very thin

Computational Cost Low Slightly higher than Prewitt Higher (due to convolution) Highest

Implementation Simple Relatively simple More complex (Gaussian + More complex (multiple
Laplacian) stages)

Typical Use Cases Simple image analysis, basic General-purpose edge Images with low noise, blob High-quality edge detection,
edge detection detection detection computer vision

4
Harris Corner Detector

Area of Significant changes in

all the directions.

Applications of the Harris Corner Detector

•Image Stitching: Finding corresponding corners in multiple images to align and stitch them together.

•Object Tracking: Tracking objects in videos by identifying and following their corners.

•3D Reconstruction: Using corners as features to reconstruct 3D models from multiple images.
5
Harris Corner Detector

6
Here M is Harris Matrix
Harris Corner Detector

0 0 1 4 9
1 0 5 7 11
1 4 9 12 16
3 8 11 14 16
8 10 15 16 20

Change in X direction
-1 0 1
-1
Change in Y direction 0
1

K is sensitivity factor; small value of K is better to find the corners. 7

Harris Corner Detector

4 7 6 4 8 8
8 8 7 8 6 7
8 6 5 6 6 4

Ix = 4^2+7^2+6^2+8^2+8^2+8^2+7^2+8^2+6^2+5^2 = 403

Iy = 4^2+8^2+8^2+8^2+6^2+7^2+6^2+6^2+4^2 = 381

Ix * Iy = 4*4, 7*8……………………… = 385 If R –ve then edge.

If R small then constant region.
401 385 If R is large then Corner
Harris Matrix (H ) or M =
385 381
Note: If k is small then it will detect the corner
in the better way.

Det (H) – k * (Trace)^2

401 x 381 – (385)^2 - 0.04 x (785)^2 = -19268.24
8
Hough Transforms: Line Detection
Motivation: Edge Linking, Line Detection, Circle Detection and other generalized shape detection

Let's consider these two points (1,2) and (2,3)

How to connect them ?

Edges are present then

how to link them.

So many lines can pass A line contains the so

through a single point. many points.

9
Hough Transforms: Line Detection
Point in Image Space will be a line in Parameter Space.
c
y Line from a non colinear point.

x m
Image Space Parameter Space c = -mx+y
Y = mx+c
m = slope
c = Intercept

Image Space Parameter Space 10

Hough Transforms: Line Detection
c Line Detection Algorithm
0 0 0 0 0
1. Quantize the parameter space.
3 0 0 0 0 0
2 0 0 0 0 0 2. Create an accumulator array A(m,c).
0 0 0 0 0
1 3. Set A(m,c) = 0, for all (m,c).
0 0 0 0 0
m 4. Fill the accumulator array and count
Original Image and 3 colinear Quantize the parameter space the frequency.
points and fill the zeros.

c
1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 1
0 1 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 1 0
0 0 1 0 0 0 0 1 0 0 1 1 1 1 1 1 1 3 1 1
0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 0 1 0 1 0
0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1
11
m
Hough Transforms: Line Detection:
Working Example:

c
Point (1,2) and (2,3) are they colinear ?

Parameter space : c = -mx+y

For (1,2) –

C = -m+2, if c=0 then m =2 1

if m=0 then c =2 output (2,2)

For (2,3) 1 1.5,3 m

2,2
C = -2m+3, if c=0 then m = 1.5
if m=0 then c = 3 output (1.5,3)
Y = mx + c
Y = x+1

12
Hough Transforms: Line Detection: Polar form
Issues:

1. Slope -∞ < m < ∞

2. It directly makes the accumulator array very large.

3.It required more memory and computation cost.

Solution:
Polar form. Ro (rho) is distance from origin and theta is the angle for the same.

13
Hough Transforms: Circle Detection
y b

x a
Image Plane Parameter Plane

For each circle point in the image plane their will be a Accumulator Array filing will be circular
circle in the parameter space. b
0 1 1 1 0
(x-a)^2 + (y-b)^2 = r^2 1 0 0 0 1
1 0 3 0 1
a,b = centre of the circle.
r = radius of the circle. 1 0 0 0 1
1 1 1 1 0
If r is known then parameter space contains 2
parameters a and b. a
14
Problems with Hough Transform

1.Computational Cost:

2.Parameter Sensitivity:

3.Memory Usage

4.Loss of Positional Accuracy: Handling of Complex Scenes:

5.Requirement for Preprocessing

15
Morphological Operations
1. Morphological operations are a set of techniques used in image processing to analyze and modify the shape and
structure of objects within an image.
2. Morphological operations are defined in terms of sets. Specifically in Image Processing, Morphology uses two types of
set of pixels. Structuring element can be specified in terms of both foreground and background pixels.

16
Dilation
Dilation generally grow or increase thickness. Full match = 1
Partial match = 1
No match = 0

For Example

0 0 0 0 0 0 0 0 1 1 0 0
0 0 1 1 0 0 1 0 1 1 1 1 0
0 1 1 1 1 0 1 0 1 1 1 1 0
0 0 1 1 0 0 1 0 1 1 1 1 0
0 0 0 0 0 0 0 0 1 1 0 0

Input Image Structuring Element Output Image

17
Dilation Examples

Application of dilation

1. Filling holes in connected components.

2. Feature Enhancement

18
Erosion
Erosion generally shrink or decrease thickness. Full match = 1
Partial match = 0
No match = 0

For Example

1 1 1 1 1 1 0 0 0 0 0 0
1 1 0 0 1 1 1 1 0 0 0 0 1
1 0 0 0 0 1 1 1 0 0 0 0 1
1 1 0 0 1 1 1 1 0 0 0 0 1
1 1 1 1 1 1 0 0 0 0 0 0

Input Image Structuring Element Output Image

19
Erosion Examples

Application of erosion

1. Noise Removal.
2. Object Separation and feature extraction
20
Opening
Erosion followed by Dilation
0 0 0 0 0 0 0
0 1 0 0 0 1 0
0 0 0 0 0 0 0
For Example
Erosion
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 0 1 1 1
1 1 1 0 1 1 1 1 1 1
1 1 1 0 1 1 1

Input Image Structuring Element 1 1 1 0 1 1 1

Erosion followed by Dilation

Application of erosion

1. Small object elimination.

2. Smoothening Contours.
3. Finding the holes or gaps.
21
Closing
Dilation followed by Erosion.
1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
For Example 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1
1 1 1 0 1 1 1 1 1
Dilation
1 1 1 0 1 1 1 1 1
1 1 1 0 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1
Input Image
Structuring Element 1 1 1 1 1 1

Application of erosion Dilation followed by Erosion

1. Filling small holes and gaps and Noise

reduction.

22
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 17th Feb to 21st Feb 2025
[email protected]
23
Hit or Miss Transform – (HMT)

1. Hit or Miss Transform is a basic tool for shape detection.

2. HMT utilizes two structuring elements.

B1 - for detecting shapes in the foreground.

B2 - for detecting shapes in the background.

24
Hit or Miss Transform – (HMT)

Detecting a shape D

Structuring element
completely fits on
several locations.

Background with one pixel

thick foreground. Structuring element fits on
one location.

25
Hit or Miss Transform – (HMT)
Q. Instead of going for two structuring elements. Can we detect the same shape by using a single structuring element ?

Solution. Use a structuring element B which is exactly same shape as D with additional border of background elements
with a width of one pixel thick.

26
Hit or Miss Transform – (HMT)
One Structuring Element

Detection of single pixel thick hole.

Detection of an upper right corner.

Detection of Multiple Features.

In this case, X is nothing but don’t care.

It may take value 0 or 1. Here don’t care
value 1 is considered.
27
Basic Morphological Algorithms
Boundary Extraction – First Erode the original image with a Structuring element and then perform a set
difference between A and its eroded output.

Pixel Value 1 Pixel Value 0 28

Basic Morphological Algorithms
Hole Filling – A hole is defined as background region surrounded by a connected border of foreground pixels. An
algorithm is developed based on dilation, complementation and Intersection.

29
Basic Morphological Algorithms
X0 X1 X2 X3

1 1 1 1 1 1 1
1 1 1 1 1
1

1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1 1
1 1 1 1

X6 X7 X8 X8 U A 30
Basic Morphological Algorithms
Connected Components – It generally used to extract all the connected components in original image A.

Here Bones are the

foreign objects
inside chicken
Comparisons of all Morphological Operations.
Morphological Applications
operations
Dilation Medical Imaging – Enhance the Visibility of Blood Vessels.
Industrial Inspection – Detect Defects or Irregularity in goods.

Erosion Shrinking - Make the objects much thinner or highlight the core part of object.
Remove Noise – Eliminates noise along the edge of the objects.

Opening Image Cleaning/ Noise Removal

Object Isolation

Closing Analyzing Satellite Imagery

Scientists use satellite images to study changes in land cover, such as deforestation or urban
growth. Sometimes, clouds or shadows can obscure parts of the image, creating gaps in the
data.

Hit or Miss Object Localization.

Texture analysis
Texture is a repeating pattern of local variation in image intensity.

Texture Analysis
Statistical Approaches or Spectral Approaches or
Techniques Techniques

It is generally based on the property of

Fourier spectrum.

(a) Smooth (b) Coarse (c) Regular

Characterization of texture as smooth, coarse and grainy.
Texture From Histogram
Grey Level Co-occurrence Matrix (GLCM)
Problems with texture from histogram –
Measure of texture computed using histogram-based method carries no information regarding to spatial relationship between
pixels.

1. Total Levels present in the input image L = 8. So

the Size of GLCM is 8 by 8.

2. Look for one pixel immediately right.

3. Check the co-occurrence of each pair and fill the

GLCM Matrix.

4. Calculate different statistical descriptor based on

the requirement's.
1 1
5 1
Grey Level Co-occurrence Matrix (GLCM)

mr and mc are mean computed along row and

column.
GLCM Example

Construct a GLCM matrix for the below

matrix

Look for one pixel immediately right.

1 7 2 1
2 3 4 5
7 2 1 7
8 2 1 7
GLCM Example
Construct a GLCM matrix for the below matrix

Look for one pixel immediately right.

Total 8 intensity Levels – Size of GLCM is 8 by 8.
1 7 2 1
2 3 4 5
7 2 1 7 1 2 3 4 5 6 7 8
8 2 1 7 1 3
2 3 1
3 1
4 1
5
6
7 2
8 1
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 17th Feb to 21st Feb 2025
[email protected]
39
Color Image Processing
Color Image Processing

Pseudo-Color Processing Full – Color Processing

Assigning Colors to grey scale intensity Image Acquired using a full color sensor like
or range of intensities. digital camera or color scanner.
Color Fundamentals

1. Basically the color human and other animal perceive in an object is determined by the nature of the light
reflected from the target.

2. Chromatic Light spans the electromagnetic spectrum from approximately 400nm to 700nm.

Three basic quantities used to describe the quality of chromatic light source are :
Radiance.
Luminance.
Brightness.
Color Fundamentals
Radiance: Total amount of energy flows from a light source. It is measured in Watt.

Luminance: Total amount of energy that an observer perceive from the light source. It is measured
in lumens (lm). For example infrared.

Brightness: it’s a subjective descriptor and difficult to measure.

Cones are the sensors in the human eye responsible for color vision.

Human eye is divided into three principle sensing categories.

65% - of all cones are sensitive to RED.

33% - of all cones are sensitive to GREEN.

2% - of cones are sensitive to BLUE.

Color Fundamentals

Because of these absorption characteristics,

human eye sees colors as variable combinations of
so called primary colors: Red, Green and Blue.

Absorption of Light By red green and Blue cones in the human

eye as a function of wavelength.
Color Fundamentals
Hue: Dominant color perceived by an observer.

Primary and secondary color of light and pigments.

Color Fundamentals
Saturation: it refers to the purity of color.

Example :
High Saturation: Lets say, if you are using pure red paint then its fully
saturated.

Low Saturation: If you are gradually mixing white paint inside the red then it
changed from pure red to faded red or pink.

Degree of saturation inversely proportional to the amount of white light

added.

Brightness : it always refer how light and dark color will appear.

High Brightness: if blue is very light almost white.

Low Brightness: If blue is very dark almost black.

Color Fundamentals
Chromaticity: Hue and saturation are taken together is called chromaticity.

Any color is characterized by its brightness and chromaticity.

Tristimulus Values – The amount of red green and blue required to form any particular color is known as tristimulus values
and it is denoted by X, Y, Z.

x,y,z are trichromatic coefficient.

Color Models
Requirement of color Models: To facilitate the specification of color in some standard way.

1. It should follow some coordinate system.

2. There should be a subspace within system, such that each color in the model is represented by a single point contained
in it.
Color Model Application Domain(s) Key Characteristics
Computer displays, digital cameras, Additive color model (combining light);
RGB image scanners device-dependent
Older printing technologies (less Subtractive color model (absorbing
CMY common now) light); device-dependent
Subtractive color model; handles black
CMYK Color printing (magazines, books, etc.) ink; device-dependent
Image editing, color analysis, computer Hue, Saturation, Intensity; more
HSI vision intuitive for color adjustments

Image editing, color picking, computer Hue, Saturation, Value; similar to HSI,
HSV graphics slightly different brightness
representation

Digital video (DVDs, Blu-ray, digital TV), Separates luminance (brightness) from
YCbCr JPEG compression chrominance (color) for efficient
compression
Color Models: RGB
Color Models RGB 1. RGB model based on cartesian coordinate system.

2. Primary colors R,G, and B are at the three corners.

3. Secondary colors like Cyan, Magenta and yellow at the

other corners.

4. Black at the origin and white at the corner which is

farthest away from the origin.

5. Assumption is that all R,G and B values are normalized

between 0 to 1.

6. Pixel Depth – No. of bits used to represent each pixel in

RGB space. Each RGB color pixel has a depth of 24bits.

7. let's say it’s an 8-bit image, then the limits of cube

along each axis becomes [0,255].

for example : White will be at a point [255,255,255]

RGB Color Cube
Color Models: HSV
Angle (in degree) Color
0-60 Red
61-120 Yellow
121-180 Green
181-240 Cyan
241-300 Blue
301-360 Magenta

This Color Model is closer to how human perceive the color.

1. Hue: It is a color portion of the model, expressed as a

number between 0 to 360 degree.

2. Saturation ranges between 0 to 1.

0 represent No color.
1 represent the primary color.

3. It represent the brightness 0 means completely black 100

is the brightest.
Color Models: YcbCr
It is most widely used in digital video and photography
systems, including DVD, Blue-ray disc, digital TV and JPEG
images.

Y (Luma) - It represent the brightness or luminance of the

image.

Cb (Blue difference chroma) – It represent the difference

between blue color and the luma.

Cr (Red difference chroma) – it is difference between red color

and chroma.

YCbCr – it allows better compression of image and video data.

 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 24th Feb to 28st Feb 2025
[email protected]
51
Pseudo Color Image Processing Pseudo color or false color image processing

Objective: Assigning colors to grey scale values based on a specified criteria.

Motivation: Human can differentiate between different colors and intensities as compared to different shades of grey.

Grey Scale Image

Color Image
Pseudo Image Processing
Intensity Slicing and Color Coding

1. Any pixel whose intensity level above the plane will be

coded in one color and any pixel below the plane will
be coded with another color.

2. Intensity levels those are lies on the plane assigning

either of the color or the third color.
Pseudo Image Processing
Intensity Slicing and Color Coding

Grey Scale Image Intensity Slicing using 8 Color coded image of the weld,
colors X-ray image of the weld where one color assign to 255
intensity levels and another color
to all other intensity levels.
Regions that appear of constant intensity in grey
scale is quite variable in the color sliced image.
It will make the quality inspector job easy, as a result
low error rate.
Pseudo Image Processing
Intensity to color transformations : it is better than the simple slicing techniques.

Apply three independent transformations on the intensity

of input pixels.

These three results are then fed separately to red, green

and blue channels of color monitors.

The output will be a composite image whose color content

is modulated by the nature of the transformation function.
Pseudo Image Processing
Bag without explosive or with explosive.
Previous transformation applied on single grey scale
image.

The below approach is combining several grey scale

image into single color composite.
Example – Multispectral image processing

Applied different Applied Similar

transformation transformation
function function
 Pseudo Image Processing
Example – Multispectral image processing

Red Green Blue

NIR of Landsat RGB Color Composite using IR,G,B RGB Color composite using R,IR,B
Basics of Full Color Image Processing
Full Color Image Processing

Process each Grey scale color Process or work with the color pixel
image components individually directly.
and then form a composite color
image
Color Transformations
Tone Correction

Image Negative

Color Complements