Image Filtering

Digital Image
An image may be defined as a two-dimensional function, f ( x, y), where x and y are spatial (plane)
coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of the
image at that point.
When x, y, and the intensity values of f are all finite, discrete quantities, we call the image a digital image.
 Image filtering
• Producing a new image where the value at a pixel in the output image is a
function of a neighborhood of the pixel location in the input image.
 Image Filtering Vs. Warping
• Image filtering changes the range (i.e. the pixel values) of an image, so
the colors of the image are altered without changing the pixel
positions.
• image warping changes the domain (i.e. the pixel positions) of an
image, where points are mapped to other points without changing
the colors.
• The goal of using filters is to modify or enhance image properties and/or
to extract valuable information from the pictures such as edges, corners,
and blobs.
 Linear Filters
• Form new image whose pixels are a weighted sum of original pixel values, using
the same set of weights at each point.
• Example: smoothing by averaging :form the average of pixels in a neighborhood.
• Smoothing with a Gaussian– form a weighted average of pixels in a neighborhood.
• Finding a derivative– form a difference of pixels in a neighborhood

Gaussian filtered Image

Gaussian Kernel
 Linear Filter Properties
Output is a linear function of the input.
• Additivity:The sum of the filtered versions of two images is equal to the filtered
version of the sum of the two images.
• Homogeneity:The filtered version of a constant multiplied by an image is equal
to the constant multiplied by the filtered version of the image.
• Shift invariance:Output is a shift-invariant function of the input (i.e. shift the
input image two pixels to the left, the output is shifted two pixels to the left)
• Linearity: The filtered version of a constant multiplied by an image is equal to
the constant multiplied by the filtered image
Use Of Linear Filters:
• Linear filters are used to improve images by: Sharpening edges, Reducing noise,
Correcting for uneven illumination, and Deconvoluting blur and motion.
 Filtering Application Noise Filtering
• Image processing is useful for noise reduction.
Common types of noise:
• Salt and pepper noise: contains random occurrences of black and white pixels.
• Impulse noise: contains random occurrences of white pixels .
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Mean filtering(Low Pass)
 Mean filtering
• Mean filtering is a widely employed technique within the realm of image processing,
serving as a crucial tool for noise reduction and smoothing. By adopting spatial filtering
principles, mean filtering aids in eliminating random variations or noise present in an
image, all while retaining its essential features. In this context, we will explore the
mechanics, advantages, and limitations of mean filtering
• The fundamental concept behind mean filtering is elegantly simple: each pixel in an
image is substituted with the average value derived from neighboring pixels confined
within a specified window or kernel. The size of this kernel determines the degree of
smoothing, with larger kernels inducing more potent smoothing effects at the potential
cost of finer detail loss.
• In essence, mean filtering involves the replacement of each pixel’s value with the
average value of its neighboring pixels, including the pixel itself. This process proves
effective in removing pixel values that diverge significantly from their surroundings.
Conceptually aligned with a convolution filter, mean filtering revolves around a kernel
that defines the shape and size of the neighborhood encompassed for mean calculation.
While a standard 3×3 square kernel is frequently employed, the utilization of larger
kernels like 5×5 squares can intensify the smoothing for more substantial noise
reduction
 Mean filtering
• Several iterations of mean smoothing exist, among which is Threshold Averaging. Here, smoothing is
contingent upon a condition: the central pixel value undergoes alteration only if the difference between its
initial value and the computed average surpasses a preset threshold. This approach efficiently reduces noise
while preserving more image detail than traditional mean filtering.
• In the realm of computational simplicity, mean filtering stands as an exemplar. Its straightforward
implementation and efficiency render it an easily accessible tool. However, mean filtering’s suitability is not
universal; images containing sharp edges or intricate details may experience edge blurring due to the
averaging procedure. Notably effective against Gaussian or salt-and-pepper noise, mean filtering harnesses
the context of surrounding pixels to mitigate random noise values.
• While mean filtering holds its place in preprocessing images for subsequent intricate tasks like edge
detection or object recognition, its significance is underscored in image restoration and enhancement
processes. Notably, in domains like medical imaging, mean filtering’s application contributes to enhanced
diagnostic precision and improved visual quality.
• In summary, mean filtering emerges as an uncomplicated yet potent technique in image processing. Its
prowess in noise reduction and texture smoothing bestows it with versatility across diverse applications.
Nevertheless, judicious selection of kernel size remains pivotal to striking a balance between noise
reduction and vital image feature preservation. When judiciously employed, mean filtering profoundly
elevates image quality and facilitates comprehensive image analysis. Despite its computational efficiency, it
is imperative to acknowledge its limitations; alternative convolution filters, such as the Gaussian smoothing
filter, present distinct trade-offs between noise reduction and detail preservation
Effect of mean filters
Mean Filters Kernels
 Generalization: Cross-Correlation
•Filtering
Let’s write this down as an equation. Assume the averaging window is
(2k+1)x(2k+1):

• We can generalize this idea by allowing different weights for different

neighboring pixels:

This is called a cross-correlation operation and written:

H is called the “filter,” “kernel,” or “mask.”

 Gaussian Filter
• Gaussian filtering is a critical tool in the field of image processing, especially for noise
reduction. By applying a Gaussian kernel, the filter gives central pixels more weight than
surrounding regions, effectively reducing noise while preserving image structure. The
essential parameter σ controls the filter’s scope and subsequent smoothing. Gaussian
filters are excellent at removing random, subtle image noise patterns, making them vital in
many image processing applications.
• Among the many linear smoothing filters used for noise reduction, the Gaussian filter
stands out as a crucial tool. It uses weights derived from the Gaussian distribution to
effectively reduce noise in images. However, before using the Gaussian filter, it is important
to preprocess the image and remove any outliers or spikes. While it is adept at handling
random noise, the Gaussian filter has its limitations. It tends to blend noise into the result,
causing indiscriminate smoothing along edges.
• In summary, the Gaussian filter is a popular linear filter for reducing image noise in image
processing. Its ability to assign accurate weights to pixels based on the Gaussian
distribution enables it to reduce random, subtle noise patterns while preserving the
underlying structure of the image. However, it is important to recognize its limitations and
take preliminary steps when dealing with specific types of noise and preserving edges.
When combined with other image filtering techniques, the Gaussian filter becomes a
valuable tool, improving image quality and producing visually pleasing results.
Gaussian Filter
Convolution
 Convolution
A convolution operation is a cross-correlation where the filter is flipped both horizontally and vertically before being
applied to the image.
Convolution vs. Correlation When do
they differ? 1D Example from Signal
Processing
 Median filters(Non linear Filter)
• A Median Filter operates over a window by selecting the median
intensity in the window

Is a median filter a kind of convolution?

Comparison: salt and pepper noise
Comparison: Gaussian noise
 Edge Detection
• Basic idea: look for a neighborhood with strong signs of change.

how to detect change?

 Differential Operators
• Attempt to approximate the gradient at a pixel via masks.
• Threshold the gradient to select the edge pixels.
What’s a gradient?
Def: the gradient of a scalar function f(x1, x2, ... xn) is denoted by ∇f (del f)
and is defined by: ∇f = (∂f/∂x1, ∂f/∂x2, . . . ∂f/∂xn)
What’s a derivative?
 Edge Detection
• In Image processing /computer vision, edges are sudden discontinuities in an image, which
can arise from surface normal, surface color, depth, illumination, or other discontinuities.
Edges are important for two main reasons.
1) Most semantic and shape information can be deduced from them, so we can
perform object recognition and analyze perspectives and geometry of an image.
2) They are a more compact representation than pixels.
• We can pinpoint where edges occur from an image's intensity profile along a row or column
of the image. Wherever there is a rapid change in the intensity function indicates an edge, as
seen where the function's first derivative has a local extrema.
 Edge Detection
• An image gradient, which is a generalization of the concept of derivative to
more than one dimension, points in the direction where intensity increases the
most. If the gradient is [δf⁄δx,δf⁄δy] or

• then the gradient direction would be θ = tan-1(δf⁄δy/δf⁄δx) or

• and the edge strength would be the gradient magnitude:

• However, plotting the pixel intensities often results in noise, making it
impossible to identify where an edge is by only taking the first derivative of
the function.
• If we apply a filter that is a derivative of a Gaussian function, we can
eliminate the image noise and effectively locate edges.
Building off of this procedure, we can design an edge detector. The optimal edge
detector must be accurate, minimizing the number of false positives and false
negatives; have precise localization, pinpointing edges at the positions where they
actually occur; and have single response, ensuring that only one edge is found
where there only is one edge.
 Apply Non-Maximum
• Suppression (NMS)
By filtering with Gaussian, we obtain image gradients, reducing the
influence of noise within image. However, this introduces a drawback
that the edges in image are no longer single-pixel, but become a thick
region due to the smoothing effect.
 Apply Non-Maximum
Suppression (NMS)
• According to the criterion: Single response, it needs to be fixed to output a
single-pixel curve (or line segment), not a region (multiple-pixels).
• In order to do so, a technique, which is called Non-Maximum Suppression, is
applied. It is to keep only the local maxima pixel and to discard non-maxima
pixels in an edge region, resulting thinner, and ideally single-pixel, edges
• The process for NMS is as follows:
• Given the current pixel, find two neighboring pixels in the same direction as the
computed orientation of current pixel.
• Compare the magnitudes from three (current and two neighboring pixels)
• If the magnitude of current pixel is the maximum, keep its magnitude. Otherwise
discard the pixel from the edge detection process.
• Repeat the above stages for each pixel
Apply Non-Maximum
Suppression (NMS)
 Hysteresis Thresholding

• As edges can be seen as boundaries between objects within image, there is often a
sharp transition in pixel intensities neighboring an edge, and therefore such pixels
have high magnitude value. Similarly, we can interpret this observation as a pixel
with high magnitude is likely to be an edge that we want to detect.
• Therefore, many detectors are built with thresholding in order to remove non-
probable pixels. It applies to Canny edge detector as well. One difference is that while
others use a single threshold, Canny edge detector use two thresholds: high and low,
and this is called Hysteresis thresholding.
• Before getting into Hysteresis thresholding, Let’s take a look what could happen when
single-value thresholding. Think of a case when a certain threshold theta is given. Any
edge whose magnitude (|grad|) is lower than theta will be discarded, and this
indicates the lost of edge in image.
• Here is an example. If the chosen theta is higher
than the lowest magnitude of true edges, we
loose them. In the following image, we lost
some edges in woman’s chin due to the
thresholding.
• In order to reduce such lost, Hysteresis
thresholding uses a high and low thresholds.
• The main idea of Hysteresis thresholding is
• Use the high threshold to find “strong edges”, which
is sure to be considered as an edge, to start edge
chain.
• Use the low threshold to find “weak edges”, which
connect empty segment between “strong edges”.
 Gradient-Based Methods of Edge detection

• Prewitt Operator:
• Wherever there is a sudden change in pixel intensities, an edge is detected by the
mask. Since the edge is defined as the change in pixel intensities, it can be calculated
by using differentiation. Prewitt mask is a first-order derivative mask. In the graph
representation of Prewitt-mask’s result, the edge is represented by the local maxima or
local minima.
• Prewitt operator detects both types of edges, these are:
Horizontal edges or along the x-axis
Vertical Edges or along the y-axis.
Wherever there is a sudden change in pixel intensities, an edge is detected by
the mask. Since the edge is defined as the change in pixel intensities, it can be
calculated by using differentiation. Prewitt mask is a first-order derivative mask.
In the graph representation of Prewitt-mask’s result, the edge is represented by
the local maxima or local minim.
• More weight means more edge detection.
• The opposite sign should be present in the mask. (+ and -)
• The Sum of the mask values must be equal to zero.
 Gradient-Based Methods of Edge detection

• Sobel Operator:
• The Sobel operator is a discrete differentiation operator that computes an approximation of the
gradient of the image intensity function. It uses convolutional masks to highlight regions with high
spatial frequency, which correspond to edges.

As the center column is of zero so it does not Above mask will find edges in
include the original values of an image but horizontal direction and it is
rather it calculates the difference of right because that zeros column is in
and left pixel values around that edge. Also horizontal direction. When you will
the center values of both the first and third convolve this mask onto an image
column is 2 and -2 respectively. it would prominent horizontal Vertical Mask of
This give more weight age to the pixel values edges in the image. The only Sobel Operator
around the edge region. This increase the difference between it is that it
edge intensity and it become enhanced have 2 and -2 as a center element Horizontal Mask
comparatively to the original image. of first and third row of Sobel Operator
 Sample Image After Applying Vertical Mask After Applying Horizontal Mask

As you can see that in the first picture on which we apply vertical mask, all the vertical edges are
more visible than the original image. Similarly in the second picture we have applied the horizontal
mask and in result all the horizontal edges are visible.
 Roberts Cross Edge Detector

The Roberts Cross operator performs a simple, quick to compute, 2-D spatial gradient measurement on an image
It thus highlights regions of high spatial frequency which often correspond to edges
The input to the operator is a grayscale image, as is the output. Pixel values at each point in the output represent the
estimated absolute magnitude of the spatial gradient of the input image at that point
How It Works

The operator consists of a pair of 2×2 convolution kernels as shown in Figure 1. One kernel is simply the other rotated
by 90°

Roberts Cross convolution kernels

 Roberts Cross Edge Detector
These kernels are designed to respond maximally to edges running at 45° to the pixel grid, one kernel for each of the two
perpendicular orientations. The kernels can be applied separately to the input image, to produce separate measurements of the
gradient component in each orientation (call these Gx and Gy). These can then be combined together to find the absolute
magnitude of the gradient at each point and the orientation of that gradient. The gradient magnitude is given by

Although typically, an approximate magnitude is computed using:

which is much faster to compute.
The angle of orientation of the edge giving rise to the spatial gradient (relative to the pixel grid orientation) is given by
Angle of 0° corresponds to a vertical orientation such that
the direction of maximum contrast from black to white
runs from left to right on the image

Using this kernel the approximate

magnitude is given by:

Using this kernel the approximate magnitude is given by:

 Roberts Cross Edge Detector
Use:
The main reason for using the Roberts Cross operator is that it is very quick to compute. Only four input pixels need to be
examined to determine the value of each output pixel, and only subtractions and additions are used in the calculation. In
addition there are no parameters to set. Its main disadvantages are that since it uses such a small kernel, it is very sensitive to
noise. It also produces very weak responses to genuine edges unless they are very sharp
 Laplacian Operator (Second derivative for sharpening)
• The Laplacian is a 2-D isotropic measure of the 2nd spatial derivative of an image.
• The Laplacian L(x,y) of an image with pixel intensity values I(x,y) is given by:

Since the input image is represented as a set of discrete pixels, we have to find a discrete convolution
kernel that can approximate the second derivatives in the definition of the Laplacian.

Using one of these kernels, the Laplacian can be calculated using standard convolution methods.
 Laplacian Operator

Laplacian kernel used to implement in equation. (b) Kernel used to implement an extension of this equation that includes the
diagonal terms. (c) and (d) Two other Laplacian kernels
 Laplacian of Gaussian (LOG)
• Because Laplacian kernels are approximating a second derivative measurement
on the image, they are very sensitive to noise.
• To overcome this image is often applied Gaussian smoothing before applying
Laplacian filter.
• This pre-processing step reduces the high frequency noise components prior to
the differentiation step.
• Since the convolution operation is associative, we can convolve the Gaussian
smoothing filter with the Laplacian filter first of all.
• Since both the Gaussian and the Laplacian kernels are usually much smaller than
the image, this method usually requires far fewer arithmetic operations.
• The LoG (`Laplacian of Gaussian') kernel can be precalculated in advance so only
one convolution needs to be performed at run-time on the image.
 Laplacian of Gaussian (LOG)
The 2-D LoG function centered on zero and with Gaussian standard deviation has the form

The 2-D Laplacian of Gaussian (LoG) function. The x and y axes are marked in standard deviations ( σ).
 Laplacian of Gaussian (LOG)
• The LoG operator calculates the second spatial derivative of an image. This
means that in areas where the image has a constant intensity (i.e. where
the intensity gradient is zero), the LoG response will be zero. In the vicinity
of a change in intensity, however, the LoG response will be positive on the
darker side, and negative on the lighter side. This means that at a
reasonably sharp edge between two regions of uniform but different
intensities, the LoG response will be:
• zero at a long distance from the edge,
• positive just to one side of the edge,
• negative just to the other side of the edge,
• zero at some point in between, on the edge itself.
 Laplacian of Gaussian (LOG)

Response of 1-D LoG filter to a step edge. The left hand graph shows a 1-D image, 200 pixels long, containing a
step edge. The right hand graph shows the response of a 1-D LoG filter with Gaussian σ= 3 pixels.
Laplacian of Gaussian (LOG)
Laplacian of Gaussian (LOG)
 Canny Edge Detector
• Cranny’s approach is based on three basic objectives:
• 1. Low error rate. All edges should be found, and there should be
no spurious responses.
• 2. Edge points should be well localized. The edges located must be
as close as possible to the true edges. That is, the distance between
a point marked as an edge by the detector and the center of the
true edge should be minimum.
• 3. Single edge point response. The detector should return only one
point for each true edge point. That is, the number of local maxima
around the true edge should be minimum. This means that the
detector should not identify multiple edge pixels where only a
single edge point exists.
 Canny Edge Detector
• Let f (x,y) denote the input image and G (x, y) denote the Gaussian
function

We form a smoothed image, fs(x, y )by convolving f and G:

This operation is followed by computing the gradient magnitude and direction (angle),
Canny Edge Detector
 Canny Edge Detector

When repeated for all values of x and y, this procedure yields a nonmaxima suppressed image g N (x,y) that is of the same
size as fs(x, y)
The final operation is to threshold g N (x,y) to reduce false edge points, in which all values below the threshold were set to 0

If we set the threshold too low, there will still be some false edges (called false positives). If the threshold is set too high,
then valid edge points will be eliminated (false negatives).

Canny’s algorithm attempts to improve on this situation by using hysteresis thresholding which a low threshold,
TL and a high threshold, TH . Experimental evidence (Canny [1986]) suggests that the ratio of the high to low
threshold should be in the range of 2:1 to 3:1.
 Canny Edge Detector
• We can visualize the thresholding operation as creating two additional
images:
Canny Edge Detector
 Harris Corner Detector
• The Harris corner detector is a fundamental algorithm in computer vision
used to extract corners from images.
• It identifies "corners" in an image, which are points where the image
intensity changes significantly in two or more directions which are valuable
features for various computer vision task.
• The Harris corner detector essentially analyzes how the image intensity
changes when a small window is shifted in different directions.
• It was first introduced by Chris Harris and Mike Stephens in 1988 upon the
improvement of Moravec’s corner detector. Compared to the previous one,
Harris’ corner detector takes the differential of the corner score into account
with reference to direction directly, instead of using shifting patches for
every 45-degree angles, and has been proved to be more accurate in
distinguishing between edges and corners
 Corner

• A corner is a point whose local neighborhood stands in two dominant and

different edge directions i.e junction of two edges.
• Corners are the important features in the image, and they are generally
termed as interest points which are invariant to translation, rotation, and
illumination.
 Corner

In the above figure, if we take the flat region then no gradient change is observed in any
direction. Similarly, in the edge region, no gradient change is observed along the edge direction.
So both flat region and edge region are bad for patch matching since they not very distinctive
(there are many similar patches in along edge in edge region). While in corner region we observe
a significant gradient change in all direction. Due this corners are considered good for patch
matching(shifting the window in any direction yield a large change in appearance) and generally
more stable over the change of viewpoint
visualization of gradients
Compute image gradients over a small region
Distribution reveals edge orientation and
magnitude
How do you quantify orientation and
magnitude?
Subtract the mean from each image gradient
 Finding corners

• Compute image gradients over small region.

• Subtract mean from each image gradient.
• Compute the covariance matrix.
• Compute the covariance matrix.
• Compute eigenvectors and eigenvalues.
• Use threshold on eigenvalues to detect corners
 Finding corners

• consider a small window around each pixel p in an image. We want to

identify all such pixel windows that are unique. Uniqueness can be
measured by shifting each window by a small amount in a given direction
and measuring the amount of change that occurs in the pixel values.
• we take the sum squared difference (SSD) of the pixel values before and
after the shift and identifying pixel windows where the SSD is large for
shifts in all 8 directions. Let us define the change function E(u,v) as
the sum of all the sum squared differences (SSD), where u,v are the x,y
coordinates of every pixel in our 3 x 3 window and I is the intensity value of
the pixel. The features in the image are all pixels that have large values of
E(u,v), as defined by some threshold.
Error function
 Error function
• We have to maximize this function E(u,v) for corner detection. That means, we
have to maximize the second term. Applying Taylor Expansion to the above
equation and using some mathematical steps, we get the final equation as

Now, we rename the summed-matrix, and put it to be M:

So the equation now becomes:

 Eigenvalues
• we want the SSD to be large in shifts for all eight directions, or conversely,
for the SSD to be small for none of the directions. By solving for the
eigenvectors of M, we can obtain the directions for both the largest and
smallest increases in SSD. The corresponding eigenvalues give us the actual
value amount of these increases. A score, R, is calculated for each window:

λ1 and λ2 are the eigenvalues of M. So the values of these eigenvalues decide whether a region is a
corner, edge or flat.
 Error surface indicates a good image feature
• When |R| is small, which happens when λ1 and λ2 are small, the
region is flat.
• When R<0, which happens when λ1>>λ2 or vice versa, the region is an
edge.
• When R is large, which happens when λ1 and λ2 are large and λ1∼λ2,
the region is a corner.
 Eigen Value
• Compute eigenvalues and eigenvectors

Compute the determinant of

Find the roots of polynomial

For each eigenvalue, solve

 Visualization as an ellipse
• M is symmetric, we have

We can visualize M as an ellipse with axis lengths determined by the eigenvalues and orientation determined by R
Interpreting eigenvalues
 Use threshold on eigenvalues to detect corners

Use the smallest eigenvalue as the response function

Eigenvalues need to be bigger than one

Harris Corner Detector
Harris Corner Detector
Corner response
Thresholded corner response
Non-maximal suppression
 Harris corner response is rotation
invariant

Ellipse rotates but its shape (eigenvalues) remains the same Corner response R is invariant to image rotation