Feature Extraction

and Analysis
Content
•Introduction
•Feature Extraction
• Shape Features - Binary Object Features
• Histogram Features
• Color Features
• Spectral Features

•Feature Analysis
• Feature Vectors and Feature Spaces
• Distance and Similarity Measures
 IMAGE FEATURE EXTRACTION

IMAGE

FEATURE TO
REPRESENTATION
OBJECT

CLASSIFICATION
FEATURE DESCRIPTION

OBJECT CLASSES OBJECT CLASSIFICATION

 FEATURE

NATURE OF NATURE OF DOMAIN INFORMATION

FEATURE OBJECT

PRESERVING
GENERAL APPLICATION EDGE BASED REGION BASED
FEATURE SPECIFIC FEATURES SPATIAL
NON
PRESERVING
PIXEL STRUCTURAL STRUCTURAL
FREQUENCY

LOCAL GLOBAL GLOBAL

GLOBAL

FEATURE CLASSIFICATION
 TYTYPES OF FEATURE EXTEXTRACTION
TYPES OF FEATURE EXTRACTION
PES OF FEATURE EXTRACTION

SPATIAL FEATURES TRANSFORM FEATURE

TEXTURE FEATURE EDGE & BOUNDARIES

SHAPE FEATURE
Introduction
The goal in image analysis is to extract useful information for solving
application-based problems.
The first step to this is to reduce the amount of image data using
methods that we have discussed before:
◦ Image segmentation
◦ Filtering in frequency domain
Introduction
The next step would be to extract features that are useful in solving
computer imaging problems.
What features to be extracted are application dependent.
After the features have been extracted, then analysis can be done.
Shape Features
Depend on a silhouette (outline) of an image
All that is needed is a binary image
Binary Object Features
In order to extract object features, we need an image that has
undergone image segmentation and any necessary morphological
filtering.
This will provide us with a clearly defined object which can be labeled
and processed independently.
Binary Object Features
After all the binary objects in the image are labeled, we can treat each
object as a binary image.
◦ The labeled object has a value of ‘1’ and everything else is ‘0’.

The labeling process goes as follows:

◦ Define the desired connectivity.
◦ Scan the image and label connected objects with the same symbol.
Binary Object Features
After we have labeled the objects, we have an image filled with object
numbers.
This image is used to extract the features of interest.
Among the binary object features include area, center of area, axis of
least second moment, perimeter, Euler number, projections, thinness
ration and aspect ratio.
Binary Object Features
In order to extract those features for individual object, we need to
create separate binary image for each of them.
We can achieve this by assigning 1 to pixels with the specified label and
0 elsewhere.
◦ If after the labeling process we end up with 3 different labels, then we need
to create 3 separate binary images for each object.
Binary Object Features – Area
The area of the ith object is defined as follows:

The area Ai is measured in pixels and indicates the relative size of the
object.
height1 width1
Ai    I ( r, c)
r 0 c 0
i
Binary Object Features – Center of Area
The center of area is defined as follows:
height1 width1
1
ri 
Ai
  rI ( r, c)
r 0 c 0
i

height1 width1
1
ci 
Ai
  cI ( r, c)
r 0 c 0
i

These correspond to the row and column coordinate of the center of

the ith object.
 Binary Object Features – Axis of Least Second
Moment
The Axis of Least Second Moment is expressed as  - the angle of the
axis relatives to the vertical axis.

 height1 width1

 2   ( r  r )(c  c ) I i ( r, c) 
1 1  
 i  tan height1 width1 r 0 c0
2  height1 width1 
   ( r  r ) I i ( r, c )    (c  c ) I i ( r, c) 
2 2

 r 0 c 0 r 0 c 0 
Binary Object Features – Axis of Least Second
Moment

This assumes that the origin is

as the center of area.
This feature provides
information about the object’s
orientation.
This axis corresponds to the line
about which it takes the least
amount of energy to spin an
object.
Binary Object Features -
Perimeter
The perimeter is defined as the total pixels that constitutes the edge of
the object.
Perimeter can help us to locate the object in space and provide
information about the shape of the object.
Perimeters can be found by counting the number of ‘1’ pixels that have
‘0’ pixels as neighbors.
Binary Object Features -
Perimeter
Perimeter can also be found by applying an edge detector to the object,
followed by counting the ‘1’ pixels.
The two methods above only give an estimate of the actual perimeter.
An improved estimate can be found by multiplying the results from
either of the two methods by π/4.
Binary Object Features – Thinness Ratio
The thinness ratio, T, can be calculated from perimeter and area.
The equation for thinness ratio is defined as follows:

 Ai 
Ti  4  2 
 i 
P
Binary Object Features – Thinness Ratio
The thinness ratio is used as a measure of roundness.
◦ It has a maximum value of 1, which corresponds to a circle.
◦ As the object becomes thinner and thinner, the perimeter becomes larger
relative to the area and the ratio decreases.
Binary Object Features – Irregularity Ratio
The inverse of thinness ration is called compactness or irregularity ratio,
1/T.
This metric is used to determine the regularity of an object:
◦ Regular objects have less vertices (branches) and hence, less perimeter
compare to irregular object of the same area.
Binary Object Features – Aspect Ratio
The aspect ratio (also called elongation or eccentricity) is defined by the
ratio of the bounding box of an object.
This can be found by scanning the image and finding the minimum and
maximum values on the row and column where the object lies.
Binary Object Features – Aspect Ratio
The equation for aspect ratio is as follows:

cmax  cmin  1
rmax  rmin  1
It reveals how the object spread in both vertical and horizontal
direction.
High aspect ratio indicates the object spread more towards horizontal
direction.
Binary Object Features – Euler Number
Euler number is defined as the difference between the number of
objects and the number of holes.
◦ Euler number = num of object – number of holes

In the case of a single object, the Euler number indicates how many
closed curves (holes) the object contains.
Binary Object Features – Euler Number

Euler number can be used in tasks such as optical character recognition

(OCR).
Binary Object Features – Euler Number

Euler number can also be found using the number of convexities and
concavities.
◦ Euler number = number of convexities – number of concavities

This can be found by scanning the image for the following patterns:

0 0 0 1
0 1 1 
  1
Convexities Concavities
Binary Object Features – Projection
The projection of a binary object, may provide useful information
related to object’s shape.
It can be found by summing all the pixels along the rows or columns.
◦ Summing the rows give horizontal projection.
◦ Summing the columns give the vertical projection.
Binary Object Features – Projection
We can defined the horizontal projection hi(r) and vertical projection
vi(c) as:
width1
hi ( r )   I ( r, c)
c 0
i

height1
vi ( c )   I ( r, c)
r 0
i

An example of projections is shown in the next slide:

Binary Object Features – Projection
Histogram Features
The histogram of an image is a plot of the gray-level values versus the
number of pixels at that value.
The shape of the histogram provides us with information about the
nature of the image.
◦ The characteristics of the histogram has close relationship with characteristic
of image such as brightness and contrast.
Histogram Features
Histogram Features
Histogram Features
Histogram Features
Histogram Features
Histogram Features
The histogram is used as a model of the probability distribution of gray
levels.
The first-order histogram probability P(g) is defined as follows:

N (g)
P( g ) 
M
P(g) : probability of gray level g in image
N(g) : number of pixel with gray level g in image
M : total number of pixel in image
Histogram Features
The features based on the first-order histogram probability are
◦ Mean
◦ Standard deviation
◦ Skew
◦ Energy
◦ Entropy.
Histogram Features – Mean
The mean is the average value, so it tells us something about the
general brightness of the image.
◦ A bright image has a high mean.
◦ A dark image has a low mean.

The mean can be defined as follows:

L 1 height1 width1
I ( r, c)
Mean  g   gP( g )   
g 0 r 0 c 0 M
Histogram Features – Standard Deviation
The standard deviation, which is also known as the square root of
the variance, tells something about the contrast.
It describes the spread in the data.
◦ Image with high contrast should have a high standard deviation.

The standard deviation is defined as follows:

L 1
g   (
g 0
g  g ) 2
P( g )
Histogram Features – Skew
The skew measures the asymmetry (unbalance) about the mean in the
gray-level distribution.
Image with bimodal histogram distribution (object in contrast
background) should have high standard deviation but low skew
distribution (one peak at each side of mean).
Histogram Features – Skew
Skew can be defined in two ways:
L 1
1
SKEW 
g 3 (g  g )
g 0
3
P( g )

g  mod
SKEW ' 
g
In the second method, the mod is defined as the peak, or highest value.
Histogram Features – Energy
The energy measure tells us something about how gray levels are
distributed.
The equation for energy is as follows:

L 1
ENERGY   P( g )
2

g 0
Histogram Features – Energy
The energy measure has a value of 1 for an image with a constant value.
This value gets smaller as the pixel values are distributed across more
gray level values.
A high energy means the number of gray levels in the image is few.
◦ Therefore it is easier to compress the image data.
Histogram Features – Entropy
Entropy measures how many bits do we need to code the image data.
The equation for entropy is as follows:
L 1
ENTROPY   P( g ) log2 P( g )
g 0
As the pixel values are distributed among more gray levels, the entropy
increases.
Color Features
Useful in classifying objects based on color.
Typical color images consist of three color planes: red, green and blue.
◦ They can be treated as three separate gray-scale images.

This approach allows us to use any of the object or histogram features

previously defined, but applied to each color band.
Color Features
However, using absolute color measure such as RGB color space is not
robust.
◦ There are many factors that contribute to color: lighting, sensors, optical
filtering, and any print or photographic process.
◦ Any change in these factors will change the absolute color measure.
◦ Any system developed based on absolute color measure will not work when
any of these factors change.
Color Features
In practice, some form of relative color measure is best to be used.
Information regarding relationship between color can be obtained by
applying the color transforms.
◦ These transforms provide us with two color components and one brightness
component.
◦ Example: HSL, SCT, Luv, Lab, YCrCb, YIQ, etc.
 TRANSFORM FEATURES

• It provides information of data in

frequency domain.
• It is used to examine specific region of
an image feature using mask.
• Transform feature can be extracted using
zonal filter or mask.
• Generally high frequency is used for
edge & boundary detection and for
orientation detection angular silts can be
used.
 u(m,n) j(k,l) ĵ(k,l)
û(m,n)
Forward Inverse
X
transform transform
i/p image

Mask g(k , l)
Spectral Images
The primary metric for spectral features (frequency-domain-based
features) is power.
Power is defined as the magnitude of the spectral component squared.
POWER  T (u, v )
2

Spectral features are useful when classifying images based on textures.

◦ Done by looking for peaks in the power spectrum.
Spectral Images
It is typical to look at power in various regions, and these regions can be
defined as rings, sectors or boxes.
We can then measure the power in a region of interest by summing the
power over the range of frequencies of interest.

  T (u, v)
2
Spectral Re gionPower 
uREGION vREGION
Spectral Images
Spectral Images
Spectral Images
The ring measure can be used to find texture:
◦ High power in small radii corresponds to smooth textures.
◦ High power in large radii corresponds to coarse texture.

The sector power measure can be used to find lines or edges in a given
direction, but the results are size invariant.
Feature Analysis
Important to aid in feature selection process
Initially, features selected based on understanding of the problem and
developer’s experience
FA then will examine carefully to see the most useful & put back through
feedback loop
To define the mathematical tools – feature vectors, feature spaces, distance
& similarity measurement
Feature Vectors
A feature vector is a method to represent an image or part of an image.
A feature vector is an n-dimensional vector that contains a set of values
where each value represents a certain feature.
This vector can be used to classify an object, or provide us with
condensed higher-level information regarding the image.
 Feature Vector
Let us consider one example:

We need to control a robotic gripper that picks parts

from an assembly line and puts them into boxes
(either box A or box B, depending on object type). In
order to do this, we need to determine:
1) Where the object is
2) What type of object it is
The first step would be to define the feature vector
that will solve this problem.
Feature Vectors
To determine where the object is:
Use the area and the center area of the object, defined by
(r,c).
To determine the type of object:
Use the perimeter of object.
Therefore, the feature vector is: [area, r, c, perimeter]
Feature Vectors
In feature extraction process, we might need to compare two feature
vectors.
The primary methods to do this are either to measure the difference
between the two or to measure the similarity.
The difference can be measured using a distance measure in the n-
dimensional space.
Feature Spaces
A mathematical abstraction which is also n-dimensional and is created
for a visualization of feature vectors
2-dimensional space
Feature vectors of x1 and x2 and two
classes represented by x and o.
Each x & o represents one sample in
feature space defined by its values of x1
x2 and x2

x1
Distance & Similarity
Measures
Feature vector is to present the object and will be used to classify it
To perform classification, need to compare two feature vectors
2 primary methods – difference between two or similarity
Two vectors that are closely related will have small difference and large
similarity
Distance Measures
Difference can be measured by distance measure in n-dimensional feature
space; the bigger the distance – the greater the difference
Several metric measurement
◦ Euclidean distance
◦ Range-normalized Euclidean distance
◦ City block or absolute value metric
◦ Maximum value
Distance Measures
Euclidean distance is the most common metric for measuring the
distance between two vectors.
Given two vectors A and B, where:

A  a1 a2 ... an 
B  b1 b2 ... bn 
Distance Measures
The Euclidean distance is given by:
n

 i i
a  b 2
 a1  b1 2  a2  b2 2  ...  an  bn 2
i 1

This measure may be biased as a result of the varying range on different

components of the vector.
◦ One component may range 1 to 5, another component may range 1 to 5000.
Distance Measures
◦ A difference of 5 is significant on the first component, but insignificant on
the second component.

This problem can be rectified by using range-normalized Euclidean

distance:

n
ai  bi  2 Ri is the range of the


i 1 Ri
2
ith component.
Distance Measures
Another distance measure, called the city block or absolute value
metric, is defined as follows:
n

a i 1
i  bi
This metric is computationally faster than the Euclidean distance but
gives similar result.
Distance Measures
The city block distance can also be range-normalized to give a range-
normalized city block distance metric, with Ri defined as before:

n
a i  bi
i 1 Ri
Distance Measures
The final distance metric considered here is the maximum value metric
defined by:

max{ a1  b1 , a2  b2 ,..., an  bn }
The normalized version:

a1  b1 a2  b2 an  bn
max{ , ,..., }
R1 R2 Rn
Similarity Measures
The second type of metric used for comparing two feature vectors is the
similarity measure.
The most common form of the similarity measure is the vector inner
product.
Using our definition of vector A and B, the vector inner product can be
defined by the following equation:
Similarity Measures
n

 a b  a b
i 1
i i 1 1  a 2 b2  ...  a n bn 
This similarity measure can also be ranged normalized:
n
ai bi  a1b1 a 2 b2 a n bn 
 2
  2  2
 ...  2


i 1 Ri  R1 R2 Rn 
Similarity Measures
Alternately, we can normalize this measure by dividing each vector
component by the magnitude of the vector.

n
ai bi a1b1  a 2 b2  ..  a n bn
 n n

n n

a b a b
i 1 2 2 2 2
j j j j
j 1 j 1 j 1 j 1
Similarity Measures
When selecting a feature for use in a computer imaging application, an
important factor is the robustness of the feature.
A feature is robust if it will provide consistent results across the entire
application domain.
For example, if we develop a system to work under any lightning
conditions, we do not want to use features that are lightning
dependent.
Similarity Measures
Another type of robustness is called RST-invariance.
◦ RST means rotation, size and translation.

A very robust feature will be RST-invariant.

◦ If the image is rotated, shrunk, enlarged or translated, the value of the
feature will not change.
Conclusion
•Feature Extraction
• Binary Object Features (Area, Center of Area, Axis of Least Second Moment,
Perimeter, Thinness Ratio, Irregularity, Aspect Ratio, Euler Number,
Projection)
• Histogram Features (Mean, Standard Deviation, Skew, Energy, Entropy)
• Color Features
• Spectral Features
•Feature Analysis
• Feature Vectors and Feature Spaces
◦ Distance and Similarity Measures (Euclidean distance, Range-normalized
Euclidean distance, City block or absolute value metric, Maximum value)
Boundary Representation
After the processes of segmentation, object
representation is done by joining the isolate
points and forming boundary. This is required for
proper image analysis.

Some of the methods are:

i. Chain code
ii. Polygonal approximations
iii. Signatures
iv. Bending energy
 Chain code
They are used to represent boundary by connecting
lines having a specific length & orientation.

4-DIRECTIONAL CODE 8-DIRECTIONAL CODE

Algorithm for chain code
ORIGINAL IMAGE 4-DIRECTIONAL CODE
301122

8-DIRECTIONAL CODE
301122
ADVANTAGE OF CHAIN CODE:
Information of interest is preserved.
It is a compact representation of an image.
A chain code is prior condition for image analysis.
It is translation invariant.
But it is not rotation & scale invariant.
It represents any change occurring in direction,
indicating the corner points.
Problems in chain code:
1) Starting point of code determines chain code.
So any change in rotation or size affects the chain
code in unnormalized form.

2. Chain code is noise sensitive.

To solve this problem , resampling of boundary is
done to smooth out small variation.
 Polygon approximation
It is a process of connecting points of interest with
approximation. Using fewest possible points this
method is applied.

Due to approximation loss of information occurs.

The main reason to use this method is to

approximately reduce the code for polygon.
There are 2 types of approximation:

1. Fit & split

2. Merging
 1. Fit & split method
This method is useful for following reasons:
i. Noise elimination
ii. Absence of relevant feature
iii. Absence of change in vertices
iv. Simplification of shape
Procedure for method :
1. Divide boundary into small segment & fit a line for
each segment.

2. Draw a line L b/w the 2 farthest points P & Q.

3. Compute the value of perpendicular line from

boundary to line segment.

4. Compare this value with threshold value. If value

exceeds threshold value then line is split.
5. Identify the vertices which is close to the inflection
points(points where curve changes its sign) of
curve.

6. Now connect all the points to obtain a polygon.

7. Repeat the above process until no split is further

required.
FIT & SPLIT PROCESS FOR
S SHAPE IMAGE
Merging process

Procedure for this method:

1. Select a starting point for of the curve.

2. Add pixels, one-by-one to line segment.
3. Let the line pass through the new pixels.
4. Compute the square error for all the points along
the segment.
5. If error is greater than threshold, begin a new line at
the current point.
dk

MERGING PROCESS FOR

S SHAPE IMAGE
Signatures
Sometimes its required to represent 2D functions in
1D for fast & easy object matching. This is done using
signature function.

It is invariant to translation but not invariant of

rotation and scaling.

For e.g. x-axis shows the distance from centroid & y

axis shows the angle. See example on next slide.
y axis

x axis x axis
Bending energy
Thank You!