Medical Image Processing (2008)

Medical Image Processing

X. Image Segmentation

Medical image segmentation is

generally difficult
• Noisy images
– Often noise-to-signal is 10%
– This is ten times N/S of camera images
• Often textured in complex ways
• Relatively poorly sampled
– Many pixels contain more than one tissue
type. (Partial Volume Effect)
• Objects of interest have complex shapes
• Signs of clinical interest are subtle

Min H. Cho
Kyung Hee Univ. 1
Medical Image Processing (2008)

Visual Human Project

X. Image Segmentation
1. Preview
2. Point, Line and Edge Detection
3. Thresholding
4. Region-Based Segmentation
5. Segmentation Using Morphological Watersheds

6. Advanced Topics : Active Contours

- Snake
- Level Set

Min H. Cho
Kyung Hee Univ. 2
Medical Image Processing (2008)

Preview
• Segmentation is to subdivide an image into
its constituent regions or objects.
• Segmentation algorithms generally are
based on one of two basis properties of
intensity values
– Discontinuity : to partition an image based on
abrupt changes in intensity (edges)
– Similarity : to partition an image into regions
that are similar according to a set of predefined
criteria. (thresholding, region growing, region
splitting/merging)

Two Principal Approaches :

Discontinuity and Similarity

Edge-based

Region-based

Many small edges

Min H. Cho
Kyung Hee Univ. 3
Medical Image Processing (2008)

Detection of Discontinuities
(Point, Line, and Edge)
• Detecting sharp, local changes in
intensities
• Isolated points, lines, and edges.
• First and second-order derivatives
– First-order (gradient operator)
• produce thicker edges
– Second-order (Laplacian operator)
• Stronger response to fine detail
• Double-edge response at ramp and
step transitions
• The sign determines whether a
transition into an edge is from light to
dark or dark to light.

Detection of Isolated Points

• Laplacian + thresholding

Min H. Cho
Kyung Hee Univ. 4
Medical Image Processing (2008)

Line Detection
• Laplacian à positive and negative lines
- Take absolute values à thick lines
- Use only the positive values

Detection of lines in a specified direction

Line detection masks

Min H. Cho
Kyung Hee Univ. 5
Medical Image Processing (2008)

Edge Models
• In practice, edges are blurred and noisy. à ramp edge
• Ramp slope ß degree of blurring ß focusing mechanism
(optics, sampling, image acquisition imperfection)

ideal ramp roof

Derivatives and Ramp Edge

• The magnitude of the 1st derivative can be used to detect the presence
of an edge.
• The sign of the 2nd derivative can be used to determine whether an
edge pixel lies on the dark or light side of an edge
• The 2nd derivative produces two values à undesirable
• The zero crossings of the 2nd derivative can be used for locating the
centers of thick edges

Min H. Cho
Kyung Hee Univ. 6
Medical Image Processing (2008)

Noise and Edge

• The 2nd derivative is more sensitive to
noise.
• Fairly little noise can have a significant
impact on the two derivatives.
• Smoothing should be serious
consideration prior to the use of
derivatives

• Edge detection steps

– Smoothing for noise reduction
– Detection of edge points
– Selection of true edges

Basic Edge Detection

- The Image Gradient
é ¶f ù
ê ú M ( x, y ) = mag(Ñf ) = g x2 + g y2 »| g x | + | g y |
é g x ù ê ¶x ú
Ñf º grad ( f ) º ê ú = ê ú æ gy ö The direction of an edge is
ë g y û ê ¶f ú a ( x, y ) = tan -1 çç ÷÷ perpendicular to a(x,y)
ê ¶y ú è gx ø
ë û

Min H. Cho
Kyung Hee Univ. 7
Medical Image Processing (2008)

Gradient Operators

¶f
gx = = f ( x + 1, y ) - f ( x, y )
¶x
¶f
gy = = f ( x, y + 1) - f ( x, y )
¶y

Example

Min H. Cho
Kyung Hee Univ. 8
Medical Image Processing (2008)

Example

Example
- Diagonal Edge Detection

Min H. Cho
Kyung Hee Univ. 9
Medical Image Processing (2008)

Combining the Gradient with

Thresholding

The Marr-Hildreth Edge Detector

• The intensity changes are not independent of image
scale and so their detection requires the use of
operators of different sizes.
• A sudden intensity change will give rise to a peak or
trough in the 1st derivative or, equivalently, to a zero
crossing in the 2nd derivative.

• An edge detector should be a differential operator

capable of being “tuned” to act at any desired
scale, so that large operators can be used to detect
blurry edges and small operators to detect sharply
focused fine detail.

Min H. Cho
Kyung Hee Univ. 10
Medical Image Processing (2008)

LoG (Laplacian of a Gaussian)

• n×n Gaussian LPF+ 3×3 Laplacian :
Convolving the image with Gaussian smoothing function first and then
computing the Laplacian of the result
• Find edges via zero-crossing.
x2 + y2
-
2s 2 Mexican hat
G ( x, y ) = e operator
¶ 2G ¶ 2G
Ñ 2 G ( x, y ) = +
¶x 2 ¶y 2
x2 + y2
é x 2 + y 2 - 2s 2 ù - 2s 2
=ê úe
ë s4 û

the coefficients
sum to zero

The Marr-Hildreth Edge Detector

• The size of Gaussian filter
– The smallest odd integer ≥ 6σ
• Zero-crossing
– The signs of at least two of its opposing neighboring
pixels must differ. (left/right, up/down, two diagonals)
– Their difference ≥ threshold

• Attractive
– Zero crossing produces thinner edges
– Noise reduction
• Drawbacks
– Zero-crossing creates closed loops. (spaghetti effect)
à a positive threshold

Min H. Cho
Kyung Hee Univ. 11
Medical Image Processing (2008)

Example

spaghetti
effect

Example

Sobel

LoG Threshold Zero-Crossings

Min H. Cho
Kyung Hee Univ. 12
Medical Image Processing (2008)

The Canny Edge Detector

• Objective
– Low error rate : as close as possible to the true
edges (less spurious responses)
– Edge points should be well localized.
– Single edge point response (only one point for
each true edge point)

Canny Edge Detection Algorithm

1. Smooth the input image with a Gaussian filter
2. Compute the gradient magnitude M(x,y) and angle images
α(x,y).
3. Apply nonmaxima suppression to M(x,y). (to thin M(x,y) which
contains wide ridges around local maxima)
4. Use double thresholding and connectivity analysis to detect and
link edges.

Min H. Cho
Kyung Hee Univ. 13
Medical Image Processing (2008)

Nonmaxima Suppression
Step 1) Specify a number of discrete orientations of the edge normal (gradient vector).
– Ex) 4 orientations (0˚,45˚,90˚,-45˚) for a 3x3 region

Step 2) Get the nonmaxima-suppressed image gN(x,y)

1) Find the direction dk that is closest to α(x,y).
2) If M(x,y) is less than at least one of its two neighbors along dk, gN(x,y)=0 (suppression);
otherwise, gN(x,y)=M(x,y)

d3

d3

d2

d1

d4

d3

Double Thresholding and Connectivity Analysis

Step 3) Threshold gN(x,y) to reduce false edge points.
1) Hysteresis thresholding (low and high thresholds)
gNH(x,y)=gN(x,y)≥TH à strong edges à valid edges with gaps
gNL(x,y)=gN(x,y)≥TL
gNL(x,y)=gNL(x,y)- gNH(x,y) à weak edges (including false edges)

2) Locate the next unvisited edge pixel, p in gNH(x,y)

3) Mark as valid edge pixels all the weak pixels in gNL(x,y) that are connected to p, and repeat
4) Append marked pixels in gNL(x,y) to gNH(x,y)
missed

TH

false

TL

Min H. Cho
Kyung Hee Univ. 14
Medical Image Processing (2008)

Example
Thresholded gradient

Marr-Hildreth Canny

Example

Min H. Cho
Kyung Hee Univ. 15
Medical Image Processing (2008)

Edge Linking and Boundary

Detection
• Obstacles to edge detection
– Noise
– Breaks in the edges due to nonuniform illumination
– Other effects introducing spurious discontinuities in
intensity values
• Require additional linking algorithms designed to
assemble edge pixels into meaningful edges and/or
region boundaries.
• Basic approaches
– Local processing
– Regional processing
– Global processing via the Hough transform

Local Processing
• Analyze the characteristics of pixels in a small
neighborhood (say, 3x3, 5x5) about every edge pixel.
• All points that are similar according to a set of
predefined criteria are linked.
• Principal properties used for establishing similarity:
(1) Magnitude of gradient vector, M(x,y)=|∇f(x,y)|
|M(s,t)-M(x,y)|≤E
(2) Direction of gradient vector, α(x,y)
|α(s,t)-α(x,y)|≤A
• Edge pixel (x,y) is linked with (s,t) if both criteria are
satisfied.

Min H. Cho
Kyung Hee Univ. 16
Medical Image Processing (2008)

Simplified Formulation
1. Compute M(x,y) and α(x,y)
2. g(x,y)= 1 if M(x,y)>TM AND |α(x,y)-A|<TA
0 otherwise
A : a specified direction
3. Scan the rows of g(x,y) and fill all gaps in each row
that do not exceed a specified length
4. To detect gaps in any direction, rotate g(x,y) by this
angle and apply step 3.

Example

Min H. Cho
Kyung Hee Univ. 17
Medical Image Processing (2008)

Regional Processing
• When knowledge is available regarding the regional
membership of pixels
• Polygonal curve fitting

Example

gradient filtering shrink

noise reduction skeleton spur reduction T=3

1D smoothing for
jaggedness reduction
T=6 T=9 1D averaging 1D averaging

Min H. Cho
Kyung Hee Univ. 18
Medical Image Processing (2008)

Global processing using the Hough

transform
• Situation
– Unlike regional processing cases, all we have is an edge
image and no knowledge about where objects of
interest might be.
– All pixels are candidates for linking and thus have to be
accepted or eliminated based on predefined global
properties.

• An approach based on whether sets of pixels lie on

curves of a specified shape.
• Once detected, these curves form the edges or
region boundaries of interest.

The Hough Transform

• Brute force : n edge pixels
– Number of possible lines = n(n-1)/2
– Distance per line = n
– Total number of distances = n2(n-2)/2 ≈ n3 à a computationally prohibitive task

• The Hough transform : Efficient algorithm to find straight lines

– When different values for a and b are considered, yi=axi+b gives all possible
lines through the point (xi,yi). The equation b=-xia+yi gives one line in the ab-
plane for a specific (xi,yi)
– ab-plane : parametric space yielding the equation of a single line
– When another point (xj,yj) is considered, b=-xja+yj represents another line in the
ab-plane.
– Suppose that these two lines intersect at the point (a’, b’), then y=a’x+b’
represents the line in the xy-plane on which both (xi,yi) and (xj,yj) lie

Min H. Cho
Kyung Hee Univ. 19
Medical Image Processing (2008)

Algorithm
1) Set all cells of ab-plane to zero
2) For every (xk, yk)
2.1) Let a=every subdivision on the a-axis
2.2) Calculate b=-xka+yk
2.3) Round off b to the nearest allotted value on the b-axis
2.4) Increment accumulator cell (a, b) with 1
ab-plane
• Problem : -∞<a<∞, -∞<b<∞
à solution : (a,b) à (ρ,θ), xcosθ+ysinθ=ρ,
-90˚≤θ≤90˚, -√2D≤ρ≤√2D (D is the image size)

ρθ-plane

Example
R

A B
S Q

Min H. Cho
Kyung Hee Univ. 20
Medical Image Processing (2008)

Generalized Hough Transform

• can be used for any function of the form
g(v,c) = 0
– v is a vector of coordinates
– c is a vector of coefficients
• Ex) Circle
– equation: (x-c1)2 + (y-c2)2 = c32
– three parameters (c1, c2, c3) : cube like cells
– accumulators of the form A(i, j, k)
– increment c1 and c2 , solve of c3 that satisfies the
equation
– update the accumulator corresponding to the cell
associated with triplet (c1, c2, c3)

Algorithm for edge linking

1) Compute the gradient of an image and threshold it to
obtain a binary image.
2) Specify subdivisions in the ρθ-plane.
3) Examine the counts of the accumulator cells for high
pixel concentrations.
4) Examine the relationship (principally for continuity)
between pixels in a chosen cell.
5) Link these pixels if gaps are smaller than threshold

• Require only that we examine pixels associated with

specific accumulator cells à a significant advantage of
global processing

Min H. Cho
Kyung Hee Univ. 21
Medical Image Processing (2008)

Example
- Extract the two edges of the principal runway
for autonomous navigation of air vehicles

Example

link criteria:
- the pixels belong to
one of the set of pixels
linked according to the
highest count
- no gaps are longer
than 5 pixels

Min H. Cho
Kyung Hee Univ. 22
Medical Image Processing (2008)

Thresholding
• g(x,y)= 1 if f(x,y)>T
0 if f(x,y)≤T

• Global thresholding
• Variable (regional) thresholding, dynamic (adaptive) thresholding
- T depends on f(x,y) and p(x,y) (on gray-level values and its neighbors)
• Multiple thresholding

• Key factors
- the separation between peaks in the histogram
- the noise content
- the relative size of objects and background
- the uniformity of the illumination source
- the uniformity of the reflectance properties
image with dark image with dark
background and background and
a light object two light objects

The Role of Noise

Min H. Cho
Kyung Hee Univ. 23
Medical Image Processing (2008)

The Role of Illumination and

Reflectance
f(x,y) = i(x,y) r(x,y)

Basic Global Thresholding

• Estimate automatically the threshold value T
1) Select an initial estimate for T. (ex: average)
2) Segment the image into regions G1 and G2 using T.
3) Compute the average intensity values m1 and m2
for the pixels in G1 and G2
4) Compute a new threshold value :
T=(m1+m2)/2
5) Repeat Steps 2 through 4 until the difference in T
in successive iterations is smaller than a predefined
parameter ΔT.

Min H. Cho
Kyung Hee Univ. 24
Medical Image Processing (2008)

Example

Tinit=mG
ΔT=0
3 iterations
with result
T=125

Otsu’s Method for Optimum Global

Thresholding
• Objective : Minimize the average error incurred in
assigning pixels to two groups (classes)

• Optimum : Maximize the between-class variance

• Basic idea :
– Well-thresholded classes should be distinct with respect
to the intensity values of their pixels
– The best (optimum) threshold gives the best separation
between classes
• Based entirely on the histogram

Min H. Cho
Kyung Hee Univ. 25
Medical Image Processing (2008)

Otsu’s Algorithm
L -1
• Normalized histogram pi=ni/MN (å pi = 1)
i =0

• Threshold the image into C1 and C2 using T=k k

• The probability that a pixel is assigned to C1, P1 (k ) = å pi
i =0
• The probability of C2, P2(k)=1- P1(k)
• Means of C1 and C2 :
1 k 1 L -1
m1 (k ) = å ipi ,
P1 (k ) i =0
m2 (k ) = å ipi
P2 (k ) i = k +1
P1m1 + P2 m2 = mG , P1 + P2 = 1

• Separability measure (goodness of threshold), η = σB2(k)/σG2

between-class variance s B2 (k ) = P1 (m1 - mG ) 2 + P2 (m2 - mG ) 2 = P1 P2 (m1 - m2 ) 2
(mG P1 - m) 2 k
= (m = å ipi )
P1 (1 - P1 ) i =0 pi

• Determine k to maximize σB2(k)

P1 P2

k i

Example

k=169 k=181, η=0.467

Min H. Cho
Kyung Hee Univ. 26
Medical Image Processing (2008)

Using Image Smoothing to Improve

Global Thresholding

Using Edges to Improve Global Thresholding

Otsu’s

5x5 averaging Otsu’s

1) Compute an edge image, g(x,y) from f(x,y)

2) Threshold the edge image using a specifed threshold
value, T to make a mask image, gT(x,y)
3) Compute a histogram of f(x,y) corresponding to gT(x,y)
4) Use the histogram to segment f(x,y) globally

Min H. Cho
Kyung Hee Univ. 27
Medical Image Processing (2008)

Example

Example
T=42

TE=5% of max
T=55

TE=99.5% of max T=115

Min H. Cho
Kyung Hee Univ. 28
Medical Image Processing (2008)

Multiple Thresholds
• Extended Otsu’s method
• (K-1) threshold values for K classes:
K
C1, C2, C3, ..., CK
• Between-class variance s B = å Pk (mk - mG ) 2
2
Pk = å pi
k =1 iÎC k

η=0.954

k1=80 k2=177

Variable Thresholding
Using Image Partioning

Min H. Cho
Kyung Hee Univ. 29
Medical Image Processing (2008)

Variable Thresholding
Based on Local Image Properties
• Local threshold, Txy=aσxy+bmxy in a neighborhood, Sxy centered at (x,y)
or Txy=aσxy+bmG

• g(x,y)=1 if the predicate, Q(local parameters) is true

0 if Q is false
ex 1) Q(σxy, mxy)=true if f(x,y)>aσxy AND f(x,y)>bmxy
false otherwise
ex 2) Q(σxy, mxy)=true if f(x,y)>Txy
false otherwise

Variable Thresholding
Based on Moving Averaging
• Moving average along scan lines, mxy
• Txy=bmxy

Otsu’s Moving average

n=20, b=0.5

Min H. Cho
Kyung Hee Univ. 30
Medical Image Processing (2008)

Region Based Segmentation

• Region Growing
- Procedure grouping pixels or subregions into larger regions
based on predefined criteria for growth.
- Start with a set of “seed” points
- Grow regions by appending to each seed those neighbors that
have predefined properties similar to the seed (such as specific
ranges of intensity or color)
• Region growing should stop when no more pixels satisfy the
criteria for inclusion in that region
• A basic algorithm is based on 8-connectivity

Region Growing Algorithm

• f(x,y) : input image, S(x,y) : seed image
1) Find all connected components in S(x,y) and erode
to one pixel; Label such pixels
2) Form an image fQ such that, at a pair of coordinates
(x,y), let fQ(x,y)=1 if the given predicate, Q=TRUE;
otherwise let fQ(x,y)=0
3) Let g be an image formed by appending to each
seed point all the 1-valued points in fQ that are 8-
connected to that seed point
4) Label each connected component in g with a
different region label (e.g., 1,2,3,…)

Min H. Cho
Kyung Hee Univ. 31
Medical Image Processing (2008)

The mechanics of the Algorithm

1. original 2. Histogram 3. Initial seed
(thresholded ①)
Defective welds

254

4. Final seed (255) 5. |①-255| 6. Histogram of ⑤

(eroded ③) Using intensity as a
measure of similarity
68 126

Q=TRUE if diff ≤T

The problem of segmenting the 7. Dual-thresholded ⑤ 8. Single-thresholded ⑤ 9. Region growing

defects cannot be solved using
dual thresholds even though the
thresholds are in the main valleys.

Advantages and Disadvantages

• Advantages：
1. Region growing methods can correctly separate the regions that have
the same properties we define
2. Region growing methods can provide the original images which have
clear edges the good segmentation results.
3. The concept is simple. We only need a small numbers of seed point to
represent the property we want, then grow the region.
4. We can determine the seed points and the criteria we want to make.
5. We can choose the multiple criteria at the same time.
6. It performs well with respect to noise.

• Disadvantage：
1. The computation is consuming.
2. Noise or variation of intensity may result in holes or over-segmentation.
3. This method may not distinguish the shading of the real images.

Min H. Cho
Kyung Hee Univ. 32
Medical Image Processing (2008)

Region Splitting and Merging

• Subdivide an image initially into a set of arbitrary, disjoint
regions and then merge and/or split the regions in an attempt to
satisfy the necessary conditions
• R: entire image region, Q: predicate
(1) Split into four disjoint quadrants any region Ri for which Q(Ri)
= FALSE
(2) Merge any adjacent regions Rj and Rk for which Q(Rj∪Rk) =
TRUE
(3) Stop when no further merging is possible

Region Splitting and Merging Algorithm

• Initially take the image as a whole to be the area of interest.
• Look at the area of interest and decide if all pixels
contained in the region satisfy some similarity constraint.
• If TRUE, then the area of interest corresponds to a region in
the image.
• If FALSE, split the area of interest (usually into four equal
sub-areas) and consider each of the sub-areas as the area
of interest in turn.

• This process continues until no further splitting occurs. In

the worst case this happens when the areas are just one
pixel in size.
• This is a divide and conquer or top down method.

Min H. Cho
Kyung Hee Univ. 33
Medical Image Processing (2008)

Example
Segment out of the image the “ring” of less dense matter surrounding the dense center

If σ>a AND 0<m<b,

then Q= TRUE

Morphological Watersheds
• To extract nearly uniform objects from the background.
• A closed contour
• Independent of shape and size
• Efficient and accurate

• Any gray-scale image can be considered as a topographic surface.

• Often applied to the gradient of an image rather than to the image itself
• If we flood this surface from its minima and, if we prevent the merging of the
waters coming from different sources, we partition the image into two different
sets: the catchment basins and the watershed lines.
• When the rising water in distinct basins is about to merge, a dam is built to
prevent the merging.

Flooding and
building dams

Min H. Cho
Kyung Hee Univ. 34
Medical Image Processing (2008)

Dam Construction
Dam construction is based on binary images

Cn-1(M1) Cn-1(M2)
· M2 Two partially flooded catchment
· M1
basins at stage n-1

Flooding at stage n, showing that water

has spilled between basins and merged;
The two connected components have
q become a single component q.

Result of dilation and dam construction

(Dilation is constrained to q)
Building a dam = Setting
points of the dam to a value
greater than the max

Gradient of The Image

• If we apply this transformation to the image
gradient, the catchment basins should
theoretically correspond to the homogeneous
grey level regions of this image.

Min H. Cho
Kyung Hee Univ. 35
Medical Image Processing (2008)

The Use of Markers

• Over-segmentation problem due to noise and other local irregularities of the gradient
• Limit the number of allowable regions by incorporating a preprocessing stage designed to
bring additional knowledge into the segmentation procedure
• Internal markers for objects of interest and external markers for the background
• Procedure for marker selection
1) Preprocessing (smoothing) to minimize the effect of small spatial details
2) Marker criteria
- A region that is surrounded by points of higher altitude
- The points in the region form a connected component
- All the points in the region have the same intensity value
3) Apply the watershed algorithm to the smoothed image (internal markers are the only allowed regional
minima)
4) The resulting watershed lines are defined as the external markers and effectively partition the image into
regions. Each region contains a single internal marker and part of the background.
5)* Apply the watershed algorithm to each individual region of the gradient of the smoothed image

Electrophoresis image Result of watershed Image showing internal Result of segmentation

segmentation markers (red regions)
(over-segmentation) and external markers
(yellow lines)

Example

Min H. Cho
Kyung Hee Univ. 36
Medical Image Processing (2008)

Example
In this case, the criterion used is not the intensity values
but the distance transform of the image

Min H. Cho
Kyung Hee Univ. 37