IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO.

6, NOVEMBER 1981 687

[7] A. Klinger and C. R. Dyer, "Experiments in picture representa-

tion using regular decomposition," Comput. Graphics Image Pro-
cessing, vol. 5, pp. 68-105, 1976.
[8] A. Klinger and M. L. Rhodes, "Organization and access of image
data by areas," IEEE Trans. Pattern Anal. Mach. Intell., vol.
PAMI-1, pp. 5 0-60, 1979.
[9] P. Naur, Ed., "Revised report on the algorithmic language Algol
IAl I%>
,L \b image 60," Commun. Ass. Comput. Mach., vol. 3, pp. 299-314,1960.
Fig. 3. Altered image for the alternative perimeter computation
[10] A. Rosenfeld and A. C. Kak, Digital Picture Processing. New
York: Academic, 1976, section 9.2.1.
method. [111 H. Samet, "Region representation: Quadtrees from boundary
codes," Commun. Ass. Comput. Mach., vol. 23, pp. 163-170,
of BLACK blocks (i.e., the image complexity) dominates the Mar. 1980.
execution time of the algorithm. It should be clear that if the [12] , "Connected component labeling using quadtrees," J. Ass.
image contains more than one region, then the algorithm will Comput. Mach., vol. 28, pp. 487-501, July 1981.
return the total perimeter of all the regions. Similarly, if
holes are present, their boundaries are also included in the
value of the perimeter obtained by this algorithm. Note that
if we first labeled the connected components (i.e., regions)
of the image [121, then the perimeter of each region could
be separately computed. Image Approximation from Gray Scale "Medial Axes"
Our algorithm examines the four adjacencies of each BLACK SHYUAN WANG, ANGELA Y. WU, AND AZRIEL ROSENFELD
node (i.e., in the N, E, S, and W directions). An alternative
is to only examine for each BLACK node the adjacent WHITE
nodes in the E and S directions. The difficulty with such an Abstract-Several types of gray-weighted "medial axes" have been
approach is that it ignores the contributions of the N and W defined. This paper shows that one of them, the min-max medial
segments of the region's boundary (e.g., CO, CI, AI, AH, BJ in axis, can be used to reconstruct good approximations to the original
Fig. 1). This can be overcome by surrounding the image by six image based on a relatively small amount of information.
WHITE blocks, as shown in Fig. 3, and for each BLACK node
examine the adjacent WHITE nodes in the E and S directions Index Terms-Image approximation, image processing, medial axis
and for each WHITE node examine the adjacent BLACK nodes transformations, pattern recognition.
in the E and S directions. These BLACK-WHITE and WHITE-
BLACK pairs make a contribution to the perimeter of the image.
It should be clear that the two methods are equivalent. How- I. INTRODUCTION
ever, depending on the particular configuration of BLACK and A binary image can be exactly reconstructed if we know the
WHITE blocks in the image, one method may be superior to centers and radii of its maximal blocks of 1's, since the set of
the other. In particular, if there are considerably more BLACK 1's is the union of these maximal blocks. This representation
nodes than WHITE nodes, then the second method may be is called the medial axis transformation (MAT) [1], since the
preferable since less adjacencies need to be explored. centers of maximal blocks should lie near the midlines of re-
The algorithm demonstrates the utility of the quadtree as gions of l's.
a desirable data structure for image representation. Computa- Several generalizations of the MAT to gray scale images have
tion of perimeter is generally achieved by use of a chain code been proposed. The binary medial axis (=set of centers of
representation. We have shown that it can be computed with maximal blocks, MA) turns out to be the set of l's whose
reasonable efficiency when the quadtree is used as the data distances (=shortest path lengths) to the set of 0's are local
structure. maxima [2]. Let us define the gray-weighted distance be-
tween two points as the smallest sum of gray levels along any
ACKNOWLEDGMENT path joining the points; then we can define the gray-weighted
The author would like to thank K. Riley for her help in pre- medial axis (GRAYMAT) [3] as the set of points whose gray-
paring this manuscript and P. Young for drawing the figures. weighted distances to the set of 0's are local maxima. This
He has also benefited greatly from discussions with C. R. Dyer definition is not entirely satisfactory, since it gives the set of
and A. Rosenfeld. 0's special status, thus in effect requiring a segmentation of the
image; it is also hard to see how to reconstruct an image from
REFERENCES its GRAYMAT.
[1] C. R. Dyer, A. Rosenfeld, and H. Samet, "Region representation: Another generalization, the SPAN (spatial piecewise approxi-
Boundary codes from quadtrees," Commun. Ass. Comput. Mach., mation by neighborhoods) [4 ], finds the largest block, centered
vol. 23, pp. 171-179, Mar. 1980. at each image point, that satisfies some criterion of homo-
[2] H. Freeman, "Computer processing of line-drawing images," geneity, and discards blocks if they are contained in other
Comput. Surveys, vol. 6, pp. 5 7-97, 1974. such blocks, thus obtaining a set of maximal blocks. If we
[3] G. M. Hunter, "Efficient computation and data structures for know the centers, radii, and mean gray levels of these SPAN
graphics," Ph.D. dissertation, Dep. Elec. Eng. Comput. Sci., blocks, we can reconstruct a good approximation to the image
Princeton Univ., Princeton, NJ, 1978.
[4] G. M. Hunter and K. Steiglitz, "Operations on using quadtrees," by superimposing blocks having these gray levels. A disad-
IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-1, pp. 145-
153, 1979. Manuscript received July 31, 1980; revised October 30, 1980. This
[5] -, "Linear transformation of pictures represented by quad- work was supported by the Defense Advanced Research Projects Agency
trees," Comput. Graphics Image Processing, vol. 10, pp. 289- and the U.S. Army Night Vision Laboratory under Contract DAAG-
296, 1979. 53-76C-0138 (DARPA Order 3206).
[6] A. Klinger, "Patterns and search statistics," in Optimizing Meth- The authors are with the Computer Vision Laboratory, Computer
ods in Statistics, J. S. Rustagi, Ed. New York: Academic, 1971. Science Center, University of Maryland, College Park, MD 20742.

0162-8828/81/1100-0687$00.75 ( 1981 IEEE

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
688 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981

vantage of this approach is the computational cost of deter-

mining the maximal blocks.
A recently proposed generalization, the GRADMAT [5],
assigns a score to each image point based on the strengths of
the gradient magnitudes (in opposite senses) at pairs of points
symmetrically located with respect to the given point. Un-
fortunately, the GRADMAT turns out to be quite sensitive
to noise. In Section IV we will see that, in consequence, ap- (ii)
proximations to the image reconstructed from the GRADMAT
are rather poor.
A final generalization is based on a characterization of the
binary MAT in terms of shrinking and expanding operations
[6]. Let S be the set of l's, let s(k) be the result of repeatedly
(k times) "expanding" S by changing all O's that are adjacent
to l's into l's, and similarly let S(-k) be the result of k steps
of "shrinking" S by changing all l's that are adjacent to O's
into O's. It can be shown that (S(-k))(1) C S(-k+ 1), and that
the difference between these two sets is just the set of MA
block centers that have radius k. We can generalize this defini-
tion to gray scale images [ 71 by using local min and max opera- (a) (b)
tions as analogs of shrinking and expanding [8]. Let I(k) be Fig. 1. Reconstruction from the MMMAT using points whose MMMAT
the result of k iterations of local max applied to the image I, vector sums are greater than M/4. (a) Original images. (b) Recon-
and let I(-k) similarly be the result of k local mins. It can be structed images. (i) Chromosomes. (ii) Terrain. (iii) Tank. (iv) Blood
shown that (I(-k))(1) S.I(- k+ 1). At each image point, we can cell.
define a vector A = (Al, * ** ., Am) whose kth component is
the value of I(-k+l) - (I(-k))(1) at that point; the array of In particular, for all nonnegative k we have (I(-k))() < I(-k+ 1).
these vectors constitutes the MMMAT (min-max MAT) of I. The m-vector MMMA T of I is the m-tuple of arrays A = (A 1,
Typically, at most points of the image all the components of * * Am), where each array is congruent with the image, and
the MMMAT will have low values, and at most of the excep- in which, for 1 < k < m, we have
tional points the values will be high only for a few consecutive A k = (-k+ i) (I(-k))(- 1)
values of k, corresponding to the natural "MMMAT radius"
at the given point. Thus, each A k is nonnegative.
The original image I = IO) can be reconstructed from I(-m) We can define scalar MMAT's in various ways. One possibil-
and the MMMAT using a process of iterative local max and ity is to sum the terms of A at each point, i.e., to use the
addition of A values, since I(-k+1) = (I(-k))(1) + Ak. How- scalar array
ever, this exact reconstruction process requires a large amount SA = A, + ***+AiM.
of information, including the entire array of MMMAT vectors
A. In Section II we will see how good approximations to the It was seen in [7 ] that the high-valued points in SA constitute
image can be constructed using only a few high MMMAT good "skeletons" of the homogeneous regions in the original
values at a relatively small number of points. image I, while most of the points in S have low values. This
means that at most points, all the components of A have low
II. METHODOLOGY values.
This section summarizes the algorithms used for constructing Another useful scalar MMMAT uses the larges term of A at
the MMMAT, simplifying it in various ways, and reconstruct- each point, i.e.,
ing (an approximation to) the image from the (simplified)
MMMAT. For ease of reading, the algorithms are described MA = max (Al1,* ,
informally, but the descriptions are sufficiently precise that This too, as seen in [7], yields good "skeletons." This implies
it is straightforward to derive their formal definitions. Results that even at those points where SA has a high value, only one
of applying these algorithms to four images are also given. (or perhaps a few) of the Ai's have high values. As we shall
now see, these observations can serve as a basis for defining
A. MMMA T Construction approximations to the MMMAT from which good approxima-
Notation: tions to I can be reconstructed.
I(O) is the original image.
I(1) is the result of a local max operation applied to I, i.e., B. Exact Reconstruction from the MMMA T
the value at each point is replaced by the highest value in the By reversing the definition Of A k, we have for 1 < k < m
neighborhood of the point. In our experiments, the neighbor-
hood used consisted of the point and its eight horizontal, (- k +) = (f&k))(I) + Ak.
vertical, and diagonal neighbors.
I(k) = ((I(1))(1) ... )(1) is the result of k iterations of the
Hence, if we are given I(-m) and 4, we can exactly recon-
struct I as follows:
local max operation applied to I.
I(-') is the result of a local min operation applied to I, I(-m+1) = (I(-m))(0) + Am
using the same neighborhood as for local max. 1(-m+2) = (I(-m+1))(1) + Am.-1
I(-k) = ((I ...))&1) )(-1) is the result of k iterations of
the local min operation applied to I.
It can be shown that, for all nonnegative integers h and k, J=
I-I° (I(l)) + Al
we have (pointwise)
In other words, we start with 1(-m), apply a local max opera-
(I(-h))(k) <" Ik-h <., (I(k))(-h). tion to it, and add Am to it pointwise to obtain I(-m+1). We

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 198168
689

.x..........x............
I x...
.x. ...x...
*.. x.

.xxxx.. X ..
x....... ........x...
888484.. xxxx.. .. X. x........... ...........
8442222224448. .84
88448 ...8888. .84
.4 4 84....
88.e 8..
.I22448.
...I....
.8...... ....x... xx X .. x.
838. .42448. 84....
...I.... x. x ..x ....X.x
.22
B..
.X. .. ....x .. ... .x...
.42 ...
.x.. x...........x..........
x.. ... ..... x. ....x..
..
.88422. x.. ..
88844..... ....8222 x..X
8442248...... .84248 4.........42
. B ..X. X.
xx. ... ... x. ... .. xx ..
82... .8444488 Kx
.84.884. xx ...
.x..x......
8 44 .
88
...8.224B .xxx.......
x............ ,X
.8. 2424
44I...

X x ... ..x
..x.. ...
x...Xx
X. x. X

48.. I...
....... 42.88................

xxx>
........88 .. 2'2448 ......... ....44 xx. xx. ....................... xxxx.
............ 844448 .. ......422 x.x ...............X
84 22 X
..........44....... ..........42 X X
..88 .............. .. 44
.a...a.. 8.88e..........B X. x..
44...... ........84 .8.........1
44 ........4...........
4288 .44448 .. x x

42406... 4222248 ..
4248 .8..........8422228........... xxx

428 .... 848 ...........42 ........... .x.. X. x..........

44 88..
e..........44 ...........
..
.....428..........44 ........... x x x

...... 22...........88 ...........

.22 ......................
8.. 22 ....
...8888 ....
... 42488 ...........
..............844448...
... x x xx

(a) (b)

Fig. 2. Skeletons of chromosome image (see Fig. 1) obtained using various techniques. (a) Thresholding. 2 indicates

points with value >M/2; 4 indicates points with M/2 > value > M14; 8 indicates points with M/4 > value > M/8. (b)
Thinning.

next apply another local max operation and add AM-,. to TABLE I
obtain If&m+2); and so on, until finally we add Al to get the NuMBERS OF SKELETON POINTS IN FoUR 64 X 64 IMAGES (SEE FIG. 1) USING
VARious TRESHOLDS (M = HIGHEST MMMAT Sum IN IMAGE)
original image IL

C. Approximate Reconstruction: Using,& M./2 M/4 M/8

at Only a Few Points Chromosomes 74 189 322
Terrain 244 582 1268
The reconstruction of I just described involves repeatedly
Tank 30 227 552
taking local maxima and adding Ai values pointwise. At a
Blood Cell 74 462 721
point for which all the Ai values are small, adding them
should not make a significant contribution to the final value;
the results would be similar if we used A4 0 at such points.
As mentioned in Section II-A, all the Ai values are in fact the image I; a high-contrast image should tend to have higher
small at most points. Thus we should be able to get good MMMAT values than a low-contrast image. We can reduce the
approximate reconstructions of I by using the A values only influence of contrast by choosing a threshold proportional to
at points where SA A1 +...- + Am is high (e.g., higher than
=
the highest SA value in the image. We can also start the recon-
some threshold), and ignoring the A values (i.e., treating them struction process with an ~S-(m)"~that is identically zero.
as zero in the reconstruction algorithm) everywhere else. To test these ideas, reconstruction experiments were per-
The choice of a threshold should depend on the contrast in formed using the four images shown in Fig. 1(a). These

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
 690 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981

........... x......................
. . .
.xx.........
......................
.x........ ........................

.x.... x.......x...x................
xxxxxxxx........x...................
.x.....xxxxxxx ...... x................

............xxxx .................... x.x...

.x.xx
........x...................
. xx...................
....x........... .. x...................
..x. x............................
....
.x............x.xx...... ............

...........x .........x.. ........

x....xxxx
. ..............x..........

......................xxxx....... .xxxxx... .. x.. .... .x. x......

.x.................... xx..... x..xxx ............x.. xxxxxxxx......
.X. xxxxx ....
.x......................x.
X....X..........X.......... ..X........xx...x..... .x......... xxx.........
X.........x...X........... ..X....... .x....... ............

X.......................x.........
..X .....................x..X....... ...... xxx...x..
........................x..X.......
.................X..............X... ......x x........
....X x.........................xx.... ......x ......X
x............................
........... .xxx
....X .x........... x
........... .......

X............................. .x............
...................

X...........xx.................. x............... . x......

...................

X............................. .................xx........... x.....

X.............................
X.............................
X. ........X.
.x.. ...X. x ............. ..x...........

.............x...............
x.x........... ..X..................
X.x............
x.. ..
.x.x...........X............
...X........... x.x................... ....x. x.............x x............
X..... .........XX.X.................
X...... X.x....... .x ................ X X. .....x...xx.. .. x..
...... .X........ x ...............
.........X......x
.X . X. x..
.X.
...x ......
..............

xxxx....... .. ...x........... ........

.xxx X ....xxxx.. .. .X.
....x.x..x.......x
X .X.........xxx
X..xxxxxxxxxxxx .... X ... .x .. .. x..............
....
.x.. .x ...
xxx..............................
..........Xxx.xxx....Xx..X... ..
..xxxxxxxx .........................
.......... x....................... X x X .(d
).X
.......... x..X.....................

(c)
Fig. 2. (Continued.) (c) Line detection (6 conditions). (d) Line detection (5 conditions).

images are, from top to bottom, part of a chromosome spread,

part of a LANDSAT image of terrain, an infrared image of a
tank, and a photomicrograph of some blood cells. Each image
is a 64 X 64 array of pixels, and each pixel has 64 possible
gray levels. We used m = 10, 15, 15, and 20 for the four
images; these numbers were chosen on the basis of the sizes
of the regions occurring in the images.
The numbers of "skeleton points" in each image having
above-threshold SA values, for various thresholds, are shown
in Table I. The thresholds used were M/2, M/4, and M/8,
where M is the highest SA value in the given image. We see
from Table I that only a fraction of the 4096 image points
are skeleton points, even for the low threshold (M/8). The
fraction is greatest for the terrain image, which is the most
complex, and it increases most rapidly for the tank image,
which is the noisiest (note also the large increase between
M/2 and M/4 for the blood cell image, probably due to the
texture in the while blood cell's nucleus).
The skeleton points whose SA values exceed M/4 give quite
good reconstructions, as seen in Fig. 1(b). (In the tank image, (a) (b) (c) (d)
the background near the top of the image has a few dark areas
which give rise to SA values between M/2 and M14 at a few Fig. 3. Same as Fig. 1 except: (a) Original image. (b) Reconstruction
points. As a result, when the threshold M14 is used, the result- using the maximum A value at each point having sum >M/4. (c) Re-
ing reconstruction has conspicuous artifacts. As we will see construction using the two largest A values at each such point. (d)
later (Fig. 3), these disappear when M/2 is used as a threshold.) Reconstruction using the three largest A values at each such point.

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981 691

TABLE II
NUMBERS OF SKELETON POINTS IN THE FOUR IMAGES AFTER THINNING
OR LINE DETECTION

Line detection
Thinning Six conditions Five conditions
Chromosomes 166 127 208
Terrain 486 324 557
Tank 276 196 264
Blood Cell 277 321 457

(a) (b) (c) (d)

Fig. 4. (a) Reconstructed images using the largest A value at points
whose MMMAT vector sums are greater than M/2. (b) Same as
(a) except that the three largest A values at each point are used.
(c) Same as (a) except that the five largest A values at each point
are used. (d) Same as (a) except that all points with MMMAT vector
sums greater than M/8 are used.

A more detailed look at the skeleton points obtained when

we threshold at M/2, M/4, and M/8 is provided by Fig. 2(a)
which shows these points for the chromosome image. Points
labeled 2 have SA values greater than M/2; points labeled 4 (a) (b)
have values between M/2 and M/4; and points labeled 8 have Fig. 5. Reconstruction using points obtained by thinning the MMMAT
values between M/4 and M/8. sums. (a) The largest A values are used. (b) The three largest A
values are used.
D. Approximate Reconstruction: Using Only a Few Ai Values
The reconstructions shown in Fig. l(b) were based on a
large amount of information; they used A values at only a
few "skeleton points," but at each such point m values were
used. As pointed out at the end of Section 1I-A, however,
at most of the skeleton points, only a few components of the
A,vector actually have high values. Thus, one should be able
to obtain good reconstructions using only the highest few
Ai's at each skeleton point, and ignoring all other Ai's (i.e.,
treating them as 0).
Fig. 3 shows reconstructions of the four images using the
same skeleton points as in Fig. 1, but using only a few (1, 2,
or 3) highest Ai values at each point. We see that good ap-
proximations are obtained using only a single value, and that
when three values are used, the results are almost the same as
when the entire A vector was used.
Fig. 4 shows that using only the skeleton points for which
SA > M/2 does not yield good approximations to the original
images. The M/4 skeleton with only one value at each point
[Fig. 3(b)] gives better results than the M/2 skeleton with five
values [Fig. 4(c)]. The M/8 skeletons have 60 to 100 percent
more points than the M/4 skeletons, but the reconstruction (a) (b)
quality is not much better [compare Fig. 4(d) with Fig. 3(b)]. Fig. 6. Same as Fig. 5 except that points satisfying a nonlinear line de-
tector (six conditions) are used.

III. MODIFICATIONS
A. Thinning
Thinning operations on binary pictures, which repeatedly
delete black border points without locally disconnecting
In a piecewise homogeneous image, the high-valued skeleton their neighborhoods, reduce objects to arcs and curves. This
points should tend to lie on a set of arcs or curves, but these concept has been generalized to gray scale images [9], where
may be somewhat thick (e.g., a region of even width yields each point's gray level is changed to the minimum of its
a skeleton that is two points thick). If we apply a thinning neighbors' gray levels, provided this does not locally "dis-
operation to the skeleton points before we reconstruct, the connect" its neighborhood, where we say that two points
resulting reconstruction should be quite similar to the original are "connected" if there exists a path between them com-
image, except for slight variations in region dimensions. posed of points that are as dark as both of the endpoints.

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
692 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981

(a)

(b) (e)

(c) (f)

(d) (g)

(a) (b) Fig. 8. Reconstruction of a noisy chromosome image. (a) Chromo-

Fig. 7. Same as Fig. 5 except that points satisfying a less strict nonlinear somes of Fig. l(a) with noise added (mean = 0, standard deviation =
line detector (flve conditions) are used. 5 on a gray scale of 0-63). (b) Reconstruction of (i) using points
with MMMAT sum >M/4 and the largest A value at each of these
points. (c) Same as (b) except that the three largest A values are
It seems preferable to apply thinning to the array of SA used. (d), (e) Same as (b), (c) except that points obtained from the
nonlinear line detector are used. (f), (g) Same as (b), (c) except that
values, rather than thresholding them first and then thinning points obtained from thinning are used.
the resulting binary array, since the former approach pre-
serves more information. Thinning allows SA values that do
not lie on high-valued "ridges" to be selectively eliminated When the process just described is iterated, elongated high-
to produce a cleaner set of arc- and curve-like loci of skeleton valued regions are thinned into arcs or curves of high-valued
points. The results of thinning are shown in Fig. 2(b) for the points. The number of iterations used should be at least half
chromosome image. the width of the widest such region. In our experiments, the
The procedure used to produce thinned skeletons can be
summarized as follows. numbers of iterations used on the four images were 4, 6, 2,
1) Construct the array of MMMAT sums SA. and 4.
2) Apply the thinning algorithm, which is defined as fol- As shown in the first part of Table II, this procedure usually
lows. Let P be any point and N(P) be the set of eight neigh- yields a somewhat lower number of skeleton points than if we
bors of P, excluding P itself. Let the points of N(P), in cylic simply threshold the original SA values at M/4. However, as
order, be Ql, ,Q8, Ql,.. .If there exist four neighbors Fig. 5 shows, the reconstructions from these thinned skeleton
points, using the largest component or largest three compo-
Qa, Qb, Qc, Qd, in cyclic order, such that Qa and Qc have nents of A at each of these points, are not as good as the re-
higher values than P, but Qb and Qd have lower values, we constructions obtained when we simply threshold SA at M/4.
leave the value of P unchanged; otherwise, we give P a new
value equal to the smallest of the values of the Qi's. It would seem that in most cases, the somewhat greater degree
(The idea underlying this algorithm is that if there exist of compression obtained by thinning the skeleton would not
be sufficient to justify the added computational cost, in view
Qa, Qb, Qc, Qd as just described, then the path Qa, P, Qc of the poorer reconstruction results.
consists of higher values than either of the paths Qa, Qa+,i
,Qc and Qc, Qc+l1, , Q; thus, if we reduced the value
B. Line Detection
of P, we would be weakening the "degree of connectedness"
of Qa and Qc within P's neighborhood. It is easily seen that A less computationally costly approach to obtaining "cleaner"
this is a generalization of a standard algorithm for thinning sets of skeleton points is to apply local line detection opera-
the set of I's in a binary picture: change the value of P from tions to the SA values. This should tend to eliminate skeleton
1 to 0 if P has at least one 0 neighbor, and if doing so does points that do not lie on lines or smooth curves, or along the
not disconnect any pair of l's in P's neighborhood.) crests of "ridges."
This algorithm is applied "in parallel" (i.e., using the old Two line detection criteria were used in our experiments.
values of the neighbors) at every point P. In order to prevent Let P(i, j) denote the value of the point P at position (i, j)
high-valued regions that are two points wide from being de- in the input array. For vertical line detection, we give P the
stroyed by a single parallel application of the algorithm, we new value
actually apply it only to a subset of the P's at a time, e.g.,
first to P's that have higher values than their north neighbors,
then to P's having higher values than their south neighbors, PVU, j)= [P(i, j +k)- (P(i - l, j +k)
k= -1
then east and then west. (This type of restriction is also used
in the binary case.) +P(i+ 1,j+k))]
Finally, in order to prevent regions that are already thin
from eroding away at their ends, we do not change the value provided the following conditions are satisfied:
of P if it is significantly higher than the values of all but one P(i,j+k)>P(i- l,j+k) for k -l,0,
of its neighbors. (In the binary case, we do not change P from 1.
1 to 0 if only one of its neighbors is 1.) P(i,j+k)>P(i+ l,j+k)

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981 693

.2.4 .8 .4 88 88 8.....8 .S. 4.8.8

8 E X . X X X... X ... X. X X
.........88 888.8 8688 8.88 8 . 88888 88848 P.xx .xX X X.......... X...............
.8 88.8.
... 8. .8 ... 4 ..... . ......... .8.8 ...........
.8 8 8 6888 88 8. 8 88
4.88 8 88
6....... 8 88.88.. 8 88 48 88.8 8. x....... .x .. ... x... ... x..... Xx .
.... . x..
. .88 6 .84 8 8 84.88 8 848.888 888 .x.. ..X X.x .....x . X.. X
B.8.86.8.848 ........ 88... 8 ... 4.8 . 8 8 88 ... X. ..... X.... X...BBvzZx@rv& X........
we@w ...... X...........o
X

.... 4.86 4.8 8 4 8 88 8

.... 8844444.48 . 8.88 ..8 ..8..... 8 6 84 8 6 8
..8.8422222448.888 8 8 8 8. 8. 6 8
844488888. 8.4 8888.88 8 .8 8 8.
888.88.8 ....8..... 84 88 8 8. 8. 8
.......88 8888 ... S.448.88 8
..... 8 8 68.. 8 888.88
8 884.8 ... 448 48 68 8 8.8. *b.
......
888.
8. 4. ......... ... 428. .88.8.48 ...
.
8.6 ............. S.
.x
..........X.X..... . x ..X......x
X.. . . .. X... X..
... 8.88.8 ..... 842888866 6 6 4 6 68 ................. 8 ........X..........
X.x.....x. 2 . . .x. . x......
.88 .... 8... 8 .8288 .8 44 6.B. 6.88 x. x4z xx .x.o*x
.......... 8 8886 ... 8.848 ..6. ... 8 88 86
8.. .. 88 ..... 484248.. 8 8. 8 .8 88648 .8 8
8... . 8 8 ........ 88888446 8 48844.88 886 x. xxx .. X xxxx . X.
.8
.8 ....8. 6.6..
888.8.8.8 8.444
88444444
8
8.8.8.8.. . 48228888.
8682424 . 8....
8.. .6.
.......X.. .. ...X. ..I ....... .. xxx . x... .X.. X.x
.... 8.. 84 .. 86 8 48 .. 4 4444448 88. 88
.. .x . X........ x......... xx . *@ x.
X. .¢ @0g xx
.8 44288.8...66.8.. 4 4 6...8 8424288 8
4228 .. 8 4 6.48 ...68 48 .8 ...8
..x x. ...x x..
......
.
8. 24 .. 88
88 ........ 44. 4 ... B.
I4
...
8
...
8. .
6.
.... 4.....
.
8. .. 86
6.S.8 8.8
X .X. X X. . X. . XX .
..

44..
88.
B a..
886
B . 88. 6.888. 6. 868884. 88.
8 8488 6..
6 . 8... B
8 ..x x........ x .x. ..x x. X . X.... .. X. X .....
... 8 88. 6 .. 8... 8..... 6. 8 4.
. 8. 488
..........
8 84 x x....... .x . .xX. ............x...
xX.. .x.x....
. . x.
.* *@s *o @o
888 .. 4 .8 8. 8. B. 88. 66 68888.... x ...........X...........X.. 8 ..x....
. . . .x ..
. ...8 . 8. 888 3. 8. 4.. 8..... 6.. 8886 .. 84. 8.88 ..... 4. 8 8
.... 8.8
.. 84.. 88..... .8 ..8 68 8. 4 88.
. 8
.8. . 88. 8. 8 8 6S. 6. 88. 8. 8. ...8... 8.. 4.... S. 84. . S. S. . BS. S.
88... 8... 48........ 886 ... . 88. 66. 4 . 86 .... 88
. 88. B. ... 8... 6. 88684. 86. 6.... 68 8.84. 8 868
4.... 88..8 ......... 6 88 8 3... .6... .....8.
B. as. 488 E8 888688.8 888.88
... x..
88 86

6... S. 68 ... 8 .X X .X ...

8..86. ...888. 6 8
see 6 8 .8. 8.. .8 8.6 6.a .8.
8. 88 4.84 8 .... 8 . 6 .. . 8 ..... 8 6......
8 ... 8.4.6686 6 88 .886 8 ....88 ........8
X..X......x X. .....X.. X.... x.
.
... 8 88.88 .8.. 88.. 8 ....... 48 ..... 8 844. .888. ..X X. X
x.I
..............

.8.4... 4 8 .884 ..... 8...........8.8 4.88888 . X X

.88. 8. 888. 8888888 .64424.8 ... 8 ..6 . 88.8 4
8.. 8 886 . 88. 8 8.. 4844444.... 8 ... 8 ... .. 86. 684. 48
..8.88 .888..
.... .. . 8 .. 8 .....
84.44.84442848
8842888
8 ..... 4 .88
88.4 8
xX. x........ ...........x
48.48
688.24
.
.
......
.... 884 . 8 ... 8888 .88.8 ... 848 .. 888. .... 88 .....8 8 8844 eX sz4X
.. 68. 88.8 .. 8 8. 48.8 88 ... 88 .. .8.4 8844 4w¢*vq
...............tb
*s X.o
.8.44. 88.88.. 88.8 .. 8.8.8 ... 6. 8 44
x.. .. X.. X..
.8. 888.8888 . 8 ..... 4 4 . 84.88..8 .8.. 8 X. ..X. ... ....X
.48.. 8 8.. 8 8 8.888 ...88 66.6. 8
444...... 6. 888. 8 6 ..... 8 88. 848 .. ...... 8848 6 .... X . .. .X .. .
848888 .... . 8 . 8 8.6. 44.4 ... 8 88.
.244... 8 6 .. 86 .... See 8 8 6 8.44442488 ...68.488 ...6B. 8.8
82448. .. 66. 88 8.6 8.. 8. .. .. 44442884. 888.84. . .48. 4
.288 . 848888 .. 888.8 .. 8. 8.. 8. 8.. 884448 8 8... S. X.
4888.8888... 6. .8... 824.8 ... 8 8 44... .8..... 8. 88. .48 ..x .x..X.
848... .88.88. 88.444 8 8. 8. 84 86 8. 8
.8 88 8. 248 .8 24.8 ... 48.
.... 8.. 8.24 .8 88 .8688.8.
......8....888.8...86 B 8.8 68 6 ..... 8.88 88.88 .6B. 4.888 xx. .X. X. X. X..
4.8.4484 ...... 8 .8.668 688. 488. 488 .8 8 88.88 . 4 88
.844448 ...... 88 88 . 8..86 8666444.8
(a) (b)
Fig. 9. Skeletons of the noisy chromosome image [Fig. 8(a)] using various techniques. (a) Thresholding. 2 indicates
points with value > M/2; 4 indicates points with M/2 > value > M/4; 8 indicates points with M/4 > value > M18. (b)
Thinning.

In the first line detection algorithm, all six of these condi- sets of skeleton points obtained in this way, using the six- and
tions were required to hold; in the second algorithm, only five-condition criteria, are displayed in Fig. 2(c) and (d) for the
five of the six conditions were required. Detection of lines chromosome image. These results are quite thin and curve-like.
in directions other than vertical is analogous; see [101 for As Table II shows, the number of skeleton points obtained
further details. We used eight directions in our experiments. using six-condition line detection is usually somewhat less
If P does not satisfy the conditions for any of the directions, than that obtained using thinning, while the number obtained
its new value is 0; if it satisfies them for one or more of the using five conditions is usually somewhat greater, roughly
directions, its new value is the highest of their values. the same as when thresholding alone is used. The skeletons
Since a point and its neighbors can respond to line detectors look much cleaner, but the reconstructions are not as good
in different directions, applying the line detection process to as those using thresholding alone, as we see from Figs. 6 and 7.
SA only once may yield lines that are somewhat thick, and Thus here again, as in the case of thinning, the extra computa-
may not eliminate small noise streaks. We can improve the tional cost and poorer results outweigh any slight improve-
results by iterating the process [ 101. In our experiments, the ments in the degree of compression.
six-condition line detector was iterated twice, and the five-
condition detector three times. C. Effects of Noise
This algorithm was applied in parallel (i.e., using the old To see how noise in the images affects reconstruction from
values of the neighbors) at every point P of the array of SA the MMMAT, we added noise (mean = 0, standard deviation =
values. Applying line detection has the advantage that points 5) to the chromosome picture, and then performed the recon-
not lying on ridge crests can be selectively eliminated. The struction process. The original and the reconstructed images

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
694 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981

numbers of points in these skeletons are only 42 and 177 (see

Table III). Fig. 11 shows that reconstructions from these small
x...
.....

...
x........
x........
numbers of points give fair approximations to the original
...... x. xxxx .. x. x.
x.X chromosome image, with much of the noise removed.
X.........
We see from these results that line detection (and thinning)
..................... X .............

................................. X..X.......................... x x
.X...
xx .xx. X...
seems to be more useful when applied after (semi) threshold-
ing, rather than before.
xxx. x.X. xx
X...xx
X X X
.~x.x.x.x.xx.x.
xxx. X IV. RECONSTRUCTION FROM THE GRADMAT
xx. xx
xxx
...X..... The GRADMAT skeleton is quite sensitive to the presence
of noise edges or irregularities in the region edges. In this
X X X X sB section we discuss possible ways to reconstruct an image
from the high score points of its GRADMAT.
Each image point p's GRADMAT value is a score based on
X .6Xb X the gradient magnitudes at pairs of points that have p as their
midpoint. If each point p stores the coordinates and gradient
~ xx x +xS¢.....................................
X magnitudes of all the pairs of points q making contributions
to its score, then one way to reconstruct an approximation
to the image is to "color" all the points between each pair of
q's with the corresponding gradient magnitude. In case a
X X point receives more than one gradient value, the maximum
.. X X X is used. However, the amount of information needed at each
....xx

X. X J6
X..x........X....X.X.....
9. xxx
X .X.x..........
xx X. ( X........ X..
point in order to do this is too large.
.x.n X .e X X
selection
.X..X
mh.x .x areshon. Alternatively, instead of trying to use all pairs of contribut-
X. .....x.....x..x..X. .X X. ing points, we can choose a radius r which provides the strong-
X X X xx
X ...x..X
X
X est response and color all the points in the disk of radius r
.X ..X ..X..x
X.
with the score of p. However, this would certainly produce
inF. X.
x..x .x.......... M/xx.
Thresholding tha thinning
w .X...x.xx...............
X. X...
.~~~~~~~~~c
objects that are too big, as too many points are colored in. A
..... xX......x
.X
.X .....X.
..xx....
modification is to color only a rectangular strip having p at its
center and length 2r where its width and orientation are deter-
mined by the responses from the points on the circle C of
X X .X X XX x
X.....
x.xx
X X radius r centered at p, i.e., all the points at distance r from p,
as follows. Divide C into 16 equal length arcs. For each pair
of opposite arcs, calculate the total score contribution from
the points on them. Let X be the sector angle corresponding
xd lin deeto eve thug
..t
use fe e poixts. esx to the longest consecutive sequence of arcs with nonzero con-
olg.x.
xi.xX ec tributions. The major axis of the rectangle is located in the
direction of X/2. Various widths have been tried in the re-
construction. Fig. 12(b) shows the reconstruction using width
2r sin (X/4). The objects are somewhat too big, and have
irregular boundaries. Fig. 12(c) shows the reconstruction
using width 2r sin (X/8); the objects are smaller, but even
more irregular. Fig. 12(d) is the reconstructed image using
rectangles of length r and width 1, i.e., a line joining the two
opposite edge points on C which give the maximal contribu-
tion to the score of p. The objects are now too small and
quite ragged. Thus we see that reconstruction from the
GRADMAT produces objects whose general shapes are good
approximations of the original objects, but the results are
apeaointl selectonimethood
using differnstrucskeeo are shromo- not as good as those using the MMMAT.
V. COMPRESSION
some picture with most of the noise removed. As we saw in the earlier sections, the MMMAT can be
Fig. 9 shows the skeletons of the image obtained by applying used for image compression, since we can often reconstruct
thresholding, thinning, and line detection to the S,& array ob- good approximations to images from a few A values at a few
tained from the noisy chromosome picture. As can be seen skeleton points. Actually, these selected MMMAT values
from Fig. 9(a), there are too few points with values >M12, provide more than just a compact approximation to the
and there are many isolated points with values between M12 image; they also provide basic information about the sizes
and M14 [compare Figs. 2(a) and 9(a)]. The skeletons ob- and constrasts of regions in the image (a high value Of Ak
tained from thinning and line detection contain curve seg- implies a high-contrast region of width 2k). Thus, image
ments that are not in the "centers" of the objects; rather, they compression is not the main reason for using the MMMAT.
~arise from points whose values are quite low, i.e., <M/4 [see
the points labeled "8" in Fig. 9(a)]. If we modify our skele-
Nevertheless, it is instructive to compare the degree of com-
pression obtained from the MMMAT to the degrees of com-
ton point selection procedure by applying the line detector or pression obtained using various image coding schemes.
the thinning operator to the SA array semithresholded at M14 Since we know the numbers of skeleton points used in each
(i.e., points with values <M/4 are set to zero), we obtain quite of our reconstructions (Tables I-III), this immediately deter-
reasonable skeletons such that most of the isolated points with mines the corresponding degrees of compression. For ex-
values >M/4 are no longer vart of the skeleton: see Fig. 10. ample, in Fig. 3(b), the numbers of skeleton points used to
This is particularly noticeable in the line detection case. The reconstruct the four images are 189, 582, 227, and 462, re-

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 198169
695

xx x x..I.
.....................

x............... ... ....

x......... x ... ..
x..... x.........
X. X
xx. xxxx.
x x.
~ ~ ~ x..

...
.x
X
x ....xx.....
X .. x.
..X. xx x.
x.. x.
x x
x.. .x .x. x
.. .xxxxxx X x.
x
X. ... .. X.
x. xx X
x .x X
X
X

x X .... x..... x.......

x.

x.. x
X ..x x...

xX.x ... X.. .. ... x....

....x x
X.
..x.. X
x X .. X ... x....
X.
x.. x xx ..x.x.... X.
X.
x. .. x
xxxx.. .x .. x... ... X. X
x x
x xxxx.
x...X
x x..
.xx. x.. .. X
x
xxx.
x xx......... x ...x
x

(a) (b)

Fig. 10. Skeletons of the noisy chromosome image obtained by applying (a) thinning operator and (b) nonlinear line
detector to the MMMAT sum picture semithresholded at M/4.

6 bits. This is a total of 22 bits per point, whereas the original

image gray. level. req'uired only 6 bits per point; but the origi'nal
(a) (b) image has 4096 points. The compressioni factor for each image
is thus

4096 X 6
(c) (d) (number of skeleton points) X 22
For the four images, the factors are thus 5.9., 1.9, 4.9, and 2.4.
Fig. 11. Reconstruction of the noisy chromosome imnage of Fig. 8(a).
Similar factors can be readily.computed for the other recon-
(a) Using skeleton points obtained by applying the nonlinear line struction experiments. Note that the four factors just com-
detector to the MMMAT sum picture semithresholded at M/4, and the puted correspond to bit rates of between' 1.0 and 2.5, bits per
largest A value at each of these points. (b) Same as (a) except that im'age point; suc'h rates are comparable with those typ-ic-ally
the three largest A values are used. (c), (d) Same as (a), (b) except obtained in' interpo'lative an'd transform coding schemes 1111.
that the thinning operator is used. The compression factor alone, of course, is inadequate to
determine the value of a coding scheme, since it tells us
spectively, and only one A value is used for each of the.se nothing about the quality of the reconstructed image. Sub-
points. In a 64 X 64 image, specifying the. c oordinates of jective evaluation, which we used informally in the preceding
a skeleton point requires 12 bits of information (6 per co- sections, is a good quality criterion for images intended for
ordinate), specifying which Ak value is used at each point human consumption; but it must he quantified by obtaining
requires about 4 bits, and specifying the A value itself requires quality ratings from a large population of human subjects. It

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.
 696 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. PAMI-3, NO. 6, NOVEMBER 1981

TABLE III varies along such an arc provides information about its shape
NUMBERS OF SKELETON POINTS OBTAINED FROM THE CHROMOSOME IMAGE (width, taper, etc.). Analogously, it should be possible to
OF FIG. 8(a) USING VARIOUS SELECTION TECHNIQUES extract certain types of "shape" information from an unseg-
mented gray level image by inspecting its generalized MAT.
Thresholding at M/4 257
Thinning 275 REFERENCES
Line detection 300 [1] H. Blum, "A transformation for extracting new descriptors of
Thinning after semithresholding at M/4 177 shape," in Models for the Perception ofSpeech and Visual Form,
Line detection after semithresholding at M/4 42 W. Wathen-Dunn, Ed. Cambridge, MA: MIT Press, 1967, pp.
362-380.
[2] A. Rosenfeld and J. L. Pfaltz, "Sequential operations in digital
picture processing," J. Ass. Comput. Mach., vol. 13, pp. 471-
494, 1966.
[3] G. Levi and U. Montanari, "A grey-weighted skeleton," Inform.
(a) Contr., vol. 17, pp. 62-91, 1970.
[4] N. Ahuja, L. S. Davis, D. L. Milgram, and A. Rosenfeld, "Piece-
wise approximation of pictures using maximal neighborhoods,"
IEEE Trans. Comput., vol. C-27, pp. 375-379, 1978.
[5] S. Wang, A. Rosenfeld, and A. Y. Wu, "A medial axis transforma-
(b) tion for grayscale pictures," Comput. Vis. Lab., Comput. Sci.
Center, Univ. of Maryland, College Park, MD, Tech. Rep. TR-
843, Dec. 1979.
[6] J. C. Mott-Smith, "Medial axis transformations," in Picture Pro-
cessing and Psychopictorics, B. S. Lipkin and A. Rosenfeld, Eds.
(c) New York: Academic, 1970, pp. 267-278.
[7] S. Peleg and A. Rosenfeld, "A min-max medial axis transforma-
tion," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3,
pp. 208-210, 1981.
[8] Y. Nakagawa and A. Rosenfeld, "A note on the use of local
min and max operations in digital picture processing," IEEE
(d) Trans. Syst., Man, Cybern., vol. SMC-8, pp. 632-635, 1978.
[9] C. R. Dyer and A. Rosenfeld, "Thinning algorithms for gray-
scale pictures," IEEE Trans. Pattern Anal. Machine Intell.,
Fig. 12. Reconstructions from the GRADMAT. (a) Original image. vol. PAMI-1, pp. 88-89, 1979.
(b) Reconstruction using rectangles. (c) Reconstruction using rect- [10] A. Rosenfeld and A. C. Kak, Digital Picture Processing. New
angies as in (b) but with sector angle reduced by a factor of 2. (d) York: Academic, 1976, pp. 310-314.
Reconstruction using lines. [11] W. K. Pratt, Digital Image Processing. New York: Wiley, 1978,
p. 729.
is customary instead to use crude objective measures of
quality, such as the mean-square error between the original
and reconstructed images [ 11], even though these are not Comments on "Nosing Around the Neighborhood:
very good predictors of judged quality. Let f be the original A New System Structure and Classification Rule for
image, f be the reconstruction, and let ,u, ,u be the means of
f, f, respectively; then the normalized mean-square error is Recognition in Partially Exposed Environments"
defined as PIERRE A. DEVIJVER
12((f - A) - ( t _ p ))2
z(f_ 1s)2
Abstract-Observations are made relative to the definition and gen-
where the denominator is the variance of f. For the four re- eration of a consistent training sample set. We show that the con-
constructions in Fig. 3(b), the values of this error are 0.71, sistency assumption is crucial to the method described in the above
1.01, 2.91, and 0.63, respectively. Note that the highest correspondence,1 and that, in general, the proposed solution to the
value, 2.91, is for the tank image, in which the reconstruction generation problem, namely k-NN editing, fails to produce consistent
has artifacts, as pointed out in the discussion of Fig. 1. sets.

VI. CONCLUDING REMARKS Index Terms-Classifiers with reject option, nearest neighbor tech-
Using a small amount of information, namely, a set of points niques, neighborhood census rule, optimum thresholds, pattern recog-
nition under partial supervision, recognition system design.
having high MMMAT values and a few components that make
strong contributions to these values, good approximations to
gray scale images can be reconstructed. Hence, these points
with their appropriate A values form a compact representa- In the above correspondence,' an extension of the basic
tion of the image. Both the MMMAT skeleton and the recon- k-NN classification rule is outlined to cope with partially
struction computation are relatively inexpensive and comput- exposed environments. The system has the capability of
able in parallel. The examples show that they give better learning originally unknown pattern classes. In short, the
approximations than reconstructions from the SPAN [4] or
GRADMAT. Manuscript received April 8, 1980; revised December 22, 1980.
The MAT is not just a compact representation of a binary The author is with the Philips Research Laboratory, B-1170 Brussels,
image; it also provides important structural and shape in- Belgium.
formation. The MAT points tend to lie on a set of arcs cor- 'B. V. Dasarathy, IEEE Trans. Pattern Anal. Machine Intell., vol.
responding to "lobes" of the set S, and the way the radius PAMI-2, pp. 67-71, Jan. 1980.

0162-8828/81/1100-0696$00.75 © 1981 IEEE

Authorized licensed use limited to: Georgetown University. Downloaded on January 11,2022 at 20:43:09 UTC from IEEE Xplore. Restrictions apply.