Peg-Free Hand Geometry Recognition

Using Hierarchical Geometry and Shape Matching

Alexandra L.N. Wong1 and Pengcheng Shi2

Department of Electronic and Electrical Engineering
Hong Kong University of Science and Technology

Abstract
We propose a feature-based hierarchical framework for
hand geometry recognition, based upon matching of geo-
metrical and shape features. Rid of the needs for pegs, the
acquisition of the hand images is simplified and more
user-friendly. Geometrical significant landmarks are ex-
tracted from the segmented images, and are used for hand
alignment and the construction of recognition features. The Figure 1. Three peg-fixed hand photos: deformed hand
recognition process is hierarchical in nature, and it employs
a Gaussian Mixture Model for the first group of features, shape (left), different placements of the same hand (middle
followed by a distance metric classification of the second and right).
group of features if necessary. The method has been tested
on a medium size dataset of 323 images with promising
results of 89% hit (true acceptance) rate and 2.2% false fix the hand placement during image acquisition. In general,
acceptance rate. the use of pegs introduces two problems. First, as shown in
Figure 1, pegs will almost definitely deform the shape of the
hand. Second, even though the pegs are fixed, the fingers
may be placed differently at different instants, and this
1 Motivation causes variability in the hand placement as illustrated in
Figure 1. These problems will degrade the performance of
Biometrics is getting more and more attention in recent hand geometry verification because they adversely affect
years for security and other purposes. So far, only finger- the features.
print has seen limited success for on-line security check,
since other biometrics verification and identification sys- In this paper, we present a novel recognition system that
tems require more complicated and expensive acquisition uses a peg-free hand image acquisition and a hierarchical
interfaces and recognition processes. geometry (length and width of the fingers) and shape (length
and location of the fingertip regions) matching for user
Hand geometry has long been used for biometric verifi- identification. Without the needs for pegs, the system has
cation and identification because of its acquisition simple acquisition interface. Users can place their hands in
convenience and good verification and identification per- arbitrary fashion and can have various extending angles
formance [1, 2, 3, 4, 5]. From anatomical point of view, between the five fingers. Geometric significant shape
human hand can be characterized by its length, width, landmarks are extracted from the segmented images, which
thickness, geometrical composition, shapes of the palm, and are further used for hand alignment and hand geometry
shape and geometry of the fingers. Earlier efforts have used recognition. Our system offers a simple and friendly ac-
combinations of these features for recognition with varying quisition interface and a good recognition rate with low
degrees of success. Traditionally, pegs are almost always false acceptance rate.
used to fix the placement of the hand, and the length, width
and thickness of the hand are then taken as features [2, 3, 4].
In [5], the outline of the hand is extracted and is represented 2 Methodology
by a group of salient points, which serve as features in the
verification process. 2.1 Hand Image Acquisition
A commercial flatbed scanner was used to acquire the
Even though the pegs often cause deformation on hand
geometry, which in turn reduces the accuracy in feature hand images. Randomly placed hands of the participants
extraction and further analysis, they are used extensively to were scanned with dark background using 150dpi scanning

Address: Department of Electronic and Electrical Engineering, Hong Kong University of Science and Technology

Clear Water Bay, Kowloon, Hong Kong. Tel: +852 2358 8529 Fax: +852 2335 0194 E-mail: [email protected]; 2 [email protected]
 Figure 3. Landmark extraction and hand alignment. Top:
Figure 2. Varying poses of hand placements and finger exten- extracted hand boundary and landmarks. Bottom: aligned
sion angles from peg-free acquisition. Top: typical (left), palm images such that all the middle fingers are vertically upward.
rotation (middle), and widely extended fingers (right). Bottom:
corresponding binary images.
of the reference axis:
quality. The fingers must be clearly separated from each  ∆y 
θ = tan −1  
 ∆x 
other in the image in order to obtain a complete hand shape.
Examples of acquired hand images are shown in the top row
of Figure 2. The reference axis is aligned to be upward vertical in order
to align the hand image with the database template to the
2.2 Landmark Extraction and Hand Alignment same orientation, and finger-wise correspondence between
Since there is clear distinction in intensity between the the input image and the template is established, using the
hand and the background by design, a binary image is ob- rotation equation:
tained through thresholding (Figure 2) and hand boundary is
easily located afterwards (Figure 3). Geometrical landmarks,  xnew   cosθ sin θ   x 
i.e. the fingertip points and the valley points between adja-  y  =  − sin θ cosθ   y 
cent fingers, are extracted by traveling along the hand  new    
boundary and searching for curvature extremities.
The aligned images are shown in bottom row of Figure 3.
A border following algorithm is applied to locate the
boundary of the binary image [6]. For each boundary point Compared to the traditional peg-fixed acquisition, our
( x, y ) , its curvature K is estimated from: method is less sensitive to the placement of the hand, and is
more user-friendly and convenient.
( x′y′′ − y′x′′ )
2

K=
( x′ + y ′2 )
2 3/2 2.3 Feature Selection
Salient features are necessary for the robust recognition
where y’, x’, y’’, and x’’ is the first- and second-order co- of hand geometry. As shown in Figure 4, the lengths of the
ordinate changes along y and x, calculated from the five fingers (L1 to L5), the widths of the four fingers (except
neighboring points of ( x, y ) . From the calculated curva- the thumb) at two locations (W1 to W 8), and the shape of the
ture information, nine landmarks are located which are three middle fingertip regions (S1 to S3) are selected for our
curvature extremities, as shown in top row of Figure 3. system.

These landmarks, instead of the traditional pegs that often Finger baselines: As illustrated in Figure 4, four be-
cause hand deformation [2, 3, 4, 5], are used to align the tween-finger valley points are located on the hand boundary.
palm to vertical displacement. A reference point and a ref- • Because the middle-finger is the only finger which does
erence axis are found first. The middle finger baseline is not have large spatial variations of its valley points for
created by connecting the second and third valley points. different placement of the hands, the middle-finger
The mid-point of this baseline is used as the reference point baseline is formed by connecting the second and third
of the palm rotation, and the axis from the reference point to (count from the left) valley points.
the middle fingertip landmark is selected as the reference • The ring finger has two valley points (third and fourth)
axis. A rotation angle is thus calculated from the orientation as well. However, since the relative heights of these two
 valley points are more sensitive to hand placement, it is
unstable to use these two valley points to form the
baseline for the ring finger.
• The baselines for the thumb, the index finger, the ring
finger, and the little finger are formed in the same
fashion. We assume that the two end points of each
baseline have the same distance from the respective
fingertip point. Using one of the respective valley
points as one of the end points (first valley point for
thumb, second for index, third for ring, and fourth for
pinky), we locate the other end point by searching for
the point which has the same distance from the fingertip
at the another side of the boundary of the finger. The
baselines are formed afterwards by connecting pairs of
end points.
Fig.4. Definitions of the hand features: finger length Li
(i=1,…,5), finger width Wj (j=1,…,8), and fingertip
Finger lengths: For each finger, the fingertip point and the region S k (k=1,…,3).
middle point of its baseline determine its finger length Li,
i=1,2,3,4.
r
Finger widths: For each finger except the thumb, the first
(
p x/u = )
 1 r uur r uur 
( ) ( )
M
c
∑ ( 2π )
T
finger width Wi, i=1, 3, 5, 7, is the width at the middle point i
exp  − x − µi Σi−1 x − µi 
Σi  2 
L/2 1/2
of the finger length, and the second finger width Wj, j=2, 4, i =1

6, 8, is the width at one-eighth way (with respective to the

finger length) from the fingertip. where ci is the weighting coefficient of each of the Gaus-
sian model, µ i is the mean vector of the each model, Σ i is
Fingertip regions: For the index, middle, and ring fingers, the covariance matrix of each model, M is the number of
the top one-eighth portion of the fingers are defined as the the models, and L is the dimension of the feature vectors.
fingertip regions. Each fingertip region is represented by a The GMM is trained by the training images of each person,
set of ordered boundary points (between 50 to 90). The and the characteristic parameters are acquired for each user
bottoms of the fingertip regions are coincident with W2, W4, in terms of ci , µi , and Σ i of size 1x M . The probability
and W6 respectively. Similar to the palm alignment, the r r
p( x / u) of an input test image x belongs to class u can
fingertip regions are also aligned by the method we men- then be calculated with the above equation.
tioned in section 2.2. The middle point of the bottom line is
found as the reference point and the axis between middle If the image is determined to pass a preset threshold of the
point and the fingertip top point is the reference axis. As GMM probability estimation of the template (we set the
long as the rotation angle is found, the fingertip regions are passing probability threshold to be 0.5 for our experiment),
aligned. The coordinates of the fingertip points are recorded, the image is further processed with the group #2 features in
and re-sampling may be required in order to match the the following step.
testing image fingertip points and the template fingertip
points. Group #2: The mean distributions of the group #2 features,
the point distributions of the three fingertip regions, are used
2.4 Hierarchical Recognition as the fingertip templates for each user in the database. The
The extracted hand features are put into two groups. number of points on the input fingertip region S k and on the
Group #1 consists of the 13 finger lengths and widths of all corresponding template TS must be the same, and linear
fingers, and group #2 consists of the 3 fingertip regions. A k

re-sampling of S k is needed if the two are different but the

sequential recognition scheme is adopted, using these two difference is within an acceptable range (10% in our im-
groups of features. plementation). If the difference is greater than 10%, it is
rejected.
Group #1: A Gaussian mixture model (GMM), based upon
statistical modeling and neural networks [2, 3], is used to For each fingertip point ( xi , yi ) on S k , it is classified as
obtain the characteristic parameters for the group #1 fea- either a failure point or a hit point, which is determined by
tures of each person: an Euclidean distance measure between ( xi , yi ) and its
corresponding template point ( xitemp , yitemp ) on TS : k
 only using Group #1 +
Group #1 Group #2
Hit Rate 1 0.8889
FAR 0.1222 0.022
Table 1. The hit rate and FAR for the testing images
using group #1 feature and GMM only, and using the
hierarchical recognition scheme of group #1 and group
#2 features

(x − x ) + ( y − y )
2 2
Di = i itemp i itemp

If the distance is larger than certain threshold (we have set it

to 2 pixels), it is a failure point. Otherwise, it is a hit point.
Fig.5. ROC curve by adjusting GMM passing threshold.

For all the fingertip points of the three fingertips, we cal-

culate the percentage of failure points. If the percentage is the hit rate drops to 0.8889 and the FAR decreases to 0.022.
higher than another threshold (say, 10%), we declare that the For security system, low FAR is often essential, even at the
input hand image does not correspond to the template and cost of lower hit rate.
should be rejected. Otherwise, it is recognized as the valid
user represented by the template. Using the second set of data, which contains 35 hand
images of impostors who are not in the training set, we
observe the distributions of the GMM probability measure,
3 Experiment and Discussion the input-template point number difference, and the failure
percentage. From this particular test, we conclude that the
323 right-hand images, grouped into two datasets, are used
GMM threshold plays an important role in reducing the
in our testing experiment of the system. The first set of data
FAR, and the thresholds for the point number difference and
contains totally 288 training and testing images of 22 people,
the failure percentage further make sure the false acceptance
12 to 15 images each. After the extraction of the landmarks
is minimized. Figure 5 shows a ROC curve which reflects
and alignment of the hand orientation, recognition features
the impact of the GMM threshold.
for each input image are computed and grouped. For each
person, 9 images are used for GMM training and template
Compared to peg-fixed hand geometry recognition sys-
calculation, and the rest are used as test images (90 in total).
tems, our system has an acceptable hit rate (true accept rate)
The second set of the data contains 35 test images from 7
of 88.89% with low FAR at 2.22%. In [2,3], GMM is used as
people, each with 5 images. This set of data is used to test
the training model for the length and width of the hand
the effects of the thresholds on the hit rate and false accept
geometry, and has achieved higher hit rate at 96% but with
rate (FAR).
higher 4.9% FAR as well. Further, it has constrained hand
placement because of the need for pegs.
For an input testing hand image from the first dataset, its
group #1 features are calculated. If this group of features
The hand geometry recognition system can be further
possesses a passing probability on the GMM testing (greater
combined with other hand biometrics for better performance.
than 0.5) against any template out of the 22 templates in
Since the acquisition is done by a commercial flatbed
total, its group #2 features are constructed and we move on
scanner, by adjusting the scanning resolution and brightness,
to the next stage of recognition. We set the threshold on the
we can acquire the palm print from the hand images as well.
difference of the number of points heuristically to be 10% as
A multi-biometrics recognition system is under develop-
mentioned earlier. If the input image can pass this test, we
ment.
move to the final step where the distance for hit point is set
at 2 pixels, and the failure percentage threshold is set at 10%.
References
If the image can pass this test, it is recognized as the user
[1] J. Ashbourn, Biometrics: Advanced Identity Verification,
represented by this template.
Springer-Verlag, New York, 2000.
[2] R. Sanchez-Reillo, “Hand geometry pattern recognition
Table 1 shows the hit rate for using group #1 features only
through Gaussian mixture modeling”, in 15th International
and for using both group #1 and group#2 with heuristic
Conference on Pattern Recognition, Vol.2, Sep, 2000. Pp.
thresholds. For group #1 only, the hit rate is 1 and the FAR is
937-940.
0.1222, which is quite high as well. If using both group #1
[3] R. Sanchez-Reillo, C. Sanchez-Avila, and A. Gon-
and #2 features, as in our hierarchical recognition system,
zalez-Marcos, “Biometric Identification Through Hand
Geometry Measurements”, IEEE Transactions on Pattern
Analysis and Machine Intelligence, 22(10), 2000. Pp:
1168-1171.
[4] A.K. Jain, A. Ross, and S. Pankanti, “A Prototype Hand
Geometry-based Verification System”, 2nd International
Conference on Audio- and Video-based Biometric Person
Authentication, Mar, 1999. Pp. 166-171.
[5] A.K. Jain and N. Duta, “Deformable matching of hand
shapes for verification”, in IEEE International Conference
on Image Processing, Oct, 1999. Pp. 857-861
[6] A. Rosenfeld and A.C. Kak, Digital Picture Processing,
Academic Press, San Diego, 1982.