2010 International Conference on Pattern Recognition

Triangle-Constraint for Finding More Good Features

Xiaojie Guo and Xiaochun Cao School of Computer Science and Technology, Tianjin University, China {xguo, xcao}@tju.edu.cn Abstract
We present a novel method for nding more good feature pairs between two sets of features. We rst select matched features by Bi-matching method as seed points, then organize these seed points by adopting the Delaunay triangulation algorithm. Finally, we use Triangle-Constraint (T-C) to increase both number of correct matches and matching score (the ratio between number of correct matches and total number of matches). The experimental evaluation shows that our method is robust to most of geometric and photometric transformations including rotation, scale change, blur, viewpoint change, JPEG compression and illumination change, and signicantly improves both number of correct matches and matching score.

(a) Matching result by OMM

(b) Matching result by TCM

1. Introduction
There are many tasks in computer vision which require effective techniques for nding more good (i.e. discriminative and accurate) correspondences between two sets of features, such as symmetry detection [5], wide baseline matching [9] and building panoramas [2]. And numerous feature extraction schemes have been proposed, like Harris corner detector [3], SIFT [7] and SURF [1], to capture distinctive and stable features. However, similarity measurements of features are limited for the existing schemes. The most popular measurement is to adopt dot product between descriptors and then compare the ratio between the rst and the second nearest neighbors against a predened threshold to decide whether they are matched or not (we denominate this measurement as the original matching method, OMM). Even though this strategy reduces inuences from most of geometric and photometric transformations, the drawback is that it sacrices a great many of
This work was supported by NSFC (No. 60905019, 50735003), SRF for ROCS, SEM, Open Projects Program of NLPR, Tianjin University 985 research fund, and SKL of PMTI.

Figure 1. Matching result comparison between the OMM and our method T-CM with the same matching threshold. To better illustrate, we only show 1/6 of correct matches in each feature set (green +s) randomly. There are 39 hits (matching score 85.97%) in (a) and 216 hits (matching score 93.11%) in (b). features that should be on the list of correct matches, which signicantly weakens the power of feature extraction schemes. Currently, a few improvements of similarity measurement have been developed, such as spectral technique [6] based on pairwise constraints, the method called Circular Earth Movers Distance (CEMD) [8] that relied on an adaptation of Earth Movers Distance, and the solution of scale and rotation invariant matching [4] which can efciently and accurately match features. These measurements improve the matching accuracy, but they neglect the importance of number of correct matches. The neglect inspires us to propose an effective similarity measurement, called Triangle-Constraint Measurement (T-CM), designed for increasing both the matching score and the number of correct matches to provide the tasks based on local image features with more good features (see Fig. 1).
1393 1397

1051-4651/10 $26.00 2010 IEEE DOI 10.1109/ICPR.2010.344

(a) Illustration of the Delaunay algorithm.

a a'

the Delaunay triangulation algorithm on the seed points from IA , where the triangles are joint but not overlapped in IA . The seed points from IB are organized according to the same order with those from IA due to the one-toone relationship between the seed points from IA and IB . An example of the Delaunay triangulation is shown in Fig. 2 (a). Note that the triangulation in IB might be not exactly the same with that in IA due to the false positive matches survived from the Bi-matching, e.g. the seed point pair 1 in Fig. 2 (a). The Triangle-Constraint described later is suitable to solve this problem.

c b

2.3
b' c'

Triangle-Constraint

(b) Illustration of the Triangle-Constraint. Figure 2. Example of the Triangle-Constraint Measurement.

2 Triangle-Constraint Measurement
In this section, we detail the ow of TriangleConstraint Measurement including seed point selection, organization of seed points and Triangle-Constraint.

2.1 Seed point selection

We adopt the Bi-matching for selecting seed point pairs (stable matches) for the rest of tasks, which is based on the observation that true positive matches of distinctive features between the reference image (IA ) and the matching image (IB ) are more likely to be bimatched (from IA to IB and inversely). In other words, if the relationship between matches is unidirectional, then it is probably to be either incorrect or unstable that needs to be discarded. Therefore, we assume that if there are few stable matches, the corresponding images are very likely to be irrelative or distorted signicantly. The features nally matched by the bi-matching method are considered as seed points, e.g. the yellow points shown in Fig. 2.

So far, we have obtained the triangles in IA with their correspondences in IB . We take the case shown in Fig. 2 (b) for example to explain how the TriangleConstraint works. For generalizing the case, we replace the vertexes 2, 6 and 8 by a, b and c (a, b and c) respectively for the left (right) triangle. The features are limited by the triangle abc and a b c , i.e. the features out of the triangles (the red *s in Fig. 2 (b)) are not considered in current case. In other words, only the sets of features inside the triangles (the s in Fig. 2 (b)) denoted as PA and PB are involved. For each feature Pi PA (e.g. the feature marked by dashed red circle) in abc , the relationship between Pi and the vertexes of abc is Pi = a + (b a) + (c a), (1)

where and are the scale coefcients of the vector (b a) and (c a) respectively. Fortunately, the three vertexes of abc are known and the parameters K can be computed easily by xa K = = ya 1 xb yb 1 1 xc xi yc yi , 1 1

(2)

2.2 Organization of seed points

As is known, triangle is the simplest and most stable polygon. Given the vertexes, the triangle is established without extra ordering conrmation of the vertexes. Therefore, we use the Delaunay triangulation algorithm to organize the seed points obtained from the seed point selection stage. It integrates the isolated points as triangles and maximizes the minimum angle of all the angles of the triangles in the triangulation, i.e. it tends to avoid skinny triangles. We only implement

where = 1 . The same parameters for the relationship between the estimated point Pe (the black point in Fig. 2 (b)) and the vertexes of a b c in IB hold true if the triangles are true positive correspondence. As a consequence, we estimate the coordinates of Pe via xe xa xb xc ye = ya yb yc . (3) 1 1 1 1 To be more robust to noises and distortions, we dene the area around the Pe within R pixels (in our experiments, R = 3) as candidate area and the features in this area as candidate features (the green s in Fig. 2 (b)). We denote C as the set of candidate features.

1394 1398

The similarity score between the Pi and the candidate feature Cj is measured by sj = 1.5(distj /R) DT Dcj , (j = 1, 2, ..., |C|), (4) i where distj is the Euclidean distance between the Cj and the Pi , Di and Dcj are the descriptors for the Pi and the Cj respectively, and | | stands for the cardinality of a set. If the maximum score of all the features in C is greater than a predened threshold , the corresponding feature pair is considered as temporary match, and there is at most one match for every feature. In our experiments, we use = 0.4 for all the evaluations. After processing all the features from PA , there is a set T containing all the temporary matches for PA and PB . The temporary matches are accepted as nal matches if they satisfy |T | > min{|PA |, |PB |}, (5)
2

(a) Leuven

(b) UBC

(c) Bikes

(d) Trees

(e) Boat

(f) Bark

(g) Graffiti

(h) Wall

Figure 3. Example images from eight categories of INRIA dataset. Evaluation criterion. The criterion of our evaluation is based on the number of correct matches and the matching score (the ratio between the number of correct matches and the number of all matches). Since the experiments are based on the same feature extraction scheme, the numbers of correspondences for the OMM and our method have no difference. Therefore, this criterion is sufcient to report the performance difference between the measurements clearly. Note that a match is dened correct if the distance between the accurate location (projected by the provided homography for each pair of relative images) and the estimated location (computed by the OMM or our method) is less than 6 pixels, incorrect otherwise. Relative image pair matching. Every category of INRIA dataset contains six images capturing the same scene with geometric or/and photometric transformations. Figure 4 shows the results in terms of the number of correct matches and the matching score for different categories. As can be seen, the number of correct matches of T-CM is considerably more than that obtained by the OMM for all eight categories. With respect to the matching score, almost all acquired from the T-CM are higher than those from the OMM except for the image sequences of Trees and Wall as shown in Fig. 4 (d) and (h). As for Trees, there are a huge amount of features (the most number of correct matches reaches about 8000 by the T-CM) that increase the possibility of accidentally considering incorrect matches as correct. Note that we dont draw the matching score for the last pair of the Wall sequence, since no seed point is obtained from the Bi-matching step and consequently there is no match between the images by using our method (only 3 unstable correct matches by the OMM), i.e. the matching score is undened ( 0 ). 0 Irrelative image pair matching. From another perspective, a good measurement has to be robust to noisy images. Therefore, there is an additional pair of irrelative images taken by ourselves to test and verify the validity of our method. As shown in Fig. 5, there are 22 hits by the OMM and 0 hit by our method.

where = 0.3 in our experiments. Otherwise, the triangle pair abc and a b c and the temporary matches between them are discarded. This straightforward strategy is based on the observation that if the pair of the triangles is true correspondence, they will approximately even perfectly satisfy the Triangle-Constraint due to the accurate vertexes, or they will be greatly different. And if all the relative triangles of a vertex are discarded, the vertex is then removed from seed points. This strategy is able to reduce wrong matches in the TriangleConstraint step and lter out the false positive matches survived from the Bi-matching step. The Triangle-Constraint Measurement is invariant to translation, rotation and scale transformations due to the accurate features obtained from the Bi-matching step. It is also invariant to afne transformation and robust to partially perspective transformation, since the TriangleConstraint itself is afne invariant.

3 Experimental Evaluation
Our method is suitable to apply to almost all the local image features. In this work, we use the popular SIFT as the example. Data set. We evaluate our method on real images with different geometric and photometric transformations and for different scene types. Figure 3 shows the rst image (the original image) in every category of INRIA dataset1 (Leuven, illumination change; UBC, JPEG compression; Bikes, blur, structured scene; Trees, blur, textured scene; Boat, scale change, structured scene; Bark, scale change, textured scene; Grafti, viewpoint change, structure scene; Wall, viewpoint change, textured scene).
1 http://www.robots.ox.ac.uk/vgg/research/afne/

1395 1399

2500

nb of correct matches

nb of correct matches

nb of correct matches

2000 1500 1000 500 2

4000 3000 2000 1000 0 60 TriangleConstraint OMM 70 80 90 100

2500 2000 1500 1000 500 0 2 3

nb of correct matches

TriangleConstraint OMM

5000

3000

TriangleConstraint OMM

10000 8000 6000 4000 2000 0 2 3

TriangleConstraint OMM

Decreasing light
100 100

JPEG compression %
100

Increasing blur
100

Increasing blur
TriangleConstraint OMM

Matching score %

Matching score %

Matching score %

99.5 99 98.5 98 97.5 2 TriangleConstraint OMM 3 4 5 6

99 98 97 96 60

98 96 94 92 2

Matching score %
5 6

95 90 85 80 2

TriangleConstraint OMM 70 80 90 100

TriangleConstraint OMM 3 4

Decreasing light

JPEG compression %

Increasing blur

Increasing blur

(a) Leuven
6000

(b) UBC
4000

(c) Bikes
2000

(d) Trees
10000

nb of correct matches

nb of correct matches

nb of correct matches

5000 4000 3000 2000 1000 0 1 1.5

nb of correct matches

TriangleConstraint OMM

TriangleConstraint OMM

TriangleConstraint OMM

3000 2000 1000 0 1

1500 1000 500 0 20

8000 6000 4000 2000 0 20 30

TriangleConstraint OMM

2.5

30

40

50

60

40

50

60

Scale changes
100 100

Scale changes
100

Viewpoint angle
100

Viewpoint angle

Matching score %

Matching score %

Matching score %

99.95 99.9 99.85 99.8 99.75 99.7 1 TriangleConstraint OMM 2 3 4

90 80 70 60 1

80 60 40 20 0 20 TriangleConstraint OMM 30 40 50 60

Matching score %

95 90 85 80 75 20 TriangleConstraint OMM 30 40 50 60

TriangleConstraint OMM 1.5 2 2.5 3

Scale changes

Scale changes

Viewpoint angle

Viewpoint angle

(e) Boat
categories from INRIA dataset.

(f) Bark

(g)Grafti

(h) Wall

Figure 4. Experiment results of number of correct matches (upper ones) and matching score (lower ones) for different

References
[1] H. Bay, T. Tuytelaars, and L. Van Gool. Surf:speeded up robust features. In ECCV, pages 404417, 2006. [2] M. Brown and D. Lowe. Recognising panoramas. In ICCV, pages 12181227, 2003. [3] C. Harris and M. J. Stephens. A combined corner and edge detector. In Alvey Vision Conference, volume 20, pages 147152, 1988. [4] H. Jiang and S. Yu. Linear solution to scale and rotation invariant object matching. In CVPR, pages 24742481, 2009. [5] S. Lee and Y. Liu. Curved glide-reection symmetry detection. In CVPR, pages 10461053, 2009. [6] M. Leordeanu and M. Hebert. A spectral technique for correspondence problems using pairwise constraints. In ICCV, volume 2, pages 14821489, 2005. [7] D. Lowe. Distinctive image features from scale-invariant keypoints. IJCV, 60(2):91110, 2004. [8] J. Rabin, J. Delon, and Y. Gousseau. Circular earth movers distance for the comparison of local features. In ICPR, pages 14, 2008. [9] T. Tuytelaars and L. Van Gool. Matching widely separated views based on afne invariant regions. IJCV, 1(59):6185, 2004.

Figure 5. Irrelative image pair. There are 22 hits by

the OMM and 0 hit by our method.

4 Conclusion
Similarity measurement is the crucial part for numerous tasks built on local image features. There are two factors for a good measurement, i.e. the matching accuracy and the number of correct matches. Unfortunately, existing methods hardly take in consideration both two factors. We have proposed a new measurement to improve the matching accuracy and the number of correct matches simultaneously. Experimental results have demonstrated that our method outperformed the OMM in terms of matching relative and irrelative image pairs.

1396 1400