Saudi Arabian License Plate Recognition System

Muhammad Sarfraz, Mohammed Jameel Ahmed, and Syed A. Ghazi

Department of Information and Computer Science,
King Fahd University of Petroleum and Minerals,
Dhahran – 31261, Saudi Arabia.
{sarfraz, jameel21, aghazi}@ ccse.kfupm.edu.sa

Abstract research projects include Esprit 5184 Locomotive [10],

software architecture thru DLL’s [1].
A License Plate Recognition (LPR) System is one kind The steps involved in recognition of a license plate
of Intelligent Transport systems and is of considerable are Image acquisition, License plate extraction,
interest because of its potential applications to areas such Segmentation, and Recognition. Image acquisition is the
as highway electronic toll collection, Traffic Monitoring first step in an LPR system. The current literature
System and so on. This paper proposes an automatic discusses different image acquisition methods. Naito et.
license plate recognition system for Saudi Arabian license al. [8] developed a sensing system, which uses two CCDs
plates. The system captures the images of the vehicles (Charge Coupled Devices) and a prism to split an incident
with a digital camera. An algorithm for the extraction of ray into two lights with different intensities. Salgado et.
license plate has been designed and an algorithm for al. [10] used a sensor subsystem having a high resolution
segmentation of characters is proposed. The performance CCD camera supplemented with a number of new digital
of the system has been investigated on real images of operation capabilities. The proposed system uses a high
about 610 vehicles captured under various illumination resolution digital camera for image acquisition.
conditions. Recognition of about 95% shows that the License plate extraction is the key step in a LPR
system is quite effective. system, which influences the accuracy of the system
significantly. Different approaches for the extraction of
the license plate depending upon the back ground color of
1. Introduction the image are presented in [13]. Hontani et. al. [4]
proposed a method for extracting characters without prior
Vehicle license plate recognition is one form of knowledge of their position and size in the image. Kim et.
automatic vehicle identification system. Real time LPR al. [5] used two neural network-based filters and a post
plays a major role in automatic monitoring of traffic rules processor to combine two filtered images in order to
and maintaining law enforcement on public roads. This locate the license plates. Park et. al. [9] devised a method
area is challenging because it requires an integration of to extract Korean license plate depending on the color of
many computer vision problem solvers, which include the plate. The proposed approach uses matching of
object detection and character recognition [5]. The vertical edges. This approach involves three steps,
automatic identification of vehicles by the contents of vertical edge detection, filtering and vertical edge
their license plates is important in private transport matching.
applications. There are many applications of such In the Segmentation phase, individual characters are
recognition systems, some of them are border crossing isolated from the license plate. Various approaches have
control, automatic parking attendant, speed limit been proposed in the literature. Nieuwoudt et. al. [7] used
enforcement, security, and customer identification region growing for segmentation of characters. Morel et.
enabling personalized services [1]. al. [6] used partial differential equations (PDE) based
There are multiple commercial license plate technique for image segmentation. The proposed
recognition systems available in the current literature approach for segmentation is based on vertical projections
[12]. Applications such as SeeCar, Perceptics, and on the extracted license plate and then counting the
Pearpoint are commercially available and some of the number of black pixels in each column.

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE
 Recognition of characters is the last phase in the LPR ™Using a video camera and a frame grabber
system. A wide variety of approaches have been (capture card) to select a frame.
considered for individual character recognition. Cowell
et. al. [2] discussed the recognition of individual Arabic
and latin characters. Their approach identifies the
characters based on the number of black pixel rows and The 6 Characters of
the License Plate
columns of the character and comparison of those values Character Identification
to a set of templates or signatures in the database. Cowell
et. al. [3] used thinning of Arabic characters to extract
6 Segmented Images
essential structural information, which may be later used with Characters
for the classification stage. Mei Yu et. al. [13] and Naito Character Isolation
et. al. [8] used template matching. In the proposed
Sub Image Containing
system, recognition is done using template matching License Plate
based on hamming distance between the characters.
The paper is organized as follows, Section 2 provides License Plate Extraction
an overview of the overall system, listing different stages Image with License
in the LPR system. The image acquisition phase is Plate
explained in Section 3. Section 4 discusses the proposed
method for license plate extraction. Section 5 gives a brief Acquiring an Image
description of the segmentation into individual characters Input Image to the
Camera
using vertical projection with pixel count. Section 6 deals
with recognition of characters based on the hamming
distance of each character. Section 7 discusses
experimental analysis and the paper is concluded in Figure 1: The Typical Structure of a License
Section 8. Plate Recognition System.

2. Structure of Typical LPR System The first method, using a conventional analog
camera, is clearly not appropriate for the LPR system
The system presented is designed to recognize since it is time consuming, tedious, and impractical. The
second method which uses a digital camera is more
license plates from the front and rear of the vehicle. Input
practical, cost effective and reliable. The third one uses a
to the system is an image acquired by a digital camera,
video camera with a frame grabber which could be used
containing the license plate and its output is the
in real life system to make the system automated, and is
characters on the license plate. The system consists of
suitable for real time processing.
four modules: Image acquisition, License plate extraction,
In the proposed system a high resolution digital
Segmentation and Recognition of individual characters.
camera is used to acquire the image. The acquired image
The structure of the system is shown in Figure 1.
is first converted to gray scaled image. Conversion to
The first task acquires the image of a vehicle
gray scale facilitates the extraction of the license plate.
containing the license plate. The second task then extracts
The gray-scaled image is shown in Figure 2(a).
the region that contains the license plate. The third task
isolates the six characters (consisting of three letters and
three numerals) and the last task identifies the individual 4. License Plate Extraction
characters.
License plate extraction is the key step in an LPR
3. Image Acquisition system, which influences the accuracy of the system
significantly. The goal of this phase is, given an input
image, to produce a number of candidate regions, with
This is the first phase in the LPR system. There are
high probability of containing a license plate. In the
basically following three ways of acquiring an image [2].
adopted approach, extraction of the license plate is
They are as follows:
divided into three steps which are explained in the
™Using a conventional analog camera and a
following subsections.
scanner.
™Using a digital camera.

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE
 (a) (a)

(b) (b)

(c) (c)

(d) (d)

(e) (e)

(f) (f)

(g) (g)

Figure 2. Results Showing Different Stages in a License Plate Recognition System (a) Original Gray
Scaled Image, (b) Vertical Edges, (c) Vertical Edge Regions, (d) Extracted License Plate, (e) Cropped
License Plate, (f) Extracted Plate in Binary Form, (g) Histogram depicting the vertical projections on (f).

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE
 4.1 Vertical Edge Detection of identifying the characters, it is preferable to divide the
extracted plate into six images. This is done because the
For the transformed gray-scaled image, its Saudi Arabian license plates consist of 6 characters, with
corresponding vertical edges are detected using Sobel or 3 letters and 3 numerals.
Prewitt edge detectors. The proposed system uses Sobel For the segmentation phase the proposed strategy
edge detector because it shows better results. The used is based on vertical projections and counting the
threshold used by the edge detector is dynamic because number of pixels in each column. In the proposed strategy
the system takes an automatic value from the algorithm. first the upper part (about 30%) of the extracted license
The Sobel edge detector uses a 3x3 mask, which is plate is cropped to remove . This is shown in
applied on the input image to give the resultant edged Figure 2(e). (Presence of bolts and logos in the license
image. Figure 2(b) shows an image with vertical edges. It plates may cause problem. Since the bolts are usually on
is observed that most of the vehicles usually have more the upper part of the plate, they are removed in the
horizontal lines than vertical lines [13]. To reduce the process of cropping and the license plates doesn’t contain
complexity of algorithm, the vertical edges are detected. any logos). The image is then converted into binary form
If two of the vertical edges are detected correctly, four (as shown in Figure 2(f)) and vertical projections are
corners of the license plate can then be located. made. Then, the number of black pixels in each column is
counted, and a histogram is plotted as shown in Figure
4.2 Filtering 2(g). The characters are segmented depending on the
transition from a crest to its corresponding trough. To
avoid unnecessary segmentation some threshold is taken.
Filtering is basically used to remove objects that do
not satisfy some specific features. In the proposed
approach, seed-filling algorithm by smith [11] is used to 6. Character Recognition
filter the unwanted objects. After filtering, the result is
shown in Figure 2(c). The process of filtering starts by This is the last phase in the LPR system. This phase
first detecting all the regions. A group of white pixels is is divided into two stages:
called a region if they are eight-connected pixels. ™Normalization of Individual characters.
Once all the pixels of a region are marked, the region ™Recognition using Template Matching.
is checked for its size and shape. Through filtering our
target is to select the regions that can serve as possible 6.1 Normalization
license plate boundaries and discard the others by filling
black colors in their place. In this phase, first the extracted characters are refined
to fit them into a window without white spaces on all the
4.3 Vertical Edge Matching four sides. Figure 3(a) shows the template of for the
extracted character ‘ ’ with respect to the above
In this phase the width to height ratio of license plate refinement. Now each character image is normalized to a
is used to match the vertical edges for finding the region size of 40x40. Figure 3(b) shows the character ‘ ’
of where there is a probability of license plate. The ratio normalized to 40x40. Normalization is done using
of width to height of Saudi Arabian license plate is about window to view port transformation. This mapping is
2:1. After filtering step, the image in Figure 2(c) is used to map every pixel of the original image to the
extracted having several vertical regions. Only two of the corresponding pixel in the normalized image. The image
regions could be the possible boundary of the license obtained is invariant to scaling because the characters
plate. The task of vertical edge matching is to find out the were fit in a rectangular region according to their height
correct pair of regions that include the license plate. To and width.
achieve this task, the width-to-height ratio of the
rectangular area between two vertical regions is compared 6.2 Template Matching
with the actual standard ratio of a license plate. The
standard ratio is taken between 1.75 : 1 to 2.25 : 1. Figure Template matching for character recognition is
2(d) shows the extracted license plate for the matched straightforward and is reliable. This method is more
vertical edges. tolerant to noise than structural analysis method. In this
approach, the templates are normalized to 40x40 pixels
5. License Plate Segmentation and stored in the database. The extracted character, after
normalization is matched with all the characters in the
After the extraction phase, the License Plate is database using hamming distance approach. This
segmented into individual characters. To ease the process approach is shown in Equation 1.

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE
 Table 1. Recognition rate for license plate extraction, segmentation and recognition.
License Plate License Plate License Plate
Extraction Segmentation Recognition
Correct 587/610 574/610 581/610
Recognition
Percentage 96.22% 94.04% 95.24%
Recognition

nrows ncols The experiments for the above system were

¦ ¦ mismatch
i =1 j =1
i, j performed under different situations, i.e. the experiments
were carried out for the following cases.
where
(1) ™License plates in normal shapes.
­° 1, if original i , j ≠ extracted i , j ™License plates that are leaned with some angle of
mismatchi , j = ®0, if original = extracted view.
i, j ™License plates that have similar color to license
°̄ i, j
plate body.
™Dirty license plates.
where nrows and ncols are the number of rows and
columns in the original and extracted images. In our case The whole system is working fine achieving high
nrows = ncols = 40, as the image is normalized to 40x40. recognition rate. Table 1 gives results for the percentage
The mismatches for each of the extracted character are recognition of license plate extraction, segmentation, and
found by comparing with the original characters in the character recognition. A failure results if a character of
database, and the character with least mismatch value is the license plate is incorrectly interpreted. This normally
taken as the recognized character. In the case of Saudi happens due to bad quality of extracted plate. It is shown
Arabian license plates, recognition is done by matching that the system has correctly recognized 581 license
each extracted character by all the 38 (28 alphabets and plates out of 610 test images. The system was
10 numerals) characters in the database. implemented on PIII (700MHz) using MATLAB6.1 [14].
The experimental results show that the shortcoming
of the proposed system is mainly due to bad quality of
input image during the acquisition stage, or unclear
detection or extraction of the edges. The segmentation
phase gives good results, because the image is converted
into binary form. In character recognition there is some
form of mis-recognition. This is due to the nuts and bolts
within the character. Otherwise, there is a standard font
(a) (b) for all the Arabic characters used in the license plates,
which do not cause any problem during recognition
Figure 6. (a) Original Template for Character ‘ ’ phase.
(b) Normalized 40x40 Template for character ‘ ’
. 8. Conclusion

7. Experimental Analysis and Results Although there are many running systems for
recognition of various plates such as Singaporean, Korean
Experiments have been performed to test the above and some European license plates, the proposed effort is
system. The system is designed in MATLAB6.1 [14] for first of its kind for Saudi Arabian license plates. The
recognition of Saudi Arabian license plates. The image to license plate recognition involves Image acquisition,
the system is a gray scale of size 640x480. The test License plate extraction, Segmentation, and Recognition
images were taken under various illumination conditions phases. Beside the use of Arabic language, Saudi Arabian
(i.e. noon and in the evening). license plates have several unique features that are taken
care in the segmentation and recognition phases. The

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE
 system is tested over a large number of car images and is [8] Naito, T., Tsukada, T., Yamada, K., Kozuka, K., and
proved to be 95 % accurate. Yamamoto, S., (2000) Robust license-plate recognition
method for passing vehicles under outside environment.
Acknowledgement IEEE Transactions on Vehicular Technology, Volume:
49 Issue: 6, pp: 2309-2319.
This work has been supported by King Fahd
University of Petroleum and Minerals. [9] Park, S. H., Kim, K. I., Jung, K., and Kim, H. J., (1999)
Locating car license plates using neural network, IEE
References
Electronics Letters, vol.35, no. 17, pp. 1475-1477.
[1] Bailey, D. G., D. Irecki, D., Lim, B. K., and Yang, L.,
(2002), Test bed for number plate recognition [10] Salagado, L., Menendez, J. M., Rendon, E., and Garcia,
applications, Proceedings of the First IEEE International N., (1999) Automatic car plate detection and recognition
Workshop on Electronic Design, Test and Applications through intelligent vision engineering, Proceedings of
(DELTA’02), IEEE Computer Society. IEEE 33rd Annual International Carnahan Conference on
Security Technology, pp. 71-76.
[2] Cowell, J. and Hussain, F., (2002), A fast recognition
system for isolated Arabic characters, Proceedings Sixth [11] Smith A.R., ., (1979) Tint Fill , Computer Graphics, Vol.
International Conference on Information and 13, no. 2, pp. 276-283.
Visualisation, IEEE Computer Society, London,
England, pp. 650-654. [12] Yoram, and Hofman, O., (2001) Hi-Tech Solutions,
http://www.htsol.com/Products/ Demo.html.
[3] Cowell, J., and Hussain, F., (2001), Extracting features
from Arabic characters, Proceedings of the IASTED [13] Yu, M., and Kim, Y. D., (2000) An approach to Korean
International Conference on COMPUTER GRAPHICS license plate recognition based on vertical edge
AND IMAGING, Honolulu, Hawaii, USA, pp. 201-206. matching, IEEE International Conference on Systems,
Man, and Cybernetics, vol. 4, pp. 2975-2980.
[4] Hontani, H., and Koga, T., (2001), Character extraction
method without prior knowledge on size and information,
[14] The MathWorks, Inc, (1993-2003), The matlab package,
Proceedings of the IEEE International Vehicle
http://www.mathworks.com/.
Electronics Conference (IVEC’01), pp. 67-72.

[5] Kim, K. K., Kim, K. I., Kim, J.B., and Kim, H. J.,
(2000), Learning based approach for license plate
recognition, Proceedings of IEEE Processing Society
Workshop on Neural Networks for Signal Processing,
vol. 2, pp: 614-623.

[6] Morel, J. and Solemini, S. (1995) Variational Methods in

Image Segmentation, Birkhauser, Boston.

[7] Nieuwoudt, C., and van Heerden, R., (1996) Automatic

number plate segmentation and recognition, in Seventh
annual South African workshop on Pattern Recognition,
pp. 88-93, IAPR.

Proceedings of the 2003 International Conference on Geometric Modeling and Graphics (GMAG’03)
0-7695-1985-7/03 $17.00 © 2003 IEEE