License Plate Recognition

Why read license plates? Is this challenging? Probably if solution is to be versatile… Introduction

Many methods Connected Component Analysis Kwaśnicka, Wawrzyniak [2002] Filtering, Edge Finding, Adaptive Threshold Enyedi, Konyha, Fazekas [2006] Sliding Windows, Histogram Kaushik, Chae, Jo [2009] Machine Learning, Binarization & Extension Chen, Yuille [2004] Machine Learning, Gradient-Based Escalera et al. Lit. Review

Localize license plate(s) Binarize license plate Extract license plate text Transform text to flat orientation Perform OCR on text Post-process OCR output Methodology

Many solutions assume near-horizontal license plate Chosen approach: Localize by training on text fragments Idea: Be greedy in identifying possible license plate text, and discard non-license plate text during post-processing Methodology

What is “license plate text”? Characteristics: Colour Intensity Distribution Entropy Energy Others… 1. Localization

First Attempt: Train on large blocks Failed Characteristics too varied to make good judgments on 1. Localization

Second Attempt: Train on small blocks Success! (well… mostly) Some amount of false positives from reflections, grilles, etc 1. Localization Positive Negative (for white-on-black) ?

Classification AdaBoost : Combines multiple weak classifiers linearly Our eventual classifier: Colour Number of regions in binarized edge thumbnail Size distribution of regions 1. Localization (31 regions) (2nd 0.87 times 1 st ) (7 regions) (2nd 0.50 times 1 st ) (1 region)

Matching back to the input image… Pre-process image by increasing contrast, sharpening to try and reduce blur* Recursively partition image into quadrants Each quadrant is resized to a small thumbnail Each thumbnail is checked against classifier, and judged to be either TEXT or NONTEXT 1. Localization *Possible blur removal approach: http://cs.nyu.edu/~fergus/research/deblur.html

Example Result 1 1. Localization

So how do we actually localize? “ Intelligent guessing” Connected region (of same granularity) 1. Localization

2. K-means clustering (used K=2, multiple iterations) 1. Localization

How many candidate regions do we get? Usually, around 10 to 50 More if no license plates are found at the default scale, and we have to subdivide the image further 1. Localization

1. Localization (35 candidate regions for this example)

Why go to all this trouble? For each candidate region, we have to attempt to extract text Therefore, the localization procedure cuts down tremendously on the amount of processing needed A brute force search would otherwise yield astronomical numbers of candidate regions, due to the sheer number of combinations 1. Localization

We resize each candidate region to around 40000 pixels, and convert it to black and white Thankfully, this is relatively simple since a license plate is supposed to have high contrast between the plate colour and letter colour We simply convert to grayscale, quantize the image to two colours, and normalize it 2. Binarization

We create an edge image (high contrast, remember) (also works whatever the license plate colour) We flood fill and keep only regions within certain dimensions relative to the edge image 3. Extraction

We expect at most some skew of the license plate (no complex nonlinear distortions) Eliminate remaining blobs unlikely to be characters 4. Transformation Too far Too short

Study the character blobs to predict whether the license plate has one or two lines Draw lines between all midpoints and see if there is a line that is close enough to all other midpoints (Justified by linearity of characters) 4. Transformation min_d_max = 68.5 px min_d_max = 2.11 px

Map the corners to a standard template long rectangle if one-lined, and fat rectangle if two-lined (a simple homography, as in Lab 3) 4. Transformation

We used Google Tesseract as suggested Unfortunately, the raw results were not as good as expected Possible explanation: License plate fonts are in general distinct from normal text font, which caused poor performance Possible solution: Train Tesseract exclusively on license plate fonts 5. OCR produces “ SGZ 35BBG ”

Standard Singapore license plates consist of two or three letters, then one to four digits, and one final letter The final letter is a checksum for the other characters 6. Post-processing

The formula is*: Let the license plate have the format [S] AB CDEF X Then, we take the value V = ((A*9)+(B*4)+(C*5)+(D*4)+(E*3)+(F*2)) mod 19 We can then determine X as follows: It is therefore very easy to check if an OCR output is a valid license plate number 6. Post-processing *Formula from: http://gangasudhan.com/blog/2008/07/nric-fin-number-suffix-checker.html B C D E G H J K L M P R S T U X Y Z A X 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 V

Given the accuracy (or lack thereof) of OCR when it comes to license plate fonts, it might be worthwhile to use this knowledge to repair poor OCR output We note that OCR performance seems to be better on letters than on digits in general Idea: If exactly one of the digits is misinterpreted as a letter, we can use the other characters (including the checksum) to correct it, since there are only ten possibilities for a digit This is less useful once two or more digits are misinterpreted… 6. Post-processing

We used Perl (with ImageMagick) for most of the coding Test system is an Intel Core 2 Duo 3GHz CPU running Windows, with 3GB of RAM Runtime for Localization: ~2 or 3 minutes for standard granularity (6 levels of recursion) Runtime for Binarization, Extraction: Varies; a few seconds per candidate region Runtime for Transformation: Varies; ~10 seconds per candidate region Runtime for OCR, Post-processing: negligible Total expected runtime: 5-10 minutes Implementation

A framework for a robust license plate recognition program has been implemented It has the following properties: Scale-invariant Deals with moderate skew Deals with multiple license plates Possibly corrects erroneous reads Future challenges: Characters that run into each other More sophisticated blur removal Increase speed by reimplementation in a faster programming language Conclusion

License Plate Recognition

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to License Plate Recognition (20)

Recently uploaded (20)

License Plate Recognition