2023 3rd Asian Conference on Innovation in Technology (ASIANCON)

Pune, India. Aug 25-27, 2023

ML Based Number Plate Recognition Model using

Computer Vision
Abhisri Dhyani Ashish Kumar Upadhyay Akash Kapoor
Department of Computer Science and Department of Computer Science and Department of Computer Science and
Engineering, Engineering, Engineering,
Apex Institute of Technology, Apex Institute of Technology, Apex Institute of Technology,
Chandigarh University, Chandigarh University, Chandigarh University,
Mohali, Punjab, India Mohali, Punjab, India Mohali, Punjab, India
[email protected] [email protected] [email protected]

Anshuman Sengar Divyanshi Bhatia Alankrita Aggarwal

Department of Computer Science and Department of Computer Science and Associate Professor
Engineering, Engineering, Department of Computer Science and
Apex Institute of Technology, Apex Institute of Technology, Engineering,
2023 3rd Asian Conference on Innovation in Technology (ASIANCON) | 979-8-3503-0228-8/23/$31.00 ©2023 IEEE | DOI: 10.1109/ASIANCON58793.2023.10270673

Chandigarh University, Chandigarh University, Apex Institute of Technology,

Mohali, Punjab, India Mohali, Punjab, India Chandigarh University,
[email protected] [email protected] Mohali, Punjab, India
[email protected]
Orcid id: 0000-0002-0931-1118

Abstract— In this research paper the Number Plate The primary objective of this research paper is to build
Recognition System (NPRS) is used which is one of the the number plate recognition model using YOLOv8 and
smartest and most high-tech detection systems. This is a system evaluate its accuracy for number plate recognition. The
of different technologies where an interface enables the system efficiency of the model will be gauged based on evaluation
to automatically detect and read the unique ID of a number parameters, and the dataset will be used to train the model
plate of vehicles from the real-time captured images. It is based on real scenarios to achieve maximum accuracy during
basically the process of extracting the pixel data from the testing.
digital images and converting it into ASCII values or the plain
text of the number plate. The system first detects the vehicle The project will focus on three primary objectives:
and then takes a real-time image of that vehicle. To get the detecting the number plate with more accuracy, detection
solution to this problem, a deep learning model known as YOU from various distances, and clarity in extracting numbers
ONLY LOOK ONCE (YOLO) and its latest version YOLOv8 using YOLOv8 modes and then achieving the highest
is used for detecting the license plates. This model’s main aim accuracy of the model by training it on the dataset. To
is to make use of different types of morphological operations so achieve these objectives, the model will be trained with more
that the specifically required number plate can be detected and datasets to improve its prediction accuracy. The model's real-
translated efficiently. Once we get the desired number plate, time performance in practical scenarios will also be
we can perform various important operations or tasks such as
considered to ensure its reliability and effectiveness.
checking it with the databases to check for stolen vehicles, for
tracking the vehicle’s movement, etc. It can help to improve Hardware specifications play a crucial role in the
public safety and enhance the efficiency of the transportation performance of an ML-based number plate recognition
system. model employing computer vision. It is recommended to use
a multi-core CPU with a minimum clock speed of 2.4 GHz, a
Keywords— NPRS, YOLOv8, high-tech detection system, dedicated graphics card, at least 16GB of RAM, and an SSD
real-time capturing, image processing, deep learning, with at least 256GB of space. Windows 10 or Linux is
morphological operations.
advised as the operating system.

I. INTRODUCTION The software specification for an ML-based number plate

The increasing number of vehicles on roads around the recognition model includes several elements such as data
world has made it necessary to improve traffic management preprocessing, feature extraction, machine learning model,
and security systems. One such system is the number plate post-processing, integration, testing and validation,
recognition model, which can identify a vehicle's registration maintenance, and support. These elements are critical in
number and other essential details using computer vision ensuring that the model performs accurately, reliably, and
technology. Implementation This technology can be used for efficiently.
various applications, such as toll collection, traffic law In conclusion, this research paper will render a computer
enforcement, and parking management. Traditional methods vision model using YOLOv8 for the detection of number
for number plate recognition have been less efficient and plate in either moving or static vehicles with real-time
often require manual intervention, making it time-consuming detection and evaluate the accuracy, precision and efficiency
and prone to errors. In recent years, there have been of the model. By training the model with more datasets and
significant developments in computer vision-based number considering real-time performance, this research aims to
plate recognition models, which have demonstrated better improve the reliability and effectiveness of number plate
accuracy and efficiency. recognition systems. The results of this research will have

979-8-3503-0228-8/23/$31.00 ©2023 IEEE 1

Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.
significant implications for traffic management and security A survey was done regarding various ANPR
systems worldwide. implementation strategies. Nearly 78 reference books'
accuracy findings were analyzed by the writers [4]. The basic
II. LITERATURE REVIEW ANPR processes include capturing images of the vehicle,
In this research paper, we address the problem of image identifying number plates, segmenting characters, and
detection in multi-normal and multilingual (LP) natural recognizing characters. The size, location, background, and
scenes [1]. The architecture of this system uses a pipeline screw of the number plate are all factors to be considered.
and two stages of deep learning. The system was first trained The survey found that the most accurate method for
to detect digital symbols in complete raw images using a detecting plates was Canny's edge detection. To get better
deep learning-based detector called YOLOv2. The system’s results, apply techniques like image binarization, CCA
second step is to use the cropped image to identify the image (Connected Component Analysis), vertical and horizontal
from the received number. In this model, two recognition projection, and character segmentation. The subsequent
machines are compared here: a segmentation approach and phase involves character recognition, which frequently
joint detection. We have proposed a new semi-automatic involves the use of Artificial Neural Networks, template
annotation and detection procedure. The proposed system is matching, or optical character recognition (OCR) techniques.
evaluated using two databases collected from real traffic It suggested a technique for character recognition (CR)
monitoring and parking access monitoring. It showed that a and driver's license validation (LPD) as a combined
system using two YOLO steps performs better in the context application with good accuracy and performance [5]. The
of multi-regular and multi-lingual coding. Some more tests system is made for Iranian licenses with varying resolutions,
are performed on a publicly available Kaggle database and settings, uncommon numbers and characters, multiple
show that the proposed system is better than other state-of- background colors, and diverse fonts. In this instance, the
the-art methods. system removes Persian characters from the input
Jordan seeks to create a precise ALPR for LP [2]. The photographs in two steps and uses a fine-tuned You Only
suggested method makes use of a YOLO3-based, two-stage Look Alone (YOLO) version 3 platform for each defined
convolutional neural network (CNN). The YOLO3 grid level. In contrast to the real system, a significant number of
architecture is modified to a sparse grid to identify small vehicle photos were gathered under testing and training
objects because the dimensions of the LP letters are situations that were both challenging and straightforward.
extremely small in comparison to the frame size. The Test findings reveal that of 5719 images, end-to-end
suggested method reduces false positives by using temporal accuracy is 95.05% [5].
data from various frames. In order to track LP cars and weed An end-to-end process for automatic license plate
out anomalies, sequential data structures are required. Since identification without pre-selection is provided by LPRNet
there isn't a Jordanian number database to my knowledge, [6]. Our method, which draws inspiration from recent
this study suggests a brand-new one named JALPR database. developments in deep neural networks, achieves up to 95%
The dataset, which is accessible online, contains actual accuracy on Chinese benchmarks: NVIDIA R GeForce
videos of moving automobiles throughout Jordan. For TMGTX 1080 at 3ms; Intel R CoreTMi7-6700K at 1.3 md
comparison, two well-known commercial software programs and CPU plate [6]. LPRNet can be taught to a finite extent
were chosen. The suggested approach obtains a recognition because it is a lightweight convolutional neural network.
accuracy of 87% in experiments on real YouTube films, LPRNet, to our knowledge, is the first actual RNN-free
compared to commercial systems' recognition accuracy of license plate recognition system. The LPRNet technique can
less than 81% for Jordanian characters [2]. be utilized to develop an internal solution for LPR as a result,
This demonstrated a reliable and effective ALPR system displaying good accuracy even when tested against Chinese
based on cutting-edge YOLO object detection [3]. For each characters.
phase of ALPR, Convolutional Neural Networks (CNN) are F. Liu et al. suggested a deep neural network-based
trained and modified to become more resilient under various license plate detection model in 2020 that includes a state-of-
circumstances (such as shifting camera, illumination, and the-art attention mechanism [8]. The attention mechanism
background). Using straightforward data augmentation helps focus attention on just the most important components
methods like reverse license plates (LP) and changed of the license plate image. The suggested model has a
characters, we segment and recognize distinctive characters 99.47% accuracy rate when applied to a dataset of Chinese
in two steps. On both datasets, the ALPR technique that was license plates [8].
used produced intriguing results. First, our system performed
better than the commercial Sighthound and OpenALPR It proposed a data-time-based detection number approach
systems (89.80% and 93.03%) in the SSIG database of 2000 (detection/recognition) instead of a single frame selection to
frames from 101 vehicle films, obtaining a recognition rate automatically detect and recognize vehicles on the road [9].
of 93.53% and 47 frames per second (FPS), respectively, and Two processes that have been proposed here, can be used to
much better than the prior findings (81.80%) [3]. Second, we improve system accuracy by querying license plate databases
propose the UFPR-ALPR dataset, a larger public dataset for (for example, the Department of Motor Vehicles database
ALPR that is aimed at a more realistic setting. This dataset containing a list of licenses issued and vehicle models). The
includes 150 movies and 4500 still images that were taken test results show that using the time-advanced approach of
while cameras and moving vehicles (including buses, trucks, our proposal can improve the key results by 15.5 percentage
cars, and motorcycles) were in motion. On our suggested points (p.p.) (an increase of 23.38%). In addition, an
database, the test version of the commercial system produced additional 7.8 p.p. was achieved by two processing methods,
a recognition rate that was under 70%. However, our system reaching a final recognition rate of 89.6% in a database with
outperformed it with a recognition rate of 78.33% and 35 5,200 image frames of 300 vehicles registered at the Federal
FPS [3]. University of Minas Gerais (UFMG) [9].

2
Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.
 It offers a numbering scheme that takes location and region proposal network were used in the approach for
recognition into account [10]. Artificial intelligence can be identifying car license plates described by K. Nishimura et
used to address the issue of losing your mouse. The text al. (2020). (RPN). Using a dataset of Japanese license plates,
recognition accuracy will increase once the location has been the suggested approach had a 98.3% accuracy rate [17]. A
determined. The use of artificial intelligence will be used to complete deep learning system for recognizing license plates
train the location and recognition systems. The Tesseract- that uses a thin network architecture for efficient inference
OCR system will be used to perform the recognition after the on mobile devices was presented by Y. Cai et al. in 2021, the
localization system has been trained using the YOLOv5 suggested model had a 97.53% accuracy rate when applied to
network. Number tracking is the first application of the dataset of Chinese license plates [17].
YOLOv5 [10].
Convolutional neural networks (CNNs) and optical
In this, we propose an analysis effect on the reinnervation character recognition (OCR) methods were proposed by A.
of the recognition pattern of the plaque and the deep nervous K. Awasthi et al. in 2020 for a license plate recognition
system using the Brazilian synthetic tile image database for system, the system's accuracy on a dataset of Indian license
object detection numbers [11]. Variations in rotation, size, plates was 96.2% [18]. A unique method for license plate
and volume are used in the suggested data set to evaluate identification and recognition based on a two-stage deep
hardness. As a result, the effect of using artificial tile shapes learning model was proposed by M. Shareman et al. in 2020.
on the accuracy of the system responsible for locating and The license plate region is in the first stage, and the
classifying real tile symbols and their recognition was characters are in the second stage. When applied to a dataset
evaluated in experiments carried out at three stages: of Iranian license plates, the suggested model had an
character segmentation, letter recognition, and number accuracy of 97.34% [18].
recognition (2.54%, 1.09%, and 2.49%, respectively). It is
only possible to achieve the digit recognition step with III. PROPOSED SYSTEM
62.47% accuracy using a trained neural network that has
been tested on real plates and with synthetic data [11]. The proposed model uses YOLOv8, trained on a
customized dataset, to detect number plates on vehicles. It is
The paper provides a thorough analysis of the most designed to handle increasing vehicle counts and improve
recent advancements and strategies in Automatic Number traffic management. The model is trained to recognize
Plate Recognition (ANPR) [13]. Computer vision (CV) number plates in diverse conditions and is deployed in a real-
systems are among the several algorithms that have been time system. Additional techniques like data augmentation
thoroughly compared and evaluated in real-time. ANPR and transfer learning enhance accuracy. The model's
technology employs recognition techniques to detect and effectiveness is evaluated using precision, recall, and mAP. It
identify traffic signs. Even with the greatest algorithm, using enables tasks such as automated toll collection, parking
an ANPR system may need additional hardware to produce management, law enforcement support, and traffic analysis.
more accurate results. Continuous monitoring and occasional retraining ensure
optimal performance in changing conditions.
In 2019, S. J. Kim et al. introduced the dual attention
network, a deep neural network with a unique design for
recognizing license plates (DAN) [15]. To increase
recognition accuracy, the DAN model employs two different
kinds of attention processes. An accuracy of 98.37% was
attained by the suggested model using a dataset of Korean
license plates [15].
The OCR approaches, which are prone to misalignment
and a variety of sizes, were first recognized by Muhammad
Tahir Qadri [16]. His modification can be utilized to improve
real-time OCR identification from varied sizes and angles. A
pre-programmed mechanism for detecting and identifying
vehicles using their license plates is displayed. a synthesis of Fig. 1. Flowchart for the Proposed System
the model's image processing methods for locating and
identifying the vehicle in the system's databases. S.Kranthi A. Image Acquisition
and K.Pranathin have proposed Automated License/Number
Plate Detection/Recognition (ANPR) as a method for A vital first step in creating a computer vision-based ML
identifying a vehicle's image and verifying its number plate model for number plate identification is image capture. This
number. ANPR is used to display vehicles that may be stolen involves collecting pictures of license plates through
or involved in criminal activity. Using ANPR is a simple scanners and cameras, among other sources. The correctness
method to spot a stolen car on the road, in a parking lot, or at of the model is significantly influenced by the clarity and
a traffic light. The project "Automatic Vehicle Number Plate resolution of the recorded photos. To increase the quality of
Recognition System Using Machine Learning" is the focus the captured photos, pre-processing techniques including
of this entire section. noise reduction, contrast enhancement, and image
segmentation can be used. The algorithm may also be
Many applications, including toll-collecting systems, accurately trained to recognize number plates using a sizable
parking management, and law enforcement, have made dataset of labeled photos. The model can work in real-world
extensive use of number plate recognition (NPR) models circumstances and achieve high identification rates thanks to
[17]. Some recent research on NPR models will be covered the application of computer vision algorithms during picture
in this literature review. Deep neural networks (DNNs) and a gathering.

3
Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.
B. Vehicle Detection morphological procedures. Once the ROI has been located, it
An ML-based number plate recognition model may be cropped and treated to enhance the image's quality.
employing computer vision must include vehicle detection. It Since improper localization can lead to false positives or
includes applying object identification techniques i.e., YOLO missing detections, it has a significant influence on the
to recognize and locate automobiles in an image or video model's performance. The model can successfully identify
stream. The model may then concentrate on the areas of and recognize number plates in a variety of situations, such
interest (ROIs) corresponding to the number plates for as poor light or high-speed settings, by applying computer
additional processing after the cars have been identified. This vision algorithms for number plate localization.
method shortens the processing time while simultaneously
increasing the model's accuracy. The number plate
recognition model can successfully identify license plates in
various detection situations, including parking lots, freeways,
and toll booths, by including vehicle detection.

Fig. 3. Training Graphs of the Trained Model

Fig. 2. Training Graphs of the Trained Model

C. Dataset Preparation
The creation of the dataset is a crucial stage in the IV. RESULT
development of a computer vision-based ML model for The accuracy of the YOLOv8 is 97.2%, which is a good
number plate recognition. It entails gathering a sizable and score and the highest of all the other YOLO models.
varied collection of number plate-labeled photos. The dataset Therefore, we have proven to be the best model for the
should include a range of situations, lighting settings, and detection of number plates and vehicle detection as
font, color, and size variants. To prevent false positives, the compared to the other YOLO models. At first, the camera
collection should contain pictures of cars with and without will be used to capture images. Then the model will perform
license plates. To expand the dataset's size and variety, data the preprocessing, training, detection, validation, inference,
augmentation techniques including flipping, rotating, and and evaluation of the dataset. Based upon all these processes
scaling might be used. When the dataset is ready, it may be the model is trained. After training the model, it is tested on
divided into training, validation, and testing sets for the the test dataset. Finally, the number plates are detected for
creation and assessment of the model. For the purpose of real-time image capturing through the camera. The last
creating a reliable and accurate number plate recognition recorded information will be displayed on the screen.
model, a well-prepared dataset is essential.

D. Training the Model

The most important stage in creating a computer vision-
based ML model for number plate identification is training
the model. The loss function can be minimized by, feeding
the labeled dataset into the model, and back-propagating the
model weights. For the training process, the Roboflow model
and Google Colab is used. Number of epochs, batch size, and
learning rate should be changed, which can lead to an
improvement in performance. The performance of the model
may be assessed after training using the validation and test
sets. To further enhance the model's performance, fine-
tuning strategies like transfer learning can be used.

E. Number Plate Localization

Fig. 4. mean Accuracy Graph (mAP) of the Trained model
The development of a computer vision-based ML-based
number plate recognition model requires number plate The above-given graph can be used to evaluate the mean
localization as a key step. It entails locating the area of accuracy of the trained model. Also, is given in Figure 4,
interest (ROI) in the picture or video frame that has the how the accuracy is increasing so rapidly.
license plate. The ROI may be located using a variety of
methods, including edge detection, thresholding, and

4
Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.
 TABLE I. TABULAR REPRESENTATION OF THE ACCURACY,
PRECISION AND RECALL ACHIEVED AFTER TRAINING AND
TESTING OF THE YOLOV8 MODEL ON THE CUSTOM DATASET

YOLO Parameters
Models Accuracy Precision Recall
YOLOv8 97.2% 97.6% 94.5%

Fig. 5. Box Loss Graph for the Trained Model

Fig. 8. Outputs of the Trained and Tested Model

Fig. 9. Real-time detection using a webcam

V. CONCLUSION
We propose a number plate recognition model using the
Fig. 6. Class Loss Graph for the Trained Model
YOLOv8 model for vehicle number plate recognition and
using our custom dataset of number plates. We have trained
our dataset using the YOLOv8 model, this model can detect
multiple cars and number plates in a single frame. The results
shown in this paper are achieved with higher accuracy as
compared to other ANPR, KNN, and CNN models.

REFERENCES
[1] Yousri Kessentinia, Mohamed Dhia Besbes, Sourour Ammar, Achraf
Chabbouh, “A Two-Stage Deep Neural Network for Multi-norm
License Plate Detection and Recognition”.
[2] Salah Alghyaline, The World Islamic Sciences and Education
University, “Real-time Jordanian license plate recognition using deep
learning”, Faculty of Information Technology, Department of
Computer Science, Tabarbour, Amman, Jordan.
[3] Rayson Laroca, Evair Severo, Luiz A. Zanlorensi, Luiz S. Oliveira,
Gabriel Resende Gonc¸alves, William Robson Schwartz and David
Menotti, “A Robust Real-Time Automatic License Plate Recognition
Based on the YOLO Detector”.
[4] Di Zang, Zhenliang Chai, Junqi Zhang, Dongdong Zhang and Jiujun
Cheng, “Vehicle license plate recognition using visual attention
model and deep learning”, Journal of Electronic Imaging 24(3),
033001 (May⁄Jun 2015).
Fig. 7. Object Loss Graph of the Trained Model [5] Ali Tourani, Asadollah Shahbahrami, Sajjad Soroori, Saeed Khazaee,
And Ching Yee Suen, “A Robust Deep Learning Approach for

5
Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.
 Automatic Iranian Vehicle License Plate Detection and Recognition [12] Irfan Kilic, Galip Aydin, “Turkish Vehicle License Plate Recognition
for Surveillance Systems”, (Life Fellow, Ieee) Using Deep Learning”, Department of Computer Engineering Firat
[6] Sergey Zherzdev, Alexey Gruzdev, “LPRNet: License Plate University Elazig, Turkey · Conference Paper · September 2018 DOI:
Recognition via Deep Neural Networks”. 10.1109/IDAP.2018.8620744
[7] Zied Selmi, Mohamed Ben Halima, Adel M. Alimi, “Deep Learning [13] Lubna; Mufti, N.; Shah, S.A.A. Automatic Number Plate
System for Automatic License Plate Detection and Recognition”, Recognition: A Detailed Survey of Relevant Algorithms. Sensors
Conference Paper · November 2017 DOI: 2021, 21, 3028
10.1109/ICDAR.2017.187 2017 14th IAPR International Conference [14] Wang, H.; Li, Y.; Dang, L.-M.; Moon, H. Robust Korean License
on Document Analysis and Recognition. Plate Recognition Based on Deep Neural Networks. Sensors 2021, 21,
[8] Wang Weihong and Tu Jiaoyang, “Research on License Plate 4140
Recognition Algorithms Based on Deep Learning in Complex [15] Jing Han 1.; Jian Yao.; Jiao Zhao.; Jingmin Tu.; Yahui Liu. Multi-
Environment”, Received March 15, 2020, accepted April 27, 2020, Oriented and Scale-Invariant License Plate Detection Based on
date of publication May 14, 2020, date of current version May 28, Convolutional Neural Networks. Sensors 2019, 19, 1175
2020. Digital Object Identifier 10.1109/ACCESS.2020.2994287. [16] J M S V Ravi Kumar et al 2021 IOP Conf. Ser.: Mater. Sci. Eng.
[9] Gabriel Resende Gonçalves “License Plate Recognition based on 1074 012012
Temporal Redundancy” Belo Horizonte Agosto de 2016. [17] V Gnanaprakash et al 2021 IOP Conf. Ser.: Mater. Sci. Eng. 1084
[10] Yu-Liang Chiang, University of Wisconsin-Milwaukee, “Real Time 012027
License Plate Detection Based on Machine Learning”, May 2021. [18] J.M Mossi, L Sanchis, and A. Albiol, "The Detection of the Parked
[11] Saulo Cardoso Barreto, Jorge Albuquerque Lambert and Fl´avio de Vehicles Using the Spatiotemporal Maps", The IEEE Transactions on
Barros VidalChapter, “Using Synthetic Images for Deep Learning the Intelligent Transportation Systems, vol.12, no.4, pp. 1277-1291,
Recognition Process on Automatic License Plate Recognition”, May 2011.
2019 · DOI: 10.1007/978-3-030-21077-9_11

6
Authorized licensed use limited to: BMS College of Engineering. Downloaded on June 03,2024 at 06:57:38 UTC from IEEE Xplore. Restrictions apply.