Vol. 5, No.

8 August 2014 ISSN 2079-8407

Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org

Review and Analysis of Pothole Detection Methods

1
Taehyeong Kim*, 2 Seung-Ki Ryu
1
Senior Researcher, Korea Institute of Civil Engineering and Building Technology, Korea (*corresponding author)
2
Research Fellow, Korea Institute of Civil Engineering and Building Technology, Korea

ABSTRACT
As one type of pavement distresses, potholes are important clues indicating structural defects of the asphalt road, and
accurately detecting these potholes is one of important tasks for determining proper strategies of asphalt-surfaced
pavement maintenance and rehabilitation. However, manually detecting and evaluating methods are expensive and time-
consuming. Thus, several efforts have been made for developing a technology which can automatically detect and
recognize potholes, which may contribute to improvement of survey efficiency and pavement quality through prior
investigation and immediate action. In this study, we investigate and analyze pothole detection methods which have
developed and propose a potential direction of developing a pothole detection method to accurately and efficiently detect
potholes.

Keywords: Pothole, Detecting, Vibration, 3D reconstruction, 2D images, Video

1. INTRODUCTION Also, pavement distress detection such as cracks,

Recently, damaged pavement like potholes are potholes, etc. mostly performed manually is a labor-
increasing due to the climate change such as heavy rains intensive and time-consuming. In order to resolve this
and snow in Korea, and thus complaints and lawsuits of problem, a new research is being conducted in Korea for
accidents related to potholes are growing. developing a technology that can detect and recognize
potholes based on images, which may contribute to
As one type of pavement distresses, a pothole is improvement of survey efficiency and pavement quality
defined as a bowl-shaped depression in the pavement by prior investigation and immediate action.
surface and minimum plan dimension is 150 mm [1].
Potholes can generate damages such as flat tire and wheel In this study, we investigate and analyze pothole
damage, impact and damage on the lower part of a detection methods which have developed until now and
vehicle, sudden braking and steering wheel operation, and further propose a potential direction of developing a
vehicle collision and major accidents. pothole detection method to accurately and efficiently
detect potholes.
Figure 1 shows the number of accidents related
to potholes in Korea from 2008 to July 2013. As shown in Fig 2, Existing methods for pothole
detection can be divided into vibration-based methods by
Yu and Yu, De Zoysa et al., Erikson et al. and Mednis et
al. [2–5], 3D reconstruction-based methods by Wang,
Chang et al., Hou et al., Li et al., Staniek, and Moazzam et
al. [6–12], and vision-based methods by Koch and
Brilakis, Jog et al., Lokeshwor et al., Koch et al., Buza et
al., and Lokeshwor et al. [13–18].

Fig 1: Number of pothole accidents in Korea from 2008 to

July 2013 (MLTM, 2013)

From 2008 to July 2013, 4,233 of accidents

related to potholes were happened in Korea. In 2008, total
number of pothole accidents was 209 and in 2013, 1,051
of pothole accidents happened until July, which is 5 times
of 2008. Among 4,223 pothole accidents, 2,961 accidents Fig 2: Classification of pothole detection methods
happened on regional highway, which is 70.1% of total
pothole accidents.

603
 Vol. 5, No. 8 August 2014 ISSN 2079-8407
Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org
2. VIBRATION-BASED POTHOLE Vibration-based method use accelerometers in
DETECTION order to detect potholes. As the advantages of a vibration-
Yu and Yu proposed the use of recent data based system, these methods require small storage and
acquisition hardware to develop a vibration-based system can be used in real-time processing. However, vibration-
for preliminary evaluation of pavement conditions [2]. based methods could provide wrong results that the hinges
The distress of the pavement such as cracks and rutting and joints of road can be detected as potholes and
impose impacting forces on the vehicle. The pavement potholes in the center of a lane cannot be detected using
surface conditions can be estimated from the recorded accelerometers due to no hit by any of the vehicle’s
responses of the testing vehicle when driving on the wheels (Erikson et al. [4]).
pavement. This system has the advantage of small storage
requirement, cost-effective and amenable for automatic Vibration-based methods for detecting potholes
real-time data processing. However, it does not provide can be summarized as shown in Table 1.
the complete details of distress characteristics as by video-
based system. Also, the service condition of the vehicle Table 1: Vision-based methods for detecting potholes
such as tire pressure should be calibrated to compare
results. Equipment
Authors Methods
& experiments
Sri Lanka has an extensive road network that Yu and Yu Using - ICP accelerometer,
spans the country and new roads are being build every [2] acceleration PC-oscilloscope
day, yet even the roads in the capital city are not records - Test driving on I-90,
maintained properly. De Zoysa et al. proposed a public USA
transportation system based sensor network (BusNet) to De Zoysa et Using - A MICAz mote with
monitor road surface condition by adding acceleration al. [3] acceleration acceleration sensor
sensor boards to the system [3]. BusNet is a sensor and BusNet - Test in Sri Lanka
network initially designed to monitor environmental Erikson et al. Using - P2 (three-axis
pollution using sensors mounted on public transport [4] accelerometer acceleration sensor and
buses. The collected acceleration readings are transmitted data GPS devices)
over the BusNet to the central collection point at the Main - 7 taxis running in Boston
Station. Their work, based on preliminary results, is still Mednis et al. Using - 4 Android OS based
in process for collection of more data sets and developing [5] accelerometer smart phones
an analytical model. sensor of - Test drive in Riga,
smartphone Latvia
Erikson et al. investigated an application of
mobile sensing to detect and report the surface conditions 3. 3D RECONSTURCTION
of roads [4]. They developed Pothole Patrol (P2) system 3D reconstruction methods can be further
gathering data from three-axis acceleration sensor and classified into 3D laser scanner methods by Chang et al.
GPS devices deployed on embedded computers in cars. [7] and Li et al. [9], stereo vision methods by Wang [6],
They deployed P2 on 7 taxis running in the Boston area. Hou et al. [8] and Staniek [10], and visualization using
Using a simple machine-learning approach, they Microsoft Kinect sensor by Joubert et al. and Moazzam et
identified potholes and other severe road surface al. [11-12].
anomalies from accelerometer data. Also, they uploaded
detections to central servers using opportunistic WiFi 3.1 3D Laser Scanner Methods
connections provided by participating open WiFi access The 3D laser scanner uses a technique that
points or, using a cellular data service, where available. employs reflected laser pulses to create accurate digital
Vibration-based methods could provide wrong results that models of existing objects. In the study by Chang et al.[7],
the joints of road or manhole can be detected as potholes the accurate 3D point cloud points with their elevations
and potholes in the center of a lane cannot be detected were captured during scanning and extracted focusing on
using accelerometers due to no hit by any of the vehicle’s specific distress features by means of a grid-based
wheels. processing approach. The experimental results indicate
that the severity and coverage of distress such as potholes
Mednis et al. proposed a mobile sensing system can be accurately and automatically quantified to
for road irregularity detection using Android OS based calculate the needed amounts of filled materials.
smartphones [5]. The selected test track is 4.4km long and
the test drive session included 10 consecutive laps. The Li et al. introduced a real-time, low-cost
technical equipment used during the test drive session inspection system to detect distress features such as
included a passenger car BMW323 touring and four rutting, shoving and potholes using high-speed 3D
different smartphones such as Samsung i5700, Samsung transverse scanning techniques consisting of an infrared
Galaxy S, HTC Desire, and HTC HD2. The Evaluation of laser line projector and a digital camera [9]. To improve
selected data processing algorithms presented true the accuracy of the system, a multi-view coplanar scheme
positive rate as high as 90% using real world data. is employed in the calibration procedure so that more

604
 Vol. 5, No. 8 August 2014 ISSN 2079-8407
Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org
feature points can be used and distributed across the field so that it is difficult to use them in a real time
of view of the camera. environment. Also, both cameras should be vary
accurately aligned since the cameras may misalign and
Laser scanning systems can detect potholes in affect the quality of the outcome if there is the vibration
real time. However, the cost of laser scanning equipment by the vehicle motion.
is still significant at vehicle-level and currently these
works are focused on the local accuracy of 3D 3.3 Kinect Sensor
measurement. Joubert et al. [11] proposed a low-cost sensor
system using Kinect sensor and a high-speed USB camera
3.2 Stereo Vision Methods to detect and analyze potholes. At the time of this study,
Wang did feasibility study to conduct the project was still in its early stages. Some tests were
comprehensive survey of pavement condition through the done to determine the viability of using Kinect to examine
use of stereovision technology [6]. In this method, two potholes.
digital cameras are used to cover a pavement surface. The
first step is to analyze 2D images from each of the two Recently, Moazzam et al. [12] used a low-cost
cameras to detect and classify any cracks. The results Kinect sensor to collect the pavement depth images and
from analyzing two image sources of the same pavement calculate the approximate volume of a pothole. Using a
are then combined so that cracks missed by one analysis low-cost Kinect sensor, the pavement depth images were
are still counted, therefore potentially achieving higher collected from concrete and asphalt roads. Meshes were
accuracy. Also, the pair of images on the same pavement generated for better visualization of potholes. Area of
surface is used to establish 3D surface model with pothole was analyzed with respect to depth. The
longitudinal and transverse profiles through geometric approximate volume of pothole was calculated using
modeling. To recover the 3D properties from given pairs trapezoidal rule on area-depth curves through pavement
of 2D images on the same pavement surface, the sequence image analysis.
of steps such as camera calibration, distortion correct,
matching stereo points, 3D reconstruct, and profile report Although it is cost effective as compared to
should be performed. industrial cameras and lasers, the use of infrared
technology based on Kinect sensor for measurement is
Hou et al. presented a method of applying the still a novel idea and further research is necessary for
stereovision technique into pavement imaging to improvement in error rate. 3D reconstruction methods for
reconstruct the 3D pavement surface from a pair of detecting potholes are summarized as shown in Table 2.
images [8]. A total of four cameras were used in two pairs
to collect pavement surface images across a 4-meter wide Table 2: 3D reconstruction methods for detecting
pavement (each pair of images covers 2 meters of the potholes
road). In this method, 4 steps such as calibration,
distortion adjustment, matching and 3D reconstruction Equipment
Authors Methods
was involved. As an experiment platform, DHDV (Digital & experiments
Highway Data Vehicle) which is a multi-system road Chang et Using 3D laser - 3D laser (MENSi GS
condition survey vehicle developed by University of al.[7] scanning 100)
Arkansas and Waylink Systems Co. Also, they presented - Simulation dataset &
preliminary results for the feasibility of applying real data set
stereovision into pavement imaging. The resolution in 3D Li et al. [9] Using 3D laser - 3D laser & digital
reconstruction can only be made as 2mm in the still scanning camera
environment and more than 5mm in the dynamic moving
environment. They mentioned that as the next step, Wang [6] Using stereo - Two cameras
application of very high-precision gyro system is needed vision - Feasibility study
for vertical motion. Hou et al. [8] Using stereo - Digital Highway Data
vision Vehicle (two
Staniek suggested the stereo vision techniques computers, GPS, DMI,
for the measurement of pavement condition with a stereo line scan and area-scan
vision system attached to a vehicle for recording of the cameras, and laser
road network conditions [10]. A detailed description of lighting devices)
the proposed solution was found in this article. As field - Preliminary test
experiments, the road section was measured on a local Staniek [10] Using stereo - Test bench
road with the length of 650m. The author concluded that vision (two cameras)
the proposed stereo vision method might be used for - Test drive in Poland
evaluation of the road network conditions in Poland. Jouber et al. Using Kinect - Kinect & USB
[11] sensor camera
Stereo vision methods need a high computational Moazzam et Using Kinect - Kinect sensor
effort to reconstruct pavement surface through the al. [12] sensor - Evaluation test in
procedure of matching feature points between two views Rawalpindi, Pakistan

605
 Vol. 5, No. 8 August 2014 ISSN 2079-8407
Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org
4. VISION-BASED POTHOLE and mesh modeling was based on the dense 3D point
DETECTION cloud reconstruction of Golparvar-Fard et al. [19] and the
Vision-based methods include 2D image-based poisson surface reconstruction approach. The proposed
approaches by Koch and Brilakis [13] and Buza et al. [17] method was validated on several actual potholes using a
and video-based approaches by Jog et al. [14], Lokeshwor Canon Vixia HD camera (20fps at 20mph). Future work
et al. [15] and Koch et al. [16]. will be real-time detection of the pothole and assessment
of the total number of frames that need to be used for 3D
4.1 2D Image-Based Approaches reconstruction.
Koch and Brilakis presented a method for
automated pothole detection in asphalt pavement images Lokeshwor et al. presented a method for
[13]. Under the proposed method, the image is first automated detection and assessment of potholes, cracks
segmented into defect and non-defect regions. The and patches from video clips of Indian highways [15]. In
potential pothole shape is then approximated according to the proposed method, first captured video clips are
the geometric characteristics of a defect region. Next, the segmented automatically into two different types of
texture of a potential region is extracted and compared frames category (frames with distress and frames without
with the texture of the surrounding non-defect region. If distress) using DFS (Distress Frames Selection)
the texture of the defect region is coarser and grainier than algorithm. Then, database of frames with distress is
the one of the surrounding surface, the region of interest is processed with CDDMC (Critical Distress Detection,
assumed to be pothole. In order to test the proposed Measurement and Classification) algorithm which
method, it was implemented in MATLAB utilizing the consists of image enhancement, image segmentation,
Image Processing Toolbox, and images were cropped visual properties extraction, detection and classification
from video files captured using a remote-controlled robot by decision logic, and quantification. Also, the decision
vehicle prototype equipped with a HP Elite Autofocus logic for potholes, cracks and patches were developed by
Webcam which was installed at an altitude of about 2 feet three main distinctive visual properties of these distresses
as shown in Fig 1. Total 120 images were collected, and such as the visual texture (standard deviation), the shape
50 images of them were used for training and others for (circularity), and the dimension (average width). A
testing. The resulting accuracy was 86% with 82% database of 1275 video frames with distress was selected
precision and 86% recall. randomly from the results obtained after applying DFS
algorithm to various video clips. As results, an overall
Recently, Buza et al. proposed a new accuracy of 97% with 95% precision and 81% recall in
unsupervised vision-based method which does not require detecting frames with potholes, overall accuracy of 94%
expensive equipment, additional filtering and training with 93% precision and 98% recall in detecting frames
phase [17]. Their method deploys image processing and with cracks, and overall accuracy of 90% with 8.5%
spectral clustering for identification and rough estimation precision and 19% recall in detecting frames with patches.
of potholes. The proposed method is divided into three If few objects such as bleedings, manholes, black colored
steps such as image segmentation, shape extraction using road markings and discoloration spots appear very similar
spectral clustering, and identification and extraction. The to potholes, cracks and patches, the proposed method
proposed method was implemented in MATLAB and could not deliver high accuracy.
tested on 50 pothole images which were selected from
Google image collection. The accuracy for estimation of a The method by Koch and Brilakis [13] was
pothole surface area was about 81%. So, this method can limited to single frames and therefore cannot determine
be used for rough estimation for repairs and rehabilitation the magnitude of potholes in the frame of video-based
of pavements. pavement assessment. In order to complement and
improve the previous method, Koch et al. presented an
4.2 Video-Based Approaches enhanced pothole-recognition method which updates the
2D image-based approaches have been focused texture signature for intact pavement regions and utilize
on only pothole detection and is limited to single frame, vision tracking to track detected potholes over a sequence
so it cannot determine the magnitude of potholes for of frames [16]. The proposed method was implemented in
assessment. To overcome the limitation of the above MATLAB and tested on 39 pavement video containing
method, video-based approaches were proposed to 10,180 frames. The resulting total recognition precision
recognize a pothole and calculate total number of potholes and recall were 75% and 84%, respectively.
over a sequence of frames. Consequently, compared with the previous method, the
texture-comparison performance was increased by 53%,
Jog et al. presented a new approach based on 2D and the computation time was reduced by 57%. They
recognition and 3D reconstruction for detection and assumed that only one pothole enters the viewport at a
measurement (width, number, and depth) of potholes time, and therefore additional work is needed for
using a monocular camera mounted on the rear of a car considering multiple potholes in the viewport.
[14]. The method of 2D recognition built upon the
previous work of Koch and Brilakis [13], and the method Vision-based methods for detecting potholes can
of 3D sparse reconstruction built upon the previous work be summarized as shown in Table 3.
of Golparvar-Fard et al. [19], and 3D dense reconstruction

606
 Vol. 5, No. 8 August 2014 ISSN 2079-8407
Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org
Table 3: Vision-based methods for detecting potholes Korea Institute of Civil Engineering and Building
Technology.
Equipment
Authors Contents
& experiments REFERENCES
Koch and 2D image- - Robot vehicle with a [1] J. S. Miller, and W. Y. Bellinger, Distress
Brilakis [13] Based rear camera (HP Elite identification manual for the long-term pavement
approach Autofocus webcam) performance program, FHWA-RD-03-031, Federal
- Tests for 120 images Highway Administration, Washington, DC, USA
(640*480) (2003).
Buza et al. 2D image- - Tests for 50 images
[17] Based from google image [2] B. X. Yu, and X. Yu, Vibration-based system for
approach collection pavement condition evaluation, In Proceedings of
Jog et al. [14] Video-based - Canon Vixia HD the 9th International Conference on Applications of
approach camera Advanced Technology in Transportation (2006),
- A few series of video 183-189.
stream (1440*810, 20fps)
Lokeshwor et Video-based - 1275 video frames [3] K. De Zoysa, C. Keppitiyagama, G. P. Seneviratne,
al. [15] approach consisting of 283 and W.W.A.T. Shihan, A public transport system
monochromatic images based sensor network for road surface condition
(1280*960) by ARRB monitoring, In Proceedings of Workshop on
NSV and 992 video Networked Systems for Developing Regions
images(640*480) by a (2007), 1-6.
digital camera in India)
Koch et al. Video-based - Robot vehicle with a [4] J. Erikson, L. Girod, and B. Hull, The pothole
[13] approach rear camera (HP Elite patrol: using a mobile sensor network for road
Autofocus webcam) surface monitoring, In Proceedings of the 6th
- 39 video clips International Conference on Mobile Systems,
(640*480, 30fps) Applications, and Services (2008), 29-39.

5. CONCLUSIONS [5] A. Mednis, G. Strazdins, R. Zviedris, G. Kanonirs,

Accurately detecting potholes is one of important and L. Selavo, Real time pothole detection using
tasks for determining proper strategies of pavement Android smartphones with accelerometers, In
maintenance and rehabilitation. However, the person Proceedings of the International Conference on
manually detecting and evaluating methods are expensive Distributed Computing in Sensor Systems and
and time-consuming. Thus, several efforts have been Workshops (2001), 1-6.
made for developing a technology which can
automatically detect and recognize potholes, which may [6] K. C. P. Wang, Challenges and feasibility for
contribute to improvement of survey efficiency and comprehensive automated survey of pavement
pavement quality through prior investigation and conditions, In Proceedings of 8th International
immediate action. Conference on Applications of Advanced
Technologies in Transportation Engineering (2004),
We investigated and analyzed the existing 531-536.
methods for pothole detection which can be divided into
vibration-based methods, 3D reconstruction-based [7] K.T. Chang, J. R. Chang, and J. K. Liu, Detection
methods, and vision-based methods. of pavement distress using 3D laser scanning
technology, In Proceedings of the ASCE
Although the vision-based methods are cost- International Conference on Computing in Civil
effective compared to 3D laser scanner methods, it may Engineering (2005), 1-11.
be difficult to accurately detect a pothole by these
methods due to the distorted signal generated by noise [8] Z. Hou, K. C. P. Wang, and W. Gong,
since they detect a pothole through analysis of the Experimentation of 3D pavement imaging through
collected image and video data. Thus, there is need to stereovision, In Proceedings of the International
develop a pothole detection method using various features Conference on Transportation Engineering (2007),
in 2D images which improve the existing pothole 376-381.
detection method and can accurately detect a pothole.
[9] Q. Li, M. Yao, X. Yao, and B. Xu, A real-time 3D
scanning system for pavement distortion
ACKNOWLEDGEMENTS inspection, Measurement Science and Technology,
This research was supported by a grant from a
Vol. 21, No. 1, (2009), 15702-15709.
Strategic Research Project (2014-0219, Development of
Pothole-Free Smart Quality Terminal) funded by the

607
 Vol. 5, No. 8 August 2014 ISSN 2079-8407
Journal of Emerging Trends in Computing and Information Sciences
©2009-2014 CIS Journal. All rights reserved.

http://www.cisjournal.org
[10] M. Staniek, Stereo vision techniques in the road
pavement evaluation, In Proceedings of the [17] Buza, E., S. Omanovic, and A. Huseinnovic, Stereo
XXVIII International Baltic Road Conference vision techniques in the road pavement evaluation,
(2013), 1-5. In Proceedings of the 2nd International Conference
on Information Technology and Computer
[11] D. Joubert, A. Tyatyantsi, J. Mphahlehle, and V. Networks, (2013), 48-53.
Manchidi, Pothole tagging system, In Proceedings
of the 4th Robotics and Mechatronics Conference [18] Lokeshwor, H., L. K. Das, and S. Goel, Robust
of South Africa(2011), 1-4. method for automated segmentation of frames
with/without distress from road surface video clips,
[12] I. Moazzam, K. Kamal, S. Mathavan, S. Usman, Journal of Transportation Engineering, Vol. 140
and M. Rahman, Metrology and visualization of (2014), 31-41.
potholes using the Microsoft Kinect sensor, In
Proceedings of the 16th International IEEE Annual [19] M. Golparvar-Fard, F. Pena-Mora, S. Savarese,
Conference on Intelligent Transportation Automated model-based progress monitoring using
Systems(2013), 1284-1291. unordered daily construction photographs and IFC
based building information models, Journal of
[13] C. Koch, and I. Brilakis, Pothole detection in Computing in Civil Engineering (2014),
asphalt pavement images, Advanced Engineering 10.1061/(ASCE)CP.1943-5487.0000205.
Informatics, Vol. 25 (2011), 507-515.
AUTHOR PROFILES
[14] G. M. Jog, C. Koch, M. Golparvar-Fard, and I. Taehyeong Kim received the degree in transportation
Brilakis, Pothole properties measurement through engineering at the University of Maryland in the U.S.
visual 2D recognition and 3D reconstruction, In Currently; he is a senior researcher at Korea Institute of
Proceedings of the ASCE International Conference Civil Engineering and Building Technology. His research
on Computing in Civil Engineering (2012), 553- interest covers intelligent transportation systems,
560. information technology, optimization, paratransit,
logistics, and simulation.
[15] H. Lokeshwor, L. K. Das, and S. K. Sud, Method
for automated assessment of potholes, cracks and Seung-Ki Ryu received the degree in electrical
patches from road surface video clips, Procedia – engineering at the ChungBuk National University in
Social and Behavioral Sciences, Vol. 104 (2013), Korea. Currently, he is a research fellow at Korea Institute
312-321. of Civil Engineering and Building Technology. His
research interest covers intelligent transportation systems,
[16] Koch, C., G. M. Jog, and I. Brilakis, Pothole information technology, ubiquitous city, construction IT
detection with image processing and spectral convergence and logistics.
clustering, Journal of Computing in Civil
Engineering, Vol. 27, No. 4, (2013), 370-378.

608