International Journal of Intelligent Systems Technologies and Applications

Survey of Road Anomalies Detection Methods

Abstract: Automatic road anomalies detection and recognition systems are
essential due to their effect on the comfort and safety of drivers and passengers.
Drivers should be aware of bad road conditions and the existence of anomalies
in routes to avoid accidents, reduce the possibility of car malfunction, and take
the most appropriate way to reach their destinations. These reasons have led to
conducting much research to automatically detect and recognize road anomalies.
The related studies can be categorized into accelerometer-based techniques and
vision-based techniques, which can also be divided based on the possibility of
using deep neural Networks techniques or not. In this paper several studies for
anomalies detection and classification have been reviewed. We also discuss several
types of road anomalies, such as potholes, cracks, and speed bumps. Additionally,
road damage detection techniques used for different types of road anomalies,
challenges, and limitations of current research.

Keywords: Road Anomalies; Computer vision; Deep learning; Image processing

1 Introduction

Good road conditions are an important sign of the development and economic growth of
countries by keeping the movements of goods and people from one place to another as easy
as possible and reducing the costs of transportation.
Road conditions also affect the ride quality and safety because bad road conditions
and anomalies increase the possibility of accidents, which cause more injuries and loss
of lives (Danilescu et al., 2015; Bello-Salau et al., 2019; LOYA and RICARDO, 2019).
So, early detection of road damage is a necessity. Road conditions should be monitored
continuously to detect anomalies and damages as early as possible, because latency in
detecting them increases the number of accidents, traffic jam, and maximizes maintenance
costs.
There are several types of anomalies to be studied which are classified into several
categories. First, damages are categorized as cracks and other damages. Cracks divided into
linear cracks and alligator cracks, each in turn are categorized into different types of cracks.
Other damages are rutting, speed pumps, potholes, manholes, separation, white line blur,
and crosswalk blur (Maeda et al., 2018).
Detecting road damage cannot be done with a manual check, because it costs time, money,
and slows down road maintenance process. So, supporting technology is needed to detect
this kind of road hazard (Ragnoli et al., 2018; Carlos et al., 2018).
Two basic data collection methodologies were mainly used to detect road damages;
vibration-based detection and computer vision-based detection. The Vibration-based
approaches read data from tri-axial accelerometer, analyze data, extract pattern features,
then predict the type of anomaly. On the other hand, the vision-based techniques capture
images for the roads, prepare them, analyze them to detect anomalies, then recognize
these anomalies. Within these two main methodologies, several machine learning, image
processing, and deep learning techniques have been proposed (Munawar et al., 2021). Deep

Copyright © 20xx Inderscience Enterprises Ltd.

2
learning technologies such as different versions of YOLO (Redmon et al., 2016) , R-CNN
detector series (Girshick et al., 2014; Girshick, 2015; Ren et al., 2015) with different CNNs
(VGG16, ResNet-152, ResNet-50 and others) as backbone, are widely used in anomalies
detection approaches for fast and accurate signal or images analysis, anomalies detection,
and classification (Ma et al., 2022; Rastogi et al., 2020).
This review paper discussed state-of-the-art research in road anomalies detection and
classification. The paper focused on studying different types of road anomalies such as:
potholes, cracks, and speed pumps, either each one alone or together in same study. Different
data collectors, methodologies and techniques were analyzed, results of these studies were
compared, and the limitations of each research have been identified and clarified. Finally,
Future plan to cover the research gap is suggested.

2 Data collection for road anomalies detection methodologies

Road anomalies detection research studies are categorized into acceleration-based and
vision-based approaches. For anomalies detection an information should be collected from
roads. Two types of data were collected to be analyzed and used for anomalies detection
accelerometer signals and images or mixed.

2.1 Accelerometer-based

For accelerometer-based approaches, most researchers use smartphone accelerometers

to collect the road data as in (Silva et al., 2017; Seraj et al., 2015)where smartphones
accelerometers were used to detect and classify road anomalies. In (Silva et al., 2017),
the authors proposed a method to detect road anomalies using smartphone sensors. They
collected data using the accelerometer. The collected data are 3-axis accelerometer, latitude,
longitude, speed, timestamp, and anomaly. A system called RoADS was implemented
in (Seraj et al., 2015) to read inertial accelerometer data. Similarly, (Pandey et al., 2022)
used inertial sensor (accelerometer and gyroscope) to collect data from IoS smartphone
installed in the car for pothole detection. Furthermore, the authors in (Martinelli et al., 2022)
equipped a car with sensors to read acceleration data. while in (Sattar et al., 2021) authors
collected raw sensor data and location information by developing a mobile app installed on a
smart device (mobile, or tablet). The collected data was linear accelerometer, rotation vector,
and location information data. These data was collected from Linear accelerometer sensor,
gyroscope, magnetometer and other sensors for road anomalies detection and classification.

2.2 Image-based

On the other hand, the vision-based approaches collects road images either using a mobile
camera or a camera installed on a car, as in (Siriborvornratanakul, 2018)\cite{ (Azhar et al.,
2016; Rasyid et al., 2019; Haq et al., 2019; Danilescu et al., 2015; Shaghouri et al., 2021;
Wang et al., 2018; Doshi and Yilmaz, 2020; Lee et al., 2021; Arya et al., 2021; Mandal
et al., 2018; Dung, 2019; Gopalakrishnan et al., 2017; Jana et al., 2022; Rao et al., 2021;
Dewangan and Sahu, 2020; Yun et al., 2019; Babu et al.).
These images were captured to detect road anomalies and distresses. Images of potholes
in the roads were captured and studied using different tools in (Siriborvornratanakul, 2018;
 Survey of Road Anomalies Detection Methods 3
Azhar et al., 2016; Rasyid et al., 2019; Haq et al., 2019; Danilescu et al., 2015; Shaghouri
et al., 2021). In (Siriborvornratanakul, 2018; Haq et al., 2019; Danilescu et al., 2015;
Shaghouri et al., 2021), cameras were fixed in the cars to collect images of the specified
roads such that in (Siriborvornratanakul, 2018) onboard in-car camera for capturing road
images while in (Danilescu et al., 2015) a camera embedded inside the car were used. The
situation is almost the same in (Shaghouri et al., 2021) , the camera was mounted on the
windshield of the car to capture road images, but the number of captured images was few,
so the authors increased the number of images by downloading images from the Internet.
In (Haq et al., 2019) , the car was equipped with a stereo apparatus, which consists of
two cameras. The videos of the road were taken from both cameras then the images were
captured from the videos. Each point of the road had right and left view to be analyzed and
construct a 3D view of the roads. A data set developed by (Koch and Brilakis, 2011) Koch
et al. were used in (Azhar et al., 2016). This data set contains 120 pavement images; 50
images for training and the other 70 images were used for testing. Also, a wireless portable
camera was used to capture images of the potholes in the roads (Rasyid et al., 2019).
In (Mandal et al., 2018; Gopalakrishnan et al., 2017; Jana et al., 2022; Rao et al., 2021;
Hacğlu and Başağa, 2022; Dung, 2019) , the authors focused on studying cracks on the
roads. Images of cracks were taken using cameras in vehicles as in (Mandal et al., 2018;
Jana et al., 2022; Rao et al., 2021; Hacğlu and Başağa, 2022). In (Mandal et al., 2018) ,
a total of 9053 images were captured from 7 local governments across Japan. Out of the
captured images 7240 images were used for training and the other 1813 were used for
testing. The captured images consist of different types of cracks to be studied. A total of
136 images were used as a data set in (Jana et al., 2022) , where images were divided as 60
percent training images and 40 percent testing images. The images varied between images
that contains cracks and a non-cracked images. While in (Rao et al., 2021) images were
collected using a camera and a data set was built. The data set contains images of size 256
× 256 divided as 2,173 training images (2044 train + 129 validation) and 377 test images.
A data set of 323 images with resolution of 4128 × 2322 were created using a smartphone
camera in (Hacğlu and Başağa, 2022).
The other type of used data sets is a public already generated data sets as in (Dung,
2019; Gopalakrishnan et al., 2017) whereas a public concrete data set of 40000 227*227
crack images, and a subset of 1056 images from the pavement distress images data set from
the Federal Highway Administration’s FHWA’s program were used respectively.
Images were captured in (Dewangan and Sahu, 2020; Yun et al., 2019; Babu et al.;
Arunpriyan et al., 2019; Patil et al., 2020) for the detection of speed bumps. In (Dewangan
and Sahu, 2020). At first, they prepared the primary setup for their study. From black paper
sheet and wooden materials, they designed and built streets and speed bumps in different
forms (marked and unmarked). After that they built their data set by capturing images for
these speed bumps, they took around 550 images. In the next step, they pre-processed the
images then used augmentation techniques to increase the number of images in the database
to be suitable to use with CNN, after these steps they had around 3400 images in the data
set. However, camera and Lidar were used in (Yun et al., 2019) to detect speed bumps. A
Raspberry pi camera were used in (Babu et al.) to collect images of unmarked speed bumps
from Indian roads. While in (Arunpriyan et al., 2019) , road images were captured using a
monocular camera feed placed in front of the vehicle.
Road images and accelerometer data was collected in (Lee et al., 2021; Sprague and
Azar, 2022) for anomalies detection, localization, and severity estimation. In these studies
car was equipped with a smartphone to collect road images from the camera and the
4
corresponding acceleration data using the smartphone accelerometer. The collected images
were used to study the presence or absence of anomalies. When an anomaly is detected
in an image the acceleration data correspondent to that image is retrieved from the server
storing all the collected data to study the severity of that anomaly if the car has passed over
it.

3 Road Anomalies and detection methods

This section presents an overview of the state-of-the art studies for automatic road anomalies
detection and classification. From the detected anomalies side, each study focused on
detecting and analyzing one or more types of road anomalies. Mainly road anomalies are
categorized as cracks, potholes, manholes, rutting, speed pumps, crosswalk blur, white line
blur, and many others. This review paper focuses on the studies of detecting potholes, cracks,
and speed pumps, either each one alone or detecting more than one type of them.
anomalies detection studies used different techniques for anomalies detection,
recognition, and classification such as signal processing, computer vision and image
processing with and without the use of DNN.

3.1 Potholes

A pothole is hole with different sizes occurs in the road because of bad weather or due to
heavy traffic load. Several studies focused on detecting potholes in the roads either from road
accelerometer signals, or from road images as in (Pandey et al., 2022; Siriborvornratanakul,
2018; Azhar et al., 2016; Rasyid et al., 2019; Haq et al., 2019; Danilescu et al., 2015;
Shaghouri et al., 2021).
In (Pandey et al., 2022) , inertial sensors (accelerometer and gyroscope) data were collected
for detecting potholes using a proposed 2-hidden layer 1-D CNN with kernel size 3 model
for features extraction, pothole detection and classification.
In (Siriborvornratanakul, 2018) , the authors proposed a road monitoring algorithm. Using
this algorithm, they can detect, recognize, track, evaluate, and estimate potholes. The
algorithm takes road images, prepares, binarizes them, and applies morphological erosion
then dilation to connect white areas and remove black noises from binarized image.
Finally, they calculated the pothole-probability score by computing two scores; first one is
calculating a difference value between mean intensities of pixels inside and outside each
contour. The second one is calculating a ratio of Nsharp/Nall, where Nsharp is the number of
sharp contour perimeter pixels and Nall is the number of all pixels in the contour perimeter.
Finally, the total score is computed as a contour acceptance score by computing the average
of the two pothole-likelihood scores. If the acceptance score of the contour is lower than a
predefined threshold, it will be rejected. The resulted contours are the detected potholes.
Machine learning algorithms were used in (Azhar et al., 2016; Rasyid et al., 2019) to
classify the images based on pothole/non-pothole images. In (Azhar et al., 2016) , the HOG
algorithm was used for features vector extraction, which in turn entered the Naïve Bayes
classifier. The Naïve Bayes assigns the label to an input image based upon the maximum
posteriori probability as inFigure 1
In (Rasyid et al., 2019) researchers used TensorFlow and OpenCV libraries to detect
potholes in images using TensorFlow with a faster RCNN inception v2 pre-trained model.
Additionally, the sensor device contains a GPS sensor, IMU sensor, external GPS antenna,
 Survey of Road Anomalies Detection Methods 5
Figure 1 Algorithmic Workflow in (Azhar et al., 2016)

and Microcontroller for control of the sensor and sending it to the Processing Unit is used
to localize the pothole. A method for 3D reconstruction of potholes using SIFT key points
detector and disparity map was used to create a 3D scene of the pothole in (Haq et al.,
2019). In (Danilescu et al., 2015) an approach is developed using morphological algorithms
for pothole detection in roads. The pothole detection algorithm goes through the following
steps:

• Cropping road segment from the collected images based on road extraction algorithm
by cropping 50% of its height and 40% of the width (20% from each side).
• Removing the noise from the original images by using Gaussian low pass filter.
• Segmenting the foreground from the background by binarizing the image using Otsu’s
algorithm.
• Skeletonization process is applied to retain the connected pixels only which represent
the potholes.

A real-time pothole detection approach was proposed in (Shaghouri et al., 2021) , where
6
authors experimented three types of object detectors on different frameworks, such as:
Single shot Multi box Detector SSD on TensorFlow framework, YOLOv3 and YOLOv4 on
Darknet framework. All the processing was done on Google Colaboratory (Colab) on images
from a combination of two different data sets. The first one is available online, the second
one is a combination of images from different sources of the internet and images captured
using camera from Lebanese roads. The experiments showed that SSD gives the worst
performance and can’t be used as a real-time detector. While on the other hand, YOLOv4
was the best results with 81% recall, 85% precision, 85.39% mAP, and a processing speed
of 20 frames per second.

3.2 Cracks

Cracks are a kind of distresses in the pavement that causes problems with traffic and
driving safety. Early detection of such deterioration is essential before it becomes too severe.
In (Mandal et al., 2018; Dung, 2019; Gopalakrishnan et al., 2017; Jana et al., 2022; Rao
et al., 2021) researchers focused on studying the presence or absence of road cracks using
different deep neural networks. YOLOv2 is used in (Mandal et al., 2018) , and YOLOv2
uses single CNN with standard layer types such as convolutional with a 3 × 3 kernel and
max pooling with a 2 × 2 kernel. To minimize the data to the shape of 13x13x125 the last
convolutional layer is used with a 1 × 1 kernel. This 13 × 13 structure represents the size
of the grid where the image gets apportioned. All these grid cells predict 5 bounding boxes,
and each box described by seven data elements: the values of x, y, width, and height; the
confidence score; road crack and no-crack probability distribution as inFigure 2

Figure 2 Classification of predicted crack examples: True Positive-(a), False Positive-(b), False
Negative-(c) obtained from YOLO v2 in (Mandal et al., 2018).

In (Dung, 2019) , the authors used Fully Convolutional Network FCN and VGG16 as the
backbone of the FCN to classify each pixel into “crack” or “non-crack” classes. Additionally,
crack density computed using pixel density ratio. While In (Gopalakrishnan et al., 2017) ,
Deep Convolutional Neural Network (DCNN) trained on the ‘big data’ ImageNet database
(VGG16) was used as a feature vector extractor then the vector was entered to several
classifiers such as: Single layer NN, Random Forest, Extremely Randomized Tree, Support
Vector Machine, and Logistic Regression to automatically detect cracks in images of Hot-
Mix Asphalt (HMA) and Portland Cement Concrete (PCC) surfaced pavement.
 Survey of Road Anomalies Detection Methods 7
A method was proposed to detect pavement cracks using Deep Learning with transfer
learning in (Jana et al., 2022). The proposed model was built based on different pre-
trained network architectures namely, Google net, Alexnet and ReseNet101. The network
was designed by tuning the training parameters such as setting the initial value of the
learning rate, initializing epochs, and choosing optimizer then a dataset of 136 images were
used to train and test the proposed model, 60% of images were used as training data the
remaining images were used for testing. The results showed that transfer learning using
pretrained Google net gave the best performance in detecting pavement cracks because of
its adaptability for each iteration, number of layers used in it, and the reduced loss.
Other studies focused on detecting cracks in concrete structure as in (Rao et al., 2021)
using different types of CNN and non-overlapping window (patches) were proposed. The
proposed approach is shown in Figure 3 . A data set was built for this study, the data set
consists of images of crack/non-crack concrete structures divided into 2,173 training images
(2044 train + 129 validation) and 377 test images of size 256 × 256 which produced 32,704
training patches, 2,074 validation patches and 6,032 test patches. The performance of the
proposed approach was tested using 15 different CNN models such as VGG, ResNets,
AlexNet, Inception networks, and others. Results showed that VGG19 gave the best accuracy
of 95%.

Figure 3 Proposed methodology in (Rao et al., 2021).

3.3 Speed bumps

A small raised built area in the road called speed bumps is used to force drivers to drive
slowly on the road. The lack of awareness of the presence of such bumps and driving fast
over them may cause fatal accidents and many other problems. Consequently, automatic
detection and alerting the driver to their presence in advance is necessary and limits these
problems.
8
Another research was conducted for detecting cracks in concrete roads using faster R-CNN
as in (Hacğlu and Başağa, 2022).
Deep learning approaches for detecting speed bumps are proposed in (Dewangan and
Sahu, 2020; Yun et al., 2019; Babu et al.; Arunpriyan et al., 2019; Patil et al., 2020).
In (Dewangan and Sahu, 2020) at first, they prepared the primary setup for their study.
From black paper sheet and wooden materials, they designed and built streets and speed
bumps in different forms (marked and unmarked). After that, they built their data set by
capturing images for these speed bumps, they took around 550 images. In the next step,
they preprocessed the images then used augmentation techniques to increase the number
of images in the database to be suitable to use with CNN, after these steps they had around
3400 images in the data set. After that, they built the CNN with 3*3 and 5*5 filters to extract
features vectors, which is transferred to the pooling layer to extract the defined size feature
vectors. The defined sized vectors are then sent to the fully connected layer for classification
of speed bumps presence. Finally, depending on the size of the boxes drawn around the
speed bump, they calculated the distance between the car and the speed bump, the proposed
methodology shown inFigure 4

Figure 4 Proposed speed bump detection using convolutional neural network (CNN) architecture
in (Dewangan and Sahu, 2020).

Also, camera and Lidar were used in (Yun et al., 2019) to detect speed bumps. First after
preprocessing grayscale images and binarizing them, the pattern of the speed bumps enters
the first detector which is a HaaR classifier to detect the regions of the image which are the
candidates of a speed bump. Then the second detector is used to filter these regions from
the first detector and find only the true speed bumps, using Lidar and camera for this task.
Lidar is used to filter objects that have heights more than the vehicle height then HOG is
used to extract features from the images, and finally the SVM classifier is used to recognize
the speed bumps. The camera and the Lidar are used to calculate the height of the speed
bump and the distance from it.
A new approach for detecting unmarked speed bumps is proposed in (Babu et al.).
The proposed method used image processing techniques to detect unmarked speed bumps
from Indian road images that are collected using a Raspberry pi camera. The images were
captured from video, then went through several steps to detect the presence or absence of
the speed bump in the image and classify the sharpness level of the detected bump. The
model works by at first preprocess the images by converting it from RGB to gray scale
 Survey of Road Anomalies Detection Methods 9
images, then apply a Gaussian filter on it to deal with illumination and blur in the image.
After that, canny edge detection algorithm was used for identifying edges in the image, and
finally Hough transform is used to identify the lines that represents a speed bump based
on the line length. The model was tested on a database of 1385 images that were captured
in different day times, the results showed the average accuracy of correct detection were
95.5%.
Authors proposed a novel method for upcoming speed pumbs detection for self-driving
cars in (Arunpriyan et al., 2019; Patil et al., 2020). In (Arunpriyan et al., 2019) speed bumps
were detected using a deep learning algorithm called SegNet. SegNet is a DNN for semantic
pixel-wise segmentation. While, GAN were used in (Patil et al., 2020) for segmenting speed
bumps by generating an output image with conditioned label, when color image is given as
input as in Figure 5 . The generator network generates fake images which are so close to
the real input images where discriminator network can’t distinguish between real and fake
images at the end of
training.

Figure 5 Proposed speed bump detection using GAN in (Patil et al., 2020).

3.4 Multiple types of anomalies

While some researchers focused on detecting only one type of road anomalies, others
took in consideration the need of detection of more than one type of anomalies. In (Wang
et al., 2018; Doshi and Yilmaz, 2020) authors used Faster R- CNN to detect and classify
damaged roads. The proposed method in (Wang et al., 2018) consists of two stages; first
input images are pre-processed and augmented, then feature extracted using ResNet-152
feature extractor. After that, feature maps enter the Region Proposal Network as an input
then a set of rectangular object proposals with their scores outputted from the network. In the
second stage, Fast R-CNN detector is used for each object proposal to extract fixed-length
features vector by the Region of Interest (RoI) pooling layer from the feature maps. Then,
10
each feature vector will go through a sequence of fully connected layers to predict the class
label of the damage and draw the bounding box. They also studied cracks, crosswalk line,
and manholes. In (Doshi and Yilmaz, 2020) an ensemble model is proposed to detect and
classify road damages (different types of cracks and potholes) efficiently using YOLOv4
object detector.
In (Lee et al., 2021) , a hybrid approach for checking the presence or absences of
road anomalies is proposed, using camera and accelerometer, as shown in figure 2. The
road surface images are captured using a smartphone camera from inside the car, the
images then entered to FCN model to detect anomalies. When the anomaly is detected,
the 3-axis acceleration of the smartphone accelerometer were measured for three seconds.
The Z-direction acceleration value used mainly to detect and identify road anomalies and
determines the severity of the detected anomalies. In general, when the size of the road
surface anomaly increased, the change of the Z-axis acceleration is expected to be larger.
They found that when no anomalies on the road, the maximum variations of the Z- axis
accelerations were less than 2 m/s\^2. However, when anomalies were detected on the
road surface, the variations of the Z-axis accelerations were mostly >2 m/s\^2. Their study
showed that if the detected anomaly is a pothole which have irregular shapes, the acceleration
varied and distributed extensively than in the other cases. But, for manholes, which have
a constant circular shape, the change in value was concentrated on a narrow range. The
severity of road-surface anomalies can be identified by comparing the images with the
maximum variation of Z-axis acceleration. Overall image acquisition process is shown in
Figure 6

Figure 6 Overall image acquisition flow and three-axis accelerations with a smartphone in (Lee
et al., 2021).

Another hybrid model was proposed in (Sprague and Azar, 2022) to asses the condition
of the asphalt roads, using a smartphone mounted on the car dash-board to simultaneously
capture a video of the road and the corresponding acceleration response of a vehicle while
driving. The system developed to analyze the accelerometer data. When an anomaly is
detected the corresponding frame from the video is analyzed and segmented using semantic
segmentation identify the detected anomaly.
Different CNN models for potholes and cracks detection were developed in (Arya et al.,
2021) using: two types of YOLOR, three types of YOLOv5 (Yl, Ym, and Ys), and Faster
RCNN with different backbones such as: VGG16, ResNet50, MobileNetv2, Inception v3,
 Survey of Road Anomalies Detection Methods 11
and finally a proposed CNN called MVGG16 which represents modification of the original
VGG16. Experiments were conducted on a data set built using smartphone camera. The
images are captured in different daylights from different countries. Experiments showed
that Faster R-CNN ResNet50 gave the best accuracy of 91.9%, whereas MobileNetV2 was
the worst with 63.1%. the results also showed that MVGG16 improved the performance of
the original VGG16 but not the best.
(Martinelli et al., 2022) proposed a real-time low-cost system for extracting information
about road conditions. The proposed system read acceleration signals from sensors equipped
on a car. These signals were analyzed using short-time Fourier transform to extract features
to distinguish different of road distresses (potholes, manholes, and cracks) using different
classifiers such as SVM. While in (Sattar et al., 2021) a near real time approach for detecting
and classifying road anomalies was proposed. The authors collected data from several
sensors, then detect and classify anomalies using modified threshold — based and machine
learning approaches (K-means clustering).

4 COMPARATIVE ANALYSIS

This section presents a comparison between the different previously mentioned road
anomalies detection approaches. Different used technologies, main features, limitations,
and results of state of the art studies of each road anomaly detection approach is discussed
bellow.

4.1 Methodology

Different approaches of road anomalies based on the collected and studied road anomalies
are presented here: accelerometer-based , vision-based , and hybrid approaches each with
different anomaly type.
For accelerometer- based approaches as (Silva et al., 2017; Seraj et al., 2015; Martinelli
et al., 2022; Sattar et al., 2021; Pandey et al., 2022) data about road anomalies were collected
using an accelerometer and a gyroscope, the collected signals then were analyzed using
different signal processing techniques. Finally, different types of classifiers were used to
detect and classify road anomalies sum accelerometer based approaches are mentioned in
Table 1 . Using accelerometers arises a problem that because of the use of accelerometer,
vehicle should go over the anomaly to detect it, which may cause a lot of damage or problems
to the var and the driver, or the car could go next to the anomaly and therefore didn’t detect
it.
On the other hand, to overcome this problem computer vision, or even hybrid models
were proposed as in (Siriborvornratanakul, 2018; Azhar et al., 2016; Rasyid et al., 2019;
Haq et al., 2019; Danilescu et al., 2015; Shaghouri et al., 2021; Wang et al., 2018; Doshi
and Yilmaz, 2020; Lee et al., 2021; Arya et al., 2021; Mandal et al., 2018; Dung, 2019;
Gopalakrishnan et al., 2017; Jana et al., 2022; Rao et al., 2021; Dewangan and Sahu, 2020;
Yun et al., 2019; Babu et al.)
for vision-based approaches,
road anomalies were detected from images of the roads. Computer vision and image
processing techniques were applied to the images to preprocess and analyze them, after
that different object detectors were used to detect the anomalies from the image. Finally,
different classifiers and DNN were used to classify the image (has anomalies or not) or
 12

Table 1 Accelerometer- based techniques for road anomalies detection

Anomly type Technology Input Real time Ref
potholes 2-hidden layer 1-D CNN with kernelsize Accelerometer and gyro- No (Pandey et al.,
3 model scope signal 2022)
Signal processingtechniques+ Tree- Smartphone No (Silva et al.,
based classifier accelerationsignal 2017)
Several Types of anomalies
Wavelet decompo-sition analysis +SVM Accelerometer and gyro- Yes (Seraj et al.,
classifier scope signal 2015)
a short-time Fouriertransform + differ-ent Accelerometer signal Yes (Martinelli
classifiers et al., 2022)
Threshold-basedand Several smart devicesensors Near real (Sattar et al.,
unsupervisedmachine as: linearaccelerometer and time 2021)
learningtechniques gyro-scope sensors
 Survey of Road Anomalies Detection Methods 13
classify the detected anomaly. Table 2 shows different vision-based techniques based on
the type of anomalies they studied.
There are other studies that used the two methods together either as a confirmation of
their results or for detecting, localizing, and estimating the severity of the detected anomaly,
and these approaches are summarized in Table 3

4.2 Limitations

Both approaches have limitations either that they focused only on one type of anomalies,
neglecting any other obstacles in the road as in (Siriborvornratanakul, 2018; Azhar et al.,
2016; Haq et al., 2019; Shaghouri et al., 2021; Jana et al., 2022; Babu et al.), or the limited
data sets they used which affects the accuracy obtained especially in the case of using deep
neural networks for classification (Azhar et al., 2016; Jana et al., 2022). Also, because some
studies made lab experiments only and did not apply their work in real-time or on real roads,
and this may affect the accuracy when applied on real roads because of illumination and other
environmental conditions as in (Dewangan and Sahu, 2020). Additionally, most research
focuses on detecting the anomaly without taking in consideration its size, height, depth,
etc. which is important to the decision the driver should take as in (Siriborvornratanakul,
2018; Azhar et al., 2016; Rasyid et al., 2019; Wang et al., 2018; Mandal et al., 2018;
Gopalakrishnan et al., 2017; Dewangan and Sahu, 2020). Finally, In the studies that used
cameras placed in the car, such as (Siriborvornratanakul, 2018; Rasyid et al., 2019; Dung,
2019) the calibration of camera’s place is a limitation for their work because if camera is
not in a specific angle it cannot take proper images to be used for anomalies detection and
may cause misclassification of these anomalies. Table 4 presents the limitations of previous
studies.

4.3 Accuracy

From the results side, the results obtained from the different studies showed difference in
performance of these approaches based on the used method for analyzing images and the
classification techniques chosen in those papers. For pothole detection different methods and
classifiers were used in (Siriborvornratanakul, 2018) without the use of machine learning
techniques sample of their results shown in Figure 7.
Other methods for pothole detection used machine learning as in (Azhar et al., 2016)
HOG feature extractor with Naïve Bayes classifier was used, the accuracy of their results
was 90%. While in (Rasyid et al., 2019) faster RCNN inception v2 pretrained model was
used an example of the results is in Figure 8
YOLOv4 was used in (Shaghouri et al., 2021) for pothole detection and gain a precision
of 85%. Other approaches are proposed for the detection of speed bumps such as (Dewangan
and Sahu, 2020; Yun et al., 2019; Babu et al.; Patil et al., 2020). The overall accuracy of
detecting speed bumps was 98.54% in (Dewangan and Sahu, 2020) with the use of proposed
CNN. However, in (Yun et al., 2019) researchers used Haar cascade classifier for detecting
candidate regions of speed bumps, then HOG feature extractor with SVM classifier are used
as second detector to detect speed bumps. The average accuracy of detecting speed bumps
was 85.2%, additionally their method takes 10ms processing time than other methods.
On the other hand, in (Babu et al.)researchers concentrated on the detection of unmarked
speed bumps, The model was tested on a database of 1385 images that were captured
in different day times, the results showed the average accuracy of correct detection were
95.5%.
Table 2 Vision- based techniques for road anomalies
Anomaly type Technology Input Real time Ref
Computer visiontechniques Images from on-boardcamera, Images No (Siriborvornratanakul,
fromvideos 2018;
Potholes Danilescu
14
et al., 2015)
HOG feature ex-tractor with Na ïveBayes Images from onlineGoogle + Images No (Azhar et al.,
classifier from awebcam in a robot 2016)
SIFT detector No (Haq et al.,
2019)
Single Shot Detec-tor (SSD) Images from differentdatasets Yes (Shaghouri
et al., 2021)
YOLOv3, YOLOv4 mages from differentdatasets Yes (Shaghouri
et al., 2021)
YOLOv2 Images from a smart-phone camera No (Mandal et al.,
2018)
Cracks
FCN with VGG16as its backbone public concrete crackdataset of 40,000 No (Dung, 2019)
227 ×227-pixel images
Different types ofCNN A subset of the pavementdistress (Gopalakrishnan
(VGG16,AlexNet,ResNet101,GoogleNet) images datasetfrom the FHWA’s, self- et al., 2017;
built road images dataset Jana et al.,
2022; Rao
et al., 2021)
Faster R-CNN Smartphone images No (Hacğlu and
Başağa, 2022)
Image processingtechniques Images from a Raspberrypi camera Yes (Dewangan
and Sahu,
Speed bumps
2020)
CNN with 3*3 and5*5 filters Images of lab preparedspeed bumps Yes (Babu et al.)
segNet Images from monocular camera Yes (Arunpriyan
et al., 2019)
Generative adver-sial network GAN Images from monocularcamera Yes (Patil et al.,
2020)
Several types of Fast and faster R-CNN Images from a smart-phone camera No (Wang et al.,
anomalies 2018; Arya
et al., 2021)
YOLOv4, YOLOv5 Road damage datasetsprovided by No (Doshi and
the IEEEBigData Cup Challenge2020, Yilmaz, 2020;
large road dam-age dataset of smart- Arya et al.,
phone images 2021)
Table 3 Hybrid techniques for road anomalies detection
Anomaly type Technology Input Real-time Ref
Potholes (De-tection and lo- Faster R-CNN GPSsensor, IMU Images from wireless portable No (Rasyid et al.,
calization) sensor camera, GPSdata 2019)
More than one type of FCN + signal pro-cessing Images from Smart-phone No (Lee et al.,
anoma-lies techniques+ Histogram camera + smart-phone 2021)
ofacceleration signals(estimate the acceleration signal
sever-ity of the anomaly)
signal process-ing mages from Smart-phone No (Sprague and
techniques+ResNet50 camera + smart-phone Azar, 2022)
acceleration signal
Survey of Road Anomalies Detection Methods

Speed bumps HOG feature ex-tractor + Haar Camera Images No (Yun et al.,
andSVM classifiers +LiDAr (for 2019)
calcu-lating distance be-tween
speed bumpand car)
15
Table 4 limitations of the state of the are road anomalies detection techniques
Road anomalies de-tection approach Limitation Ref
Accelorometer-based approaches 1- vehicle should go over theanomaly to detect it, which maycause (Silva et al., 2017; Seraj et al.,
a lot of damage or prob-lems to the car and the drive 2015; Martinelli et al., 2022;
2- vehicle could go next tonot over the anomaly so theanomaly Sattar et al., 2021; Pandey
will be missed be-cause there will be no variationin the et al., 2022)
acceleration data toindicate anomaly presencehere.
16

Focusing on one type of anomaliesneglecting any other types (Siriborvornratanakul, 2018;

Azhar et al., 2016; Rasyid
vision-based approaches et al., 2019; Haq et al.,
2019; Danilescu et al., 2015;
Shaghouri et al., 2021;
Mandal et al., 2018; Dung,
2019; Gopalakrishnan et al.,
2017; Jana et al., 2022; Rao
et al., 2021; Hacğlu and
Başağa, 2022; Dewangan and
Sahu, 2020; Yun et al., 2019;
Babu et al.; Arunpriyan et al.,
2019; Patil et al., 2020)
Limited data sets (Azhar et al., 2016; Jana
et al., 2022; Patil et al., 2020;
Hacğlu and Başağa, 2022)
Lab experiments only (Dewangan and Sahu, 2020)
Focusing on detecting the anomaly without estimating its severity (Siriborvornratanakul, 2018;
level Azhar et al., 2016; Rasyid
et al., 2019; Wang et al.,
2018; Mandal et al., 2018;
Gopalakrishnan et al., 2017;
Dewangan and Sahu, 2020;
Patil et al., 2020; Arunpriyan
et al., 2019; Hacğlu and
Başağa, 2022)
Calipration of camera in the car (Siriborvornratanakul, 2018;
Rasyid et al., 2019; Dung,
2019; Doshi and Yilmaz,
2020; Lee et al., 2021; Arya
et al., 2021; Mandal et al.,
2018; Gopalakrishnan et al.,
2017; Jana et al., 2022; Rao
et al., 2021; Dewangan and
Sahu, 2020; Yun et al., 2019;
Babu et al.)
 Survey of Road Anomalies Detection Methods 17

Figure 7 One pothole detection result (Siriborvornratanakul, 2018).

Figure 8 Pothole detection result (Rasyid et al., 2019).

18
IOU score were calculated in (Patil et al., 2020) for estimating how close is the output
segmented speed pump to the input one. Mean IOU score was 93.8%, some of GAN results
for peed bumps detection are shown in Figure 9

Figure 9 Speed bump detection using GAN (Patil et al., 2020).

For crack detection different deep neural networks were used. YOLOv2 was used
in (Mandal et al., 2018) it achieved an average F1 score of 0.8780 for distress detection. It
gained high accuracy when talking about alligator cracks detection, but it had problems with
recognition of two types of transverse cracks with F1 scores of 0.7137 and 0.6885. FCN with
VGG16, Inceptionv3, and ResNet50 as backbone was tested in (Dung, 2019) the obtained
accuracy was almost 99.9 for VGG16 and InceptionV3-based classifiers in contrast with
97.5% when ResNet classifier was used. However, in (Gopalakrishnan et al., 2017) different
deep NN are tested for crack detection as Single layer NN, Random Forest, Extremely
Randomized Tree, Support Vector Machine, and Logistic Regression. The Best accuracy is
90% gained when using Single NN, while the worst one is 86% when using Random Forest.
While in (Rao et al., 2021) after testing different types of CNN models, the results showed
that VGG19 gave the best accuracy of 95%. Faster R-CNN as used in (Hacğlu and Başağa,
2022) for concrete road cracks detection. The model accuracy was 100% if it is tested in
a sunny day. The condition is different when testing it in sunset, the results became 50%
of the original accuracy. From the illumination perspective the detection accuracy differs
so that about 70% and 15% of cracks were detected at 6:00–7:00 pm and at 7:00–8:00 pm,
respectively. Consequently, the accuracy decreased as the darkness increased.
 Survey of Road Anomalies Detection Methods 19
Finally, some studies take in consideration different types of anomalies for example
in (Seraj et al., 2015) FCN model was used for classification of different types of anomalies
with accuracy of 87.05% if all 39 extracted features were used. But to enhance the results
WEKA was used to determine the most important features, WEKA outputted 10 features.
The accuracy of the FCN model improved to become 87.56. Also, in (Arya et al., 2021)
different CNN models using: two types of YOLOR, three types of YOLOv5 (Yl, Ym, and
Ys), and Faster RCNN with different backbones such as: VGG16, ResNet50, MobileNetv2,
Inception v3, and finally a proposed CNN called MVGG16 which represents modification
of the original VGG16 were tested for road damages detection. Experiments showed that
Faster R-CNN ResNet50 gave the best accuracy of 91.9%, whereas MobileNetV2 was the
worst with 63.1%. While in (Sprague and Azar, 2022) the proposed hybrid model achieved
84% recall and 88% precision rates. Despite the good performance this approach have
limited and poor performance when tested with small acceleration response anomalies ,
such as traverse cracks.

5 Conclusion and future work

The presence of anomalies on roads causes severe harm to the people and vehicles and
affects economies. So, automatic detection of these anomalies is a necessity. Road anomalies
detection techniques are divided into two main approaches: vision-based approach, and
accelerometer- based approach. This paper has presented a review of different state-of-the-
art approaches and techniques for road anomalies detection. Different developed models
and used technologies are discussed in this paper. In addition, comparisons between these
studies are conducted, limitations were identified and their results were mentioned.
To overcome the discussed problems, our goal is to develop a new model for road
anomalies detection. The proposed methodology will work on finding different types of
road anomalies, determine their measurements, and transfer this information to maps such
as Google maps or WAZE. The proposed technique will start by collecting images of roads
anomalies using drones. Then prepare these images and analyze them. Computer vision and
machine learning techniques will be used for feature extraction and classification process.
Finally, show these results to the driver on maps.

References

Arunpriyan, J., Variyar, V. V., Soman, K. P., and Adarsh, S. (2019). ‘Real-time speed
bump detection using image segmentation for autonomous vehicles’. In International
Conference on Intelligent Computing, Information and Control Systems, pp. pages 308–
315.

Arya, D., Maeda, H., Ghosh, S. K., Toshniwal, D., Mraz, A., Kashiyama, T., and Sekimoto,
Y. (2021). ‘Deep learning-based road damage detection and classification for multiple
countries’. Automation in Construction, Vol 132, pp. 103935.

Azhar, K., Murtaza, F., Yousaf, M. H., and Habib, H. A. (2016). ‘Computer vision based
detection and localization of potholes in asphalt pavement images’. In 2016 IEEE
Canadian Conference on Electrical and Computer Engineering (CCECE), pp. pages 1–5.
20
Babu, C. N. K., Priya, W. D., and Srihari, T. ‘Real-Time Detection of Unmarked Speed
Bump for Indian Roads’. European Journal of Molecular & Clinical Medicine, Vol
7, No 4, pp. 2020.

Bello-Salau, H., Onumanyi, A. J., Salawudeen, A. T., Mu’azu, M. B., and Oyinbo, A. M.
(2019). ‘An Examination of Different Vision based Approaches for Road Anomaly
Detection’. In 2019 2nd International Conference of the IEEE Nigeria Computer Chapter
(NigeriaComputConf), pp. pages 1–6.

Carlos, M. R., Aragón, M. E., González, L. C., Escalante, H. J., and Martínez, F.
(2018). ‘Evaluation of detection approaches for road anomalies based on accelerometer
readings—Addressing who’s who’. IEEE Transactions on Intelligent Transportation
Systems, Vol 19, No 10, pp. 3334–3343.

Danilescu, D., Lodin, A., Grama, L., and Rusu, C. (2015). ‘Road anomalies detection
using basic morphological algorithms’. Carpathian Journal of Electronic and Computer
Engineering, Vol 8, No 2, pp. 15.

Dewangan, D. K. and Sahu, S. P. (2020). ‘Deep learning-based speed bump detection model
for intelligent vehicle system using Raspberry Pi’. IEEE sensors journal, Vol 21, No 3,
pp. 3570–3578.

Doshi, K. and Yilmaz, Y. (2020). ‘Road damage detection using deep ensemble learning’.
In 2020 IEEE International Conference on Big Data (Big Data), pp. pages 5540–5544.

Dung, C. V. (2019). ‘Autonomous concrete crack detection using deep fully convolutional
neural network’. Automation in Construction, Vol 99, pp. 52–58.

Girshick, R. (2015). ‘Fast r-cnn’. In Proceedings of the IEEE international conference on

computer vision, pp. pages 1440–1448.

Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014). ‘Rich feature hierarchies
for accurate object detection and semantic segmentation’. In Proceedings of the IEEE
conference on computer vision and pattern recognition, pp. pages 580–587.

Gopalakrishnan, K., Khaitan, S. K., Choudhary, A., and Agrawal, A. (2017). ‘Deep
convolutional neural networks with transfer learning for computer vision-based data-
driven pavement distress detection’. Construction and building materials, Vol 157, pp.
322–330.

Hacğlu, K. and Başağa, H. B. (2022). ‘Concrete road crack detection using deep learning-
based faster R-CNN method’. Iranian Journal of Science and Technology, Transactions
of Civil Engineering, Vol 46, No 2, pp. 1621–1633.

Haq, M. U. U., Ashfaque, M., Mathavan, S., Kamal, K., and Ahmed, A. (2019). ‘Stereo-
based 3D reconstruction of potholes by a hybrid, dense matching scheme’. IEEE Sensors
Journal, Vol 19, No 10, pp. 3807–3817.

Jana, S., Thangam, S., Kishore, A., Kumar, V. S., and Vandana, S. (2022). ‘Transfer
learning based deep convolutional neural network model for pavement crack detection
from images’. International Journal of Nonlinear Analysis and Applications, Vol 13, No
1, pp. 1209–1223.
 Survey of Road Anomalies Detection Methods 21
Koch, C. and Brilakis, I. (2011). ‘Pothole detection in asphalt pavement images’. Advanced
Engineering Informatics, Vol 25, No 3, pp. 507–515.

Lee, T., Chun, C., and Ryu, S.-K. (2021). ‘Detection of road-surface anomalies using a
smartphone camera and accelerometer’. Sensors, Vol 21, No 2, pp. 561.

LOYA, C. and RICARDO, M. (2019). ‘A MACHINE LEARNING APPROACH FOR

SMARTPHONE-BASED SENSING OF ROADS AND DRIVING STYLE’.

Ma, N., Fan, J., Wang, W., Wu, J., Jiang, Y., Xie, L., and Fan, R. (2022). ‘Computer vision for
road imaging and pothole detection: A state-of-the-art review of systems and algorithms’.
arXiv preprint arXiv:2204.13590.

Maeda, H., Sekimoto, Y., Seto, T., Kashiyama, T., and Omata, H. (2018). ‘Road damage
detection using deep neural networks with images captured through a smartphone’. arXiv
preprint arXiv:1801.09454.

Mandal, V., Uong, L., and Adu-Gyamfi, Y. (2018). ‘Automated road crack detection using
deep convolutional neural networks’. In 2018 IEEE International Conference on Big
Data (Big Data), pp. pages 5212–5215.

Martinelli, A., Meocci, M., Dolfi, M., Branzi, V., Morosi, S., Argenti, F., Berzi, L.,
and Consumi, T. (2022). ‘Road Surface Anomaly Assessment Using Low-Cost
Accelerometers: A Machine Learning Approach’. Sensors, Vol 22, No 10, pp. 3788.

Munawar, H. S., Hammad, A. W., Haddad, A., Soares, C. A. P., and Waller, S. T. (2021).
‘Image-based crack detection methods: A review’. Infrastructures, Vol 6, No 8, pp. 115.

Pandey, A. K., Iqbal, R., Maniak, T., Karyotis, C., Akuma, S., and Palade, V. (2022).
‘Convolution neural networks for pothole detection of critical road infrastructure’.
Computers and Electrical Engineering, Vol 99, pp. 107725.

Patil, S. O., Variyar, V. S., and Soman, K. P. (2020). ‘Speed bump segmentation an
application of conditional generative adversarial network for self-driving vehicles’. In
2020 Fourth International Conference on Computing Methodologies and Communication
(ICCMC), pp. pages 935–939.

Ragnoli, A., Blasiis, M. R. D., and Benedetto, A. D. (2018). ‘Pavement distress detection
methods: A review’. Infrastructures, Vol 3, No 4, pp. 58.

Rao, A. S., Nguyen, T., Palaniswami, M., and Ngo, T. (2021). ‘Vision-based automated crack
detection using convolutional neural networks for condition assessment of infrastructure’.
Structural Health Monitoring, Vol 20, No 4, pp. 2124–2142.

Rastogi, R., Kumar, U., Kashyap, A., Jindal, S., and Pahwa, S. (2020). ‘A Comparative
Evaluation of the Deep Learning Algorithms for Pothole Detection’. In 2020 IEEE 17th
India Council International Conference (INDICON), pp. pages 1–6.

Rasyid, A., Albaab, M. R. U., Falah, M. F., Panduman, Y. Y. F., Yusuf, A. A., Basuki, D. K.,
Tjahjono, A., Budiarti, R. P. N., Sukaridhoto, S., and Yudianto, F. (2019). ‘Pothole visual
detection using machine learning method integrated with internet of thing video streaming
platform’. In 2019 International Electronics Symposium (IES), pp. pages 672–675.
22
Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). ‘You only look once: Unified,
real-time object detection’. In Proceedings of the IEEE conference on computer vision
and pattern recognition, pp. pages 779–788.

Ren, S., He, K., Girshick, R., and Sun, J. (2015). ‘Faster r-cnn: Towards real-time object
detection with region proposal networks’. Advances in neural information processing
systems, Vol 28,.
Sattar, S., Li, S., and Chapman, M. (2021). ‘Developing a near real-time road surface
anomaly detection approach for road surface monitoring’. Measurement, Vol 185, pp.
109990.
Seraj, F., van der Zwaag, B. J., Dilo, A., Luarasi, T., and Havinga, P. (2015). ‘RoADS: A
road pavement monitoring system for anomaly detection using smart phones’.
Shaghouri, A. A., Alkhatib, R., and Berjaoui, S. (2021). ‘Real-Time Pothole Detection
Using Deep Learning’. arXiv preprint arXiv:2107.06356.

Silva, N., Soares, J., Shah, V., Santos, M. Y., and Rodrigues, H. (2017). ‘Anomaly detection
in roads with a data mining approach’. Procedia computer science, Vol 121, pp. 415–422.
Siriborvornratanakul, T. (2018). ‘An automatic road distress visual inspection system using
an onboard in-car camera’. Advances in Multimedia, Vol 2018,.

Sprague, W. and Azar, E. R. (2022). ‘Integrating acceleration signal processing and image
segmentation for condition assessment of asphalt roads’. Canadian Journal of Civil
Engineering, Vol 99, No 999, pp. 1–13.
Wang, W., Wu, B., Yang, S., and Wang, Z. (2018). ‘Road damage detection and classification
with Faster R-CNN’. In 2018 IEEE international conference on big data (Big data), pp.
pages 5220–5223.
Yun, H.-S., Kim, T.-H., and Park, T.-H. (2019). ‘Speed-bump detection for autonomous
vehicles by lidar and camera’. Journal of Electrical Engineering & Technology, Vol
14, No 5, pp. 2155–2162.