VDIS: A System for Morphological Detection and Identification of Vehicles in RGB Images

Journal of Mason Graduate Research
Volume 2 Number 2 Year 2015
© George Mason University, Fairfax, VA
ISBN: 2327-0764
Pages: 82-95
VDIS: A System for Morphological Detection and
Identification of Vehicles in RGB Images
RON MAHABIR
KEVIN GONZALES
JARED SILK
Department of Geography and Geoinformation Science
George Mason University
With the growth of urban centers worldwide, the number of vehicles in and around
these areas has also increased. Traffic-related data plays an important role in
spatial planning, for example, optimizing road networks and in the estimation or
simulation of air and noise pollution. This information is important as it reflects
the changes taking place around us. Additionally, data collected can be used for a
wide array of applications including law enforcement, fleet management, and
supporting other analyses at varying scales. In this paper, we present a method for
the detection and identification of vehicles from low altitude, high spatial resolution
Red Blue Green (RGB) images, utilizing both object spectra and image
morphology. Results show an identification performance upwards of 62% with
false positives occurring from the use of images with sun glare and vehicles with
similar spectra values.
VDIS: A System for Morphological Detection and Identification of Vehicles in RGB Images
With the increasing growth of urban centers around the world, the number of vehicles
within and near these areas is also on the rise (Gaurav and Khisty, 1998). This has led to issues
in urban areas, including increases in traffic congestion, accident rates, air pollution, and
commute times (Drakakis-Smith, 2000). As a result, data on vehicles plays an important role in
many transportation and urban planning applications, provides a useful indicator of expanding
economic growth and consumption (East-West Center, 2002), and can provide useful insight on

VDIS 83
crisis or disaster severity. There is, therefore, considerable attention being placed on the
development of intelligent traffic control systems (Eslami and Faez, 2010; Stilla et al., 2004).
Presently, systems to capture transportation data mainly use ground sensors like induction
loops, bridge sensors and stationary cameras (Hasan et al., 2014; Eikvil et al., 2009; Zheng and
Li, 2007). These sensors are often located long distances from each other (Eslami and Faez,
2010) and provide only local measurement of traffic (Stilla et al., 2004). This configuration,
along with the limited viewing angles of most sensors in use today, leads to gaps in data
collection. Larger areas of collected data could be useful to better understand traffic dynamics
(Eikvil et al., 2009); however, this may be cost prohibitive if relying on traditional sensor
networks alone. Newer methods of collecting transportation data use sensors mounted on
various air and spaceborne platforms, which produce images that represent data captured for a
wider geographic area (Mahabir and Al-Tahir, 2008; Poli, 1999) along with the possibility for
multiple viewing angles.
Images captured from sensors like other forms of data must be processed in order to be
useful in any context. The goal of such processing is the extraction of information permitting the
detection and discrimination of vehicles in various image sets. High and very high resolution,
less than 4 meter ground sampling distance (Moeller 2005), visible Red Green Blue (RGB)
imagery is more readily accessible because of Google Earth and similar image visualization
platforms. Therefore, RGB images offer a suitable option, or at least an alternative, for
collecting transportation data.
In this study we examine the problem of vehicle detection and identification from RGB
images. The study area is a major highway in the United States for which several segments of
road were available. We propose a method using object spectra and image morphology for first
detecting the locations of vehicles in RGB images, followed by an identification process to
separate vehicles. In section 2, an overview of data and software used is given. Section 3
discusses the methodology, while section 4 presents the results and analysis from various
experiments. Finally, section 5 concludes this paper with a brief discussion on areas for future
research.

VDIS 84
Data and Software
Data on four segments of a highway in the United States were collected from the Center
for Geospatial Intelligence and Geoinformatics at George Mason University, Fairfax, Virginia.
Segments were both overlapping and disjointed, and represented different sequential images of
the road at different times. The data were collected at low altitude and provided in RGB format
(.jpg) at a spatial resolution of about 1m, with atmospheric and radiometric corrections already
applied on acquisition. The geospatial software package ENVI, version 4.7 (ITT Visual
Information Solutions, Colorado, USA), was used for processing the data.
Methods
The methodology used is divided into two main parts: vehicle detection followed by
vehicle identification. Figure 1 gives an overview of the main components of the Vehicle
Detection and Identification System (VDIS) and the first two sub-sections provide a summary of
each component applied to a control image. In the third sub-section, references are given with
respect to the various steps used to test the VDIS method.
Vehicle detection
Training classes consisting of pixels representing regions outside of roads were first
selected from the RGB image. These training classes were then used as input into a
parallelepiped classification algorithm to create a binary mask for extracting the areas of interest
(AOI), that is, areas containing roads and vehicles only. Following the removal of non-road
areas, morphological operations of opening followed by dilation were applied to the output to
further remove unwanted noise from the AOI binary mask.
Next, the parallelepiped classifier was used once again with the RGB image. Also
included as inputs were the use of the binary AOI mask of vehicles and roads, and selected road
ROI pixels, used for creating a composite binary mask for eliminating both roads and the non-
AOI areas that existed outside the extent of roads. Results of this process created a binary mask
with vehicle regions being detected with values of 1 and contained within the extent of roads.
Noise from small objects and the median between road lanes contained within vehicle regions is
also present in the output of this stage of the process. A median image filter, typically used for
removing noise in images, could not be used at this point because of similarity in the size of
objects represented by noise and by some positive AOI output. To remove noise the values of

VDIS 85
the output were first inverted. This resulted in the locations of vehicles now being represented as
regions of 0 values. Next, a morphological chain of operations – opening and eroding, followed
dilation, were used to completely remove unwanted noise.
In the final stage of the vehicle detection process, the inverted and cleaned image is used
to create a binary mask. This mask consisted of the combined product of the output of the
morphological opening operation used to clean the data and its improved successor.
Morphological dilation was then applied to the mask. The output of this final step of the
detection process was a mask that showed regions where vehicles were located in the image with
pixel values of 1, and everything else with pixel values of 0.
Vehicle Identification
Object spectra for various vehicles in the control image were used to build a spectral
library. Two colors for vehicles were chosen: red and white. The spectral library was then used
as input into a minimum distance classification process in order to detect vehicles represented by
different colors. Red vehicles were targeted first. A median filter was applied to the result of the
classification process to reduce noise, and histogram stretching was applied to threshold the
output and further reduce unwanted noise. A second median filter was then used to remove
noise from the new output, following which the mask resulting from the vehicle detection
process was applied. The final output of this process produced vehicle regions with values of 1,
identifying red color vehicles, and pixel values of 0 for everything else. The processes outlined
were then repeated to locate white vehicles in the image scene.
Testing
The vehicle detection and vehicle identification processes were applied to three test RGB
images collected for different segments along the same road network as the control image.
Confusion matrices were then generated for each of the test cases for determining the
identification performance of VDIS.
Results and Analysis
Figures 2 to 7 show the outputs of the various steps outlined in section 3 for the control
image used. Figure 6 shows that one region (highlighted in red) was wrongly detected as a
vehicle region. This figure also shows that the shadows of vehicles are detected as regions. In
Figure 7, all vehicles matching the color vehicle being searched for (highlighted in green) were

VDIS 86
identified using VDIS without any misidentifications. The omission and false positive results of
using VDIS on three different test images for the detection process are shown on Table 1, while
Figures 8 to 10 show the end results of the identification process. For each image in Figures 8 to
10, the associated confusion matrix used for deriving identification performance is also shown.
The results in Table 1 show that the detection process missed smaller vehicles, like
motorcycles. It also wrongly detected part of a truck’s shadow as a separate vehicle region. In
Figures 8 to 10, the overall identification performance values of test images were 75%, 83%, and
62%, respectfully. Closer examination of the results showed that the RGB image with the lowest
identification results (Image 3) contained a larger number of vehicles of similar spectral value
and higher sun glare content.
Discussion and conclusion
Transportation data, for example, on traffic density, location and trajectory, is a valuable
resource for understanding the dynamics of road networks and the interlinked processes taking
place around them. This is especially important since the number of vehicles on roadways has
been increasing. Analysis of such data has already led to critical findings supporting changes in
policy, including traffic modeling and simulation, emission reduction, efficient use of
infrastructure, and extension planning of road networks (Kozempel and Reulke, 2009;
Gerhardinger et al., 2005). China, for example, has implemented an auction and lottery of
license plates to control the number of road vehicles (Feng and Li, 2013). This system was
introduced in response to the growth of greenhouse gases emitted from the vehicle population. In
the United States, the increasing number of vehicles has been linked with degradation of air
quality and aquatic sediments (Van Metre et al., 2000). In Turkey, work by Yalmaz et al. (2009)
suggests that the increase in road vehicles has contributed considerably to increases in city
temperature, a leading cause of urban heat islands. The problem of increasing numbers of
vehicles is expected to be especially acute in Asian countries, where road congestion and
accident levels are some of the worst in the world (East-West Center, 2002) and where growing
poverty continues to be a daily challenge (Lall et al.,2008). Vehicle data can provide
information on how well an affected area is recovering from disaster or crisis (Gerhardinger et
al,. 2005). Images collected at various locations are especially useful in law enforcement to

VDIS 87
assist in tracking selected objects across multiple scenes which vary with location. It is therefore
for critical to collect information on transportation networks.
This paper presented a method for the detection and identification of vehicles from low
altitude RGB images, using a series of morphological operations and classifications based on
object spectra, to uniquely target selected color vehicles. Images can be captured at both high
spatial and temporal resolutions, important properties for separating vehicles from other objects
in images. Such applications of data capture are especially useful during mass events or
disasters, which result in huge congestions or road blocks. Additionally, snapshots along entire
road networks could offer more insight into the distribution of vehicles in a region and overcome
challenges in using traditional counting equipment (Eikvil et al., 2009). Results show that the
VDIS method performs well: upwards of 62% accuracy in identifying vehicles in images that
contain lower amounts of sun glare and higher contrast between vehicular spectra. In the image
with the lowest identification performance there exists a strong presence of vegetation compared
to other images; however, this would not affect final results, because the series of morphological
operations were used to remove the median separating roadways, where vegetation usually
exists. The processing of the images is not straightforward. For example, there are problems
with heterogeneity of vehicle properties (e.g. body styles and colors), uncontrolled lighting,
partial occlusions (e.g. by trees), and other undesired phenomena (e.g. strong sunlight reflections
from windshields) (Krawiec et al. 2010). Also, separation of objects becomes increasing
difficult with increased degradation of image quality, leading to cross contamination of object
spectra values. One possible solution for overcoming this challenge is the acquisition of spectral
signatures for a wider variety of objects contained in images, that is, extending data collection
along more electromagnetic wavelengths. However, this option is not always available and
provides a challenge for developing countries around the world with already constrained
budgets. Reduction in the detection results across test images occurred as a result of similar
spectral values of vehicles and their respective shadows. This becomes a problem since
operations were arranged in such a way as to first capture vehicles and their shadows as a single
object. This larger (combined) object was then used to remove smaller noise objects. The
detection process also did not pick up regions representing smaller vehicles, like motorcycles.
The settings used for morphological processes would have identified these small areas as noise

VDIS 88
and removed them. Furthermore, shadowed areas behind trucks in the image were wrongly
detected as separate objects.
The findings of this research suggest various possibilities for VDIS. It can first be used
as an image compression method for reducing the amount of data being stored. This is
especially helpful if only the locations of vehicles are of interest. VDIS can also be implemented
as a low budget approach for the detection of vehicles to supplement or corroborate data from
existing sources, or data currently being collected, such as data collected by traditional sensor
networks. Furthermore, VDIS may even provide a suitable option for processing data collected
between successive high quality data collection events, including natural disasters, such as
floods, or riot events, such as the Occupy Wall Street movement, using low altitude airborne
platforms such as unmanned aerial vehicles.
Finally, several areas for future work were identified for improving VDIS. The method
can benefit from becoming more flexible, compared to the rigid chained sequence of
morphological operations used in this study. This will require investigation into other methods
used for detecting vehicles in imagery, including the use of background subtraction and Kalman
filters (Jun-Wei et al., 2006; Morris and Trivedi, 2006; Tseng et al., 2002; Young-Kee and Ho,
1999), shape features (She et al., 2004), bayesian background transformation (Sharma et al.,
2006), multiple kernel learning (Liang et al., 2012), and pairing of headlights in the case of
nighttime imagery (Cheng and Hsu, 2011). The goal would be working towards a fully
automated approach as demonstrated by Hinz et al. (2003) and Der et al. (2004). Second, only
limited vehicular spectra were collected and used for both calibration and testing purposes.
Given the wide spectrum of colors of vehicles on roadways today, the use of much larger
spectral libraries and better quality images with higher spatial and spectral resolution would be
expected to improve the performance of VDIS. Third, a greater number of test samples, using
more images, needs to be used to further test the robustness of VDIS in order to provide more
conclusive results. Finally, the approach used to detect and identify vehicle regions could be
enhanced by better segmentation approaches.
Author Note
The authors would like to thank Dr. Anthony Stefanidis and Dr. Caixia Wang from the
Center for Geospatial Intelligence and Geoinformatics Research Lab, George Mason University,
Fairfax, Virginia for discussion, data and for making this research possible.

VDIS 89
References
Cheng, H. & Hsu, S. (2011). Intelligent highway traffic surveillance with self-diagnosis abilities.
Intelligent Transportation Systems, 12(4), 1462–1472.
Der, S., Chan, A., Nasrabadi, N & Kwon, H. (2004). Automated vehicle detection in forward-
looking infrared imagery. Applied optics, 43(2), 333-348.
Drakakis-Smith, D (2000). Third World Cities (2nd edition). Routledge, London, UK, p. 208.
East-West Center (2002). The future of population in Asia. East-West Center. Retrieved from
http://www.eastwestcenter.org/fileadmin/stored/misc/FuturePop01Frontmatter.pdf.
Eikvil, L., Aurdal, L. & Koren, H. (2009). Classification-based vehicle detection in high-
resolution satellite images. International Journal of Photogrammetry and Remote
Sensing, 64(1), 65-72.
Eslami, M. & Karim, F. (2010). Automatic traffic monitoring from satellite images using
artificial immune system. Structural, Syntactic, and Statistical Pattern Recognition.
Lecture Notes in Computer Science, 6218, 170-179.
Feng, S. & Li, Q. (2013). Car Ownership Control in Chinese Mega Cities: Shanghai, Beijing and
Guangzhou. Journeys. pp. 40-49.
Gerhardinger, A., Ehrlich, D. & Pesaresi, M. (2005). Vehicles detection from very high
resolution satellite imagery. International Archives of the Photogrammetry, Remote
Sensing and Spatial Information Science, Symposium conducted in Vienna, Austria.
Gauray, R. & Khisty, C.J. (1998). Urban transportation in developing countries. Trends, impacts
and potential systemic strategies. Symposium conducted at the meeting of the
Transportation Research Board, Washington, D.C.
Hasan, M., Saha, G., Hoque, A. & Majumder, M.B. (May, 2014). Smart traffic control system
with application of image processing techniques. Paper presented at the International
Conference on Informatics, Electronics & Vision (ICIEV), Dhaka, Bangladesh.
Hinz, S., Schlosser, C. & Reitberger, J. (2003) Automatic car detection in high resolution urban
scenes based on an adaptive 3D-model. Remote Sensing and Data Fusion over Urban
Areas, 2003. Symposium conducted at the 2nd GRSS/ISPRS Joint Workshop, Berlin,
Germany.
Institute of Geodesy and Photogrammetry. (1999). Airborne and satellite sensors. Zurich,
Switzerland: Poli.
Jun-Wei H., Yu, S., Chen, Y. & Hu, W. (2006). Automatic traffic surveillance system for
vehicle tracking and classification. Intelligent Transportation Systems, 7(2), 175–187.
Jung, Y. & Ho, Y. (1999). Traffic parameter extraction using video-based vehicle tracking.
International Conference on Intelligent Transportation Systems. pp. 764–769.
Kozempel, K. & Reulke, R. (2009). Fast vehicle detection and tracking in image bursts.
International Archives of the Photogrammetry, Remote Sensing and Spatial Information
Science, 38 (3/W4), 175-180.
Krawiec, K., Kukawka, B., & Maciejewski, T. (2010). Evolving cascades of voting feature
detectors for vehicle detection in satellite imagery. Proceedings of IEEE Congress on
Evolutionary Computation, Symposium conducted at CEC 2010, Barcelona, Spain.
Lall, S.V., Lundberg, M.K.A., & Shalizi, Z. (2008). Implications of alternate policies on welfare
of slum dwellers: evidence from Pune, India. Journal of Urban Economics, 63(1), 56-73.
Liang, P., Teodoro, G., Ling, H., Blasch, H., Chen, G. & Bai, L. (2012). Multiple kernel learning
for vehicle detection in wide area motion imagery. 15th International Conference on
Information Fusion. pp. 1629-1636.

VDIS 90
Mahabir, R, & Al-Tahir, R. (2008). The role of spatial data infrastructure in the management of
land degradation in small tropical Caribbean Islands. Paper presented at the Tenth
International Conference for Spatial Data Infrastructure, St. Augustine, Trinidad.
Moeller, M.S. (2005). Remote Sensing for the Monitoring of Urban Growth Patterns. ISPRS
Journal of Photogrammetry and Remote Sensing. 36-8 (W27), 6.
Morris, B. & Trivedi, M. (2006). Robust classification and tracking of vehicles in traffic video
streams. International conference on Intelligent Transportation Systems. pp.1078–1083.
Sharma, G., Merry, C., Goel, J. P. & McCord, M. (2006). Vehicle detection in 1‐m resolution
satellite and airborne imagery. International Journal of Remote Sensing, 27(4), 779-797.
She, K., Bebis, G., Gu, H. & Miller. R. (2004). Vehicle tracking using on-line fusion of color and
shape features. International IEEE Conference on Intelligent Transportation Systems, pp.
731-736.
Stilla, U.A., Michaelsen, E. B., Soergel, U.B, Hinz, S.C. & Enderd, J. (2004). Airborne
Monitoring of vehicle activity in urban areas. International Archives of Photogrammetry
and Remote Sensing, 35(B3): 973-979.
Tseng, B.L., Lin, C. & Smith. J.R. (2002). Real-time video surveillance for traffic monitoring
using virtual line analysis. Paper presented at IEEE International Conference on
Multimedia and Expo.
Van Metre, P., Mahler, B.J. & Furlong, E.T. (2000). Urban sprawl leaves its PAH signature.
Environmental Science and Technology, 34, 4064-4070.
Yilmaz, S., Süleyman T., Yildiz, N.D. & Yilmaz, H. (2009). Human population growth and
temperature increase along with the increase in urbanization, motor vehicle numbers and
green area amount in the sample of Erzurum city, Turkey. Environmental monitoring and
assessment, 14(1-4), 205-213.
Zheng, H. & Li, L. (2007). An artificial immune approach for vehicle detection from high
resolution space imagery. International Journal of Computer Science and Network
Security, 7(2), 67-72.
Tables and Figures
Table 1: Results of detection process for test images
Test image Wrongly identified regions Missed regions
1 1 (part of truck shadow) 1 (motorcycle)
2 1 (vehicle with high sun glare) 1 (motorcycle)
3 0 0

VDIS 91
Figure 1. Vehicle detection and identification system (VDIS)
1
Region of interest; 2
Area of interest
Selection of 1ROI classes
for classification
Isolation of AOI
(Mask)
Separation of roads from
2AOI
(Regions w/ vehicles)
Removal of noise
Detection of vehicle
regions
Vehicle identification
(from RGB image)
Selection of object
spectra values
Vehicle Detection
Vehicle Identification

VDIS 92
Figure 3. Mask showing the
isolation of AOI in black
Figure 2. RGB image
Figure 4: Removal of non-
image elements (contains
noise)
Figure 5: AOI with noise
removed

VDIS 93
Figure 6. Vehicle regions (left) with clipped RGB image (right)
Figure 7. Red and white cars detected
RedWhite

VDIS 94
Figure 8. Reported results for test image 1
Actual
Predicted
Red White Other Total
Red 1 0 0 1
White 0 2 1 3
Other 0 1 0 1
Total 1 3 1 4
Figure 9. Reported results for test image 2
Actual
Predicted
Red 2 0 0 2
White 0 3 1 4
Other 0 0 0 0
Total 2 3 1 6
Identification performance = 3/4= 75%
Identification performance=5/6= 83%

VDIS 95
Figure 10: Reported results for test image 3
Figure 10. Reported results for test
Actual
Predicted
Red 1 0 1 2
White 0 4 2 6
Other 0 0 0 0
Total 1 4 3 8
Identification performance=5/8= 62%

VDIS: A System for Morphological Detection and Identification of Vehicles in RGB Images

More Related Content

What's hot

Similar to VDIS: A System for Morphological Detection and Identification of Vehicles in RGB Images

More from rsmahabir

Recently uploaded

VDIS: A System for Morphological Detection and Identification of Vehicles in RGB Images