AKBALIK Et Al 2024 Engine Fault Detection by Sound Analysis and Machine Learning
AKBALIK Et Al 2024 Engine Fault Detection by Sound Analysis and Machine Learning
sciences
Article
Engine Fault Detection by Sound Analysis and
Machine Learning
Ferit Akbalık 1, *, Abdulnasır Yıldız 2 , Ömer Faruk Ertuğrul 3 and Hasan Zan 4
Abstract: Traditional vehicle fault diagnosis methods rely heavily on the expertise of mechanics
or diagnostic tools available at service centers, which can be costly, time-consuming, and may not
always provide accurate results. This study presents a comprehensive vehicle fault diagnosis frame-
work, which utilized Mel-Frequency Cepstral Coefficients (MFCCs), Discrete Wavelet Transform
(DWT)-based features, and the Extreme Learning Machine (ELM) classifier. To address the limita-
tions of previous works, the proposed framework leverages a large, diverse dataset encompassing
various vehicle models and real-world operating conditions. Significantly improved robustness
and generalizability of the fault diagnosis system were achieved. The results of the experiments
demonstrate the superiority of the MFCC-based features combined with the ELM classifier, achieving
the highest performance metrics in terms of accuracy, precision, recall, F1-score, macro F1-score, and
weighted F1-score, which are 92.17%, 92.24%, 92.22%, 92.10%, and 92.06%, respectively. Slightly
lower performance was obtained while employing the DWT-based features compared to employing
MFCC-based features. Additionally, frequency analysis was conducted to identify specific frequency
bins, which are the most indicative of different fault types in providing valuable guidance for future
Citation: Akbalık, F.; Yıldız, A.;
diagnostic efforts. Overall, the proposed framework provides a reliable and practical solution for
Ertuğrul, Ö.F.; Zan, H. Engine Fault accurate vehicle fault detection, paving the way for future advancements in automotive diagnostics.
Detection by Sound Analysis and
Machine Learning. Appl. Sci. 2024, 14, Keywords: vehicle fault detection; extreme learning machines; mel-frequency cepstral coefficients;
6532. https://doi.org/10.3390/ wavelet transform
app14156532
there has been a surge of interest in machine learning-based fault diagnosis systems [5–7].
Leveraging advancements in artificial intelligence and data analytics, these systems aim to
automate fault detection processes, enhance accuracy, and enable proactive maintenance
strategies [8,9].
The general structure of machine learning-based fault diagnosis methods encompasses
several stages, i.e., data acquisition, preprocessing, feature extraction, and classification [10,11].
Data acquisition involves gathering relevant information from vehicles, such as sensor
readings and diagnostic codes, and recording relevant signals [12]. Preprocessing focuses
on cleaning and organizing the data to eliminate noise and inconsistencies [13]. Feature ex-
traction aims to identify key parameters or characteristics indicative of potential faults [14].
Finally, classification algorithms are applied to categorize the data into different fault types
or states [15].
Two primary types of signals, which are vibration and sound signals, are employed in
fault diagnosis [16]. Vibration signals capture mechanical movements and dynamics within
the vehicle and provide insights into structural integrity and component performance. On
the other hand, sound signals reflect acoustic emissions associated with engine operation
and component interactions and offer valuable clues about the health and functionality of
various vehicle parts.
Several studies have explored fault diagnosis using vibration signals and demonstrate
the versatility and effectiveness of this approach. For example, Jegadeeshwaran and Sug-
umaran [17] presented a method of vibration-based continuous monitoring system and
analysis using a machine learning approach. Their study focused on fault diagnosis in
hydraulic braking systems by acquiring vibration signals from a piezoelectric transducer
under both good and faulty brake conditions. They employed decision tree algorithms to
identify the most relevant features among different faulty conditions and they achieved a
classification accuracy of 97.45%. Similarly, Barbieri et al. [18] aimed to identify damages
and diagnose damaged components in automotive gearboxes by comparing vibration
signals of damaged and undamaged systems. They employed various signal analysis tech-
niques (e.g., wavelet transform and mathematic morphology) to verify damage presence
and used a signal processing technique combining pattern spectrum and selective filtering
for component failure identification. Jafarian et al. [19] explored vibration analysis for fault
detection in an internal combustion engine and focused on detecting faults related to pop-
pet valve clearance and incomplete combustion. They utilized four accelerometers on the
engine body, applied the Principal Component Analysis (PCA) technique for data analysis,
and achieved high efficiency in fault classification and detection. These studies collectively
highlight the wide applicability and effectiveness of vibration-based fault diagnosis in
diverse automotive systems. A summary of studies employing vibration signals is given in
Table 1.
Table 1. Cont.
Although high success rates have been reported in vibration signals, it is hard to
implement them in a real-word system. Therefore, a practical method is required to
distinguish the faults. Sound signals offer distinct advantages in fault diagnosis due to their
ease of recording using commonly available devices such as cell phones or microphones.
Therefore, this makes sound analysis a practical and cost-effective approach to diagnosing
vehicle faults. Studies conducted by various researchers further exemplify the potential
of sound-based fault diagnosis in automotive systems. For instance, Madain et al. [23]
identified distinct sounds associated with specific engine malfunctions and developed
an algorithm using sound techniques in diagnosis. They reported high error detection
rates through analysis of engine sound samples collected from a laboratory environment.
Similarly, Jian-Da Wu and Chiu-Hong Liu [24] developed a fault diagnosis system for
internal combustion engines based on the discrete wavelet transform technique applied to
sound emission signals and showcased its effectiveness in fault recognition under diverse
engine operating conditions. Another study that delved into the application of acoustic
signal processing methods for assessing internal combustion engine technical conditions
proposed new algorithms for automatic detection of valve clearance issues based on
acoustic signal components [25]. Additionally, Mofleh et al. [26] conducted a study aimed
at detecting faults in spark-ignition engines using acoustic signals and an Artificial Neural
Network (ANN) system. It highlighted the high potential of ANN-based fault detection in
internal combustion engines using acoustic signals, particularly in identifying simulated
spark plug and misfire faults. These studies collectively underscore the practicality and
efficacy of sound-based fault diagnosis methods in the automotive industry and offer
valuable insights for developing reliable diagnostic systems. A summary of such studies
employing sound signals is provided in Table 2.
Despite recent advancements, current studies in vehicle fault diagnosis often en-
counter significant drawbacks. These limitations include the reliance on data recorded
in controlled laboratory settings, which may not fully represent real-world vehicle con-
ditions. Furthermore, many studies are constrained to specific car models or fault types,
limiting the generalizability and applicability of their findings. There is a pressing need
for a comprehensive dataset that encompasses diverse vehicle models, real-world vehicle
conditions, and a wide range of fault scenarios to enhance the effectiveness and accuracy
of fault diagnosis systems.
Appl. Sci. 2024, 14, 6532 4 of 18
Our study directly addresses these challenges by leveraging a diverse and extensive
dataset comprising real-life vehicle sounds. This dataset captures a wide array of vehicle
conditions various vehicle models, and an extensive range of fault scenarios encountered
in everyday driving. We employ advanced signal processing techniques, including Mel-
Frequency Cepstral Coefficients (MFCCs), Wavelet Transform, and Relief-F methods, for
robust feature extraction and feature selection, while using Extreme Learning Machines
(ELM) for the classification.
The summary of our study’s approach and key contributions is as follows:
• Instead of employing a laboratory-collected dataset, comprehensive data were col-
lected from real-life vehicle conditions and diverse vehicle models.
• In order to increase the success of the proposed approach, advanced signal processing
techniques, which are MFCC, Wavelet Transform, and Relief-F, were employed.
• A thorough frequency analysis was conducted in each fault type, and specific fre-
quency components, which are associated with different types of faults, were identified.
This paper is structured as follows. Section 2 gives details about utilized data collection
methods and focuses on how sound signals were acquired from vehicular systems under
various operating conditions. Section 3 elaborates on employed methodology encompass-
ing employed signal processing techniques and feature extraction methodologies that are
used in vehicle fault diagnosis based on sound signals. In Section 4, we present the obtained
experimental results, which includes performance evaluations of the employed diagnostic
system, comparisons with existing methods in the literature, and a detailed frequency
This paper is structured as follows. Section 2 gives details about utilized data
collection methods and focuses on how sound signals were acquired from vehicular
systems under various operating conditions. Section 3 elaborates on employed
methodology encompassing employed signal processing techniques and feature
Appl. Sci. 2024, 14, 6532 extraction methodologies that are used in vehicle fault diagnosis based on sound signals.
5 of 18
In Section 4, we present the obtained experimental results, which includes performance
evaluations of the employed diagnostic system, comparisons with existing methods in the
literature, and a detailed frequency analysis of each identified fault type. Furthermore, the
analysis of each identified fault type. Furthermore, the implications of our findings and
implications of our findings and insights gained from the experimental outcomes are
insights gained from the experimental outcomes are discussed. Finally, Section 5 serves
discussed. Finally, Section 5 serves as the conclusion of this study and summarizes key
as the conclusion of this study and summarizes key contributions made in this research,
contributions made in this research, and proposes directions for future research and
and proposes directions for future research and development in the field of vehicle fault
development in the field of vehicle fault diagnosis by sound signal analysis.
diagnosis by sound signal analysis.
2.2.Dataset
Dataset
Audio
Audiosignal
signalrecordings
recordingswere
werecollected
collectedfrom
fromvehicles,
vehicles,which
whichwere
wereserviced
servicedatatofficial
official
Ford
Ford or Toyota service centers and ensured a diverse range of cars fromthese
or Toyota service centers and ensured a diverse range of cars from thesereputable
reputable
brands.
brands. A A cellphone served as
cellphone served asthe
therecording
recordingdevice,
device, which
which captured
captured sounds,
sounds, while
while the
the cars
cars
werewere stationary,
stationary, and and
theirtheir engines
engines werewere idling
idling at ideal
at ideal operating
operating temperatures.
temperatures. As
As seen
seen in Figure
in Figure 1, cellphone
1, the the cellphone
waswas positioned
positioned 15 cm15 cm above
above the the hood
hood andand centered,
centered, withwith
the
the
hoodhood closed
closed to mimic
to mimic real-world
real-world conditions.
conditions. Engine
Engine sounds
sounds werewere recorded
recorded fors each,
for 30 30 s
each, sampling
sampling at a frequency
at a frequency of 48 of
kHz48 to
kHz to capture
capture detailed
detailed acoustic
acoustic information.
information.
Setupfor
Figure1.1.Setup
Figure foraudio
audiosignal
signalrecording.
recording.
Professional mechanics diagnosed the cars as either healthy or with one of the follow-
ing faults: spark plug issues, airflow irregularities, electrical malfunctions, engine/turbo
problems, or front-end problems. The distribution of each diagnostic class is outlined in
Table 3.
Spark Plug Issue: Typically related to ignition problems, which result in misfires,
rough idling, and decreased engine performance.
Airflow Irregularities: Pertaining to issues with the air intake system that affect engine
combustion and efficiency.
Electrical Malfunctions: Encompassing faults within the vehicle’s electrical system
that directly impact engine performance. This may include issues with sensors, wiring, or
other electrical components that affect engine operation and efficiency.
Engine/Turbo Problems: Referring to issues within the engine or turbocharger system
that impact power delivery and overall engine performance.
Spark Plug Issue: Typically related to ignition problems, which result in misfires,
rough idling, and decreased engine performance.
Airflow Irregularities: Pertaining to issues with the air intake system that affect
engine combustion and efficiency.
Electrical Malfunctions: Encompassing faults within the vehicle’s electrical system
Appl. Sci. 2024, 14, 6532 6 of 18
that directly impact engine performance. This may include issues with sensors, wiring, or
other electrical components that affect engine operation and efficiency.
Engine/Turbo Problems: Referring to issues within the engine or turbocharger system
that impact power
Front-End delivery and
Problems: overall issues
Including engine with
performance.
steering, suspension, or other components
affecting the vehicle’s front-end operation. with steering, suspension, or other
Front-End Problems: Including issues
components affecting
The collected the vehicle’s
dataset front-end
comprises audiooperation.
signals, which were collected from a wide range
The collected dataset comprises audio signals, which were collected from a wide
of gasoline-engine vehicles such as Ford Focus (2014–2021), Ford Kuga (2020–2021), Ford
range of gasoline-engine vehicles such as Ford Focus (2014–2021), Ford Kuga (2020–2021),
Ecosport (2021), Ford Mondeo (2016), Toyota Corolla (2015), and Toyota Auris (2010). It was
Ford Ecosport (2021), Ford Mondeo (2016), Toyota Corolla (2015), and Toyota Auris (2010).
aimed to ensure
It was aimed diversity
to ensure in vehicle
diversity models
in vehicle models to to
capture
capturea acomprehensive rangeofof engine
comprehensive range
sounds and fault
engine sounds andtypes.
fault types.
In addition,
In addition,ananexample
example signal
signal forfor
eacheach vehicle
vehicle diagnostic
diagnostic class
class is is provided
provided in Figure 2
in Figure
in order
2 in ordertotodemonstrate
demonstrate thethecharacteristics
characteristics of of
thethe recorded
recorded audio audio signals
signals forclass.
for each each class.
Figure 2.
Figure 2.Example
Exampleaudio
audiosignals for for
signals eacheach
vehicle diagnostic
vehicle class. class.
diagnostic
3. Methodology
3.1. Overview
This paper presents a framework comprising data acquisition, preprocessing, feature
extraction, fine-tuning, and classification, as illustrated in Figure 3. Data acquisition
details are explained in Section 2. In the preprocessing phase, each audio signal was
normalized and trimmed to 5 s to ensure consistency. Feature extraction was performed
using Mel-Frequency Cepstral Coefficients (MFCCs) and Discrete Wavelet Transform (DWT)
techniques, capturing important acoustic characteristics for fault diagnosis. The fine-tuning
process involves Grid Search optimization to find optimal hyperparameters for machine
learning models, while Extreme Learning Machines (ELM) are utilized for efficient and
accurate classification of vehicle fault diagnostic classes. Additionally, Fourier transform-
based feature extraction and Relief-f feature selection algorithm are employed for frequency
analysis of each fault type, aimed at uncovering crucial frequency components associated
with different fault types.
(DWT) techniques, capturing important acoustic characteristics for fault diagnosis. The
fine-tuning process involves Grid Search optimization to find optimal hyperparameters
for machine learning models, while Extreme Learning Machines (ELM) are utilized for
efficient and accurate classification of vehicle fault diagnostic classes. Additionally,
Fourier transform-based feature extraction and Relief-f feature selection algorithm are
Appl. Sci. 2024, 14, 6532 7 of 18
employed for frequency analysis of each fault type, aimed at uncovering crucial frequency
components associated with different fault types.
Figure 3.
Figure 3. Methodology
Methodologyblock diagram
block for vehicle
diagram fault fault
for vehicle diagnosis using sound
diagnosis using signals
sound and machine
signals and
learning.
machine learning.
3.2.Feature
3.2. FeatureExtraction
Extraction
Feature extraction
Feature extraction isis aacritical
critical step
stepin
inaudio
audiosignal
signalprocessing
processingfor
forfault
faultdiagnosis
diagnosisin
in
vehicle
vehiclesystems.
systems.This
Thissection
sectionoutlines
outlinestwo twowidely
widelyused
usedtechniques:
techniques:MFCC
MFCCand andDWT.
DWT.
MFCC is a prominent technique in audio signal processing. The process involves
several steps [30]:
• Windowing the audio signal into short segments using Equation (1).
2πn
W (n) = 0.54 − 0.46 cos N−1 ≥ n ≥ 0 (1)
N−1
• Applying the Discrete Fourier Transform (DFT) to each frame to convert the time-
domain signal into the frequency domain. DFT can be defined for a frame X compris-
ing of N samples as in Equation (2).
N −1 2πjkn
Xn = ∑ Xk e − N , n = 0, 1, 2, 3 . . . , N − 1 (2)
k =0
where Ck represents the k-th MFCC coefficient, N represents the total number of mel-
frequency filters, Sn represents the energy of the n-th mel-frequency filter bank, and k
is the index of the MFCC coefficient, usually ranging from 0 to k − 1.
These MFCC coefficients capture essential spectral features of the audio signal, such
as pitch, timbre, and formants, which are crucial for fault diagnosis in vehicle systems. In
this study, the mean of each coefficient over each frame was calculated as a feature. Details
regarding MFCC parameters are provided in Section 4.
DWT is a powerful tool for analyzing signals in both time and frequency domains
simultaneously [31]. The process involves the following:
• Decomposing the audio signal into different frequency bands using wavelet functions,
such as Daubechies and Symlets wavelets. For an input signal x (t), the approximation
coefficients A j and detail coefficients D j at level j are computed using:
where h and g are the low-pass and high-pass filter coefficients, respectively, corre-
sponding to the wavelet function.
• Extracting features from each level of decomposition.
These features provide insights into the time–frequency characteristics of the signal,
aiding in fault detection and classification in vehicle systems. In this study, we calculated
the energy, standard deviation, and entropy of each coefficient to serve as features for the
classification task. Details regarding the decomposition levels and wavelets employed are
given in Section 4.
H = g (W · X + b ) (7)
Y = H·β (8)
β = H + ·T (9)
For the classification of vehicle fault diagnostic classes, we employed ELM, a machine
learning algorithm based on a single hidden layer feedforward neural network
architecture [32–35]. In the ELM algorithm, the input layer weights and thresholds are
assigned randomly, while the output layer weights are calculated based on these
assignments. ELM training consists of two parts: (1) generating random hidden layer
Appl. Sci. 2024, 14, 6532 parameters from a predefined range, and (2) calculating the generalized inverse output 9 of 18
weight matrix [36]. ELM is popular due to its fast learning speed, generalization ability,
and simplicity. As illustrated in Figure 4, the ELM inputs map the features to the hidden
where H is theare
layer, which matrix of hidden
then passed on layer
to theoutputs, Y denotes
output layer. the output
The output fromvector, H + represents
ELM learning can be
the Moore–Penrose
used pseudoinverse
for various tasks of the matrix
such as classification, H, and Tand
regression, is the target vector.
clustering.
Figure4.4.Structure
Figure Structureof
ofan
anELM
ELMmodel.
model.
To optimize
ELM transforms input vectorof 𝑥our
the performance = ELM
𝑥 , 𝑥 classifier,
, … , 𝑥 into
we utilized
the hiddenGrid Search,
layer a hyperpa-
representation
rameter
using the optimization
weight matrix 𝑊 and[37].
technique the bias Search𝑏.
Gridvector systematically
Each neuronexplores a predefined
in the hidden set of
layer uses an
hyperparameters, evaluating each combination using 5-fold cross-validation
activation function 𝑔(. ) . The connections between the hidden layer outputs and the to determine
the optimal
output layerparameters that yield
are represented theweights
by the highest classification
𝛽. In ELM, the accuracy.
weightsThis
andmethod allows
bias values for
for a thorough examination of the model’s performance across various
the hidden layer are randomly assigned and kept fixed. The weights 𝛽 for the output settings, ensuring
the selection
layer of the
are learned mostthe
using effective parameter
least squares set. The
method. The relationship
specific parameters
betweenused for Grid
the input and
Search,
output such
of theas the is
ELM number of hidden
calculated neurons and
as summarized activation(7)–(9).
in Equations functions, are outlined in
Table 4. This exhaustive search process ensures that our model is finely tuned, enhancing
its robustness and reliability in delivering 𝐻 =accurate
𝑔(𝑊 ∙ 𝑋 + 𝑏) diagnosis results.
fault (7)
1 N TPi
N i∑
Precision ( PR) = (11)
=1
TPi + FPi
where N is the number of classes, TPi is the number of true positives for class i and FPi is
the number of false positives for class i.
1 N TPi
N i∑
Recall ( RE) = (12)
=1
TPi + FNi
1 N 2· PRi · REi
N i∑
Macro F1-Score = (13)
=1
PRi + REi
N
2· PR · RE
Weighted F1-Score = ∑ wi PRi +i REii (14)
i =1
where PRi and REi are the precision and recall for class i, respectively. Additionally,
confusion matrices, which display the model’s classification results, were constructed
to provide a detailed view of how well the classifier distinguishes between fault classes.
This visual representation is crucial for understanding the classifier’s strengths and areas
needing improvement.
to distinguishing between faulty and healthy conditions. These relevant bins highlight
specific frequency components that are key indicators of each fault type.
Table 5. Performance results for MFCC-Based features. Values given for window length are in
seconds. 50% window overlap is used for all experiments. The bold indicates the highest score.
Best ELM
MFCC Parameters Performance Results
Hyperparameters
Number of Window Activation Number of Macro Weighted
Precision Recall Accuracy
Coefs. Length Function Neurons F1-Score F1-Score
5 0.02 tribas 100 86.19 85.60 85.70 85.72 85.71
10 0.02 sig 100 90.09 89.24 89.34 89.31 89.29
20 0.02 sin 100 89.95 90.02 89.91 89.88 90.00
30 0.02 sin 100 89.81 89.73 89.51 89.45 89.64
5 0.03 radbas 80 85.74 84.59 84.73 84.73 84.64
10 0.03 sig 90 89.71 89.83 89.69 89.65 89.64
20 0.03 sin 100 92.24 92.22 92.10 92.06 92.14
30 0.03 sin 90 90.43 89.77 89.65 89.59 89.64
5 0.04 tribas 100 88.37 87.67 87.87 87.98 87.86
10 0.04 sig 90 91.45 91.12 91.11 91.11 91.07
20 0.04 sig 100 91.14 90.79 90.69 90.61 90.71
30 0.04 sig 80 89.67 89.37 89.16 89.00 89.29
The results, which are given in Table 5, indicate that using 20 MFCC coefficients with a
window length of 0.03 s and a sine activation function yielded the highest performance with
a precision of 92.24%, recall of 92.22%, F1-score of 92.10%, and accuracy of 92.14%. This
demonstrates that the choice of MFCC parameters and ELM hyperparameters significantly
impacts the classification performance.
From the table, it can be observed that the number of MFCC coefficients and window
length significantly affect the performance of the proposed method up to a certain point.
Specifically, increasing the number of MFCC coefficients from 5 to 20 generally leads to
an improvement in precision, recall, F1-score, and accuracy. However, further increasing
the number of coefficients to 30 does not seem to result in any significant improvement.
Similarly, increasing the window length from 0.02 to 0.03 or 0.04 generally leads to an
improvement in performance, with the best results achieved at a window length of 0.03
for most MFCC parameter configurations. However, the choice of activation function
and the number of neurons in the ELM classifier also seem to play a role in achieving the
best performance.
10 0.03 sig 90 89.71 89.83 89.69 89.65 89.64
20 0.03 sin 100 92.24 92.22 92.10 92.06 92.14
30 0.03 sin 90 90.43 89.77 89.65 89.59 89.64
5 0.04 tribas 100 88.37 87.67 87.87 87.98 87.86
10 0.04 sig 90 91.45 91.12 91.11 91.11 91.07
Appl. Sci. 2024, 14, 6532 12 of 18
20 0.04 sig 100 91.14 90.79 90.69 90.61 90.71
30 0.04 sig 80 89.67 89.37 89.16 89.00 89.29
The
Theconfusion
confusionmatrix
matrixfor
forthe
themodel
modelwithwith2020
MFCC
MFCC coefficients
coefficients and a window
and a window length of
length
0.02 s, shown in Figure 5, provides additional insights into the model’s performance.
of 0.02 s, shown in Figure 5, provides additional insights into the model’s performance. Each
cell
Each in cell
the matrix
in the represents the number
matrix represents the of instances
number of for which the
instances for true class
which theis true
represented
class is
by the row and the predicted class is represented by the column.
represented by the row and the predicted class is represented by the column.
Figure 5. Confusion
Figure Confusionmatrix
matrixforfor
20 MFCC coefficients
20 MFCC and window
coefficients lengthlength
and window of 0.02ofs using
0.02 sMFCC-
using
based features.
MFCC-based features.
Overall, the confusion matrix confirms that the model performs well across most
fault types, with particularly high accuracy for airflow, front-end, and spark plug faults.
Specifically, the model achieves the highest performance in detecting airflow faults with
an accuracy of 98.1%, correctly identifying 51 out of 52 cases. However, the model shows
the lowest performance in identifying healthy cases, with an accuracy of 80.0%, correctly
predicting 40 out of 50 instances. This suggests that healthy cases are more prone to being
misclassified as faults, indicating an area for further refinement.
The results of the experiments using MFCC-based features demonstrate the effective-
ness of the proposed method for fault diagnosis in vehicles. The method can accurately
diagnose different types of faults using the extracted MFCC features and the ELM classifier.
The results also provide insights into the optimal combination of MFCC parameters and
ELM hyperparameters, which can be used to improve the performance of the method
in future studies. These findings demonstrate the model’s effectiveness in vehicle fault
diagnosis and highlight areas for further optimization.
Table 6. Performance results for DWT-based features. The bold indicates the highest score.
Best ELM
DWT Parameters Performance Results
Hyperparameters
Activation Number of Macro Weighted
Level Wavelet Precision Recall Accuracy
Function Neurons F1-Score F1-Score
2 db4 sig 80 82.11 81.09 81.30 81.69 81.43
3 db4 sin 90 82.84 80.63 81.11 81.43 81.07
4 db4 sin 100 78.46 77.44 77.24 77.25 77.50
5 db4 sin 100 81.72 80.95 80.79 80.98 81.07
2 db8 sig 90 82.22 80.99 81.19 81.47 81.43
3 db8 sin 70 83.51 82.42 82.64 82.77 82.86
4 db8 sin 100 77.57 77.16 76.94 77.09 77.14
5 db8 sin 100 79.62 78.05 78.29 78.25 78.21
2 db20 sin 100 84.88 83.27 83.54 83.52 83.57
3 db20 sin 80 83.00 82.37 82.45 82.46 82.50
4 db20 sin 100 77.68 75.93 76.00 76.04 76.07
5 db20 sin 100 80.57 79.93 79.97 80.21 80.00
2 sym3 sig 80 81.13 80.58 80.48 80.63 80.71
3 sym3 sin 90 84.36 83.32 83.46 83.52 83.57
4 sym3 sin 100 74.76 74.39 74.19 74.05 74.29
5 sym3 sig 50 78.48 77.46 77.43 75.55 77.50
2 sym8 sin 80 83.19 82.33 82.42 82.45 82.50
3 sym8 sin 70 84.17 83.60 83.72 83.86 83.93
4 sym8 sin 100 76.20 75.77 75.37 75.52 75.71
5 sym8 sin 100 77.10 75.98 76.07 76.26 76.07
From the table, it is clear that different combinations of DWT parameters and ELM hy-
perparameters result in varying levels of performance. The highest accuracy was achieved
using a decomposition level of 3 and a sym8 wavelet with a sine activation function, yielding
a precision of 84.17%, recall of 83.60%, F1-score of 83.72%, and accuracy of 83.93%. This
suggests that the choice of wavelet and decomposition level significantly influences the
classification performance.
The confusion matrix in Figure 6 provides further insights into the classification
performance for the best configuration (decomposition level of 3 and sym8 wavelet). The
model achieves high accuracy for electrical faults (93.5%) and front-end faults (91.7%),
indicating strong performance in these categories. However, the model shows lower
accuracy for spark plug faults (65.0%), suggesting that distinguishing spark plug faults
from other fault types remains a challenge. Additionally, healthy cases are identified
with an accuracy of 74.0%, indicating some misclassification into fault categories, which
highlights an area for further improvement.
Overall, the results of the experiments using DWT-based features demonstrate the
effectiveness of the proposed method for fault diagnosis in vehicles. While the method
shows strong performance in certain fault categories, it achieved lower performance overall
compared to MFCC-based features. These findings provide insights into the optimal
combination of DWT parameters and ELM hyperparameters and highlight areas for further
optimization to enhance the performance of the method in future studies.
performance for the best configuration (decomposition level of 3 and sym8 wavelet). The
model achieves high accuracy for electrical faults (93.5%) and front-end faults (91.7%),
indicating strong performance in these categories. However, the model shows lower
accuracy for spark plug faults (65.0%), suggesting that distinguishing spark plug faults
from other fault types remains a challenge. Additionally, healthy cases are identified with
Appl. Sci. 2024, 14, 6532 14 of 18
an accuracy of 74.0%, indicating some misclassification into fault categories, which
highlights an area for further improvement.
Figure6.6.Confusion
Figure Confusion matrix
matrix for for decomposition
decomposition levellevel of 3sym8
of 3 and andwavelet
sym8 wavelet using DWT-based
using DWT-based features.
features.
4.3. Results for Frequency Analysis
Overall,
In the results
this section, of theofexperiments
the results the frequency using DWT-based
analysis, which wasfeatures demonstrate
conducted the
to identify
effectiveness
the of the
most relevant proposed
frequency method for
components forfault
each diagnosis
fault type,inarevehicles. While
presented. the method
Different types
shows
of engine strong
faultsperformance
often exhibit in certain
distinct fault categories,
frequency components.it achieved lower performance
These components emerge
overall compared to MFCC-based features. These findings provide insights
due to the engine’s physical structure and operational principles, with each fault generating into the
optimalsounds
unique combination of DWT at
or vibrations parameters and ELM ranges.
specific frequency hyperparameters
Therefore, and highlightthese
examining areas
for furthercomponents
frequency optimization is to enhance
crucial the performance
for accurate of the method
fault diagnosis in future
using sound studies.
analysis. Table 7
comprehensively outlines the relationship between the fault categories and their associated
4.3. Resultsgroups.
frequency for Frequency Analysis
In this section, the results of the frequency analysis, which was conducted to identify
Table 7. The
the most most relevant
relevant frequencies
frequency for eachfor
components fault
eachtype.
fault type, are presented. Different types
of engine faults often exhibit distinct frequency components. These components emerge
Most Relevant Frequency Bins (kHz)
Fault Type due to the engine’s physical structure and operational principles, with each fault
generating Bin 1
unique soundsBinor2 vibrations Bin
at 3specific frequency
Bin 4 ranges. Bin 5
Therefore,
Spark Plug Issues examining these
5.2–5.3frequency components
9.4–9.5 is crucial for accurate
12.4–12.5 fault diagnosis14.1–14.2
13.2–13.3 using sound
Airflow Irregularities analysis. Table
3.0–3.1 9.1–9.2 outlines the
7 comprehensively 9.2–9.3
relationship10.0–10.1 12.2–12.3
between the fault categories
Engine/Turbo Problems and their associated
0.7–0.8 4.8–4.9
frequency groups. 9.5–9.6 12.1–12.2 12.5–12.6
Front-End Problems 6.6–6.7 8.5–8.6 8.9–9.0 11.6–11.7 13.5–13.6
Electrical Malfunctions 2.1–2.2 11.2–11.3 11.4–11.5 12.1–12.2 14.3–14.4
For spark plug issues, the most relevant frequency bin is 5.2 to 5.3 kHz, indicating that
monitoring this high-frequency range is crucial for accurate detection. Airflow irregularities
are most prominently indicated by the 3.0 to 3.1 kHz bin, highlighting the need for precise
analysis in this low-frequency range. Engine/turbo problems are best identified by the
0.7 to 0.8 kHz bin, suggesting these faults manifest through specific low-frequency sounds.
Front-end problems are primarily associated with the 6.6 to 6.7 kHz bin, essential for
identifying issues related to components such as the suspension or chassis. Electrical
malfunctions affecting the engine show the highest relevance at the 2.1 to 2.2 kHz bin,
possibly due to distinctive noise patterns. These findings underscore the importance of
focusing on these key frequencies for accurate fault detection.
Appl. Sci. 2024, 14, 6532 15 of 18
Overall, the frequency analysis confirms that different fault types are associated with
specific and most relevant frequency bins. The identification of these relevant frequency
bins provides valuable guidance for future diagnostic efforts, suggesting that including
these specific frequencies in the analysis can lead to better and more reliable fault detection
outcomes. This insight into the frequency components of various faults enhances our
understanding and ability to diagnose vehicle issues more accurately and efficiently.
5. Conclusions
The proposed study presents a comprehensive vehicle fault diagnosis framework
utilizing MFCC-based features and DWT-based features. The results demonstrate the
efficacy of MFCC features combined with an ELM classifier, achieving the highest perfor-
mance metrics. DWT-based features, while effective, showed slightly lower performance
compared to MFCC features. Frequency analysis identified specific frequency bins most
indicative of different fault types, providing valuable guidance for future diagnostic efforts.
Additionally, by addressing the limitations of previous studies through the introduction of
a large, diverse dataset encompassing various vehicle models and real-world operating
conditions, we have significantly improved the robustness and generalizability of our fault
diagnosis system. This framework provides a reliable and practical solution for accurate
vehicle fault detection, paving the way for future advancements in automotive diagnostics.
Author Contributions: Methodology, F.A. and H.Z.; Software, H.Z.; Data curation, F.A.; Writing—review
& editing, A.Y. and Ö.F.E.; Supervision, A.Y. and Ö.F.E. All authors have read and agreed to the
published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The datasets generated during and analyzed during the current study
are available from the corresponding author upon reasonable request. The data are not publicly
available due to confidentiality constrains.
Acknowledgments: The numerical calculations reported in this paper were fully performed at
TUBITAK ULAKBIM, High Performance and Grid Computing Center (TRUBA resources).
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Edmonds, E. One-in-Three U.S. Drivers Cannot Pay for an Unexpected Car Repair Bill. American Automobile Association, 2017.
Available online: https://newsroom.aaa.com/2017/04/one-three-u-s-drivers-cannot-pay-unexpected-car-repair-bill (accessed
on 28 April 2024).
2. Karaman, E.; Rende, H.; Akşahin, M.F. Recognition of Vehicles from Their Engine Sound. Mühendis Ve Makina 2019, 60, 148–164.
3. Xu, L.; Wang, T.; Xie, J.; Yang, J.; Gao, G. A Mechanism-Based Automatic Fault Diagnosis Method for Gearboxes. Sensors 2022, 22,
9150. [CrossRef] [PubMed]
4. Liu, Y.; Zhang, J.; Ma, L. A fault diagnosis approach for diesel engines based on self-adaptive WVD, improved FCBF and
PECOC-RVM. Neurocomputing 2016, 117, 600–611. [CrossRef]
5. Wang, Y.; Ma, Q.; Zhu, Q.; Liu, X.; Zhao, L. An intelligent approach for engine fault diagnosis based on Hilbert–Huang transform
and support vector machine. Appl. Acoust. 2014, 75, 1–9. [CrossRef]
6. Feng, Z.; Zhang, D.; Zuo, M.J. Planetary Gearbox Fault diagnosis via Joint Amplitude and Frequency Demodulation Analysis
Based on Variational Mode Decomposition. Appl. Sci. 2017, 7, 775. [CrossRef]
7. López-Torres, C.; Riba, J.-R.; Garcia, A.; Romeral, L. Detection of Eccentricity Faults in Five-Phase Ferrite-PM Assisted Synchronous
Reluctance Machines. Appl. Sci. 2017, 7, 565. [CrossRef]
8. Qu, Y.; He, M.; Deutsch, J.; He, D. Detection of Pitting in Gears Using a Deep Sparse Autoencoder. Appl. Sci. 2017, 7, 515.
[CrossRef]
9. Gao, C.; Xue, W.; Ren, Y.; Zhou, Y. Numerical Control Machine Tool Fault Diagnosis Using Hybrid Stationary Subspace Analysis
and Least Squares Support Vector Machine with a Single Sensor. Appl. Sci. 2017, 7, 346. [CrossRef]
10. Lupea, I.; Lupea, M.; Coroian, A. Helical Gearbox Defect Detection with Machine Learning Using Regular Mesh Components and
Sidebands. Sensors 2024, 24, 3337. [CrossRef]
11. Moshrefi, A.; Tawfik, H.H.; Elsayed, M.Y.; Nabki, F. Industrial Fault Detection Employing Meta Ensemble Model Based on
Contact Sensor Ultrasonic Signal. Sensors 2024, 24, 2297. [CrossRef]
12. Morenas, J.d.L.; Moya-Fernández, F.; López-Gómez, J.A. The Edge Application of Machine Learning Techniques for Fault
Diagnosis in Electrical Machines. Sensors 2023, 23, 2649. [CrossRef]
13. Qu, N.; Wei, W.; Hu, C. Series Arc Fault Detection Based on Multimodal Feature Fusion. Sensors 2023, 23, 7646. [CrossRef]
[PubMed]
Appl. Sci. 2024, 14, 6532 17 of 18
14. Yang, X.; Yang, J.; Jin, Y.; Liu, Z. A New Method for Bearing Fault Diagnosis across Machines Based on Envelope Spectrum and
Conditional Metric Learning. Sensors 2024, 24, 2674. [CrossRef]
15. Abid, A.; Khan, M.T.; Iqbal, J. A review on fault detection and diagnosis techniques: Basics and beyond. Artif. Intell. Rev. 2021, 54,
3639–3664. [CrossRef]
16. Baydar, N.; Ball, A. Detection of Gear Failures via Vibration and Acoustic Signals Using Wavelet Transform. Mech. Syst. Signal
Process. 2003, 17, 787–804. [CrossRef]
17. Jegadeeshwaran, R.; Sugumaran, V. Method and Apparatus for Fault Diagnosis of Automobile Brake System Using Vibration
Signals. Recent Patents Signal Process. 2013, 3, 2–11. [CrossRef]
18. Barbieri, N.; Barbieri, G.d.S.V.; Martins, B.M.; Barbieri, L.d.S.V.; de Lima, K.F. Analysis of automotive gearbox faults using
vibration signal. Mech. Syst. Signal Process. 2019, 129, 148–163. [CrossRef]
19. Jafarian, K.; Darjani, M.; Honarkar, Z. Vibration analysis for fault detection of automobile engine using PCA technique. In
Proceedings of the 2016 4th International Conference on Control, Instrumentation, and Automation (ICCIA), Qazvin, Iran, 27–28
January 2016.
20. Ahmed, R.; El Sayed, M.; Gadsden, S.A.; Tjong, J.; Habibi, S. Automotive Internal-Combustion-Engine Fault Detection and
Classification Using Artificial Neural Network Techniques. IEEE Trans. Veh. Technol. 2015, 64, 21–33. [CrossRef]
21. Taghizadeh-Alisaraei, A.; Mahdavian, A. Fault detection of injectors in diesel engines using vibration time-frequency analysis.
Appl. Acoust. 2019, 143, 48–58. [CrossRef]
22. Wang, C.; Zhang, Y.; Zhong, Z. Fault diagnosis for diesel valve trains based on time–frequency images. Mech. Syst. Signal Process.
2008, 22, 1981–1993. [CrossRef]
23. Madain, M.; Al-Mosaiden, A.; Al-khassaweneh, M. Fault diagnosis in vehicle engines using sound recognition techniques. In
Proceedings of the 2010 IEEE International Conference on Electro/Information Technology, Normal, IL, USA, 20–22 May 2010.
24. Wu, J.-D.; Liu, C.-H. Investigation of engine fault diagnosis using discrete wavelet transform and neural network. Expert Syst.
Appl. 2008, 35, 1200–1213. [CrossRef]
25. Figlus, T.; Liščák, Š.; Wilk, A.; Łazarz, B. Condition monitoring of engine timing system by using wavelet packet decomposition
of a acoustic signal. J. Mech. Sci. Technol. 2014, 28, 1663–1671. [CrossRef]
26. Mofleh, A.; Shmroukh, A.; Ghazaly, N. Fault Detection and Classification of Spark Ignition Engine Based on Acoustic Signals and
Artificial Neural Network. Int. J. Mech. Prod. Eng. Res. Dev. 2020, 10, 5571–5578.
27. Navea, R.; Sybingco, E. Design and Implementation of an Acoustic-Based Car Engine Fault Diagnostic System in the Android
Platform. In Proceedings of the International Research Conference in Higher Education, Manila, Philippines, 26–28 October 2013.
28. Siegel, J.; Kumar, S.; Ehrenberg, I.; Sarma, S. Engine Misfire Detection with Pervasive Mobile Audio. In Proceedings of the Joint
European Conference on Machine Learning and Knowledge Discovery in Databases, Turin, Italy, 19–23 September 2016.
29. Yılmaz, G.; Mete, N.F.; Umugabekazi, U.; Aydemir, Ç. Dalgacık Dönüşümü ve Özbağlanım Model Parametreleri Öznitelikleri ile
Otomobil Motor Seslerinden Arıza Tespiti. J. Investig. Eng. Technol. 2020, 3, 48–54.
30. Abdul, Z.K.; Al-Talabani, A.K. Mel Frequency Cepstral Coefficient and its Applications: A Review. IEEE Access 2022, 10,
122136–122158. [CrossRef]
31. Weeks, M.; Bayoumi, M. Discrete Wavelet Transform: Architectures, Design and Performance Issues. J. VLSI Signal Process. Syst.
Signal Image Video Technol. 2003, 35, 155–178. [CrossRef]
32. Huang, G.; Huang, G.-B.; Song, S.; You, K. Trends in extreme learning machines: A review. Neural Netw. 2015, 64, 32–48.
[CrossRef] [PubMed]
33. Chegni, A.M.; Ghavami, B.; Eftekhari, M. A GPU-based accelerated ELM and deep-ELM training algorithms for traditional and
deep neural networks classifiers. Intell. Syst. Appl. 2022, 15, 200098. [CrossRef]
34. Qureshi, S.A.; Hussain, L.; Alshahrani, H.M.; Abbas, S.R.; Nour, M.K.; Fatima, N.; Khalid, M.I.; Sohail, H.; Mohamed, A.; Hilal,
A.M. Gunshots Localization and Classification Model Based on Wind Noise Sensitivity Analysis Using Extreme Learning Machine.
IEEE Access 2022, 10, 87302–87321. [CrossRef]
35. Huang, G.-B.; Zhu, Q.-Y.; Siew, C.-K. Extreme learning machine: Theory and applications. Neurocomputing 2006, 70, 489–501.
[CrossRef]
36. Zhang, J.; Li, Y.; Xiao, W.; Zhang, Z. Robust extreme learning machine for modeling with unknown noise. J. Frankl. Inst. 2020, 357,
9885–9908. [CrossRef]
37. Bashir, M.B.; Latiff, M.S.B.A.; Coulibaly, Y.; Yousif, A. A survey of grid-based searching techniques for large scale distributed data.
J. Netw. Comput. Appl. 2016, 60, 170–179. [CrossRef]
38. Górny, K.; Kuwałek, P.; Pietrowski, W. Increasing Electric Vehicles Reliability by Non-Invasive Diagnosis of Motor Winding
Faults. Energies 2021, 14, 2510. [CrossRef]
39. Kira, K.; Rendell, L.A. A Practical Approach to Feature Selection. In Machine Learning Proceedings 1992; Sleeman, D., Edwards, P.,
Eds.; Morgan Kaufmann: San Francisco, CA, USA, 1992; pp. 249–256.
Appl. Sci. 2024, 14, 6532 18 of 18
40. Azadi, S.; Soltani, A. Fault detection of vehicle suspension system using wavelet analysis. Veh. Syst. Dyn. 2009, 47, 403–418.
[CrossRef]
41. Kemalkar, A.K.; Bairagi, V.K. Engine fault diagnosis using sound analysis. In Proceedings of the 2016 International Conference on
Automatic Control and Dynamic Optimization Techniques (ICACDOT), Pune, India, 9–10 September 2016.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.