biomedicines

Article
Artificial Intelligence for Cardiac Diseases Diagnosis and
Prediction Using ECG Images on Embedded Systems
Lotfi Mhamdi 1, * , Oussama Dammak 2 , François Cottin 3,4 and Imed Ben Dhaou 5,6

1 Biotechnology Institute of Monastir, Environment Street, Monastir 5000, Tunisia

2 Department of Mathematics, College of First Common Year, Um Al Qura University,
Mecca 21955, Saudi Arabia
3 CIAMS EA 4532, Paris-Saclay University, 91405 Orsay, France
4 CIAMS EA 4532, Orleans University, 45067 Orleans, France
5 Department of Computer Science, Hekma School of Engineering, Computing and Informatics,
Dar Al-Hekma University, Jeddah 22246, Saudi Arabia
6 Department of Technology, Higher Institute of Computer Sciences and Mathematics, University of Monastir,
Monastir 5000, Tunisia
* Correspondence: [email protected]

Abstract: The electrocardiogram (ECG) provides essential information about various human cardiac
conditions. Several studies have investigated this topic in order to detect cardiac abnormalities for
prevention purposes. Nowadays, there is an expansion of new smart signal processing methods,
such as machine learning and its sub-branches, such as deep learning. These popular techniques
help analyze and classify the ECG signal in an efficient way. Our study aims to develop algorithmic
models to analyze ECG tracings to predict cardiovascular diseases. The direct impact of this work
is to save lives and improve medical care with less expense. As health care and health insurance
costs increase in the world, the direct impact of this work is saving lives and improving medical
Citation: Mhamdi, L.; Dammak, O.;
care. We conducted numerous experiments to optimize deep-learning parameters. We found the
Cottin, F.; Dhaou, I.B. Artificial
same validation accuracy value of about 0.95 for both MobileNetV2 and VGG16 algorithms. After
Intelligence for Cardiac Diseases
implementation on Raspberry Pi, our results showed a small decrease in accuracy (0.94 and 0.90
Diagnosis and Prediction Using
ECG Images on Embedded Systems.
for MobileNetV2 and VGG16 algorithms, respectively). Therefore, the main purpose of the present
Biomedicines 2022, 10, 2013. research work is to improve, in an easy and cheaper way, real-time monitoring using smart mobile
https://doi.org/10.3390/ tools (mobile phones, smart watches, connected T-shirts, etc.).
biomedicines10082013
Keywords: ECG images; cardiac arrhythmia classification; healthcare; deep learning; Raspberry
Academic Editors: Yu-Te Wu and
Wan-Yuo Guo

Received: 19 June 2022

Accepted: 13 July 2022 1. Introduction
Published: 19 August 2022
In medicine and health care in general, electrophysiological signals provide informa-
Publisher’s Note: MDPI stays neutral tion on the state of health. Among the electrophysiological data available, the electrocardio-
with regard to jurisdictional claims in gram (ECG) turns out to be one of the most relevant and, therefore, worth studying. The
published maps and institutional affil- variability of its cyclical behavior can provide evidence of emotional and behavioral distur-
iations. bances and changes or cardiovascular pathologies. By studying their characteristics, these
signals also remain relevant and non-invasive and help to provide effective diagnostics for
heart disease.
Nowadays, there is a variety of portable gadgets which offer the possibility of de-
Copyright: © 2022 by the authors.
tecting rare cardiac episodes due to their practical ambulatory properties, while ensuring
Licensee MDPI, Basel, Switzerland.
continuous monitoring of the ECG during long periods. Detecting these episodes early in
This article is an open access article
our daily lives increases the chance of a better lifestyle and comfort for humans.
distributed under the terms and
Automatic detection in using signal processing and pattern recognition methods can
conditions of the Creative Commons
constitute, therefore, an asset to provide relevant information without having recourse to
Attribution (CC BY) license (https://
hospital resources which are costly, time-consuming, and require much effort [1,2]. Also,
creativecommons.org/licenses/by/
4.0/).
with a better understanding of certain pathologies and the development of diagnostic

Biomedicines 2022, 10, 2013. https://doi.org/10.3390/biomedicines10082013 https://www.mdpi.com/journal/biomedicines

Biomedicines 2022, 10, 2013 2 of 16

methods and therapies allied to the evolution of technology in the medical field, better
healthcare expectations have emerged in terms of efficiency. It is also clear that fast and
personalized support is seen as the best reference. As a result, the demand for more efficient
medical systems is increasing day by day.
In response, artificial intelligence (AI) has been used in fusion with the medical field by
taking advantage of its ability to learn from a dense and complex database to find probable
links between the various co-existing parameters. Those technologies are useful in assisting
practicing physicians with the decision-making processes, not only in diagnosis but also in
treatment, by monitoring patients and studying drug efficacy tests [3–5].
Heart abnormalities can be identified by the diagnosis of cardiac rhythm irregularities
which are the main cause of cardiac arrhythmia. These abnormalities are responsible for
anatomical changes in the atria and ventricles structure. Therefore, they produce changes
in their activation, depolarization, and repolarization and ECG waveform morphology will
change, causing irregularity [6].
In this same context, traditional methods of machine learning have become less reliable
in detecting patterns and, therefore, in discerning pathological events. In order to tackle
this problem, deep learning has been introduced in the medical field and has established
itself with unparalleled efficiency [7,8]. ECG signals offer the possibility of using the Deep
Neural Network (DNN) for pattern recognition and decision-making due to its cyclical
behavior [9,10]. Thus, the genesis of artificial intelligence (AI) has opened new doors in
several fields of science. This new way of reflecting human–machine interaction has proved
to be ambitious and controversial and interest in it gradually increases each year with the
evolution of automatic calculation technology [11].
It is important to note that the alarming situation of deaths caused by cardiovascular
diseases (CVD), or sudden deaths, remain very worrying on a global scale. This pushes
scientific research to find a better way to anticipate these direct consequences. In this
context, early detection of arrhythmia is an important clinical step that can save lives. The
commonest method to detect cardiac arrhythmia uses the electrocardiogram (ECG), which
measures the electrical activity of the heart.
The main objective of the present work is to develop an automated technique for the
diagnosis of cardiac arrhythmia and, thereafter, to implement it on an embedded system.
We performed a two-category classification of ECG recordings (cardiac arrhythmia, and
healthy person) by using ECG images from the publicly available arrhythmia database
published by Khan’s team [12,13].
Our study will explore the technique of deep learning using deep convolutional neural
network (CNN) models, first to detect cardiovascular disease from images of electrocar-
diogram (ECG) tracings and, second, to predict any type of cardiac arrhythmia. For this
purpose, we used trained algorithms and, thereafter, we implemented the most suitable
one, subsequently, on a Raspberry Pi 4B 8 Go. We worked with deep-learning pretrained
models (MobileNet V2 and VGG16) using CNN with Tensorflow and Keras as backend.
Therefore, the objective of this work is as follows:
• construction of a deep-learning algorithm model for application to ECG signals;
• detection and classification of abnormal patterns (spectra) on ECG signals; and
• validation and improvement of the predictive model on a new database.

2. Materials and Methods

Our goal was to develop an algorithm able to detect and classify data into two dif-
ferent categories (normal for healthy people and cardiac arrhythmia for persons with
cardiac pathologies). The learning will be directly on the ECG data without any signifi-
cant preprocessing.
Biomedicines 2022, 10, 2013 3 of 16

2.1. Data Used

In this study we used a newly released public database [12,13]. It has about 928 images
of ECG tracings, representing four different subject classes belonging to cardiac arrhyth-
mia category:
• Normal: for subjects who do not suffer from any pathology;
• MI: for subjects with a myocardial infarction (MI);
• HMI: for subjects with a history of myocardial infarction (recovered from a MI); and
• ABH: for subjects suffering from cardiac arrhythmias (abnormal heartbeat).
Ultimately, we will have 741 ECG images for the training data and 187 for the test data.
For this purpose, we conducted two pre-trained deep neural networks (DNNs), Mo-
bilNetV22 and VGG16, which have been adapted with our data for better performance.
Various parameters have been changed (number and type of layers, number of filters per
layer, dropout, batch size, learning rate, etc.) to find the best desired model. The database
was divided into training and testing data, with a rate of 80% and 20%, respectively. The
breakdown of the total number of ECG images used for each class is shown in Table 1.

Table 1. Data description.

Class Training Dataset Test Dataset Total Dataset

Normal 227 57 284
MI 191 48 239
HMI 137 35 102
ABH 186 47 233
Total dataset 741 187 928

2.2. Preprocessing
Before passing the images to the model for training, we resized them (244.244), then
normalized (1/255) to adapt them to the requirements of the algorithm. It is very important
to note that it has been shown that deep-learning technique requires significantly more
training data than other machine-learning approaches [14]. In case the images and few
data are available, the augmentation technique can be used to duplicate images to increase
the amount of data. Indeed, this technique makes it possible to generate new versions
of the same image by applying different image-processing operations (zoom, stretch,
contrast, etc.). However, it is not always useful and can distort learning if you change some
parameters necessary for image recognition.
In this paper, we have made the choice of six image processing parameters (brightness,
contrast, gamma, hue, saturation, and central-crop) without affecting the information and
avoiding distorting the learning of the algorithm. The new images resulting from the
augmentation technique will be taken into account by the model at the time of training;
this is an online augmentation that does not require saving the new data to the computer
hard drive. We compared various image-processing parameters, the best ones with better
performance made to images during augmentation are as follows:
• brightness (0.2): brightness adjustment;
• contrast (0.6): contrast adjustment;
• gamma (gamma = 3, gain = 2): control of the overall brightness and the blue-green–red
of the image;
• hue (0.9): hue that controls the colors red, yellow, green, and blue;
• saturation (0.2): adjustment of the hue–white light mixture; and
• central-crop (0.92): division of the area of interest.
Biomedicines 2022, 10, 2013 4 of 16

2.3. Hyper-Parameters Used

Hyper-parameters are pre-defined parameters provided to the algorithm to control
learning and increase its performance. We can cite the learning rate, the number and
size of the hidden layers, etc. In the present study, we have fixed the value of some
hyper-parameters as follows:
• learning rate (0.01);
• dropout (0.1);
• Conv2D filters (min = 64, max = 128, step = 32);
• batch size = 32; and
• optimizer (Adam(1 × 10−5 )).
We chose moderately sized architectures for both DNN models. For each experiment,
less than 100 epochs were enough to reach an excellent accuracy. We ran several trials of
experiment with a CNN and pooling layer, including number of filters 32, 64, and 128.
We used 0.01 as the best efficient learning rate for all deep-learning architectures for our
experiments, after considering other learning-rate tests and the factors of training time and
computational cost.
We tried different network architectures to find the optimum one for the input data
we used, and we chose two CNN models which are the most suitable for our study. The
corresponding architectures for both models are shown in Figures 1–4.

Figure 1. MobileNet V2 architecture for transfer learning.

Biomedicines 2022, 10, 2013 5 of 16

Figure 2. MobileNet V2 architecture for fine-tuning.

Figure 3. VGG16 architecture for transfer learning.

Biomedicines 2022, 10, 2013 6 of 16

Figure 4. VGG16 architecture for fine-tuning.

2.4. Network Architecture

In this study, we used a single classification scheme with four classes (normal, myocar-
dial infarction, recovered myocardial infarction, and abnormal heart rhythm).
We performed two deep-learning algorithms (MobileNet V2 and VGG16) to classify
ECG images and predict cardiac arrhythmia. The corresponding MobileNet V2 model
architecture representations are shown in Figures 1 and 2 for transfer learning and fine-
tuning steps, respectively. VGG16 architecture representations are shown in Figures 3 and 4.
They correspond, respectively, to transfer learning and fine-tuning step.
We split the data into 80% for training and 20% for tests. Our algorithms will serve as
basic models for classification and prediction. For each model studied, the following steps
were followed:
• We applied transfer learning by taking the classic pre-trained model chosen without its
last classification layer (Table 2). We replaced this layer by another one, more adapted
to the new data; and
• We applied fine-tuning process and we made some adjustments to increase the model
performance after the transfer learning step. The idea was to retrain the previously
used transfer learning model after making some modification on its layers numbers,
types, and new hyper parameters (learning rate, batch size, activation, and op-
timizer). In our case, we excluded the six last layers from any adjustment and
modified only the rest of our model by applying the learning process only on the
last 23 layers (Table 3).
Biomedicines 2022, 10, 2013 7 of 16

Table 2. MobileNet Transfer Learning summary.

Layer Type Output Shape Param

mobilenet_1.00_224 (Functional) (None, 7, 7, 1024) 3,228,864
global_average_pooling2d (None, 1024) 0
dropout (Dropout) (None, 1024) 0
dense (Dense) (None, 1024) 1,049,600
dense_1 (Dense) (None, 1024) 1,049,600
dense_2 (Dense) (None, 512) 524,800
dense_3 (Dense) (None, 512) 262,656
dense_4 (Dense)
Total params: 6,118,085 (None, 5) 2565
Trainable params: 2,889,221
Non-trainable params: 3,228,864

Table 3. MobileNet Fine Tuning summary.

Layer Type Output Shape Param

mobilenet_1.00_224 (Functional) (None, 7, 7, 1024) 3,228,864
global_average_pooling2d (None, 1024) 0
dropout_1 (Dropout) (None, 1024) 0
dense_5 (Dense) (None, 1024) 1,049,600
dense_6 (Dense) (None, 1024) 1,049,600
dense_7 (Dense) (None, 512) 524,800
dense_8 (Dense) (None, 512) 262,656
dense_9 (Dense)
Total params: 6,117,572 (None, 4) 2052
Trainable params: 2,888,708
Non-trainable params: 3,228,864

Regarding the VGG16 model, the adjustment was applied to the entire transfer learning
model (Table 4), excluding the last six layers, too. Then we trained only the last nine layers
to perform the fine-tuning step (Table 5).

Table 4. VGG16 Transfer Learning summary.

Layer Type Output Shape Param

vgg16 (Functional) (None, 7, 7, 512) 14,714,688
flatten_1 (Flatten) (None, 25,088) 0
dense_4 (Dense) (None, 1024) 25,691,136
dense_5 (Dense) (None, 512) 524,800
dense_6 (Dense) (None, 512) 262,656
dense_7 (Dense)
Total params: 41,195,332 (None, 4) 2052
Trainable params: 26,480,644, non-trainable params: 14,714,688

Table 5. VGG16 Fine Tuning summary.

Layer Type Output Shape Param

vgg16 (Functional) (None, 7, 7, 512) 14,714,688
flatten_1 (Flatten) (None, 25,088) 0
dense (Dense) (None, 1024) 25,691,136
dense_1 (Dense) (None, 512) 524,800
dense_2 (Dense) (None, 512) 262,656
dense_3 (Dense)
Total params: 41,195,332 (None, 4) 2052
Trainable params: 26,480,644, non-trainable params: 14,714,688
Biomedicines 2022, 10, 2013 8 of 16

We trained both DNN to detect normal cardiac rhythm as well as abnormalities and to
make predictions, thereafter. The model’s parameters were learned using only the training
set and the design choices depended on maximizing the performance on the validation data
set. We used Keras and TensorFlow as backend, dedicated for deep-learning processing
and implemented in Python, and we ran the training data and test data on “Google Colab”
cloud service.
Deep-learning algorithms contain different parameters, such as learning rate and
number of units, which heavily affect performance [15]. Indeed, it is very important
to choose optimal values for these parameters. In this work, we tried several trails of
experiments to identify the best ones.
In our study, the learning process went very well and there was no need to use the
automating hyper-parameters technique—Keras tuner, which takes too long.

2.5. Performance Metrics

In this paper, two deep-learning CNN models were developed and evaluated using
the classification report, which gives us accuracy values, recall, and the f1-score. Accuracy is
the measure of the total number of correct predictions among all predictions. Recall shows
false-negative occurrence; it is a sign of the failure of the model to predict. In other words, it is
the correct prediction of actual positive cases. F1-score is an average of recall and precision.
Our method performance was evaluated with five metrics (sensitivity, specificity,
accuracy, precision, and f1-score) given in equations below [16]:

TP
Sensitivity =
( TP + FN )

TN
Specificity =
( TN + FP)
( TP + TN )
Accuracy =
( TP + FP + FN + TN )
TP
Precision =
( TP + FP)
2 ∗ (Sensitivity ∗ Precision)
F1-score =
(Sensitivity + Precision)
where TP, TN, FP, and FN are, respectively, true positive, true negative, false positive, and
false negative detected ECG images. The confusion matrix, corresponding to each learning,
was also drawn up to give us an idea about the sensitivity and specificity of the algorithm
when predicting the test data.
After learning, the model with the best accuracy will be converted to a portable version
(TensorFlow Lite) and implemented on a Raspberry Pi 4B 8GB.

2.6. Raspberry Pi
The implementation of the studied models was made for the purpose of a later use on
connected objects. We used a Raspberry Pi 4B 8Go, given its robustness which allows it to
work with algorithms of this type. It is an ARM-based single-board nano-computer.

3. Results
We used MobileNet V2 and VGG16 algorithms for cardiovascular-disease diagnosis
and prediction. We calculated the accuracy corresponding to each of the four classes, to
evaluate and compare both models. Our work showed excellent accuracy; this is very
important for easily detecting and differentiating the different classes. The highest accuracy
was found in the case of Myocardial Infarction (MI) and Previous History of MI (HMI) for
both models. A summary of the performance metrics found for every class, in both models,
is given in Section 3.2.
Biomedicines 2022, 10, 2013 9 of 16

3.1. Data Augmentation

The data augmentation result is given on the image plate below (Figure 5). Images are
easily readable without affecting their information. The learning process can be performed
on the go with the newly-built ECG images without any fear of missing any information
needed for better learning.

Figure 5. ECG images modifications after the data augmentation process.

3.2. Classification Report for MobileNet V2 and VGG16

To evaluate the model, performance metrics values for each class are given in the
tables below (Tables 6 and 7) for MobileNet V2.
Table 6. Classification report for MobileNet V2 transfer learning.

Precision Recall f1-Score Support

Normal 0.95 0.87 0.91 47
Abnormal heartbeat (ABH) 0.88 0.86 0.87 35
Previous history of MI (HMI) 0.98 1.00 0.99 48
Myocardial infarction (MI) 0.90 0.96 0.93 57
accuracy 0.93 187
macro avg 0.93 0.92 0.93 187
weighted avg 0.93 0.93 0.93 187
Biomedicines 2022, 10, 2013 10 of 16

Table 7. Classification report for MobileNet V2 fine-tuning.

Precision Recall f1-score Support

Normal 1.00 0.89 0.94 47
Abnormal heartbeat (ABH) 0.91 0.91 0.91 35
Previous history of MI (HMI) 1.00 1.00 1.00 48
Myocardial infarction (MI) 0.90 0.98 0.94 57
accuracy 0.95 187
macro avg 0.95 0.95 0.95 187
weighted avg 0.95 0.95 0.95 187

Accuracy has increased from 0.93 to 0.95 after the fine-tuning step. It represents a
small evolution but is very useful for getting better performance from the model.
Concerning the VGG16 model, classification reports are shown in Tables 8 and 9. One
can see the same accuracy value for both models after the fine-tuning step (0.95). On
the other hand, the accuracy increased better when we applied the fine-tuning process
on the VGG16 model (from 0.91 to 0.95) compared to the one with MobileNet V2 (from
0.93 to 0.95).

Table 8. Classification report for VGG16 transfer learning.

Precision Recall f1-score Support

Normal 0.95 0.77 0.85 47
Abnormal heartbeat (ABH) 0.91 0.89 0.90 35
Previous history of MI (HMI) 0.89 1.00 0.94 48
Myocardial infarction (MI) 0.90 0.96 0.93 57
accuracy 0.91 187
macro avg 0.91 0.90 0.90 187
weighted avg 0.91 0.91 0.91 187

Table 9. Classification Rerort for VGG16 fine-tuning.

Precision Recall f1-score Support

Normal 0.95 0.87 0.91 47
Abnormal heartbeat (ABH) 0.94 0.89 0.91 35
Previous history of MI (HMI) 1.00 1.00 1.00 48
Myocardial infarction (MI) 0.90 1.00 0.95 57
accuracy 0.95 187
macro avg 0.95 0.94 0.94 187
weighted avg 0.95 0.95 0.95 187

3.3. Confusion Matrix

The confusion matrix is a representation of the five metrics (true positive, false posi-
tive, false negative, and true negative) used to evaluate the model performance in every
performed class.
In the present work, looking on the confusion matrix corresponding to MobileNet
V2 after the fine-tuning step (Figure 6), the myocardial infarction (MI) class showed the
highest average score of about 56. On the other hand, the abnormal heartbeat class (ABH)
presented the smallest average score of about 32.
Biomedicines 2022, 10, 2013 11 of 16

Figure 6. Confusion matrix for the MobileNet V2 model.

Concerning VGG16 after the fine-tuning step, we found the same observations as for
the MobileNet V2 model (myocardial infarction class with the highest average score of
about 57 and abnormal heartbeat class (ABH) with the smallest average score of about 31)
(Figure 7).

Figure 7. Confusion matrix for the VGG16 model.

In order to demonstrate the reproducibility of the model architecture to new data,

we tested it on the test data. The model performance demonstrated excellent results and,
therefore, one can make conclusions about the ability of our DNN-based approach to
generalize to a new set of ECG records from a different data.

3.4. Raspberry Pi
We evaluated the performance of MobileNet V2 and VGG16 using different evaluation
metrics. The accuracy reached 95% for both models. After implementing the TFLite version
(portable version), for both models, on a Raspberry Pi 4 B, 8Go, the prediction on the test
data showed excellent accuracy values equal to 94% and 92%, with an execution time of
0.16 and 0.24 s for MobileNet V2 and VGG16, respectively. Moreover, an optimized version
of both TFLite models will be studied in our next work. Indeed, this version will have a
smaller size to gain faster speed on wearable devices without losing performance.

4. Discussion
In recent papers, the state-of-the-art study on deep learning has shown satisfactory
results in detection, classification, and prediction tasks on medical images in general, and
on ECG in particular. Indeed, cardiac arrhythmia is the result of anomalies occurring in
the heart. These anomalies are reflected as deviation of ECG waveform from its normal
shape and size and cause abnormal activation, depolarization, and repolarization, after
anatomical changes in the structure of atria and ventricles.
In a recent study [17], the authors used 10,000 recorded ECG images from a pub-
lic arrhythmia database and performed two different class scenarios. They used a one-
Biomedicines 2022, 10, 2013 12 of 16

dimensional convolutional neural network (1D-CNN) and a long short-term memory

network (LSTM) block was added to this model for sequence learning. After training, the
accuracy values achieved for the reduced and merged rhythm classes were about 92.24%
and 96.13%, respectively. This combination gave better performances in other different
works [18–20].
In another study [21], the authors used the MIT-BIH arrhythmia database. They
obtained five-fold cross-validation accuracy of 0.834 in distinguishing normal and abnormal
cardiac arrhythmia, from ECG images with CNN-LSTM. Moreover, according to these
authors, the accuracy obtained by other hybrid architectures of deep-learning algorithms
was comparable to the CNN-LSTM.
Safdarian et al. performed a study about the myocardial infarction (heart attack), a
dangerous cardiac pathology [22]. They used a data from a single lead ECG myocardial
(MI) and achieved about 94.74% accuracy. Sharma et al. conducted a multiscale eigenspace
analysis on 12-lead ECG data and the accuracy was about 96% [23].
Mohammedzadeh et al. achieved an excellent accuracy of 99.38% with an MIT BIH data
set for arrhythmia classification [24]. In another study, the ECG arrhythmia MIT database
was classified using artificial neural network (ANN) with an accuracy of about 96.77% [25].
In a recent study [26], the authors presented a DNN model trained in a data set
with more than two million labeled exams, for recognizing six types of abnormalities in
12-lead ECG recordings, and representative of both rhythmic and morphological ECG
abnormalities. They obtained f1-scores above 80% and specificity over 99%. Compared
to analysis studied in a single-lead setup, the present results show ECG analysis based
on DNNs generalizes well to 12-lead exams, giving an excellent result very closer to the
standard clinical practice. Moreover, Hannun et al. developed a deep neural network
(DNN) to classify 12 cardiac rhythm classes using about 91,000 single-lead ECGs [27]. The
results found showed a ROC (Receiver Operating characteristic Curve) of 0.97 and an
average f1-score of 0.83. They demonstrated that the deep-learning model used was able to
classify different arrhythmia types from single-lead ECGs, with high performance closer to
that of cardiologists.
Baloglu et al. developed an efficient automated deep-learning model to distinguish
between 10 different myocardial infarction classes and normal ECG records on 12-lead
ECG signals [28]. They used an open-access Physiobank (PTB) ECG database and found
a classification performance over 99%. This model is easily implementable on portable
healthcare devices.
Acharya et al. proposed a method able to detect cardiac abnormalities and classify
them to four different classes [29]. The classification results achieved were about 97.98%,
99.61%, and 94.84% for accuracy, sensitivity, and specificity, respectively.
In a recent paper, the authors proposed a data augmentation method called RandECG
to classify ECG with deep neural networks (DNN) [30]. They applied various transfor-
mation techniques and selected the suitable ones for ECG. Tested on two different data
sets, the efficacy of RandECG has been improved up to 3.51% compared to the data sets
before augmentation.
Usually, ECGs are recorded on paper, which can be noisy. Therefore, digitizing these
records into a high-quality readable signal is critical for diagnosis and further analysis [31]. In
this paper, the authors proposed a powerful deep-learning approach for ECG digitization. The
approach achieved excellent segmentation and very high concordance with the first true signal.
Du et al. suggested a fine-grained multi-label ECG framework (FM-ECG) able to find
out the critical fine-grained parts from the ECG records and further abnormalities [32]. This
framework has shown an excellent performance of making multi-label classification in
ECG images. The authors used CNN-RNN (Recurrent Neural Network)-like framework,
which utilizes CNNs to extract important parts via fine-grained learning, and uses RNNs
to recurrently infer the abnormal categories considering the class dependencies. They used
12-lead ECG images from two different data sets. The experimental results demonstrated
Biomedicines 2022, 10, 2013 13 of 16

that FM-ECG is an excellent framework to help clinicians to detect abnormalities from

ECG records.
The project of Koshti et al. consists of a monitoring system for detecting cardiac
abnormalities from the MIT-BIH database [33]. The system analyses the signal, extracts
features, and detects abnormal conditions (arrhythmia). Thereafter, Raspberry Pi 4B sends
the results of the ECG signals to the web server. The framework used achieved about 95.4%
success rate.
In Cheikhrouhou’s team’s work [34], the authors proposed a modular 1D-CNN ap-
proach, which uses a trained machine-learning model for predicting cardiac abnormalities,
based on ECG records captured from IoT (Internet of Things) wearable devices. This
approach achieved 99.46% of accuracy using the MIT-BIH Arrhythmia database. They
applied the GridSearch algorithm with the cross-validation method.
Patil and Bhole presented a real-time ECG monitoring system using the Raspberry Pi
system [35]. Indeed, ECG records can be viewed at any place at any time. This important
concept is used to detect the heart defect and abnormalities in ECG images, without
involving an expensive ECG machine nor automatic human intervention. The availability
of real-time patient’s data anywhere anytime helps to take a quick diagnosis and, therefore,
to make more efficient and faster decisions which are crucial to saving patients’ lives.
Granados et al. proposed an IoT (Internet of Things) platform; useful to implement a
real-time analysis and the use of cloud deep neural network, for cardiovascular diagnosis
through ECG classification [36]. The major goal of this study is to use 12-lead ECG signals
to diagnose cardiovascular diseases, as well as different types of correlated abnormalities.
ECG classification was performed by a CNN classifier and Tensorflow library, implemented
on a GPU-accelerated machine. This IoT platform is very useful to increase healthcare ser-
vices efficiency, by conducting home treatment instead of using valuable hospital resources.
Therefore, these results can be used to ensure better lifestyle habits for people.
It is well known that preprocessing steps are of great importance in reaching an ex-
cellent image classification. Abadi’s team assumes that deep-learning techniques require
significantly more training data [14]. In the case of few images being available, the augmen-
tation technique can be used to duplicate images to increase the amount of data. Therefore,
data augmentation is used for improving classification performance, especially in images.
It should increase data amount and diversity by adding random changes based on aug-
mentation techniques. In this context, Nonaka and Seita explored a data augmentation
technique suitable for ECG data [30]. In fact, this technique has successfully improved
atrial fibrillation classification with single-lead ECG data, with no need to change the DNN
architecture. The authors used the public CinC/Challenge 2017 data set, which consists
of 8528 samples classed to four classes. The results found shown that the ECG augmenta-
tion is effective in classifying atrial fibrillations. Moreover, the classification accuracy has
been improved.
Compared to all these studies, our present work has shown remarkable results on
the detection, classification, and prediction of the four classes studied and achieved high
accuracy. Furthermore, model evaluation with the test dataset has achieved results compa-
rable to those made by clinicians. Therefore, the performed models could be used to help
clinicians for cardiac diagnosis and decision-making. However, we should note that none
of our performed deep-learning CNN algorithms may substitute for clinicians’ handiwork.
In addition, there are several limitations for this study. Indeed, we used a small
database for training and testing steps, which is insufficient regarding the thousands of
hyper-parameters in the deep-learning approach. Moreover, an independent test dataset is
missing which is considered another limitation. Furthermore, when selecting deep-learning
hyper-parameters, we did not take into account optimization techniques.

5. Conclusions
Cardiac abnormalities may cause dangerous damages to the heart and even lead to
death. Therefore, their rapid and accurate diagnosis is important to avoid deaths. For this
Biomedicines 2022, 10, 2013 14 of 16

purpose, clinicians had recourse to ECG interpretation which is critical for diagnosing car-
diac arrhythmia and requires expertise and is time-consuming. Thus, automatic diagnosis
using computers may be useful for this task.
In this paper, we proposed a new automatic deep-learning model to diagnose, classify,
and predict cardiac arrhythmias based on 12-lead ECG images interpretation. We used the
convolutional neural network (CNN), a very famous deep-learning technique for images
classification. Importantly, both trained models with the proposed architecture achieved
excellent performance with an accuracy over 95.00%. Therefore, with this performance, we
envisage using these models for automated diagnosis in intensive-care units and wearable
devices for better health-care monitoring.
In future studies, we will evaluate the model performance on the different cardiac
arrhythmia’s datasets. Moreover, we will use the ECG signals of various leads to find
the exact affected heart location. Further work will consider using more deep-learning
techniques and hyper-parameter optimization approaches.

Author Contributions: Conceptualization, L.M.; methodology, L.M.; software, L.M.; validation, L.M.
and I.B.D.; formal analysis, F.C.; investigation, L.M.; resources, L.M. and I.B.D.; data curation, L.M.;
writing—original draft preparation, L.M.; writing—review and editing, L.M.; visualization, L.M.;
supervision, I.B.D.; project administration, L.M. and I.B.D.; funding acquisition, O.D. and F.C. All
authors have read and agreed to the published version of the manuscript.
Funding: This research was funded by the Deanship of Scientific Research at Umm Al Qura Univer-
sity, grant number [22UQU4340610DSR01] And the APC was funded by Dar Al-Hekma University
and CIAMS Laboratory, Paris-Saclay University.
Data Availability Statement: The database used is free and published for public use on the following
links: https://data.mendeley.com/datasets/gwbz3fsgp8/1; https://data.mendeley.com/datasets/
gwbz3fsgp8/2 (accessed on 15 November 2021).
Acknowledgments: We would like to thank the Deanship of Scientific Research at Um Al Qura
University for supporting this work, grant number [22UQU4340610DSR01]. The R package repository
was developed as part of the first author’s master thesis accessed on 11 December 2021 at the
University of Sfax, College of Medicine.
Conflicts of Interest: The authors declare that they have no conflict of interest.

Abbreviations

The following abbreviations are used in this manuscript:

ECG Electrocardiogram
AI Artificial Intelligence
DL Deep Learning
CVD Cardiovascular Diseases
CNN Convolutional Neural Network
MI Myocardial infarction
HMI History of Myocardial Infarction
ABH Abnormal Heartbeat
DNN Deep Neural Network
TP True Positive
TN True Negative
FP False Positive
FN False Negative
LSTM Long Short-Term Memory Network
RNN Recurrent Neural Network
IoT Internet of Things
Biomedicines 2022, 10, 2013 15 of 16

References
1. Brown, B.H.; Smallwood, R.H.; Barber, D.C.; Lawford, P.; Hose, D. Medical Physics and Biomedical Engineering: Medical Science
Series; CRC Press: Boca Raton, FL, USA, 2017.
2. Silipo, R.; Marchesi, C. Artificial neural networks for automatic ECG analysis. IEEE Trans. Signal Process. 1998, 46, 1417–1425.
[CrossRef]
3. Agliari, E.; Barra, A.; Barra, O.A.; Fachechi, A.; Vento, L.F.; Moretti, L. Detecting cardiac pathologies via machine learning on
heart-rate variability time series and related markers. Sci. Rep. 2020, 10, 8845. [CrossRef] [PubMed]
4. Dudchenko, A.; Ganzinger, M.; Kopanitsa, G. Machine Learning Algorithms in Cardiology Domain: A Systematic Review. Open
Bioinform. J. 2020, 13, 25–40. [CrossRef]
5. Zhang, X.; Gu, K.; Miao, S.; Zhang, X.; Yin, Y.; Wan, C.; Yu, Y.; Hu, J.; Wang, Z.; Shan, T.; et al. Automated detection of
cardiovascular disease by electrocardiogram signal analysis: A deep learning system. Cardiovasc. Diagn. Ther. 2020, 10, 227–235.
[CrossRef]
6. Swapna, G.; Ghista, D.N.; Martis, R.J.; Ang, A.P.C.; Sree, S.V. ECG Signal generation and heart rate variability signal extraction:
Signal processing, features detection, and their correlation with cardiac diseases. J. Mech. Med. Biol. 2012, 12, 12. [CrossRef]
7. Chaddad, A.; Kucharczyk, M.J.; Desrosiers, C.; Okuwobi, I.P.; Katib, Y.; Zhang, M.; Rathore, S.; Sargos, P.; Niazi, T. Deep Radiomic
Analysis to Predict Gleason Score in Prostate Cancer. IEEE Access 2020, 8, 167767–167778. [CrossRef]
8. Han, X.; Hu, Y.; Foschini, L.; Chinitz, L.; Jankelson, L.; Ranganath, R. Deep learning models for electrocardiograms are susceptible
to adversarial attack. Nat. Med. 2020, 26, 360–363. [CrossRef]
9. Caldas, W.L.; Madeiro, J.P.V.; Mattos, C.L.C.; Gomes, J.P.P. A New Methodology for Classifying QRS Morphology in ECG Signals.
In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020; pp. 1–9.
[CrossRef]
10. Corradi, F.; Buil, J.; De Canniere, H.; Groenendaal, W.; Vandervoort, P. Real Time Electrocardiogram Annotation with a Long
Short Term Memory Neural Network. In Proceedings of the 2019 IEEE Biomedical Circuits and Systems Conference (BioCAS),
Nara, Japan, 5 December 2019; pp. 1–4. [CrossRef]
11. Johnson, K.W.; Soto, J.T.; Glicksberg, B.S.; Shameer, K.; Miotto, R.; Ali, M.; Ashley, E.; Dudley, J.T. Artificial Intelligence in
Cardiology. J. Am. Coll. Cardiol. 2018, 71, 2668–2679. [CrossRef]
12. Khan, A.H.; Hussain, M.; Malik, M.K. ECG Images dataset of Cardiac and COVID-19 Patients. Data Brief 2021, 34, 106762.
[CrossRef]
13. Khan, A.H.; Hussain, M. ECG Images dataset of Cardiac Patients. Mendeley Data V2, 2021. [CrossRef]
14. Abadi, M.; Chu, A.; Goodfellow, I.; McMahan, H.B.; Mironov, I.; Talwar, K.; Zhang, L. In Proceedings of the 2016 ACM SIGSAC
Conference on Computer and Communications Security, Vienna, Austria, 24–28 October 2016; pp. 308–318.
15. Fan, J.; Ma, C.; Zhong, Y. A Selective Overview of Deep Learning. Stat. Sci. 2021, 36, 264–290. [CrossRef]
16. Powers, D.M. Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. arXiv
2020, arXiv:2010.16061. [CrossRef]
17. Yildirim, O.; Talo, M.; Ciaccio, E.J.; Tan, R.S.; Acharya, U.R. Accurate deep neural network model to detect cardiac arrhythmia on
more than 10,000 individual subject ECG records. Comput. Methods Programs Biomed. 2020, 197, 105740. [CrossRef] [PubMed]
18. Murat, F.; Yildirim, O.; Talo, M.; Baloglu, U.B.; Demir, Y.; Acharya, U.R. Application of deep learning techniques for heartbeats
detection using ECG signals-analysis and review. Comput. Biol. Med. 2020, 120, 103726. [CrossRef] [PubMed]
19. Oh, S.L.; Ng, E.Y.; San Tan, R.; Acharya, U.R. Automated diagnosis of arrhythmia using combination of CNN and LSTM
techniques with variable length heart beats. Comput. Biol. Med. 2018, 102, 278–287. [CrossRef] [PubMed]
20. Lih, O.S.; Jahmunah, V.; San, T.R.; Ciaccio, E.J.; Yamakawa, T.; Tanabe, M.; Kobayashi, M.; Faust, O.; Acharya, U.R. Comprehensive
electrocardiographic diagnosis based on deep learning. Artif. Intell. Med. 2020, 103, 101789. [CrossRef]
21. Swapna, G.; Soman, K.P.; Vinayakumar, R. Automated detection of cardiac arrhythmia using deep learning techniques.
Procedia Comput. Sci. 2018, 132, 1192–1201. [CrossRef]
22. Safdarian, N.; Dabanloo, N.J.; Attarodi, G. A New Pattern Recognition Method for Detection and Localization of Myocardial
Infarction Using T-Wave Integral and Total Integral as Extracted Features from One Cycle of ECG Signal. J. Biomed. Sci. Eng. 2014,
07, 818–824. [CrossRef]
23. Sharma, L.N.; Tripathy, R.K.; Dandapat, S. Multiscale Energy and Eigenspace Approach to Detection and Localization of
Myocardial Infarction. IEEE Trans. Biomed. Eng. 2015, 62, 1827–1837. [CrossRef]
24. Mohammadzadeh-Asl, B.; Setarehdan, S.K. Neural network based arrhythmia classification using Heart Rate Variability signal.
In Proceedings of the 14th European Signal Processing Conference, Florence, Italy, 4–8 September 2006; pp. 1–4.
25. Vishwa, A.; Lal, M.K.; Dixit, S.; Vardwaj, P. Clasification of Arrhythmic ECG Data Using Machine Learning Techniques. Int. J.
Interact. Multimedia Artif. Intell. 2011, 1, 67. [CrossRef]
26. Ribeiro, A.L.P.; Ribeiro, M.H.; Paixão, G.M.M.; Oliveira, D.M.; Gomes, P.R.; Canazart, J.A.; Ferreira, M.P.S.; Andersson, C.R.;
Macfarlane, P.W.; Meira, W.; et al. Automatic diagnosis of the 12-lead ECG using a deep neural network. Nat. Commun. 2020,
11, 1–9. [CrossRef]
27. Hannun, A.Y.; Rajpurkar, P.; Haghpanahi, M.; Tison, G.H.; Bourn, C.; Turakhia, M.P.; Ng, A.Y. Cardiologist-level arrhythmia
detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 2019, 25, 65–69. [CrossRef]
Biomedicines 2022, 10, 2013 16 of 16

28. Baloglu, U.B.; Talo, M.; Yildirim, O.; Tan, R.S.; Acharya, U.R. Classification of myocardial infarction with multi-lead ECG signals
and deep CNN. Pattern Recognit. Lett. 2019, 122, 23–30. [CrossRef]
29. Acharya, U.R.; Fujita, H.; Oh, S.L.; Hagiwara, Y.; Tan, J.H.; Adam, M. Application of deep convolutional neural network for
automated detection of myocardial infarction using ECG signals. Inf. Sci. 2017, 415–416, 190–198. [CrossRef]
30. Nonaka, N.; Seita, J. RandECG: Data Augmentation for Deep Neural Network based ECG classification. In Proceedings of the
35th Annual Conference of the Japanese Society for Artificial Intelligence, Online, 8–11 June 2021; p. 8.
31. Li, Y.; Qu, Q.; Wang, M.; Yu, L.; Wang, J.; Shen, L.; He, K. Deep learning for digitizing highly noisy paper-based ECG records.
Comput. Biol. Med. 2020, 127, 104077. [CrossRef] [PubMed]
32. Du, N.; Cao, Q.; Yu, L.; Liu, N.; Zhong, E.; Liu, Z.; Shen, Y.; Chen, K. FM-ECG: A fine-grained multi-label framework for ECG
image classification. Inf. Sci. 2020, 549, 164–177. [CrossRef]
33. Koshti, M.; Ganorkar, S.; Chiari, L. IoT based health monitoring system by using Raspberry Pi and ECG signal. Int. J. Innov. Res.
Sci. Eng. Technol. 2016, 5, 8977–8985.
34. Cheikhrouhou, O.; Mahmud, R.; Zouari, R.; Ibrahim, M.; Zaguia, A.; Gia, T.N. One-Dimensional CNN Approach for ECG
Arrhythmia Analysis in Fog-Cloud Environments. IEEE Access 2021, 9, 103513–103523. [CrossRef]
35. Patil, P.; Bhole, K. Real time ECG on internet using Raspberry Pi. In Proceedings of the International Conference on Communica-
tion, Computing and Internet of Things (IC3IoT), Chennai, India, 15–17 February 2018; pp. 267–270.
36. Granados, J.; Westerlund, T.; Zheng, L.; Zou, Z. IoT Platform for Real-Time Multichannel ECG Monitoring and Classification with
Neural Networks. In Proceedings of the International Conference on Research and Practical Issues of Enterprise Information
Systems, Shanghai, China, 18–20 October 2017; pp. 181–191.