Performance Evaluation of Deep Learning Models in Detection of Distributed Denial of Service Attacks 2
Performance Evaluation of Deep Learning Models in Detection of Distributed Denial of Service Attacks 2
Abstract - A Distributed Denial of S ervice (DDoS ) attack The goal of a Denial of Service (DoS) assault is to flood a
occurs when an intruder or a group of attackers attempts to target server with traffic, rendering it unusable. Unlike other
prevent legitimate users from accessing a service. A DoS attack is forms of assaults, the primary goal of a DoS attacker is to
carried out by a single system, while a DDoS attack is carried out degrade or bring down a target rather than steal information.
by several systems. DDoS attacks can be directed at various OS I DDoS (Distributed DoS) is a type of DoS assault in which
layers. Deep learning has played a significant role in the growth several systems launch attacks at the same time, exhausting the
of intrusion detection solutions in recent years. The primary goal resources on the target system. DDoS assaults can be launched
of this work is to detect and identify DDoS attacks in the OS I using botnets of malware-infected computers. The greatest
model's application, network, and transport layers using deep
learning models. The proposed models have been tested against
DDoS attack on GitHub to date occurred in February of 2018.
the CICDDoS 2019 dataset. The CICDDoS 2019 data set contains
Incoming traffic was identified at a rate of 1.3 gigabytes per
network and transport layer DDoS attacks. For the second during this attack. Fortunately, GitHub has a DDoS
CICIDDOS 2019 dataset, DNN, LS TM, and GRU achieved an security tool that detected the attempt within 10 minutes and
average accuracy of 99.32%, 99.4%, and 92.5%, respectively. was able to halt it immediately. This was the world's greatest
The proposed models' performance is compared to that of a few DDoS attack, lasting approximately 20 minutes. According to a
other higher-accuracy models and found that the proposed survey, DoS/DDoS assaults rose by more than 542 percent in
models have higher accuracy with fewer epochs. the first quarter of 2020 as compared to the fourth quarter of
2019. This is attributed to a surge in disruptive cyber behaviour
Keywords— Distributed Denial of Service (DDoS), Deep Neural during the COVID-19 pandemic, according to NexusGuard
Network, Long-Short-Term-memory, CICIDS2017, CICDDoS2019, analysts.
Accuracy
DDoS attacks can be classified into three categories.
Volume-based attacks, Protocol or transport/network-layer
I. INT RODUCT ION DDoS attacks, and Application-layer attacks are the three types
An Artificial Neural Network (ANN) is a system that can of DDoS attacks. Volume-based attacks overload a resource,
learn and make intelligent and wise decisions using algorithms. such as a server, by flooding it with bogus traffic. Attacks
Deep Learning is a form of artificial neural network. A deep using ICMP (Internet Control Message Protocol), UDP (User
learning model uses neural networks with several hidden layers Datagram protocol), and spoofed-packet floods are all possible.
of nodes. The number of layers in the neural network is Protocol or transport/network-layer DDoS attacks, such as
referred to as "deep" here. Between input and output, a set of SYN floods and Smurf DDoS, send a large number of packets
hidden layers perform feature recognition and extraction. Deep to a targeted server. Overflowing applications with bogusly
learning was created primarily to manage large quantities of generated requests are used to initiate application-layer attacks.
data and run complex algorithms in order to improve DDoS attacks may also look like non-malicious traffic. Even a
performance. Both feature extraction and classification are large number of genuine requests from genuine users will bring
done with Deep Learning models. the server to a halt. As a result, understanding these attacks
necessitates traffic flow analysis to determine what is actually
In Deep Neural Network (DNN), data flows from the input happening.
layer to the output layer and there are no loops or cycles
created. The information is just going forward. A Recurrent The impact of a DDoS attack is becoming more dangerous
Neural Network (RNN) is a type of ANN (Artificial Neural due to recent advancements in network and communication
Networks) in which the nodes' connections forms a directed technologies. As a result, research into DDoS attack detection
graph. Long-Short-Term-Memory (LSTM) has been developed is becoming increasingly important. Many relevant research
to solve the vanishing gradient problem in RNN. Input gate, projects have been completed recently, and progress has been
output gate, and forget gate make up an LSTM. The GRU made. Machine learning is a common technique for detecting
(Gated Recurrent Unit) is a newer form of RNN that looks a lot DDoS attacks using statistical features. Machine learning
like an LSTM. The GRUs did away with the cell state and methods, on the other hand, use shallow representation models,
instead used the hidden state to move data. There are only two which do not result in improved accuracy. However, deep
gates on it: a reset gate and an update gate. learning models have recently demonstrated their ability to
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
differentiate DDoS traffic from benign traffic by distinguishing system was able to distinguish between network scanning,
high-level features from low-level features. probing and basic DDoS attacks [5].
In addition, it is discovered that the majority of DDoS Doshi et al. presented a packet-level DoS detection system
attack detection approaches use the NSL KDD and KDDCUP that reliably distinguishes between standard and DoS attack
99 datasets to assess performance. In this work, it is suggested traffic collected from IoT devices [6]. A small number of
to use recent datasets, CICDDoS2019. To train and identify the features were found and us ed to reduce the computational
types of attacks in these scenarios, three models are built using overhead. The authors tested five different machine learning
DNN, LSTM and GRU. DDoS attacks can be divided into classifiers on a data set of regular and DoS attack traffic
network/transport layer attacks or application layer attacks obtained from IoT devices. Doshi et al. demonstrated that using
depending on which layer is attacked by a DDoS attacker. IoT-specific network behaviours and sufficient feature
CICDDoS2019 dataset contains traces of application and selection can result in high accuracy of DDoS attacks detection
transport/network layers attacks. The rationale behind using in IoT network traffic using a variety of machine learning
these methods is briefly described below: algorithms, including neural networks. Elsayed et al. used the
CICDDoS2019 data set to build a DDoSNET, which is an IDS
Due to its structure, which incorporates feature extraction
for detecting DDoS attacks in a Software Defined Network
and classification operations, and its layers, the DNN model environment. RNN and Auto Encoder were used in this attempt
can classify rapidly and accurately even with tiny samples.
[7]. Back propagation neural networks were developed by
Since LSTM and GRU are capable of learning all the long-term
Jiang et al. for anomaly detection, and this framework
dependencies in the dataset, the trends in the traffic could be combined traffic and user activity features obtained from web
easily analyzed. This would help to classify the normal and
server logs. The performance was evaluated using the
attack traffic. The novel feature of this work is that Bayesian CICIDS2017 data set [8].
Optimization is used to find the more appropriate values for
hyper-parameter used in the models. To the best of our Shone et al. suggested an unsupervised feature learning
knowledge, in the recent attempts, Bayesian Optimization has algorithm called the Non-symmetric Deep Auto Encoder
net been used for hyper-parameter tuning. The goal of the (NDAE). The Random Forest algorithm was used to classify
feature selection task is to filter out the irrelevant or redundant this model, which was constructed using stacked NDAEs [9].
variables given the ones already selected and select only the Two methods for detecting Distributed Reflection Denial of
ones providing collectively unique information for the outcome Service (DrDoS) attacks in the Internet of Things were
of interest leading to an optimal predictive model in terms of proposed in [10]. They used a hybrid Intrusion Detection
performance. The most significant advantage of RNN models System (IDS) to detect IoT-DoS attacks in the first method,
like LSTM and GRU is that, it is not needed to manually and a deep learning model based on LSTM was trained with
extract features from the dataset as done in traditional models. the most recent data set to detect different types of DrDoS in
During training, the network gains the ability to extract the second method. [11] proposes a deep classification model
characteristics. All that is required is the definition of the focused on flow data to detect slow DoS attacks on HTTP. The
neural network architecture as well as a labelled dataset. While classifier is evaluated using the CICIDS2017 data collection.
the earlier hidden layers learn from the dataset, the last hidden The results show that the classifier has a 99.61% accuracy. A
layer contains new engineered features. DNN is suggested as a deep learning model for detecting
DDoS attacks on a set of packets collected from network traffic
The remaining part of the paper is set out as follows:
in the study [12]. In this work, the attack types were classified
Section 2 examines the deep learning models developed to with an accuracy rate of 94.57%.
detect DDoS attacks using contemporary and CICDDoS2019
datasets. A brief summary of the dataset is provided in Section On the CICDDoS2019 dataset, which contains details about
3. Section 4 describes how DNN, LSTM and GRU are used to reflection-based and exploitation-based attacks, the attempt
classify the dataset In Section 5, the proposed model's success [13] explores the effect of data balancing algorithms in the
is analyzed and the conclusion is presented in Section 6. network traffic classification issue on various forms of
distributed denial of service attacks. Data balancing algorithms
II. RELATED WORKS such as naive random sampling, synthetic minority sampling,
and adaptive synthetic sampling have been shown to be
Hsieh and Chan et al. proposed a DDoS detection approach successful in detecting network attacks. This work showed a
based on big data and neural networks. The detection system maximum accuracy of 98.62%. [14] propos ed an Improved
was built with R on Apache Spark [1]. Ferrag et al. (2020) Deep Sparse Autoencoder-based Approach with two hidden
explored various deep learning models for cyber security layers for detecting DDoS attacks. The main goal of this work
intrusion detection. The two new real-time traffic data sets is to use an autoencoder to extract representative features from
CSECICIDS2018 and the Bot-IoT have been reviewed in this the CICDDoS2019 dataset, reduce classification error, and
work [2]. Corin et al. suggested a method called "LUCID" that correctly detect DDoS attacks. This work obtained 98%
used CNN properties to classify traffic flows as malicious or detection accuracy. Furthermore, due to DDOS attacks,
benign [3]. Asad et al. used a deep learning method to build a misusing the cloud architecture's services and resources has
framework for detecting DDoS attacks. On the CICIDS2017 become an issue. To address this issue, the authors of [15] used
data collection, this system used a feed-forward back- the hidden Markov Model to observe network traffic and the
propagation network to accurately identify application layer Random Forest to classify detected attacks from normal flow.
DDoS attacks, with an accuracy of 98% [4]. Anthi et al. used From the reviews, it is believed that deep learning has
machine learning algorithms to build an IDS for the IoT. The
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
potential to detect recent types of DDoS attacks and can be application, network, and transport layers, including a deep
well explored to improve the detection accuracy. However, the neural network developed with the feed-forward back-
problem of finding best values of hyper-parameters was not propagation technique and a recurrent neural network built
considered, as well as the question of the influence of balance with LSTM and GRU. The architecture of the proposed work is
on the accuracy of the constructed classifiers. In this article, the
shown in Fig. 1. The functions and structure of each
use of Bayesian Optimization is considered to find best values
for hyper-parameters. component in presented below.
A. Preprocessing
III. DAT ASET DESCRIPT ION
Pre-processing is a method for getting data ready for use
Given the nature of DoS/DDoS attacks, this section offers a
before analysis. The data may be usually raw or unprocessed.
short rundown of benchmark datasets for developing
DoS/DDoS attack detection systems. DARPA, KDD, NSL- When in 'unprocessed' form, the data is of little value for
KDD, and others are examples of publicly available datasets analysis. Thus, pre-processing is applied before handing it to a
that have been used as benchmark datasets. CICDDoS2019 classification algorithm. In order to better classify the
[16] is the datasets used in this study to design and evaluate CICDDoS2019 data, the following processing has been done:
deep learning models. This dataset contains traces of DDoS The 'infinity' value, which had misled the model's training, was
attacks on the application and transport/network layers and replaced with '1′, and the lines holding the 'NaN' values were
include the most recent and benign common attacks. Flow removed. Eight categorical features (Source IP, Flow ID,
length, total forward packets, total backward packets, and so on Source Port, Destination IP, Destination Port, Protocol,
are among the 86 features [16]. The dataset is divided into Timestamp, Similar HTTP) that do not contribute to training
seven categories: benign, SYN, Portmap, NETBIOS, UDP,
and nine features containing only '0′ value were removed from
UDP-LAG and LDAP. Table 1 lists the various types of attacks
and provides a brief overview of each. the dataset and the model was trained with 69 features. For
DDoS attack detection and categorization, the dataset is
labelled into two classes: 'BENIGN' traffic is labelled '0′ while
T ABLE 1: T YP ES OF ATTACKS IN CICDDO S2019
the attacks are labelled '1′. Since all the features should be in
Types of Description the same range, scaling is done using different scaling
attacks techniques, such as standard scalar and MinMax Scalar.
MinMax Scalar is used in this work to translate all the features
LDAP A DDoS reflection attack in which reques ts
to the same set. Then, using label encoding, the categorical
are sent to various computers on the internet
values of the class labels are replaced with a numeric value
using a spoofed source IP address, and the
between 0 and the number of class labels. The class label for
responses are directed to the spoofed address
ICDDOS2019 is 0 (for benign traffic) and 1 for attack traffic.
rather than the actual sender.
All the types of attacks are marked as attack traffic. The data
UDP The attacker sends packets containing UDP
packets to random ports on the host. set is now able to be fed to the deep learning model after pre-
processing. Below, the deep learning models used in this work
Portmap Attacker forwards the client to a port number are explained to classify the chosen dataset.
to communicate with the Remote Procedure
Call.
NETBIOS Malicious hackers have access to some kind
of malign software to enable them to target
or monitor the shared document.
SYN A SYN-DOS attack is a form of denial-of-of-
service attack in which an attacker rapidly
establishes a link but then hangs it
immediately Waiting for half-open
connections could slow down the service.
UDP-LAG An UDP-based attack disrupts the connection
between the client and the s erver This attack
is used in video games where players want to
halt or delay the progress of their opponents'
units.
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
Hyper-Parameters Values
Optimizer Adam
Loss Binary_crossentropy
Fig 4. LSTM/GRU Architecture
Activation Sigmoid
Learning rate 0.001
Batch Size 1024
V. RESULT S AND DISCUSSION Dropout 0.1
The experimental results obtained for the proposed neural Number of hidden layers 1,2,3
network models are presented and discussed in this section.
Google Co-laboratory and Python Programming are used to The performance of the proposed models has been tested.
train and test the proposed models. Co-laboratory is a fully The experiments have been carried out using the tuned hyper-
cloud-based Jupyter notebook environment that doesn't parameters presented in Table 3, which yielded the best
require. The models used here have been trained on a GPU training results. The values of different performance metrics
DELL EMC 740 with 128 GB RAM and 32GB GPU are shown in Table 4. From Table 4, it is understood that
RAM. Table 2 summarizes the details of experimental LSTM provides better classification accuracy for
setup. CICDDoS2019 data set.
In CICDDoS2019 data set, the six labels of the attack class Only the models with high accuracy were used for
have been translated into 'attacks.' So, in the output layer, comparison. However, these models are found to have more
binary cross entrophy is used as a loss function. The data set hidden layers or epochs. The proposed models were able to
CICDDoS2019 is divided into training and test sets in an 80/20 provide comparable precision with fewer layers or epochs. It is
ratio (80 percent for training, 20 percent for tes ting). Training believed that fine-tuning hyper parameters has resulted in
data is given for the proposed neural network models. Their improved accuracy. In addition, since GPU is used, the
performance against DDoS attacks for test data sets were proposed models took less time to practice. Table 5 shows the
assessed after training of DNN, LSTM and GRU. Accuracy, training time taken by each the models developed in this work.
Precision, F1 Score and Recall have been used as metrics to
evaluate the performance of the developed models. The hyper- T ABLE 5. T raining T ime
parameter settings are used in Table 3 to run the built models.
Models T raining time /epoch (ms)
DNN 163
LST M 180
GRU 105
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
T ABLE 6 P ERFORMANCE EVALUATION OF DNN, LST M AND GRU ON CICDDO S2019 DATA SET
Epochs =20
Models DNN LST M GRU
Number of Layers 1 2 3 1 2 3 1 2 3
T raining Accuracy (%) 99.79 99.8 99.8 99.1 99.32 99.41 92.25 92.51 92.73
T esting Accuracy(%) 99.79 99.79 99.8 99.02 99.43 99.38 92.87 93.07 93.55
Precision (%) 99.7 99.7 99.7 98.8 99.04 98.83 93.8 93.9 94.1
Recall (%) 99.9 99.9 99.9 98.56 98.3 99.12 91.8 92.67 92.9
F1-Score (%) 99.8 99.8 99.8 99.01 99.3 99.23 92.9 93.29 93.56
It is not possible to compare the training time of the proposed models place appropriately. Using s ampling techniques like SMOTE,
with other models in the literature as these works have not mentioned random sampling etc, the dataset may be balanced and then
the details of GPU, if used. Table 6 shows the values obtained for training can be carried out as a future work. The dataset contains
different performance metrics for varied number of epochs. The several types of attacks, and this work attempts to identify whether
performance of the proposed models is accessed against each the traffic is benign or attack, but does not identify the types of
one of the two classes. The indices TP, FP, TN, and FN are attack. This could also be addressed in future.
used to calculate the values of various performance metrics
VI. CONCLUSION AND FUT URE WORK
such as accuracy, precision, specificity, and sensitivity [17]. To
find the values of these indices, the confusion matrix obtained In this work, the use of deep learning models is investigated to
during training and testing the models are used. The confusion create a DDoS attack detection system in this work since deep
matrix, as known, is a visualization tool used in supervised learning has shown its promise in many domains. The recent
learning and can be used to see how well the prediction results researches on the topic is evaluated and it is discovered that
correspond to the actual data. Fig. 5 depicts the confusion deep learning models can be used to boost the performance of
matrix obtained while training the GRU for one epoch. The existing systems. To boost accuracy even further, a DNN has
diagonal elements of the confusion matrix represent correct been created with a collection of hidden layers, as well as
classifications. The predicted classes are represented by X axis, RNN-based LSTM and GRU. The models are analyzed in a
and the actual classes are represented by Y axis. Similarly, the number of network configurations parameters. These models
values for performance metrics have been calculated for all the DNN, LSTM and GRU gave an accuracy of 99.32%, 99.4%
models for different number of epochs. and 92.5% respectively for CICDDoS2019 data set. In the
future, the models can be expanded to include very recent data
sets with a large number of DDoS attack traces. Other learning
models, such as auto encoder and stacked auto encoder, will
be tried in the future to increase accuracy and discover new
types of DDoS attacks. Also, sampling techniques like SM OTE can
be applied on the dataset to address the data imbalance issues. Since
the CICDDoS2019 dataset contains more than 80 features and all the
features will not contribute for classification, feature selection
algorithms may be employed to reduce the feature set and improve
the training accuracy.
A CKNOWLEDGMENT
The authors acknowledge and thank the Department of
Science and Technology (Government of India) for
sanctioning the research grant (Ref. No.SR/FST/COLLEGE-
096/2017 dated 16.01.2018) under Fund for Improvement of
Fig. 5. Confusion Matrix – GRU S&T Infrastructure (FIST) program for completing this work.
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.
Proceedings of the Fifth International Conference on Electronics, Communication and Aerospace Technology (ICECA 2021)
IEEE Xplore Part Number: CFP21J88-ART; ISBN: 978-1-6654-3524-6
Authorized licensed use limited to: UNIVERSITY PUTRA MALAYSIA. Downloaded on January 10,2023 at 07:54:51 UTC from IEEE Xplore. Restrictions apply.