Improved Classification for Pneumonia Detection

using Transfer Learning with GAN based Synthetic

2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence) | 978-1-6654-1451-7/20/$31.00 ©2021 IEEE | DOI: 10.1109/Confluence51648.2021.9377062

Image Augmentation
Devansh Srivastav, Akansha Bajpai Dr. Prakash Srivastava
Department of Computer Science and Engineering, Department of Computer Science and Engineering,
Amity University KIET Group of Institutions
Noida, India Ghaziabad, India
[email protected], [email protected] [email protected]

Abstract—Deep learning techniques have found their appli- efficiency [7]. Since the introduction of generative adversarial
cations in various domains, and they are being widely used in networks (GAN), the issue of under-sampled datasets has also
medical treatments and diagnostics. To diagnose diseases viz. been resolved as the synthetic data, similar to the original data,
pneumonia, the examination of chest X-ray images are often
conducted, and the efficiency of diagnosis can be significantly can be generated with a very low loss percentage.
improved with the use of computer-aided diagnostic systems. In this paper, a computer-aided diagnostics model based on
Deep learning algorithms are used in this paper for the clas- deep learning is proposed to diagnose pneumonia by classifica-
sification of chest X-ray images to diagnose pneumonia. Deep tion of chest X-ray images. GANs were utilised to oversample
convolutional generative adversarial networks were trained for the dataset and the concept of transfer learning was applied
augmentation of synthetic images to oversample the dataset for
the model to perform better. Then transfer learning was used with with CNN to classify with higher accuracies. Comparative
convolutional neural networks by utilising VGG16 as the base analysis was also performed between the proposed model and
model for image classification. The model was able to achieve the naı̈ve CNN models.
94.5% accuracy on the validation set. In comparison with the The remaining paper is organised as follows: Section II
naı̈ve models, the accuracy of the proposed model was found to describes about the related work. Section III reflects the
be significantly higher.
Index Terms—pneumonia detection, DCGAN, chest X-ray, dataset description and the proposed method is presented in
deep learning, computer-aided diagnostics, transfer learning Section IV. The discussion of results is presented in Section
V and Section VI presents the conclusion along with future
I. I NTRODUCTION directions.

Pneumonia is an upper respiratory infection that affects II. R ELATED W ORK

the lungs. Primarily, pneumonia can be classified as bacterial, Several studies have been performed to enhance deep
viral and fungal [1]. Some common symptoms include fever learning applications in the medical field and to diagnose
combined with cough and sputum production which are similar pneumonia. Each of these studies have their advantages and
to some other non-infectious respiratory diseases [2]. disadvantages.
The most common method for the detection of pneumonia Using the chest X-ray images, features have been extracted
is by using the chest X-ray [3]. Solutions based on deep in [8] to perform image classification using deep transfer
learning are being preferred for medical diagnosis these days learning. The transfer learning models were optimised using
[4]. They have sometimes even outperformed radiologists in stochastic gradient descent and activation maps were plotted
terms of the accuracy of prediction [5]. Convolutional neural for each individual network.
networks (CNN) show promising results for classification of Five different transfer learning neural network architectures,
images and are being widely applied in biomedical image namely, AlexNet, DenseNet121, InceptionV3, ResNet18 and
diagnostic systems. The flow of information and gradients in GoogLeNet were combined to form an ensemble in [9] for
CNN makes the optimization tractable [6]. By utilising CNN, pneumonia classification. It was observed that the model
powerful classification models can be built to classify chest performance could be further increased by increasing the size
X-ray images to diagnose pneumonia. of the dataset.
The CNN model performance can be further improved by Pneumonia diagnosis was performed by analysing radio-
utilising the approach of transfer learning wherein, a model graphs by machine learning in [10]. The CNN model was
is trained for a particular task to perform a similar task using also used to classify bacterial and viral pneumonia using
a predefined set of weights and other parameters. It not only radiographs. It was found that DenseNet201 outperformed
saves the computational resources but also increases the model SqueezeNet in classification of chest X-ray images.

978-1-6654-1451-7/21/$31.00 2021
c IEEE 433

Authorized licensed use limited to: Frankfurt University of Applied Sciences. Downloaded on June 17,2022 at 15:28:37 UTC from IEEE Xplore. Restrictions apply.
 The CNN models were used in [11] to classify paediatric
CXRs for pneumonia detection. Heat maps were generated and
superimposed on the input to localise the region of interest.
Finally, lung masks were applied to the generated heat maps
to localise the region of interest specific to lungs.
For the evaluation and training of CNN, a cross-sectional
design was used in [12] to detect where the radiographs were
acquired. In 3 out of 5 natural comparisons, the CNN model
achieved better internal performance.
A deep neural network based on Mask-RCNN was utilised
in [13] for pixel-wise segmentation, identification and localiza-
tion of pneumonia in chest X-ray images. The model merges
Fig. 1. Visualtization of Chest X-ray Dataset.
bounding boxes from multiple models to obtain better results.
Features were extracted and combined using the mRMR
method of feature selection in [14]. These features were given transfer learning was used with CNN to differentiate between
as inputs to the decision tree, KNN and SVM classifiers. The the chest X-ray images as healthy or with pneumonia.
model accuracy suggests that mRMR method was effective for
dimensionality reduction of the feature set. A. Synthetic Image Generation using DCGAN
The performance of various CNN models was evaluated
in [15] on chest X-ray images. The hyperparameters were In the DCGAN architecture, as shown in the “Fig. 2”, both
further fine-tuned to improve the performance of the model. discriminator and generator networks are deep CNN. The role
DenseNet-169 was found to perform better than the other of the discriminator is to recognise between the genuine and
models when the rbf kernel was used. counterfeit image samples being generated by the generator.
An automated algorithm was proposed in [16] for paediatric It outputs probability of the inputted sample being a real
pneumonia using end-to-end learning. Using transfer learning, sample. The role of the generator is to synthesise such image
the model was initialised to prevent negative noise impact. samples that the discriminator will consider as real with high
Deep learning frameworks integrate residual and convolved probability [19].
images to detect childhood pneumonia.
A study of six CNN models was conducted in [17] with a
learning rate of 0.0001 over 500 epochs. GoogLeNet, LeNet
and VGG16 obtained higher accuracies for pneumonia detec-
tion using chest X-ray images.
In all these studies, it was observed that availability of a
larger dataset can significantly improve the model results. The
same can be done using GANs as proposed further in this
paper.
III. DATASET D ESCRIPTION
The dataset [18] contains 5856 images of chest X-rays
segmented into two classes. One class contains chest X-ray Fig. 2. Architecture of DCGAN.
images of a healthy person while the other class contains that
of a person with pneumonia. These chest X-ray images were The generator gets an input vector of 100 numbers randomly
graded by medical experts and were taken during the routine generated from a simple distribution. The network contains
check-up of patients [8]. The sample images for each class are four fully connected transposed convolutional layers, also
shown in “Fig. 1”. known as fractionally-strode convolutional layers. The ReLU
activation function and batch normalisation are applied to each
IV. P ROPOSED M ETHOD of the fully connected layers and the tanh activation function
The chest X-ray dataset [18] is imbalanced and contains is applied to the output layer.
fewer X-ray images of a healthy person. Training the neural Discriminator is a CNN with four fully connected con-
network on an imbalanced dataset would result in less accuracy volutional layers. LeakyReLU activation function and batch
due to high false positive rate as the model will be biased normalisation are applied to each of the fully connected layers
towards the dominant class. To overcome this issue, DCGANs and the output layer is being applied with the sigmoidal
were used to augment synthetic images of the minority class activation function. Batch normalisation significantly improves
in order to oversample the dataset. Image augmentation tech- the optimization of neural networks and is highly effective in
niques were then applied to the images in the original dataset the case of DCGANs [20]. The particular batch is normalised
as well as to the images generated by DCGANs. Finally, using its own statistical features [21].

434 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence 2021)

Authorized licensed use limited to: Frankfurt University of Applied Sciences. Downloaded on June 17,2022 at 15:28:37 UTC from IEEE Xplore. Restrictions apply.
 Equation “(1)” represents a minimax loss function which B. Image Classification and Transfer Learning
the generator tries to minimise while the discriminator tries For training the CNN model, the first step was to augment
to maximise in order to optimise the training process. In the images. The original dataset images, as well as the images
the equation, D(x) is the output probability estimated by synthetically generated using DCGAN, were augmented as per
the discriminator that the real sample is real, G(z) is the the parameters shown in TABLE I. Augmentation of images
generator’s output at a given noise z, D(G(z)) is the output helps to enhance the image recognition model in terms of
probability of the discriminator that the fake sample is real performance and accuracy [23].
and E is the expectation value [22].
TABLE I
Ex [log(D(x))] + Ez [log(1 − D(G(z)))] (1) PARAMETERS FOR I MAGE AUGMENTATION

Parameter Value
The DCGAN model was trained over 120 epochs with
randomly generated noise for the generator. The initial, inter- Rotation Range 20
mediate, and final stages of the GAN training can be visualised Rescale 1./255
in “Fig. 3”. “Fig. 4” reflects the training loss of the generator Zoom 0.2
and discriminator. It also depicts the comparison between the Width Shift Range 0.2
real images and the fake images being generated by DCGAN.
Shear 0.2
Horizontal Flip True
Height Shift Range 0.2
Standard Normalization True

In order for the model to perform better, VGG16 was used

as the base classifier to use the concept of transfer learning.
Consequently, all the augmented images were processed as
per the desired setting for VGG16. The images were resized
Fig. 3. Different Stages of DCGAN Training. to an input size of (244,244) and were loaded in a batch
of 32. For VGG16, average pooling was used and ImageNet
weights loaded. The architecture of the VGG16 model can be
visualised in “Fig. 5”.
Further, the base model was modified to better suit the
particular use case. The layers of the base model were flattened
and two hidden layers with 128 and 256 neurons each were
added. The ReLU activation function and dropout regulari-
sation rate of 20% were applied. Regularisation enables the
model to generalise the result and avoid overfitting of data
[26]. For the output layer, sigmoid activation function was used
and the neural network was compiled using adam optimiser
to optimise binary cross entropy loss function and metrics for
accuracy.
The model was trained over 25 epochs and was able to
achieve 94.5% accuracy with 16.2% loss on the validation set.
The accuracy and loss on the train and test set is visualised in
“Fig. 6”.

V. R ESULTS AND D ISCUSSIONS

The accuracy achieved by the proposed model was found
to be 94.5% on validation set to classify pneumonia on the
basis of chest X-ray images. A comparative analysis was also
conducted amongst the proposed model, a naı̈ve CNN model
and CNN model based on VGG16 without GAN based syn-
thetic image generation. The comparison of these three models
can be seen in “Fig. 7”. The naı̈ve CNN model could only
Fig. 4. Training Loss of GAN and Comparison of Real and Fake Images. achieve accuracy of 79.7% while the model based on VGG16
without GAN based augmentation could only achieve accuracy

2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence 2021) 435

Authorized licensed use limited to: Frankfurt University of Applied Sciences. Downloaded on June 17,2022 at 15:28:37 UTC from IEEE Xplore. Restrictions apply.
 Fig. 6. Accuracy and Loss during CNN training.

of 91.3%. The proposed model shows significant improvement

in performance due to dataset oversampling. When synthetic
images were generated using GAN’s to populate minority
class samples, the model no longer remains biased towards
the dominant class and thereby the performance is improved.
All the hyperparameters were fine-tuned using grid search.

VI. C ONCLUSION AND F UTURE W ORK

In this study, a deep learning based computer-aided diagnos-

tics model was proposed for pneumonia detection using chest
X-ray images. The transfer learning approach was utilised
and VGG16 was used as the base model for classification.
To enhance the model performance, synthetic images were
augmented using DCGANs to balance the dataset. The model
Fig. 5. Architecture of VGG16 as described in [24] and proposed in [25].
achieved an accuracy of 94.5%.
For future work, it is planned to improve the model accuracy
and also to classify pneumonia as viral, bacterial or fungal.
It is also planned to take the patient’s medical history into
consideration to predict the probability of pneumonia.

436 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence 2021)

Authorized licensed use limited to: Frankfurt University of Applied Sciences. Downloaded on June 17,2022 at 15:28:37 UTC from IEEE Xplore. Restrictions apply.
 [14] M. Toğaçar, B. Ergen, Z. Cömert, and F. Özyurt, “A Deep Feature
Learning Model for Pneumonia Detection Applying a Combination of
mRMR Feature Selection and Machine Learning Models.” IRBM, vol.
41, no. 4, pp. 212-222, 2020, doi: 10.1016/j.irbm.2019.10.006.
[15] D. Varshni, K. Thakral, L. Agarwal, R. Nijhawan and A. Mittal,
”Pneumonia Detection Using CNN based Feature Extraction,” 2019
IEEE International Conference on Electrical, Computer and Communi-
cation Technologies (ICECCT), Coimbatore, India, 2019, pp. 1-7, doi:
10.1109/ICECCT.2019.8869364.
[16] G. Liang and L. Zheng, “A transfer learning method with deep
residual network for pediatric pneumonia diagnosis.” Computer Meth-
ods and Programs in Biomedicine, vol. 187, p. 104964, 2020, doi:
10.1016/j.cmpb.2019.06.023.
[17] S. V. Militante, N. V. Dionisio and B. G. Sibbaluca, ”Pneumonia
Detection through Adaptive Deep Learning Models of Convolutional
Neural Networks,” 2020 11th IEEE Control and System Graduate
Research Colloquium (ICSGRC), Shah Alam, Malaysia, 2020, pp. 88-
93, doi: 10.1109/ICSGRC49013.2020.9232613.
Fig. 7. Comparison of proposed model with naı̈ve model [18] D. Kermany, K. Zhang, M. Goldbaum, “Large Dataset of Labeled Opti-
cal Coherence Tomography (OCT) and Chest X-ray Images”, Mendeley
Data, V3, 2018, doi: 10.17632/rscbjbr9sj.3.
[19] M. Frid-Adar, I. Diamant, E. Klang, M. Amitai, J. Goldberger, and
R EFERENCES H. Greenspan, “GAN-based synthetic medical image augmentation for
increased CNN performance in liver lesion classification.” Neurocom-
puting, vol. 321, pp. 321-331, 2018, doi: 10.1016/j.neucom.2018.09.013.
[1] R. J. José and J. S. Brown, “Opportunistic bacterial, viral and fungal
[20] A. Radford, L. Metz and S. Chintala, “Unsupervised Representation
infections of the lung.” Medicine, vol. 44, no. 6, pp. 378-383, 2016, doi:
Learning with Deep Convolutional Generative Adversarial Networks”,
10.1016/j.mpmed.2016.03.015.
2015, arXiv:1511.06434.
[2] J. G. Bartlett and L. M. Mundy, “Community-Acquired Pneumonia.” [21] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, X.
New England Journal of Medicine, vol. 333, no. 24, pp. 1618-1624, Chen, “Improved Techniques for Training GANs,” In Advances in neural
1995, doi: 10.1056/nejm199512143332408. information processing systems, pp. 2234-2242, 2016.
[3] World Health Organization, “Standardization of interpretation of [22] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
chest radiographs for the diagnosis of pneumonia in children,” S. Ozair, A. Courville and Y. Bengio, “Generative Adversarial Nets,”
WHO/V&B/01.35, 2001. In Advances in neural information processing systems, pp. 2672-2680,
[4] K. Kallianos, “How far have we come? Artificial intelligence for chest 2014.
radiograph interpretation.” Clinical Radiology, vol. 74, no. 5, pp. 338- [23] C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmen-
345, 2019, doi: 10.1016/j.crad.2018.12.015. tation for Deep Learning.” Journal of Big Data, vol. 6, no. 1, 2019, doi:
[5] A. Hosny, C. Parmar, J. Quackenbush, L. H. Schwartz, and H. J. W. L. 10.1186/s40537-019-0197-0.
Aerts, “Artificial intelligence in radiology.” Nature Reviews Cancer, vol. [24] S. Tammina, “Transfer learning using VGG-16 with Deep Convolutional
18, no. 8, pp. 500-510, 2018, doi: 10.1038/s41568-018-0016-5. Neural Network for Classifying Images.” International Journal of Scien-
[6] P. Rajpurkar, J. Irvin, K. Zhu, B. Yang, H. Mehta, T. Duan, D. Ding, tific and Research Publications (IJSRP), vol. 9, no. 10, p. p9420, 2019,
A. Bagul, C. Langlotz, K. Shpanskaya, M.p. Lungren and A.y. Ng, doi: 10.29322/ijsrp.9.10.2019.p9420.
“Chexnet: Radiologist-level pneumonia detection on chest X-rays with [25] K. Simonyan, A. Zisserman, “Very deep convolutional networks for
deep learning,” 2017, arXiv:1711.05225. large-scale image recognition.”, 2014, arXiv:1409.1556.
[7] K. Weiss, T. M. Khoshgoftaar, and D. Wang, “A survey of transfer [26] K. Doshi, “Synthetic Image Augmentation for Improved Classification
learning.” Journal of Big Data, vol. 3, no. 1, 2016, doi: 10.1186/s40537- using Generative Adversarial Networks”, 2019, arXiv:1907.13576.
016-0043-6.
[8] M. F. Hashmi, S. Katiyar, A. G. Keskar, N. D. Bokde, and Z. W.
Geem, “Efficient Pneumonia Detection in Chest Xray Images Using
Deep Transfer Learning,” Diagnostics, vol. 10, no. 6, p. 417, Jun. 2020,
doi: 10.3390/diagnostics10060417.
[9] V. Chouhan, S. K. Singh, A. Khamparia, D. Gupta, P. Tiwari, C.
Moreira, R. Damaševičius, and V. H. C. de Albuquerque, “A Novel
Transfer Learning Based Approach for Pneumonia Detection in Chest
X-ray Images,” Applied Sciences, vol. 10, no. 2, p. 559, Jan. 2020, doi:
10.3390/app10020559.
[10] T. Rahman, M. E. H. Chowdhury, A. Khandakar, K. R. Islam, K. F.
Islam, Z. B. Mahbub, M. A. Kadir, and S. Kashem, “Transfer Learning
with Deep Convolutional Neural Network (CNN) for Pneumonia Detec-
tion Using Chest X-ray,” Applied Sciences, vol. 10, no. 9, p. 3233, May
2020, doi: 10.3390/app10093233.
[11] S. Rajaraman, S. Candemir, I. Kim, G. Thoma, and S. Antani, “Visual-
ization and Interpretation of Convolutional Neural Network Predictions
in Detecting Pneumonia in Pediatric Chest Radiographs,” Applied Sci-
ences, vol. 8, no. 10, p. 1715, Sep. 2018, doi: 10.3390/app8101715.
[12] J. R. Zech, M. A. Badgeley, M. Liu, A. B. Costa, J. J. Titano, and E.
K. Oermann, “Variable generalization performance of a deep learning
model to detect pneumonia in chest radiographs: A cross-sectional
study.” PLOS Medicine, vol. 15, no. 11, p. e1002683, 2018, doi:
10.1371/journal.pmed.1002683.
[13] A. K. Jaiswal, P. Tiwari, S. Kumar, D. Gupta, A. Khanna, and
J. J. Rodrigues, “Identifying pneumonia in chest X-rays: A deep
learning approach.” Measurement, vol. 145, pp. 511-518, 2019, doi:
10.1016/j.measurement.2019.05.076.

2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence 2021) 437

Authorized licensed use limited to: Frankfurt University of Applied Sciences. Downloaded on June 17,2022 at 15:28:37 UTC from IEEE Xplore. Restrictions apply.