Deep Learning To Detect Pancreatic Cystic Lesions On Abdominal Computed Tomography Scans: Development and Validation Study
Deep Learning To Detect Pancreatic Cystic Lesions On Abdominal Computed Tomography Scans: Development and Validation Study
Original Paper
Maria Montserrat Duh1*, MD; Neus Torra-Ferrer1*, MD; Meritxell Riera-Marín2, MSc; Dídac Cumelles2, BSc; Júlia
Rodríguez-Comas2, PhD; Javier García López2, PhD; Mª Teresa Fernández Planas1*, MD
1
Department of Radiology, Consorci Sanitari del Maresme (Hospital de Mataró), Mataró, Spain
2
Scientific and Technical Department, Sycai Technologies SL, Barcelona, Spain
*
these authors contributed equally
Corresponding Author:
Mª Teresa Fernández Planas, MD
Department of Radiology
Consorci Sanitari del Maresme (Hospital de Mataró)
Carretera de Cirera, 230
Mataró, 08304
Spain
Phone: 34 674152399 ext 754
Email: mfernandezpl@csdm.cat
Abstract
Background: Pancreatic cystic lesions (PCLs) are frequent and underreported incidental findings on computed tomography
(CT) scans and can evolve to pancreatic cancer—the most lethal cancer, with less than 5 months of life expectancy.
Objective: The aim of this study was to develop and validate an artificial deep neural network (attention gate U-Net, also named
“AGNet”) for automated detection of PCLs. This kind of technology can help radiologists to cope with an increasing demand of
cross-sectional imaging tests and increase the number of PCLs incidentally detected, thus increasing the early detection of
pancreatic cancer.
Methods: We adapted and evaluated an algorithm based on an attention gate U-Net architecture for automated detection of
PCL on CTs. A total of 335 abdominal CTs with PCLs and control cases were manually segmented in 3D by 2 radiologists with
over 10 years of experience in consensus with a board-certified radiologist specialized in abdominal radiology. This information
was used to train a neural network for segmentation followed by a postprocessing pipeline that filtered the results of the network
and applied some physical constraints, such as the expected position of the pancreas, to minimize the number of false positives.
Results: Of 335 studies included in this study, 297 had a PCL, including serous cystadenoma, intraductal pseudopapillary
mucinous neoplasia, mucinous cystic neoplasm, and pseudocysts . The Shannon Index of the chosen data set was 0.991 with an
evenness of 0.902. The mean sensitivity obtained in the detection of these lesions was 93.1% (SD 0.1%), and the specificity was
81.8% (SD 0.1%).
Conclusions: This study shows a good performance of an automated artificial deep neural network in the detection of PCL on
both noncontrast- and contrast-enhanced abdominal CT scans.
KEYWORDS
deep learning; pancreatic cystic lesion; neural networks; precursor lesions; pancreatic cancer; computed tomography; magnetic
resonance; cancer; radiologist; technology
This type of cancer can originate from precursor cystic lesions retrospective and anonymized. The study was approved by the
[4]. Pancreatic cystic lesions (PCL) are increasingly common hospital Institutional Ethical Review Board under code 90/20
incidental findings on abdominal imaging tests. Studies have as an observational retrospective single-center study, and the
shown that up to 70% of PCLs are diagnosed incidentally on requirement for informed consent was waived.
computed tomography (CT) scans due to unrelated symptoms,
making CT scans the first accessible source of information.
Study Population
These previously undetected cystic lesions are found on 3% of A total of 297 abdominal, thoracoabdominal, or pelvic CT scans
abdominal CT examinations [5,6] and 13%-21% of abdominal acquired at Hospital de Mataró between 2010 and 2021 and
magnetic resonance imaging studies [7,8]. However, autopsy diagnosed with a PCL as well as 38 CT scans as controls were
studies have evidenced a much higher prevalence, revealing selected for the study. All CT scan images were subjectively
that up to 50% of the older population may present at least one checked for quality and absence of relevant respiratory artifacts,
pancreatic cyst [6]. which could cause misdiagnosis in the abdominal region. The
exclusion criteria were underaged patients, artifacts or bad
PCLs have a wide diversity, and their differential diagnosis quality in the CT scan image, and patients having undergone
includes nonneoplastic cysts (pseudocysts) and neoplastic ones. surgery in the past to treat the PCL and having a prothesis in
Neoplastic lesions encompass benign lesions, such as serous the pancreas that affects the image. Importantly, patients
cystadenomas (SCA), to mucinous lesions, such as mucinous diagnosed with pancreatic adenocarcinoma or any kind of tumor
cystic neoplasms (MCN), and intraductal papillary mucinous in the pancreas were also excluded from the study.
neoplasm (IPMN), which may progress to PC. Therefore,
identifying precancerous mucin–producing cysts offers a unique Of note, a CT image is considered “bad quality” if there is
opportunity for early detection and prevention of PC. Once a movement or blurriness in it (mostly in the abdominal area,
PCL is found, patients are recommended to follow up a lifelong where the pancreas is located). Studies that included these types
surveillance program with imaging modalities (magnetic of images were excluded from the training and testing set
resonance imaging or CT) to identify early-stage cancer or because they would impact the learning process of the network
high-grade dysplasia [9,10]. Consequently, correct management or the testing in a negative way, which could then lead to false
of PCL may prevent progression to pancreatic cancer, while negatives or false positives.
reducing the need for lifelong screening and related costs. The final study population consisted of 136 patients: 73 male
In this complex scenario, automated detection of pancreatic (178 studies; mean age 67.75, SD 10.74 years) and 63 female
precursor lesions could increase the detection of this (157 studies; mean age 73.52, SD 10.67 years). A mean of 2
underreported entity and help with a proper surveillance of these (SD 1.4) CT studies and a median of 2.4 studies were available
patients. A limited number of publications regarding this topic per patient.
have been released in recent years, most of them in an
Patients’ Characteristics
experimental offline setting and applying different
methodologies [11]. Additionally, although existing methods From the whole cohort of 136 patients, 9 (6.5%) of them had a
of automated analysis have shown to be accurate for images of confirmed diagnosis through endoscopic ultrasound–guided
individual organs, they still struggle to deal with the variability fine needle aspiration or surgical resection of the lesion. In the
of structures, shape, and location of abdominal organs [12]. other 16 patients, no material or insufficient yield was extracted
Artificial intelligence (AI)–based algorithms have shown to evaluate the specimen. The rest of the patients were diagnosed
promising results in the detection of preneoplastic lesions in by a minimum of 2 experienced radiologists, taking into
the pancreas [13,14], but they are still far from implementation consideration the complete clinical record and the evolution of
in the clinical practice. the patient.
The aim of this study was to develop and test an artificial deep Patients with the following PCLs were included in the study:
neural network (AGNet) [15] for automated detection of PCLs. IPMN, MCN, SCA, and pseudocysts. A total of 14 (4.2%) of
This kind of technology can help radiologists to cope with an the lesions were not classified in the above classification due
increasing demand of cross-sectional imaging tests and increase to unspecified imaging characteristics and were categorized as
the number of PCLs incidentally detected, thus increasing the cyst (Table 1). The number of studies (CT scans) with PCLs
early detection of pancreatic cancer. distributed by age and sex is shown in Figure 1.
Data sets were further divided between the training set (a subset
Methods to train the model) and the testing set (a subset to test the trained
model). The final training data set comprised 93 patients,
Ethical Considerations representing a total of 241 CT scans, and the final testing data
Our research adhered to the ethical principles outlined in the set comprised 43 patients, representing a total of 94 CT scans.
1975 Declaration of Helsinki. The data used in this study were PCLs were distributed proportionally in both data sets.
Figure 1. Number of studies (CT scans) with pancreatic cystic lesion distributed by age and sex (x-axis).
supposed to be. The cropping was not too harsh to avoid the used for the following semantic segmentation study. The image
possibility of eliminating the pancreas from the CT image being analysis pipeline is depicted in Figure 2.
Figure 2. Diagram of the steps implemented in the pipeline. (A) Preprocessing. (B) Logits. (C) Postprocessing. (D) Output.
Figure 3. Illustration of the additive attention gate [15]. Reproduced from the cited source which is published under Creative Commons Attribution
4.0 International License [21].
Figure 4. Scheme of the deep neural network architecture [15]. Fl: Feature map in the layer l; H: height; W: width; D: dimension; Conv 3x3: convolution
operation with a 3x3 kernel; ReLu: rectified linear unit operation. Reproduced from the cited source which is published under Creative Commons
Attribution 4.0 International License [21].
PCLs in abdominal CT scans. The programming language used One of the main metrics used to evaluate the effectiveness of
was Python and the framework for the model development was this method was the sensitivity or true positive rate. This is
PyTorch. The sensitivity for all cases was 93.1% (SD 0.1%), something to highlight since it is better to have a false positive
and the specificity was 81.8% (SD 0.1%). than a false negative in this study due to the consequences of
obtaining each one: for a false positive, a review of the detection
Additionally, due to the small amount of some subtypes of
would be needed, but for a false negative, the consequences are
pancreatic cysts in the training database (Figure 6), we
much worse because a PCL can exist and not be detected. If we
considered it reasonable to divide the whole cohort of patients
compare the most dangerous group and the least dangerous
into 2 big groups: on the one hand, the most dangerous cyst
group, meaning the one that can easily evolve to pancreatic
types, bearing malignant potential (IPMN and MCN), and on
cancer versus the one that cannot evolve to pancreatic cancer
the other hand, the ones with malignant potential close to 0
as easily, it is a remarkable fact that the sensitivity is almost
(PCYST and SCA). If we consider this classification, the global
10% higher for the dangerous group, which makes the network
specificity and sensitivity for the detection of the most dangerous
even more efficient. Having a better true positive rate for the
group were 81.8% and 97.0%, respectively, while for the least
most dangerous group rather than for the least dangerous group
dangerous ones, they were 81.8% and 89.0%, respectively.
is a highlight of this study.
Figure 5. Illustration of the qualitative results obtained. Each pair of images belongs to a patient with a pancreatic cystic lesion. The left image of the
pair is the ground truth, while the right one is the outcome of this method. The pixels that belong to the pancreas are painted in green and the ones for
the pancreatic cystic lesion in red.
Figure 6. Example of the types of pancreatic cysts included in this research. (1) Serous cystadenoma, marked in yellow. (2) Mucinous cystic neoplasm,
marked in red. (3) Intraductal papillary mucinous neoplasm, marked in yellow. (4) Pseudocyst, marked in red). Pancreas is depicted in green.
Authors' Contributions
MMD, NTF, and MTFP (Hospital de Mataró, Consorci Sanitari del Maresme, Barcelona, Spain) were responsible for data
collection, anonymization, experiment design, and result validation. MRM, DC, JRC, and JGL performed the experiments. All
authors contributed to the writing, revision, and final approval of the manuscript.
Conflicts of Interest
JGL and JRC are founders of Sycai Technologies and declare significant ownership. MRM and DC are employed by Sycai
Technologies. The other authors report no conflicts of interest.
References
1. Ilic M, Ilic I. Epidemiology of pancreatic cancer. World J Gastroenterol 2016 Nov 28;22(44):9694-9705 [FREE Full text]
[doi: 10.3748/wjg.v22.i44.9694] [Medline: 27956793]
2. Branca Vergano L, Monesi M, Vicenti G, Bizzoca D, Solarino G, Moretti B. Posterior approaches in malleolar fracture:
when, why and how. J Biol Regul Homeost Agents 2020;34(3 Suppl. 2):89-95. [Medline: 32856446]
3. Mizrahi JD, Surana R, Valle JW, Shroff RT. Pancreatic cancer. Lancet 2020 Jun 27;395(10242):2008-2020. [doi:
10.1016/S0140-6736(20)30974-0] [Medline: 32593337]
4. Chernyak V, Flusberg M, Haramati LB, Rozenblit AM, Bellin E. Incidental pancreatic cystic lesions: is there a relationship
with the development of pancreatic adenocarcinoma and all-cause mortality? Radiology 2015 Jan;274(1):161-169 [FREE
Full text] [doi: 10.1148/radiol.14140796] [Medline: 25117591]
5. Laffan TA, Horton KM, Klein AP, Berlanstein B, Siegelman SS, Kawamoto S, et al. Prevalence of unsuspected pancreatic
cysts on MDCT. AJR Am J Roentgenol 2008 Sep;191(3):802-807 [FREE Full text] [doi: 10.2214/AJR.07.3340] [Medline:
18716113]
6. Ip IK, Mortele KJ, Prevedello LM, Khorasani R. Focal cystic pancreatic lesions: assessing variation in radiologists'
management recommendations. Radiology 2011 Apr;259(1):136-141. [doi: 10.1148/radiol.10100970] [Medline: 21292867]
7. Lee KS, Sekhar A, Rofsky NM, Pedrosa I. Prevalence of incidental pancreatic cysts in the adult population on MR imaging.
Am J Gastroenterol 2010 Sep;105(9):2079-2084. [doi: 10.1038/ajg.2010.122] [Medline: 20354507]
8. Zhang X, Mitchell DG, Dohke M, Holland GA, Parker L. Pancreatic cysts: depiction on single-shot fast spin-echo MR
images. Radiology 2002 May;223(2):547-553. [doi: 10.1148/radiol.2232010815] [Medline: 11997566]
9. Elta GH, Enestvedt BK, Sauer BG, Lennon AM. ACG clinical guideline: diagnosis and management of pancreatic cysts.
Am J Gastroenterol 2018 Apr;113(4):464-479. [doi: 10.1038/ajg.2018.14] [Medline: 29485131]
10. European Study Group on Cystic Tumours of the Pancreas. European evidence-based guidelines on pancreatic cystic
neoplasms. Gut 2018 May;67(5):789-804 [FREE Full text] [doi: 10.1136/gutjnl-2018-316027] [Medline: 29574408]
11. Springer S, Masica DL, Dal Molin M, Douville C, Thoburn CJ, Afsari B, et al. A multimodality test to guide the management
of patients with a pancreatic cyst. Sci Transl Med 2019 Jul 17;11(501):eaav4772 [FREE Full text] [doi:
10.1126/scitranslmed.aav4772] [Medline: 31316009]
12. Cai J, Lu L, Zhang Z, Xing F, Yang L, Yin Q. Pancreas segmentation in MRI using graph-based decision fusion on
convolutional neural networks. Med Image Comput Comput Assist Interv 2016:442-450. [doi:
10.1007/978-3-319-46723-8_51] [Medline: 28083570]
13. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 2018
Dec;18(8):500-510 [FREE Full text] [doi: 10.1038/s41568-018-0016-5] [Medline: 29777175]
14. Abel L, Wasserthal J, Weikert T, Sauter AW, Nesic I, Obradovic M, et al. Automated detection of pancreatic cystic lesions
on CT using deep learning. Diagnostics (Basel) 2021 May 19;11(5):901 [FREE Full text] [doi: 10.3390/diagnostics11050901]
[Medline: 34069328]
15. Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, et al. Attention U-Net: learning where to look for the
pancreas. ArXiv Preprint posted online April 11, 2018. [doi: 10.48550/arxiv.1804.03999]
16. Fedorov A, Beichel R, Kalpathy-Cramer J, Finet J, Fillion-Robin J, Pujol S, et al. 3D Slicer as an image computing platform
for the Quantitative Imaging Network. Magn Reson Imaging 2012 Nov;30(9):1323-1341 [FREE Full text] [doi:
10.1016/j.mri.2012.05.001] [Medline: 22770690]
17. Huo Y, Tang Y, Chen Y, Gao D, Han S, Bao S, et al. Stochastic tissue window normalization of deep learning on computed
tomography. J Med Imag 2019 Oct 1;6(04):1. [doi: 10.1117/1.jmi.6.4.044005]
18. Weng W, Zhu X. INet: convolutional networks for biomedical image segmentation. IEEE Access 2021;9:16591-16603.
[doi: 10.1109/access.2021.3053408]
19. Kingma D, Ba J. Adam: a method forstochastic optimization. ArXiv Preprint posted online Dec 22, 2014. [doi:
10.48550/arxiv.1412.6980]
20. Cornell RB, Nissley SM, Horwitz AF. Cholesterol availability modulates myoblast fusion. J Cell Biol 1980 Sep;86(3):820-824
[FREE Full text] [doi: 10.1083/jcb.86.3.820] [Medline: 7410480]
21. Attribution 4.0 International (CC BY 4.0). Creative Commons. URL: https://creativecommons.org/licenses/by/4.0/ [accessed
[2023-03-16]]
22. Seçkin AM, Berens P. Test-time Data Augmentation for Estimation of Heteroscedastic Aleatoric Uncertainty in Deep
Neural Networks. Test-time Data Augmentation for Estimation of Heteroscedastic Aleatoric Uncertainty in Deep Neural
Networks 2022 Jul 11:1-9.
23. Anta JA, Martínez-Ballestero I, Eiroa D, García J, Rodríguez-Comas J. Artificial intelligence for the detection of pancreatic
lesions. Int J Comput Assist Radiol Surg 2022 Oct 11;17(10):1855-1865. [doi: 10.1007/s11548-022-02706-z] [Medline:
35951286]
Abbreviations
AG: attention gate
AI: artificial intelligence
CT: computed tomography
IPMN: intraductal pseudopapillary mucinous neoplasia
HU: Hounsfield unit
MCN: mucinous cystic neoplasm
PCL: pancreatic cystic lesion
SCA: serous cystadenoma
TTA: test-time augmentation
Edited by K El Emam; submitted 01.07.22; peer-reviewed by F Maleki, W Klement; comments to author 17.08.22; revised version
received 02.09.22; accepted 11.11.22; published 17.03.23
Please cite as:
Duh MM, Torra-Ferrer N, Riera-Marín M, Cumelles D, Rodríguez-Comas J, García López J, Fernández Planas MT
Deep Learning to Detect Pancreatic Cystic Lesions on Abdominal Computed Tomography Scans: Development and Validation Study
JMIR AI 2023;2:e40702
URL: https://ai.jmir.org/2023/1/e40702
doi: 10.2196/40702
PMID:
©Maria Montserrat Duh, Neus Torra-Ferrer, Meritxell Riera-Marín, Dídac Cumelles, Júlia Rodríguez-Comas, Javier García
López, Mª Teresa Fernández Planas. Originally published in JMIR AI (https://ai.jmir.org), 17.03.2023. This is an open-access
article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/),
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR
AI, is properly cited. The complete bibliographic information, a link to the original publication on https://www.ai.jmir.org/, as
well as this copyright and license information must be included.