Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks

Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional... The 2019 novel coronavirus disease (COVID-19), with a starting point in China, has spread rapidly among people living in other countries and is approaching approximately 101,917,147 cases worldwide according to the statistics of World Health Organization. There are a limited number of COVID-19 test kits available in hospitals due to the increasing cases daily. Therefore, it is necessary to implement an automatic detection system as a quick alternative diagnosis option to prevent COVID-19 spreading among people. In this study, five pre-trained convolutional neural network-based models (ResNet50, ResNet101, ResNet152, InceptionV3 and Inception-ResNetV2) have been proposed for the detection of coronavirus pneu- monia-infected patient using chest X-ray radiographs. We have implemented three different binary classifications with four classes (COVID-19, normal (healthy), viral pneumonia and bacterial pneumonia) by using five-fold cross-validation. Considering the performance results obtained, it has been seen that the pre-trained ResNet50 model provides the highest classification performance (96.1% accuracy for Dataset-1, 99.5% accuracy for Dataset-2 and 99.7% accuracy for Dataset-3) among other four used models. Keywords Coronavirus · Bacterial pneumonia · Viral pneumonia · Chest X-ray radiographs · Convolutional neural network · Deep transfer learning 1 Introduction syndrome (MERS-CoV) and severe acute respiratory syn- drome (SARS-CoV). COVID-19 is a new species discovered The coronavirus disease (COVID-19) pandemic emerged in 2019 and has not been previously identified in humans in Wuhan, China, in December 2019 and became a serious [4]. COVID-19 causes lighter symptoms in about 99% of public health problem worldwide [1, 2]. Until now, no spe- cases, according to early data, while the rest is severe or cific drug or vaccine has been found against COVID-19 [2 ]. critical [5]. As of January 31, 2021, the total number of The virus that causes COVID-19 epidemic disease is called worldwide cases of coronavirus is 103,286,991 including severe acute respiratory syndrome coronavirus-2 (SARS- 2,232,776 deaths. Of these, the number of active patients is CoV-2) [3]. Coronaviruses (CoV) are a large family of 26,127,156 [6]. Nowadays the world is struggling with the viruses that cause diseases such as Middle East respiratory COVID-19 epidemic. Deaths from pneumonia developing due to the SARS-CoV-2 virus are increasing day by day. Chest radiography (X-ray) is one of the most important * Ceren Kaya methods used for the diagnosis of pneumonia worldwide [7]. [email protected]; [email protected] Chest X-ray is a fast, cheap [8] and common clinical method Ali Narin [9–11]. The chest X-ray gives the patient a lower radiation [email protected] dose compared to computed tomography (CT) and magnetic Ziynet Pamuk resonance imaging (MRI) [11]. However, making the correct [email protected] diagnosis from X-ray images requires expert knowledge and Department of Electrical and Electronics Engineering, experience [7]. It is much more difficult to diagnose using Zonguldak Bulent Ecevit University, Zonguldak 67100, a chest X-ray than other imaging modalities such as CT or Turkey MRI [8]. Department of Biomedical Engineering, Zonguldak Bulent Ecevit University, Zonguldak 67100, Turkey Vol.:(0123456789) 1 3 1208 Pattern Analysis and Applications (2021) 24:1207–1220 By looking at the chest X-ray, COVID-19 can only be 3.3, respectively. Performance metrics are given in detail diagnosed by specialist physicians. The number of special- in Sect. 3.4. Obtained experimental results from proposed ists who can make this diagnosis is less than the number of models and discussion are presented in Sects. 4 and 5, normal doctors. Even in normal times, the number of doc- respectively. Finally, in Sect. 6, the conclusion and future tors per person is insufficient in countries around the world. works are summarized. According to data from 2017, Greece ranks first with 607 doctors per 100,000 people. In other countries, this number is much lower [12].2 Related works In case of disasters such as COVID-19 pandemic, demanding health services at the same time, collapse of the Studies diagnosed with COVID-19 using chest X-rays have health system is inevitable due to the insufficient number of binary or multiple classifications. Some studies use raw data, hospital beds and health personnel. Also, COVID-19 is a while others have feature extraction process. The number of highly contagious disease, and doctors, nurses and caregiv- data used in studies also varies. Among the studies, the most ers are most at risk. Early diagnosis of pneumonia has a preferred method is convolutional neural network (CNN). vital importance both in terms of slowing the speed of the Apostolopoulos and Bessiana used a common pneumonia, spread of the epidemic by quarantining the patient and in the COVID-19-induced pneumonia, and an evolutionary neural recovery process of the patient. network for healthy differentiation on automatic detection of Doctors can diagnose pneumonia from the chest X-ray COVID-19. In particular, the procedure called transfer learn- more quickly and accurately thanks to computer-aided diag- ing has been adopted. With transfer learning, the detection nosis (CAD) [8]. Use of artificial intelligence methods is of various abnormalities in small medical image datasets increasing due to its ability to cope with enormous datasets is an achievable goal, often with remarkable results [15]. exceeding human potential in the field of medical services Based on chest X-ray images, Zhang et al. aimed to develop [13]. Integrating CAD methods into radiologist diagnos- a deep learning-based model that can detect COVID-19 with tic systems greatly reduces the workload of doctors and high sensitivity, providing fast and reliable scanning [16]. increases reliability and quantitative analysis [11]. CAD Singh et al. classified the chest computed tomography (CT) systems based on deep learning and medical imaging are images from infected people with and without COVID-19 becoming more and more research fields [13, 14]. using multi-objective differential evolution (MODE)-based In this study, we have proposed an automatic CAD CNN [17]. Jaiswal et al. proposed DenseNet201-based deep prediction of COVID-19 using a deep convolutional neu- transfer learning model on chest CT images to classify the ral network-based pre-trained transfer models and chest patients as COVID-19 infected or not [14]. In the study of X-ray images. For this purpose, we have used ResNet50, Chen et  al, they proposed Residual Attention U-Net for ResNet101, ResNet152, InceptionV3 and Inception- automated multi-class segmentation technique to prepare ResNetV2 pre-trained models to obtain higher prediction the ground for the quantitative diagnosis of lung infection accuracies for three different binary datasets including X-ray on COVID-19-related pneumonia using CT images [18]. images of normal (healthy), COVID-19, bacterial and viral Adhikari’s study suggested a network called “Auto Diag- pneumonia patients. nostic Medical Analysis” trying to find infectious areas to The novelty and originality of proposed study are summa- help the doctor better identify the diseased part, if any. Both rized as follows: (1) The proposed models have end-to-end X-ray and CT images were used in the study. It has been rec- structure without manual feature extraction, selection and ommended DenseNet network to remove and mark infected classification. (2) The performances of the COVID-19 data areas of the lung [19]. In the study by Alqudah et al., two across normal, viral pneumonia and bacterial pneumonia different methods were used to diagnose COVID-19 using classes were significantly higher. (3) It has been studied with chest X-ray images. The first one used AOCTNet, MobileNet more data than many studies in the literature. (4) It has been and ShuffleNet CNNs. Secondly, the features of their images studied and compared with 5 different CNN models. (5) A have been removed and they have been classified using soft- high-accuracy decision support system has been proposed max classifier, K nearest neighbor (kNN), support vector to radiologists for the automatic diagnosis and detection of machine (SVM) and random forest (RF) algorithms [20]. patients with suspected COVID-19 and follow-up. Khan et al. classified the chest X-ray images from normal, The flow of the manuscript is organized as follows: The bacterial and viral pneumonia cases using the Xception work done in the field of deep learning techniques on chest architecture to detect COVID-19 infection [21]. Ghoshal and X-ray and CT images for COVID-19 disease is presented Tucker used the dropweights-based Bayesian CNN model in Sect. 2. Dataset is expressed in detail in Sect. 3.1. Deep using chest X-ray images for the diagnosis of COVID-19 transfer learning architecture, pre-trained models and experi- [22]. Hemdan et  al. used VGG19 and DenseNet models mental setup parameters are described in Sects. 3.2 and to diagnose COVID-19 from X-ray images [23]. Ucar and 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1209 Korkmaz worked on X-ray images for COVID-19 diagnosis X-ray images. Distribution of images per class in created and supported the SqueezeNet model with Bayesian opti- datasets is given in Table 1. mization [24]. In the study conducted by Apostopolus et al., The data augmentation method was used with scaling they performed automatic detection from X-ray images using factor = 1./255, shear range = 0.1, zoom range = 0.1 and CNNs with transfer learning [25]. Sahinbas and Catak used horizontal flipping enabled in training dataset. All images X-ray images for the diagnosis of COVID-19 and worked on were resized to 224 × 224 pixel size in the datasets. In Fig. 1, VGG16, VGG19, ResNet, DenseNet and InceptionV3 mod- representative chest X-ray images of normal (healthy), els [26]. Medhi et al. used X-ray images as feature extrac- COVID-19, bacterial and viral pneumonia patients are given, tion and segmentation in their study, and then, COVID-19 respectively. was positively and normally classified using CNN [27]. Barstugan et al. classified X-ray images for the diagnosis of 3.2 Architecture of deep transfer learning COVID-19 using five different feature extraction methods that are Grey-Level Cooccurrence Matrix (GLCM), Local Deep learning is a sub-branch of the machine learning field, Directional Patterns (LDP), Grey-Level Run Length Matrix inspired by the structure of the brain. Deep learning tech- (GLRLM), Grey-Level Size Zone Matrix (GLSZM) and niques used in recent years continue to show an impressive Discrete Wavelet Transform (DWT). The obtained features performance in the field of medical image processing, as in were classified by SVM. During the classification process, many fields. By applying deep learning techniques to medi- two-fold, five-fold and ten-fold cross-validation methods cal data, it is tried to draw meaningful results from medical were used [28]. Punn and Agarwal worked on X-ray images data. and used ResNet, InceptionV3, Inception-ResNet models to Deep learning models have been used successfully in diagnose COVID-19 [29]. Afshar et al. developed deep neu- many areas such as classification, segmentation and lesion ral network (DNN)-based diagnostic solutions and offered an detection of medical data. Analysis of image and signal data alternative modeling framework based on Capsule Networks was obtained with medical imaging techniques such as mag- that can process on small datasets [30]. netic resonance imaging (MRI), computed tomography (CT) In our previous study in March 2020, we used ResNet50, and X-ray with the help of deep learning models. As a result InceptionV3 and Inception-ResNetV2 models for the diag- of these analyzes, detection and diagnosis of diseases such as nosis of COVID-19 using chest X-ray images. However, diabetes mellitus, brain tumor, skin cancer and breast cancer since there were not enough data on COVID-19, we were are provided in studies with convenience [35–41]. only able to train through 50 normal and 50 COVID-19 posi- A convolutional neural network (CNN) is a class of tive cases [31]. Therefore, to overcome the issues associated deep neural networks used in image recognition problems with our previous study [31], proposed study was recon- [42]. Coming to how CNN works, the images given as ducted by increasing the number of data and deep transfer input must be recognized by computers and converted into learning models to classify COVID-19-infected patients. a format that can be processed. For this reason, images are first converted to matrix format. The system determines which image belongs to which label based on the differ - 3 Materials and methods ences in images and therefore in matrices. It learns the effects of these differences on the label during the training 3.1 Dataset phase and then makes predictions for new images using them. CNN consists of three different layers that are a con- In this study, chest X-ray images of 341 COVID-19 patients volutional layer, pooling layer and fully connected layer to have been obtained from the open source GitHub repository perform these operations effectively. The feature extraction shared by Dr. Joseph Cohen et al. [32]. This repository is process takes place in both convolutional and pooling lay- consisting chest X-ray/computed tomography (CT) images ers. On the other hand, the classification process occurs in of mainly patients with acute respiratory distress syndrome (ARDS), COVID-19, Middle East respiratory syndrome (MERS), pneumonia, severe acute respiratory syndrome Table 1 Number of images per class for each dataset (SARS). 2800 normal (healthy) chest X-ray images were Datasets/classes Bacterial COVID-19 Normal Viral pneumonia selected from “ChestX-ray8” database [33]. In addition, pneumo- 2772 bacterial and 1493 viral pneumonia chest X-ray images nia were used from Kaggle repository called “Chest X-Ray Dataset-1 – 341 2800 – Images (Pneumonia)” [34]. Dataset-2 – 341 – 1493 Our experiments have been based on three binary cre- Dataset-3 2772 341 – – ated datasets (Dataset-1, Dataset-2 and Dataset-3) with chest 1 3 1210 Pattern Analysis and Applications (2021) 24:1207–1220 Fig. 1 Representative chest X-ray images of normal (healthy) (first row), COVID-19 (second row), bacterial (third row) and viral pneumonia (fourth row) patients fully connected layer. These layers are examined sequen- l l−1 l−1 l tially in the following. x = f w ∗ y + b (1) j j a j a=1 l l−1 where x is the jth feature map in layer l, w indicates jth j j 3.2.1 Convolutional layer l−1 kernels in layer l − 1 , y represents the ath feature map in layer l − 1 , b indicates the bias of the jth feature map in layer Convolutional layer is the base layer of CNN. It is respon- l, N is number of total features in layer l − 1 , and (∗) repre- sible for determining the features of the pattern. In this sents vector convolution process. layer, the input image is passed through a filter. The values resulting from filtering consist of the feature map. This layer applies some kernels that slide through the pattern 3.2.2 Pooling layer to extract low- and high-level features in the pattern [43]. The kernel is a 3 × 3- or 5 × 5-shaped matrix to be trans- The second layer after the convolutional layer is the pooling formed with the input pattern matrix. Stride parameter is layer. Pooling layer is usually applied to the created fea- the number of steps tuned for shifting over input matrix. ture maps for reducing the number of feature maps and net- The output of convolutional layer can be given as: work parameters by applying corresponding mathematical 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1211 computation. In this study, we used max-pooling and global weights of models trained on different datasets are trans- average pooling. The max-pooling process selects only the ferred to the new model [45, 46]. Apart from the transferred maximum value by using the matrix size specified in each parts, the learning process is also carried out through the feature map, resulting in reduced output neurons. There is newly added layers. It is stated that it is particularly success- also a global average pooling layer that is only used before ful even in few datasets [47]. In addition, this method used the fully connected layer, reducing data to a single dimen- allows to obtain results faster with lower calculation cost. sion. It is connected to the fully connected layer after global In the analysis of medical data, one of the biggest dif- average pooling layer. The other intermediate layer used is ficulties faced by researchers is the limited number of avail- the dropout layer. The main purpose of this layer is to pre- able datasets. Deep learning models often need a lot of vent network overfitting and divergence [44]. data. Labeling this data by experts is both costly and time- consuming. The biggest advantage of using transfer learn- 3.2.3 Fully connected layer ing method is that it allows the training of data with fewer datasets and requires less calculation costs. With the transfer Fully connected layer is the last and most important layer of learning method, which is widely used in the field of deep CNN. This layer functions like a multilayer perceptron. Rec- learning, the information gained by the pre-trained model tified linear unit (ReLU) activation function is commonly on a large dataset is transferred to the model to be trained. used on fully connected layer, while softmax activation func- In this study, we built deep CNN-based ResNet50, tion is used to predict output images in the last layer of fully ResNet101, ResNet152, InceptionV3 and Inception- connected layer. Mathematical computation of these two ResNetV2 models for the classification of COVID-19 activation functions is as follows: chest X-ray images to three different binary classes (Binary Class-1 = COVID-19 and normal (healthy), Binary Class-2 0, if x < 0 = COVID-19 and viral pneumonia and Binary Class-3 ReLU(x)= (2) x, if x ≥ 0 = COVID-19 and bacterial pneumonia). In addition, we applied transfer learning technique that was realized by e using ImageNet data to overcome the insufficient data and Softmax(x )= x (3) training time. The schematic representation of conven- y=1 tional CNN including pre-trained ResNet50, ResNet101, where x and m represent input data and the number of ResNet152, InceptionV3 and Inception ResNetV2 models classes, respectively. Neurons in a fully connected layer have for the prediction of normal (healthy), COVID-19, bacterial full connections to all activation functions in previous layer. and viral pneumonia patients is depicted in Fig. 2. It is also available publicly for open access at https:// git hub. com/ dr cer 3.2.4 Pre‑trained models enkaya/ COVID- 19- Detec tionV2. Training convolutional neural network (CNN) models with • ResNet50 millions of parameters from scratch is not only very time- Residual neural network (ResNet) model is an consuming, but also requires equipment with high per- improved version of CNN. ResNet adds shortcuts formance. To overcome these problems, parameters and between layers to solve a problem. Thanks to this, it Fig. 2 Schematic representation of pre-trained models for the prediction of normal (healthy), COVID-19, bacterial and viral pneumonia patients 1 3 1212 Pattern Analysis and Applications (2021) 24:1207–1220 prevents the distortion that occurs as the network gets ResNet101, ResNet152, InceptionV3 and Inception- deeper and more complex. In addition, bottleneck blocks ResNetV2) were pre-trained with random initialization are used to make training faster in the ResNet model [48]. weights by optimizing the cross-entropy function with adap- ResNet50 is a 50-layer network trained on the ImageNet tive moment estimation (ADAM) optimizer ( 1 = 0.9 and dataset. ImageNet is an image database with more than 2 = 0.999 ). The batch size, learning rate and number of 14 million images belonging to more than 20,000 catego- epochs were experimentally set to 3, 1e 5 and 30, respec- ries created for image recognition competitions [49]. tively, for all experiments. All datasets used were randomly InceptionV3 split into two independent datasets with 80% and 20% InceptionV3 is a kind of convolutional neural network for training and testing, respectively. As cross-validation model. It consists of numerous convolution and maxi- method, k-fold was chosen and results were obtained accord- mum pooling steps. In the last stage, it contains a fully ing to 5 different k values (k = 1–5) as shown in Fig. 3. connected neural network [50]. As with the ResNet50 model, the network is trained with ImageNet dataset. 3.4 Performance metrics Inception-ResNetV2 The model consists of a deep convolutional network Five criteria were used for the performances of deep transfer using the Inception-ResNetV2 architecture that was learning models. These are: trained on the ImageNet-2012 dataset. The input to the model is a 299 × 299 image, and the output is a list of TP + TN Accuracy = (4) estimated class probabilities [51]. TP + TN + FP + FN ResNet101 and ResNet152 ResNet101 and ResNet152 consist of 101 and 152 lay- TP Recall = (5) ers, respectively, due to stacked ResNet building blocks. TP + FN You can load a pre-trained version of the network trained on more than a million images from the ImageNet data- TN Specificity = (6) base [49]. As a result, the network has learned rich fea- TN + FP ture representations for a wide range of images. The net- work has an image input size of 224 × 224. TP Precision = (7) TP + FP 3.3 Experimental setup 2 ∗ Precision * Recall F1-score = Python programming language was used to train the pro- (8) Precision + Recall posed deep transfer learning models. All experiments were performed on Google Colaboratory (Colab) Linux server TP, FP, TN and FN given in Eqs. (4)–(8) represent the num- with the Ubuntu 16.04 operating system using the online ber of true positive, false positive, true negative and false cloud service with Central Processing Unit (CPU), Tesla negative, respectively. For Dataset-1, given a test dataset K80 Graphics Processing Unit (GPU) or Tensor Process- and model, TP is the proportion of positive (COVID-19) that ing Unit (TPU) hardware for free. CNN models (ResNet50, is correctly labeled as COVID-19 by the model; FP is the Fig. 3 Visual display of testing and training datasets for five- fold cross-validation 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1213 proportion of negative (normal) that is mislabeled as positive 0.4 InceptionV3 (COVID-19); TN is the proportion of negative (normal) that ResNet50 ResNet101 is correctly labeled as normal; and FN is the proportion of ResNet152 positive (COVID-19) that is mislabeled as negative (normal) 0.3 InceptionResNetV2 0.04 by the model. 0.02 0.2 4 Experimental results 20 25 30 In this paper, we performed 3 different binary classifications 0.1 with 4 different classes (COVID-19, normal, viral pneumo- nia and bacterial pneumonia). Five-fold cross-validation method has been used in order to get a robust result in this study performed with 5 different pre-trained models that 51015202530 are InceptionV3, ResNet50, ResNet101, ResNet152 and Epoch Inception-ResNetV2. While 80% of the data is reserved for training, the remaining 20% is reserved for testing. All this Fig. 5 Binary Class-1: comparison of training loss values of 5 differ - ent models for fold-4 process continued until each 20% part was tested. Consider- ing the training times of all models, for Dataset-1 (Binary Class-1), the total training times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained given in Figs. 4 and 5. It is clear that the performance of the ResNet50 model is better than the other models. It can be models were 16027 s, 14638 s, 17841 s, 18802 s and 23078 s, respectively. For Dataset-2 (Binary Class-2), the total said that the ResNet50 model reaches lower values among the loss values of other models. Detection performance training times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained models on test data is shown in Fig.  6. While a lot of oscillation is observed in some models, some models are more sta- were 12241 s, 9948 s, 13089 s, 14923 s and 19336 s, respec- tively. Finally, for Dataset-3 (Binary Class-3), the total train- ble. The ResNet50 model appears to have less oscillation after the 15th epoch. Comprehensive performance values ing times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained models were 15801 s, for each fold value of each model are given in Table 2. As seen from Table 2, the detection of the ResNet50 model in 14386 s, 17658 s, 18581 s and 22865 s, respectively. Firstly, the accuracy and loss values in the training the COVID-19 class is significantly higher than the other models. ResNet50 and ResNet101 have the highest overall process obtained for the models applied to Dataset-1 that includes Binary Class-1 (COVID-19/normal classes) are 0.9 0.98 1 0.8 0.96 0.995 0.7 0.94 0.99 20 22 24 26 28 30 0.6 InceptionV3 0.92 ResNet50 InceptionV3 ResNet50 ResNet101 0.9 0.5 ResNet101 ResNet152 ResNet152 InceptionResNetV2 InceptionResNetV2 0.88 0.4 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 4 Binary Class-1: comparison of training accuracy of 5 different Fig. 6 Binary Class-1: comparison of testing accuracy of 5 different models for fold-4 models for fold-4 1 3 Accuracy values Accuracy values Loss 1214 Pattern Analysis and Applications (2021) 24:1207–1220 Table 2 All performances of 5 Models/fold Confusion matrix and performance results (%) different models on each fold for COVID-19/normal binary TP TN FP FN ACC REC SPE PRE F1 classification InceptionV3 Fold-1 60 519 41 8 92.2 88.2 92.7 59.4 71.0 Fold-2 60 547 13 8 96.7 88.2 97.7 82.2 85.1 Fold-3 65 560 0 3 99.5 95.6 100 100 97.7 Fold-4 57 524 36 11 92.5 83.8 93.6 61.3 70.8 Fold-5 67 538 22 2 96.2 97.1 96.1 75.3 84.8 Total/average 309 2688 112 32 95.4 90.6 96.0 73.4 81.1 ResNet50 Fold-1 65 511 49 3 91.7 95.6 91.3 57.0 71.4 Fold-2 59 545 15 9 96.2 86.8 97.3 79.7 83.1 Fold-3 62 556 4 6 98.4 91.2 99.3 93.9 92.5 Fold-4 59 543 17 9 95.9 86.8 97.0 77.6 81.9 Fold-5 68 549 11 1 98.1 98.6 98.0 86.1 91.9 Total/average 313 2704 96 28 96.1 91.8 96.6 76.5 83.5 ResNet101 Fold-1 50 543 17 18 94.4 73.5 97.0 74.6 74.1 Fold-2 49 559 1 19 96.8 72.1 99.8 98.0 83.1 Fold-3 68 541 19 0 97.0 100 96.6 78.2 87.7 Fold-4 33 554 6 35 93.5 48.5 98.9 84.6 61.7 Fold-5 67 553 7 2 98.6 97.1 98.8 90.5 93.7 Total/average 267 2750 50 74 96.1 78.3 98.2 84.2 81.2 ResNet152 Fold-1 15 547 13 53 89.5 22.1 97.7 53.6 31.3 Fold-2 55 556 4 13 97.3 80.9 99.3 93.2 86.6 Fold-3 55 555 5 13 97.1 80.9 99.1 91.7 85.9 Fold-4 34 539 21 34 91.2 50.0 96.3 61.8 55.3 Fold-5 64 528 32 5 94.1 92.8 94.3 66.7 77.6 Total/average 223 2725 75 118 93.9 65.4 97.3 74.8 69.8 Inception-ResNetV2 Fold-1 59 538 22 9 95.1 86.8 96.1 72.8 79.2 Fold-2 59 523 37 9 92.7 86.8 93.4 61.5 72.0 Fold-3 38 553 7 10 97.2 79.2 98.8 84.4 81.7 Fold-4 48 516 44 20 89.8 70.6 92.1 52.2 60.0 Fold-5 64 542 18 5 96.3 92.8 96.8 78.0 84.8 Total/average 268 2672 128 53 94.2 83.5 95.4 67.7 74.8 TP, true positive; TN, true negative; FP, false positive (FP); FN, false negative; ACC, accuracy; REC, recall; SPE, specificity; PRE, precision (PRE); and F1, F1-score performance with 96.1%. It is obvious that the excess of the first 3 epochs of the ResNet50 model. Detailed perfor - normal data results in higher performance in all models. mances of the models are given in Table 3. It is clearly seen Secondly, when the results obtained for the data in Binary that quite high values are reached for each fold value. While Class-2 (COVID-19/viral pneumonia classes) are evaluated, 99.4% was reached in the detection of COVID-19, it is seen the training performances of the models given in Figs. 7 and that 99.5% was reached in the detection of viral pneumonia. 8 are quite high. It can be said that the accuracy values and In the last study, the detection success of Binary Class-3 loss values of the ResNet50 and ResNet101 models perform (COVID-19/bacterial pneumonia classes) was investigated. better than the other models. Performance values obtained The performances of 5 different models on both training and through test data are shown in Fig.  9. Here, the models’ test data are given in Figs. 10, 11 and 12. As in other studies, results on the test data are generally more stable. There is no it is clearly seen that the ResNet50 model exhibits higher oscillation except when there is excessive oscillation only in training performance. InceptionV3 model is seen to exhibit 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1215 0.9 0.8 0.95 0.995 0.7 0.9 0.6 0.99 20 22 24 26 28 30 InceptionV3 InceptionV3 0.5 ResNet50 ResNet50 0.85 ResNet101 ResNet101 0.4 ResNet152 ResNet152 InceptionResNetV2 InceptionResNetV2 0.3 0.8 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 9 Binary Class-2: comparison of testing accuracy of 5 different Fig. 7 Binary Class-2: comparison of training accuracy of 5 different models for fold-4 models for fold-4 in the literature. In binary classification, it is common to 0.4 InceptionV3 distinguish COVID-19 positive from COVID-19 negative. In ResNet50 addition, it is very important to distinguish viral and bacte- ResNet101 ResNet152 rial pneumonia patients, which are other types of diseases 0.3 InceptionResNetV2 0.04 affecting the lung, from COVID-19 positive patients. There are a limited number of studies in the literature that work with multiple classes. Narayan Das et al. conducted stud- 0.2 0.02 ies for 3 different classes (COVID-19 positive, pneumonia and other infection). The researchers used 70% of the data for the training, the remaining 10% for validation and 20% 0.1 20 25 30 for the test. As a result, they obtained 97.40% accuracy over test data with extreme version of Inception (Xception) CNN model [9]. Singh et al. proposed a two-class study 51015202530 using limited data. They reported their performances by Epoch dividing the dataset at different training and testing rates. They achieved the highest accuracy of 94.65 ∓ 2.1 at 70% training–30% testing rates. In their study, they set the CNN Fig. 8 Binary Class-2: comparison of training loss values of 5 differ - ent models for fold-4 hyper-parameters using multi-objective adaptive differential evolution (MADE) [52]. Afshar et al. conducted their studies using a method called COVID-CAPS with multi-class (nor- increasing performance toward the end of the epoch number. mal, bacterial pneumonia, non-COVID viral pneumonia and COVID-19) studies. They achieved 95.7% accuracy with the When the detailed results given in Table 4 are evaluated, it can be said that the InceptionV3 model has a performance approach without pre-training and 98.3% accuracy with pre- trained COVID-CAPS. However, although their sensitivity of 100% in the detection of COVID-19, while the overall performance is also said to be the ResNet50 model which values are not as high as general accuracy, they detected the sensitivity without using pre-training and with using pre- has a high success. trained COVID-CAPS as 90% and 80%, respectively [30]. Ucar and Korkmaz carried out multi-class (normal, 5 Discussion pneumonia and COVID-19 cases) work with deep Bayes- SqueezeNet. They obtained the average accuracy value of The use of artificial intelligence-based systems is very com - 98.26%. They worked with 76 COVID-19 data [24]. Sahi- nbas and Catak worked with 5 different pre-trained models mon in detecting those caught in the COVID-19 epidemic. As given in Table 5, there are many studies on this subject (VGG16, VGG19, ResNet, DenseNet and InceptionV3). 1 3 Accuracy values Loss Accuracy values 1216 Pattern Analysis and Applications (2021) 24:1207–1220 Table 3 All performances of 5 Models/fold Confusion matrix and Performance results (%) different models on each fold for COVID-19/viral pneumonia TP TN FP FN ACC REC SPE PRE F1 binary classification InceptionV3 Fold-1 68 292 6 0 98.4 100 98.0 91.9 95.8 Fold-2 68 292 6 0 98.4 100 98.0 91.9 95.8 Fold-3 67 295 4 1 98.6 98.5 98.7 94.4 96.4 Fold-4 68 290 9 0 97.5 100 97.0 88.3 93.8 Fold-5 69 299 0 0 100 100 100 100 100 Total/average 340 1468 25 1 98.6 99.7 98.3 93.2 96.3 ResNet50 Fold-1 68 297 1 0 99.7 100 99.7 98.6 99.3 Fold-2 68 293 5 0 98.6 100 98.3 93.2 96.5 Fold-3 68 298 1 0 99.7 100 99.7 98.6 99.3 Fold-4 66 299 0 2 99.5 97.1 100 100 98.5 Fold-5 69 299 0 0 100 100 100 100 100 Total/average 339 1486 7 2 99.5 99.4 99.5 98.0 98.7 ResNet101 Fold-1 61 294 4 7 97.0 89.7 98.7 93.8 91.7 Fold-2 62 293 5 6 97.0 91.2 98.3 92.5 91.9 Fold-3 68 295 4 0 98.9 100 98.7 94.4 97.1 Fold-4 65 298 1 3 98.9 95.6 99.7 98.5 97.0 Fold-5 45 299 0 24 93.5 65.2 100 100 78.9 Total/average 301 1479 14 40 97.1 88.3 99.1 95.6 91.8 ResNet152 Fold-1 63 291 7 5 96.7 92.6 97.7 90.0 91.3 Fold-2 67 293 5 1 98.4 98.5 98.3 93.1 95.7 Fold-3 66 298 1 2 99.2 97.1 99.7 98.5 97.8 Fold-4 56 299 0 13 96.5 81.2 100 100 89.6 Fold-5 58 298 1 10 97.0 85.3 99.7 98.3 91.3 Total/average 310 1479 14 31 97.5 90.9 99.1 95.7 93.2 Inception-ResNetV2 Fold-1 68 283 15 0 95.9 100 95.0 81.9 90.1 Fold-2 68 267 31 0 91.5 100 89.6 68.7 81.4 Fold-3 68 278 21 0 94.3 100 93.0 76.4 86.6 Fold-4 41 296 3 27 91.8 60.3 99.0 93.2 73.2 Fold-5 69 293 6 0 98.4 100 98.0 92.0 95.8 Total/average 314 1417 76 27 94.4 92.1 94.9 80.5 85.9 They achieved 80% accuracy with VGG16 as their binary COVID-19 data. They found the detection accuracy classifier performances. They worked with 70 COVID of 95.18% with the confidence-aware anomaly detec- positives and 70 COVID negatives in total [26]. Khan et al. tion (CAAD) model [16]. Apostopolus et  al. obtained worked with normal, pneumonia-bacterial, pneumonia- an accuracy of 93.48% using a total of 224 COVID-19 viral and COVID-19 chest X-ray images. As a result, they data with the VGG-19 CNN model for their 3 classes achieved 89.6% overall performance with the model they (COVID-19–bacterial–normal) study [25]. Narin et  al. named CoroNet. They used 290 COVID-19 data. They used 50 COVID-19/50 normal data in their study, where worked with more COVID-19 data than many studies [21]. they achieved 98% accuracy with ResNet50 [31]. In many Medhi et al. achieved 93% overall performance value in their studies in the literature, researchers have studied a limited study using deep CNN. They worked with 150 pieces of number of COVID-19 data. In this study, the differentia- COVID-19 data [27]. tion performance of 341 COVID-19 data from each other In another study, Zhang and his colleagues performed was investigated with 3 different studies. In the study, 5 binary and multi-class classifications containing 106 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1217 0.98 0.9 0.96 0.998 0.8 0.94 0.996 20 22 24 26 28 30 InceptionV3 0.92 InceptionV3 ResNet50 0.7 ResNet50 ResNet101 ResNet101 ResNet152 0.9 ResNet152 InceptionResNetV2 InceptionResNetV2 0.6 0.88 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 12 Binary Class-3: comparison of testing accuracy of 5 different Fig. 10 Binary Class-3: comparison of training accuracy of 5 differ - models for fold-4 ent models for fold-4 increase in the work density of radiologists. In these manual 0.4 InceptionV3 diagnoses and determinations, the expert’s tiredness may ResNet50 increase the error rate. It is clear that decision support sys- ResNet101 tems will be needed in order to eliminate this problem. Thus, 0.3 ResNet152 a more effective diagnosis can be made. The most impor - InceptionResNetV2 0.03 tant issue that restricts this study is to work with limited data. Increasing the data, testing it with the data in many 0.2 0.02 different centers will enable the creation of more stable sys- 0.01 tems. In future studies, the features will be extracted using image processing methods on X-ray and CT images. From 0.1 20 25 30 these extracted features, the features that provide the best separation between classes will be determined and perfor- mance values will be measured with different classification 51015202530 algorithms. In addition, the results will be compared with Epoch deep learning models. Apart from this, the results of the study will be tested with data from many different centers. In a future study, studies will be conducted to determine the Fig. 11 Binary Class-3: comparison of training loss values of 5 dif- ferent models for fold-4 demographic characteristics of patients and COVID-19 pos- sibilities with artificial intelligence-based systems. different CNN models were compared. The most important points in the study can be expressed as follows: ∙6 Conclusion There are no manual feature extraction, feature selection and classification in this method. It was realized end to end ∙ Early prediction of COVID-19 patients is vital to prevent directly with raw data. The performances of the COVID-19 data across normal, viral pneumonia and bacterial pneumo- the spread of the disease to other people. In this study, we ∙ proposed a deep transfer learning-based approach using nia classes were significantly higher. It has been studied with more data than many studies in the literature. ∙ It has chest X-ray images obtained from normal, COVID-19, ∙ bacterial and viral pneumonia patients to predict COVID- been studied and compared with 5 different CNN models. A high-accuracy decision support system has been proposed 19 patients automatically. Performance results show that to radiologists for the automatic diagnosis and detection of patients with suspected COVID-19 and follow-up. From another point of view, considering that this pan- demic period affects the whole world, there is a serious 1 3 Accuracy values Loss Accuracy values 1218 Pattern Analysis and Applications (2021) 24:1207–1220 Table 4 All performances of Models/fold Confusion matrix and Performance results (%) 5 different models on each fold for COVID-19/bacterial TP TN FP FN ACC REC SPE PRE F1 pneumonia binary classification InceptionV3 Fold-1 68 554 0 0 100 100 100 100 100 Fold-2 68 551 3 0 99.5 100 99.5 95.8 97.8 Fold-3 68 541 13 0 97.9 100 97.7 84.0 91.3 Fold-4 68 532 23 0 96.3 100 95.9 74.7 85.5 Fold-5 69 521 34 0 94.6 100 93.9 67.0 80.2 Total/average 341 2699 73 0 97.7 100 97.4 82.4 90.3 ResNet50 Fold-1 68 554 0 0 100 100 100 100 100 Fold-2 67 551 3 1 99.4 98.5 99.5 95.7 97.1 Fold-3 68 554 0 0 100 100 100 100 100 Fold-4 65 555 0 3 99.5 95.6 100 100 97.7 Fold-5 69 552 3 0 99.5 100 99.5 95.8 97.9 Total/average 337 2766 6 4 99.7 98.8 99.8 98.3 98.5 ResNet101 Fold-1 42 554 0 26 95.8 61.8 100 100 76.4 Fold-2 33 553 1 35 94.2 48.5 99.8 97.1 64.7 Fold-3 68 554 0 0 100 100 100 100 100 Fold-4 14 554 1 54 91.2 20.6 99.8 93.3 33.7 Fold-5 22 555 0 47 92.5 31.9 100 100 48.4 Total/average 179 2770 2 162 94.7 52.5 99.9 98.9 68.6 ResNet152 Fold-1 9 554 0 59 90.5 13.2 100 100 23.4 Fold-2 64 552 2 4 99.0 94.1 99.6 97.0 95.5 Fold-3 26 554 0 42 93.2 38.2 100 100 55.3 Fold-4 28 554 1 40 93.4 41.2 99.8 96.6 57.7 Fold-5 47 502 53 22 88.0 68.1 90.5 47.0 55.6 Total/average 174 2716 56 167 92.8 51.0 98.0 75.7 60.9 Inception-ResNetV2 Fold-1 60 540 14 8 96.5 88.2 97.5 81.1 84.5 Fold-2 39 552 2 29 95.0 57.4 99.6 95.1 71.6 Fold-3 66 547 7 2 98.6 97.1 98.7 90.4 93.6 Fold-4 34 551 4 34 93.9 50.0 99.3 89.5 64.2 Fold-5 42 536 19 27 92.6 60.9 96.6 68.9 64.6 Total/average 241 2726 46 100 95.3 70.7 98.3 84.0 76.8 ResNet50 pre-trained model yielded the highest accuracy early stage, this study gives insight on how deep transfer among five models for used three different datasets (Data- learning methods can be used. In subsequent studies, the set-1: 96.1%, Dataset-2: 99.5% and Dataset-3: 99.7%). In classification performance of different CNN models can be the light of our findings, it is believed that it will help tested by increasing the number of COVID-19 chest X-ray radiologists to make decisions in clinical practice due to images in the dataset. the higher performance. In order to detect COVID-19 at an 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1219 Table 5 The performance comparison literature about COVID-19 diagnostic methods using chest X-ray images Previous study Data type Methods/classifier Number of classes Accuracy (%) Narayan Das et al. [9] X-ray Xception 3 97.40 Singh et. al. [52] X-ray MADE-based CNN 2 94.65 ∓ 2.1 Afshar et al. [30] X-ray Capsule Networks 4 95.7 Ucar and Korkmaz [24] X-ray Bayes-SqueezeNet 3 98.26 Khan et al. [21] X-ray CoroNet 4 89.60 Sahinbas and Catak [26] X-ray VGG16, VGG19, ResNet 2 80 DenseNet and InceptionV3 Medhi et al. [27] X-ray Deep CNN 2 93 Zhang et al. [16] X-ray CAAD 2 95.18 Apostopolus et al. [25] X-ray VGG-19 3 93.48 Narin et al. [31] X-ray InceptionV3, ResNet50, Inception-ResNetV2 2 98 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Normal) 96.1 ResNet152, Inception-ResNetV2 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Viral Pne.) 99.5 ResNet152, Inception-ResNetV2 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Bacterial Pne.) 99.7 ResNet152, Inception-ResNetV2 Author’s contribution AN and CK were involved in the study concep- 7. Jaiswal AK, Tiwari P, Kumar S, Gupta D, Khanna A, Rodrigues tion and acquisition of dataset. AN conducted the experiments and JJ (2019) Identifying pneumonia in chest X-rays: a deep learning analyzed the results. AN, CK and ZP wrote the manuscript and revised approach. Measurement 145:511–518 the draft critically. All authors reviewed the manuscript. 8. Antin B, Kravitz J, Martayan E (2017) Detecting pneumonia in chest X-rays with supervised learning. http:// cs229. stanf ord. edu/ proj2 017/ final- repor ts/ 52312 21. pdf Availability of data and materials Correspondence and requests for data 9. Narayan Das N, Kumar N, Kaur M, Kumar V, Singh D (2020) and materials should be addressed to C.K. Automated deep transfer learning-based approach for detection of COVID-19 infection in chest X-rays. IRBM. https:// doi. org/ Declaration 10. 1016/j. irbm. 2020. 07. 001 10. Ayan E, Ünver HM (2019) Diagnosis of pneumonia from chest Conflicts of interest The authors declare that they have no conflict of X-ray images using deep learning. In: Scientific meeting on elec- interest. trical-electronics and biomedical engineering and computer sci- ence (EBBT), Istanbul, Turkey, pp 1–5. https:// doi. org/ 10. 1109/ EBBT. 2019. 87415 82 11. Gaál G, Maga B, Lukács A (2020) Attention U-Net based adversarial architectures for chest X-ray lung segmentation. References arXiv:2003.10304 12. https:// tr. euron ews. com. Accessed 12 Dec 2020 1. Roosa K, Lee Y, Luo R, Kirpich A, Rothenberg R, Hyman JM et al 13. Liang G, Zheng L (2020) A transfer learning method with deep (2020) Real-time forecasts of the COVID-19 epidemic in China residual network for pediatric pneumonia diagnosis. Comput from February 5th to February 24th, 2020. Infect Dis Model Methods Programs Biomed 187:104964 5:256–263 14. Jaiswal A, Gianchandani N, Singh D, Kumar V, Kaur M 2. Yan L, Zhang H-T, Xiao Y, Wang M, Sun C, Liang J et  al (2020) Classification of the COVID-19 infected patients using (2020) Prediction of criticality in patients with severe COVID- DenseNet201 based deep transfer learning. J Biomol Struct 19 infection using three clinical features: a machine learning- Dyn. https:// doi. org/ 10. 1080/ 07391 102. 2020. 17886 42 based prognostic model with clinical data in Wuhan. medRxiv 15. Apostolopoulos ID, Mpesiana TA (2020) Covid-19: automatic 2020.02.27.20028027 detection from x-ray images utilizing transfer learning with con- 3. Stoecklin SB, Rolland P, Silue Y, Mailles A, Campese C, Simon- volutional neural networks. Phys Eng Sci Med 43:635–640 don A et  al (2020) First cases of coronavirus disease 2019 16. Zhang J, Xie Y, Li Y, Shen C, Xia Y (2020) COVID-19 screen- (COVID-19) in France: surveillance, investigations and control ing on chest X-ray images using deep learning based anomaly measures, January 2020. Eurosurveillance 25(6):2000094 detection. arXiv:2003.12338v1 4. Coronavirus (2020) World Health Organization. https://www .who. 17. Singh D, Kumar V, Kaur M (2020) Classification of COVID-19 int/ health- topics/ coron avirus patients from chest CT images using multi-objective differen- 5. McKeever A (2020) Here’s what coronavirus does to the body. tial evolution-based convolutional neural networks. Eur J Clin National geographic. https:// www. natio nalge ograp hic. com/ scien Microbiol Infect Dis 39:1379–1389. https:// doi. org/ 10. 1007/ ce/ 2020/ 02/ here- is- what- coron avirus- does- to- the- body/ s10096- 020- 03901-z 6. COVID-19 Coronavirus Pandemic (2020) Worldometer. https:// www. world omete rs. info/ coron avirus/ 1 3 1220 Pattern Analysis and Applications (2021) 24:1207–1220 18. Chen X, Yao L, Zhang Y (2020) Residual attention U-Net for 36. Saba T, Mohamed AS, El-Affendi M, Amin J, Sharif M (2020) automated multi-class segmentation of COVID-19 chest CT Brain tumor detection using fusion of hand crafted and deep learn- images. arXiv:2004.05645v1 ing features. Cogn Syst Res 59:221–230 19. Adhikari NCD (2020) Infection severity detection of CoVID19 37. Dorj UO, Lee KK, Choi JY, Lee M (2018) The skin cancer classi- from X-rays and CT scans using artificial intelligence. Int J fication using deep convolutional neural network. Multimed Tools Comput (IJC) 38(1):73–92 Appl 77(8):9909–9924 20. Alqudah AM, Qazan S, Alqudah A (2020) Automated systems 38. Kassani SH, Kassani PH (2019) A comparative study of deep for detection of COVID-19 using chest X-ray images and light- learning architectures on melanoma detection. Tissue Cell weight convolutional neural networks. https://doi. or g/10. 21203/ 58:76–83 rs.3. rs- 24305/ v1 39. Ribli D, Horváth A, Unger Z, Pollner P, Csabai I (2018) Detecting 21. Khan AI, Shah JL, Bhat MM (2020) Coronet: a deep neural and classifying lesions in mammograms with deep learning. Sci network for detection and diagnosis of COVID-19 from chest Rep 8:4165 X-ray images. Comput Methods Programs Biomed 196:105581 40. Celik Y, Talo M, Yildirim O, Karabatak M, Acharya UR (2020) 22. Ghoshal B, Tucker A (2020) Estimating uncertainty and inter- Automated invasive ductal carcinoma detection based using deep pretability in deep learning for coronavirus (COVID-19) detec- transfer learning with whole-slide images. Pattern Recogn Lett tion. arXiv:2003.10769v1 133:232–239 23. Hemdan EED, Shouman MA, Karar ME (2020) COVIDX-Net: 41. Yu X, Kang C, Guttery DS, Kadry S, Chen Y, Zhang Y-D (2020) a framework of deep learning classifiers to diagnose COVID-19 ResNet-SCDA-50 for breast abnormality classification. In: IEEE/ in X-ray images. arXiv:2003.11055 ACM transactions on computational biology and bioinformatics, 24. Ucar F, Korkmaz D (2020) COVIDiagnosis-Net: deep Bayes- pp 1–8. https:// doi. org/ 10. 1109/ TCBB. 2020. 29865 44 SqueezeNet based diagnosis of the coronavirus disease 2019 42. Jmour N, Zayen S, Abdelkrim A (2018) Convolutional neural (COVID-19) from X-ray images. Med Hypotheses 140:109761 networks for image classification. In: International conference on 25. Apostolopoulos ID, Aznaouridis SI, Tzani MA (2020) Extract- advanced systems and electric technologies (IC\_ASET), Ham- ing possibly representative COVID-19 Biomarkers from X-ray mamet, Tunisia, pp 397–402 images with deep learning approach and image data related to 43. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature pulmonary diseases. J Med Biol Eng 40:462–469 521:436–444. https://www .natio nalg eog raphic. com/ scien ce/ 2020/ 26. Sahinbas K, Catak FO (2020) Transfer learning based convolu-02/ here- is- what- coron avirus- does- to- the- body/2 tional neural network for COVID-19 detection with X-ray images. 44. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov https:// www. ozgur catak. org/ files/ papers/ covid 19- deep- learn ing. R (2014) Dropout: a simple way to prevent neural networks from pdf overfitting. J Mach Learn Res 15:1929–1958 27. Jamil M, Hussain I (2020) Automatic detection of COVID-19 45. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans infection from chest X-ray using deep learning. medRxiv. https:// Knowl Data Eng 22(10):1345–1359 doi. org/ 10. 1101/ 2020. 05. 10. 20097 063 46. Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer 28. Barstugan M, Ozkaya U, Ozturk S (2020) Coronavirus (COVID- learning using computational intelligence: a survey. Knowl Based 19) classification using CT images by machine learning methods. Syst 80:14–23 arXiv:2003.09424 47. Huang Z, Pan Z, Lei B (2017) Transfer learning with deep con- 29. Punn NS, Agarwal S (2020) Automated diagnosis of COVID-19 volutional neural network for sar target classification with limited with limited posteroanterior chest X-ray images using fine-tuned labeled data. Remote Sens 9(9):907 deep neural networks. https:// www. who. int/ health- topics/ coron 48. Wu Z, Shen C, Van Den Hengel A (2019) Wider or deeper: revis- avirus6 iting the ResNet model for visual recognition. Pattern Recogn 30. Afshar P, Heidarian S, Naderkhani F, Oikonomou A, Plataniotis 90:119–133 KN, Mohammadi A (2020) COVID-CAPS: a capsule network- 49. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S et al based framework for identification of COVID-19 cases from X-ray (2015) ImageNet large scale visual recognition challenge. Int J images. arXiv:2004.02696v2 Comput Vis 115:211–252 31. Narin A, Kaya C, Pamuk Z (2020) Automatic detection of coro- 50. Ahn JM, Kim S, Ahn KS, Cho SH, Lee KB, Kim US (2018) A navirus disease (COVID-19) using X-ray images and deep con- deep learning model for the detection of both advanced and early volutional neural networks. arXiv:2003.10849v1 glaucoma using fundus photography. PLoS ONE 13(11):e0207982 32. Cohen JP, Morrison P, Dao L (2020) COVID-19 image data col- 51. Byra M, Styczynski G, Szmigielski C, Kalinowski P, Michalowski lection. arXiv:2003.11597 L, Paluszkiewicz R et al (2018) Transfer learning with deep con- 33. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) volutional neural network for liver steatosis assessment in ultra- ChestX-Ray8: hospital-scale chest X-ray database and bench- sound images. Int J Comput Assist Radiol Surg 13(12):1895–1903 marks on weakly-supervised classification and localization of 52. Singh D, Kumar V, Yadav V, Kaur M (2020) Deep convolutional common thorax diseases. In: IEEE conference on computer vision neural networks based classification model for COVID-19 infected and pattern recognition (CVPR), Honolulu, pp 3462–3471. https:// patients using chest X-ray images. Int J Pattern Recognit Artif doi. org/ 10. 1109/ CVPR. 2017. 369 Intell. https:// doi. org/ 10. 1142/ S0218 00142 15100 46 34. Mooney P (2018) Chest X-ray images (Pneumonia). Kaggle Repository. https:// www. kaggle. com/ pault imoth ymoon ey/ chest- Publisher’s Note Springer Nature remains neutral with regard to xray- pnemo nia jurisdictional claims in published maps and institutional affiliations. 35. Yildirim O, Talo M, Ay B, Baloglu UB, Aydin G, Acharya UR (2019) Automated detection of diabetic subject using pre-trained 2D-CNN models with frequency spectrum images extracted from heart rate signals. Comput Biol Med 113:103387 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Pattern Analysis and Applications Pubmed Central

Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks

Narin, Ali; Kaya, Ceren; Pamuk, Ziynet
Pattern Analysis and Applications , Volume 24 (3) – May 9, 2021
Download PDF
14 pages

Loading next page...
 
/lp/pubmed-central/automatic-detection-of-coronavirus-disease-covid-19-using-x-ray-images-DhMrsN3y1z

References (95)

Publisher
Pubmed Central
Copyright
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2021
ISSN
1433-7541
eISSN
1433-755X
DOI
10.1007/s10044-021-00984-y
Publisher site
See Article on Publisher Site

Abstract

The 2019 novel coronavirus disease (COVID-19), with a starting point in China, has spread rapidly among people living in other countries and is approaching approximately 101,917,147 cases worldwide according to the statistics of World Health Organization. There are a limited number of COVID-19 test kits available in hospitals due to the increasing cases daily. Therefore, it is necessary to implement an automatic detection system as a quick alternative diagnosis option to prevent COVID-19 spreading among people. In this study, five pre-trained convolutional neural network-based models (ResNet50, ResNet101, ResNet152, InceptionV3 and Inception-ResNetV2) have been proposed for the detection of coronavirus pneu- monia-infected patient using chest X-ray radiographs. We have implemented three different binary classifications with four classes (COVID-19, normal (healthy), viral pneumonia and bacterial pneumonia) by using five-fold cross-validation. Considering the performance results obtained, it has been seen that the pre-trained ResNet50 model provides the highest classification performance (96.1% accuracy for Dataset-1, 99.5% accuracy for Dataset-2 and 99.7% accuracy for Dataset-3) among other four used models. Keywords Coronavirus · Bacterial pneumonia · Viral pneumonia · Chest X-ray radiographs · Convolutional neural network · Deep transfer learning 1 Introduction syndrome (MERS-CoV) and severe acute respiratory syn- drome (SARS-CoV). COVID-19 is a new species discovered The coronavirus disease (COVID-19) pandemic emerged in 2019 and has not been previously identified in humans in Wuhan, China, in December 2019 and became a serious [4]. COVID-19 causes lighter symptoms in about 99% of public health problem worldwide [1, 2]. Until now, no spe- cases, according to early data, while the rest is severe or cific drug or vaccine has been found against COVID-19 [2 ]. critical [5]. As of January 31, 2021, the total number of The virus that causes COVID-19 epidemic disease is called worldwide cases of coronavirus is 103,286,991 including severe acute respiratory syndrome coronavirus-2 (SARS- 2,232,776 deaths. Of these, the number of active patients is CoV-2) [3]. Coronaviruses (CoV) are a large family of 26,127,156 [6]. Nowadays the world is struggling with the viruses that cause diseases such as Middle East respiratory COVID-19 epidemic. Deaths from pneumonia developing due to the SARS-CoV-2 virus are increasing day by day. Chest radiography (X-ray) is one of the most important * Ceren Kaya methods used for the diagnosis of pneumonia worldwide [7]. [email protected]; [email protected] Chest X-ray is a fast, cheap [8] and common clinical method Ali Narin [9–11]. The chest X-ray gives the patient a lower radiation [email protected] dose compared to computed tomography (CT) and magnetic Ziynet Pamuk resonance imaging (MRI) [11]. However, making the correct [email protected] diagnosis from X-ray images requires expert knowledge and Department of Electrical and Electronics Engineering, experience [7]. It is much more difficult to diagnose using Zonguldak Bulent Ecevit University, Zonguldak 67100, a chest X-ray than other imaging modalities such as CT or Turkey MRI [8]. Department of Biomedical Engineering, Zonguldak Bulent Ecevit University, Zonguldak 67100, Turkey Vol.:(0123456789) 1 3 1208 Pattern Analysis and Applications (2021) 24:1207–1220 By looking at the chest X-ray, COVID-19 can only be 3.3, respectively. Performance metrics are given in detail diagnosed by specialist physicians. The number of special- in Sect. 3.4. Obtained experimental results from proposed ists who can make this diagnosis is less than the number of models and discussion are presented in Sects. 4 and 5, normal doctors. Even in normal times, the number of doc- respectively. Finally, in Sect. 6, the conclusion and future tors per person is insufficient in countries around the world. works are summarized. According to data from 2017, Greece ranks first with 607 doctors per 100,000 people. In other countries, this number is much lower [12].2 Related works In case of disasters such as COVID-19 pandemic, demanding health services at the same time, collapse of the Studies diagnosed with COVID-19 using chest X-rays have health system is inevitable due to the insufficient number of binary or multiple classifications. Some studies use raw data, hospital beds and health personnel. Also, COVID-19 is a while others have feature extraction process. The number of highly contagious disease, and doctors, nurses and caregiv- data used in studies also varies. Among the studies, the most ers are most at risk. Early diagnosis of pneumonia has a preferred method is convolutional neural network (CNN). vital importance both in terms of slowing the speed of the Apostolopoulos and Bessiana used a common pneumonia, spread of the epidemic by quarantining the patient and in the COVID-19-induced pneumonia, and an evolutionary neural recovery process of the patient. network for healthy differentiation on automatic detection of Doctors can diagnose pneumonia from the chest X-ray COVID-19. In particular, the procedure called transfer learn- more quickly and accurately thanks to computer-aided diag- ing has been adopted. With transfer learning, the detection nosis (CAD) [8]. Use of artificial intelligence methods is of various abnormalities in small medical image datasets increasing due to its ability to cope with enormous datasets is an achievable goal, often with remarkable results [15]. exceeding human potential in the field of medical services Based on chest X-ray images, Zhang et al. aimed to develop [13]. Integrating CAD methods into radiologist diagnos- a deep learning-based model that can detect COVID-19 with tic systems greatly reduces the workload of doctors and high sensitivity, providing fast and reliable scanning [16]. increases reliability and quantitative analysis [11]. CAD Singh et al. classified the chest computed tomography (CT) systems based on deep learning and medical imaging are images from infected people with and without COVID-19 becoming more and more research fields [13, 14]. using multi-objective differential evolution (MODE)-based In this study, we have proposed an automatic CAD CNN [17]. Jaiswal et al. proposed DenseNet201-based deep prediction of COVID-19 using a deep convolutional neu- transfer learning model on chest CT images to classify the ral network-based pre-trained transfer models and chest patients as COVID-19 infected or not [14]. In the study of X-ray images. For this purpose, we have used ResNet50, Chen et  al, they proposed Residual Attention U-Net for ResNet101, ResNet152, InceptionV3 and Inception- automated multi-class segmentation technique to prepare ResNetV2 pre-trained models to obtain higher prediction the ground for the quantitative diagnosis of lung infection accuracies for three different binary datasets including X-ray on COVID-19-related pneumonia using CT images [18]. images of normal (healthy), COVID-19, bacterial and viral Adhikari’s study suggested a network called “Auto Diag- pneumonia patients. nostic Medical Analysis” trying to find infectious areas to The novelty and originality of proposed study are summa- help the doctor better identify the diseased part, if any. Both rized as follows: (1) The proposed models have end-to-end X-ray and CT images were used in the study. It has been rec- structure without manual feature extraction, selection and ommended DenseNet network to remove and mark infected classification. (2) The performances of the COVID-19 data areas of the lung [19]. In the study by Alqudah et al., two across normal, viral pneumonia and bacterial pneumonia different methods were used to diagnose COVID-19 using classes were significantly higher. (3) It has been studied with chest X-ray images. The first one used AOCTNet, MobileNet more data than many studies in the literature. (4) It has been and ShuffleNet CNNs. Secondly, the features of their images studied and compared with 5 different CNN models. (5) A have been removed and they have been classified using soft- high-accuracy decision support system has been proposed max classifier, K nearest neighbor (kNN), support vector to radiologists for the automatic diagnosis and detection of machine (SVM) and random forest (RF) algorithms [20]. patients with suspected COVID-19 and follow-up. Khan et al. classified the chest X-ray images from normal, The flow of the manuscript is organized as follows: The bacterial and viral pneumonia cases using the Xception work done in the field of deep learning techniques on chest architecture to detect COVID-19 infection [21]. Ghoshal and X-ray and CT images for COVID-19 disease is presented Tucker used the dropweights-based Bayesian CNN model in Sect. 2. Dataset is expressed in detail in Sect. 3.1. Deep using chest X-ray images for the diagnosis of COVID-19 transfer learning architecture, pre-trained models and experi- [22]. Hemdan et  al. used VGG19 and DenseNet models mental setup parameters are described in Sects. 3.2 and to diagnose COVID-19 from X-ray images [23]. Ucar and 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1209 Korkmaz worked on X-ray images for COVID-19 diagnosis X-ray images. Distribution of images per class in created and supported the SqueezeNet model with Bayesian opti- datasets is given in Table 1. mization [24]. In the study conducted by Apostopolus et al., The data augmentation method was used with scaling they performed automatic detection from X-ray images using factor = 1./255, shear range = 0.1, zoom range = 0.1 and CNNs with transfer learning [25]. Sahinbas and Catak used horizontal flipping enabled in training dataset. All images X-ray images for the diagnosis of COVID-19 and worked on were resized to 224 × 224 pixel size in the datasets. In Fig. 1, VGG16, VGG19, ResNet, DenseNet and InceptionV3 mod- representative chest X-ray images of normal (healthy), els [26]. Medhi et al. used X-ray images as feature extrac- COVID-19, bacterial and viral pneumonia patients are given, tion and segmentation in their study, and then, COVID-19 respectively. was positively and normally classified using CNN [27]. Barstugan et al. classified X-ray images for the diagnosis of 3.2 Architecture of deep transfer learning COVID-19 using five different feature extraction methods that are Grey-Level Cooccurrence Matrix (GLCM), Local Deep learning is a sub-branch of the machine learning field, Directional Patterns (LDP), Grey-Level Run Length Matrix inspired by the structure of the brain. Deep learning tech- (GLRLM), Grey-Level Size Zone Matrix (GLSZM) and niques used in recent years continue to show an impressive Discrete Wavelet Transform (DWT). The obtained features performance in the field of medical image processing, as in were classified by SVM. During the classification process, many fields. By applying deep learning techniques to medi- two-fold, five-fold and ten-fold cross-validation methods cal data, it is tried to draw meaningful results from medical were used [28]. Punn and Agarwal worked on X-ray images data. and used ResNet, InceptionV3, Inception-ResNet models to Deep learning models have been used successfully in diagnose COVID-19 [29]. Afshar et al. developed deep neu- many areas such as classification, segmentation and lesion ral network (DNN)-based diagnostic solutions and offered an detection of medical data. Analysis of image and signal data alternative modeling framework based on Capsule Networks was obtained with medical imaging techniques such as mag- that can process on small datasets [30]. netic resonance imaging (MRI), computed tomography (CT) In our previous study in March 2020, we used ResNet50, and X-ray with the help of deep learning models. As a result InceptionV3 and Inception-ResNetV2 models for the diag- of these analyzes, detection and diagnosis of diseases such as nosis of COVID-19 using chest X-ray images. However, diabetes mellitus, brain tumor, skin cancer and breast cancer since there were not enough data on COVID-19, we were are provided in studies with convenience [35–41]. only able to train through 50 normal and 50 COVID-19 posi- A convolutional neural network (CNN) is a class of tive cases [31]. Therefore, to overcome the issues associated deep neural networks used in image recognition problems with our previous study [31], proposed study was recon- [42]. Coming to how CNN works, the images given as ducted by increasing the number of data and deep transfer input must be recognized by computers and converted into learning models to classify COVID-19-infected patients. a format that can be processed. For this reason, images are first converted to matrix format. The system determines which image belongs to which label based on the differ - 3 Materials and methods ences in images and therefore in matrices. It learns the effects of these differences on the label during the training 3.1 Dataset phase and then makes predictions for new images using them. CNN consists of three different layers that are a con- In this study, chest X-ray images of 341 COVID-19 patients volutional layer, pooling layer and fully connected layer to have been obtained from the open source GitHub repository perform these operations effectively. The feature extraction shared by Dr. Joseph Cohen et al. [32]. This repository is process takes place in both convolutional and pooling lay- consisting chest X-ray/computed tomography (CT) images ers. On the other hand, the classification process occurs in of mainly patients with acute respiratory distress syndrome (ARDS), COVID-19, Middle East respiratory syndrome (MERS), pneumonia, severe acute respiratory syndrome Table 1 Number of images per class for each dataset (SARS). 2800 normal (healthy) chest X-ray images were Datasets/classes Bacterial COVID-19 Normal Viral pneumonia selected from “ChestX-ray8” database [33]. In addition, pneumo- 2772 bacterial and 1493 viral pneumonia chest X-ray images nia were used from Kaggle repository called “Chest X-Ray Dataset-1 – 341 2800 – Images (Pneumonia)” [34]. Dataset-2 – 341 – 1493 Our experiments have been based on three binary cre- Dataset-3 2772 341 – – ated datasets (Dataset-1, Dataset-2 and Dataset-3) with chest 1 3 1210 Pattern Analysis and Applications (2021) 24:1207–1220 Fig. 1 Representative chest X-ray images of normal (healthy) (first row), COVID-19 (second row), bacterial (third row) and viral pneumonia (fourth row) patients fully connected layer. These layers are examined sequen- l l−1 l−1 l tially in the following. x = f w ∗ y + b (1) j j a j a=1 l l−1 where x is the jth feature map in layer l, w indicates jth j j 3.2.1 Convolutional layer l−1 kernels in layer l − 1 , y represents the ath feature map in layer l − 1 , b indicates the bias of the jth feature map in layer Convolutional layer is the base layer of CNN. It is respon- l, N is number of total features in layer l − 1 , and (∗) repre- sible for determining the features of the pattern. In this sents vector convolution process. layer, the input image is passed through a filter. The values resulting from filtering consist of the feature map. This layer applies some kernels that slide through the pattern 3.2.2 Pooling layer to extract low- and high-level features in the pattern [43]. The kernel is a 3 × 3- or 5 × 5-shaped matrix to be trans- The second layer after the convolutional layer is the pooling formed with the input pattern matrix. Stride parameter is layer. Pooling layer is usually applied to the created fea- the number of steps tuned for shifting over input matrix. ture maps for reducing the number of feature maps and net- The output of convolutional layer can be given as: work parameters by applying corresponding mathematical 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1211 computation. In this study, we used max-pooling and global weights of models trained on different datasets are trans- average pooling. The max-pooling process selects only the ferred to the new model [45, 46]. Apart from the transferred maximum value by using the matrix size specified in each parts, the learning process is also carried out through the feature map, resulting in reduced output neurons. There is newly added layers. It is stated that it is particularly success- also a global average pooling layer that is only used before ful even in few datasets [47]. In addition, this method used the fully connected layer, reducing data to a single dimen- allows to obtain results faster with lower calculation cost. sion. It is connected to the fully connected layer after global In the analysis of medical data, one of the biggest dif- average pooling layer. The other intermediate layer used is ficulties faced by researchers is the limited number of avail- the dropout layer. The main purpose of this layer is to pre- able datasets. Deep learning models often need a lot of vent network overfitting and divergence [44]. data. Labeling this data by experts is both costly and time- consuming. The biggest advantage of using transfer learn- 3.2.3 Fully connected layer ing method is that it allows the training of data with fewer datasets and requires less calculation costs. With the transfer Fully connected layer is the last and most important layer of learning method, which is widely used in the field of deep CNN. This layer functions like a multilayer perceptron. Rec- learning, the information gained by the pre-trained model tified linear unit (ReLU) activation function is commonly on a large dataset is transferred to the model to be trained. used on fully connected layer, while softmax activation func- In this study, we built deep CNN-based ResNet50, tion is used to predict output images in the last layer of fully ResNet101, ResNet152, InceptionV3 and Inception- connected layer. Mathematical computation of these two ResNetV2 models for the classification of COVID-19 activation functions is as follows: chest X-ray images to three different binary classes (Binary Class-1 = COVID-19 and normal (healthy), Binary Class-2 0, if x < 0 = COVID-19 and viral pneumonia and Binary Class-3 ReLU(x)= (2) x, if x ≥ 0 = COVID-19 and bacterial pneumonia). In addition, we applied transfer learning technique that was realized by e using ImageNet data to overcome the insufficient data and Softmax(x )= x (3) training time. The schematic representation of conven- y=1 tional CNN including pre-trained ResNet50, ResNet101, where x and m represent input data and the number of ResNet152, InceptionV3 and Inception ResNetV2 models classes, respectively. Neurons in a fully connected layer have for the prediction of normal (healthy), COVID-19, bacterial full connections to all activation functions in previous layer. and viral pneumonia patients is depicted in Fig. 2. It is also available publicly for open access at https:// git hub. com/ dr cer 3.2.4 Pre‑trained models enkaya/ COVID- 19- Detec tionV2. Training convolutional neural network (CNN) models with • ResNet50 millions of parameters from scratch is not only very time- Residual neural network (ResNet) model is an consuming, but also requires equipment with high per- improved version of CNN. ResNet adds shortcuts formance. To overcome these problems, parameters and between layers to solve a problem. Thanks to this, it Fig. 2 Schematic representation of pre-trained models for the prediction of normal (healthy), COVID-19, bacterial and viral pneumonia patients 1 3 1212 Pattern Analysis and Applications (2021) 24:1207–1220 prevents the distortion that occurs as the network gets ResNet101, ResNet152, InceptionV3 and Inception- deeper and more complex. In addition, bottleneck blocks ResNetV2) were pre-trained with random initialization are used to make training faster in the ResNet model [48]. weights by optimizing the cross-entropy function with adap- ResNet50 is a 50-layer network trained on the ImageNet tive moment estimation (ADAM) optimizer ( 1 = 0.9 and dataset. ImageNet is an image database with more than 2 = 0.999 ). The batch size, learning rate and number of 14 million images belonging to more than 20,000 catego- epochs were experimentally set to 3, 1e 5 and 30, respec- ries created for image recognition competitions [49]. tively, for all experiments. All datasets used were randomly InceptionV3 split into two independent datasets with 80% and 20% InceptionV3 is a kind of convolutional neural network for training and testing, respectively. As cross-validation model. It consists of numerous convolution and maxi- method, k-fold was chosen and results were obtained accord- mum pooling steps. In the last stage, it contains a fully ing to 5 different k values (k = 1–5) as shown in Fig. 3. connected neural network [50]. As with the ResNet50 model, the network is trained with ImageNet dataset. 3.4 Performance metrics Inception-ResNetV2 The model consists of a deep convolutional network Five criteria were used for the performances of deep transfer using the Inception-ResNetV2 architecture that was learning models. These are: trained on the ImageNet-2012 dataset. The input to the model is a 299 × 299 image, and the output is a list of TP + TN Accuracy = (4) estimated class probabilities [51]. TP + TN + FP + FN ResNet101 and ResNet152 ResNet101 and ResNet152 consist of 101 and 152 lay- TP Recall = (5) ers, respectively, due to stacked ResNet building blocks. TP + FN You can load a pre-trained version of the network trained on more than a million images from the ImageNet data- TN Specificity = (6) base [49]. As a result, the network has learned rich fea- TN + FP ture representations for a wide range of images. The net- work has an image input size of 224 × 224. TP Precision = (7) TP + FP 3.3 Experimental setup 2 ∗ Precision * Recall F1-score = Python programming language was used to train the pro- (8) Precision + Recall posed deep transfer learning models. All experiments were performed on Google Colaboratory (Colab) Linux server TP, FP, TN and FN given in Eqs. (4)–(8) represent the num- with the Ubuntu 16.04 operating system using the online ber of true positive, false positive, true negative and false cloud service with Central Processing Unit (CPU), Tesla negative, respectively. For Dataset-1, given a test dataset K80 Graphics Processing Unit (GPU) or Tensor Process- and model, TP is the proportion of positive (COVID-19) that ing Unit (TPU) hardware for free. CNN models (ResNet50, is correctly labeled as COVID-19 by the model; FP is the Fig. 3 Visual display of testing and training datasets for five- fold cross-validation 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1213 proportion of negative (normal) that is mislabeled as positive 0.4 InceptionV3 (COVID-19); TN is the proportion of negative (normal) that ResNet50 ResNet101 is correctly labeled as normal; and FN is the proportion of ResNet152 positive (COVID-19) that is mislabeled as negative (normal) 0.3 InceptionResNetV2 0.04 by the model. 0.02 0.2 4 Experimental results 20 25 30 In this paper, we performed 3 different binary classifications 0.1 with 4 different classes (COVID-19, normal, viral pneumo- nia and bacterial pneumonia). Five-fold cross-validation method has been used in order to get a robust result in this study performed with 5 different pre-trained models that 51015202530 are InceptionV3, ResNet50, ResNet101, ResNet152 and Epoch Inception-ResNetV2. While 80% of the data is reserved for training, the remaining 20% is reserved for testing. All this Fig. 5 Binary Class-1: comparison of training loss values of 5 differ - ent models for fold-4 process continued until each 20% part was tested. Consider- ing the training times of all models, for Dataset-1 (Binary Class-1), the total training times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained given in Figs. 4 and 5. It is clear that the performance of the ResNet50 model is better than the other models. It can be models were 16027 s, 14638 s, 17841 s, 18802 s and 23078 s, respectively. For Dataset-2 (Binary Class-2), the total said that the ResNet50 model reaches lower values among the loss values of other models. Detection performance training times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained models on test data is shown in Fig.  6. While a lot of oscillation is observed in some models, some models are more sta- were 12241 s, 9948 s, 13089 s, 14923 s and 19336 s, respec- tively. Finally, for Dataset-3 (Binary Class-3), the total train- ble. The ResNet50 model appears to have less oscillation after the 15th epoch. Comprehensive performance values ing times of InceptionV3, ResNet50, ResNet101, ResNet152 and Inception-ResNetV2 pre-trained models were 15801 s, for each fold value of each model are given in Table 2. As seen from Table 2, the detection of the ResNet50 model in 14386 s, 17658 s, 18581 s and 22865 s, respectively. Firstly, the accuracy and loss values in the training the COVID-19 class is significantly higher than the other models. ResNet50 and ResNet101 have the highest overall process obtained for the models applied to Dataset-1 that includes Binary Class-1 (COVID-19/normal classes) are 0.9 0.98 1 0.8 0.96 0.995 0.7 0.94 0.99 20 22 24 26 28 30 0.6 InceptionV3 0.92 ResNet50 InceptionV3 ResNet50 ResNet101 0.9 0.5 ResNet101 ResNet152 ResNet152 InceptionResNetV2 InceptionResNetV2 0.88 0.4 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 4 Binary Class-1: comparison of training accuracy of 5 different Fig. 6 Binary Class-1: comparison of testing accuracy of 5 different models for fold-4 models for fold-4 1 3 Accuracy values Accuracy values Loss 1214 Pattern Analysis and Applications (2021) 24:1207–1220 Table 2 All performances of 5 Models/fold Confusion matrix and performance results (%) different models on each fold for COVID-19/normal binary TP TN FP FN ACC REC SPE PRE F1 classification InceptionV3 Fold-1 60 519 41 8 92.2 88.2 92.7 59.4 71.0 Fold-2 60 547 13 8 96.7 88.2 97.7 82.2 85.1 Fold-3 65 560 0 3 99.5 95.6 100 100 97.7 Fold-4 57 524 36 11 92.5 83.8 93.6 61.3 70.8 Fold-5 67 538 22 2 96.2 97.1 96.1 75.3 84.8 Total/average 309 2688 112 32 95.4 90.6 96.0 73.4 81.1 ResNet50 Fold-1 65 511 49 3 91.7 95.6 91.3 57.0 71.4 Fold-2 59 545 15 9 96.2 86.8 97.3 79.7 83.1 Fold-3 62 556 4 6 98.4 91.2 99.3 93.9 92.5 Fold-4 59 543 17 9 95.9 86.8 97.0 77.6 81.9 Fold-5 68 549 11 1 98.1 98.6 98.0 86.1 91.9 Total/average 313 2704 96 28 96.1 91.8 96.6 76.5 83.5 ResNet101 Fold-1 50 543 17 18 94.4 73.5 97.0 74.6 74.1 Fold-2 49 559 1 19 96.8 72.1 99.8 98.0 83.1 Fold-3 68 541 19 0 97.0 100 96.6 78.2 87.7 Fold-4 33 554 6 35 93.5 48.5 98.9 84.6 61.7 Fold-5 67 553 7 2 98.6 97.1 98.8 90.5 93.7 Total/average 267 2750 50 74 96.1 78.3 98.2 84.2 81.2 ResNet152 Fold-1 15 547 13 53 89.5 22.1 97.7 53.6 31.3 Fold-2 55 556 4 13 97.3 80.9 99.3 93.2 86.6 Fold-3 55 555 5 13 97.1 80.9 99.1 91.7 85.9 Fold-4 34 539 21 34 91.2 50.0 96.3 61.8 55.3 Fold-5 64 528 32 5 94.1 92.8 94.3 66.7 77.6 Total/average 223 2725 75 118 93.9 65.4 97.3 74.8 69.8 Inception-ResNetV2 Fold-1 59 538 22 9 95.1 86.8 96.1 72.8 79.2 Fold-2 59 523 37 9 92.7 86.8 93.4 61.5 72.0 Fold-3 38 553 7 10 97.2 79.2 98.8 84.4 81.7 Fold-4 48 516 44 20 89.8 70.6 92.1 52.2 60.0 Fold-5 64 542 18 5 96.3 92.8 96.8 78.0 84.8 Total/average 268 2672 128 53 94.2 83.5 95.4 67.7 74.8 TP, true positive; TN, true negative; FP, false positive (FP); FN, false negative; ACC, accuracy; REC, recall; SPE, specificity; PRE, precision (PRE); and F1, F1-score performance with 96.1%. It is obvious that the excess of the first 3 epochs of the ResNet50 model. Detailed perfor - normal data results in higher performance in all models. mances of the models are given in Table 3. It is clearly seen Secondly, when the results obtained for the data in Binary that quite high values are reached for each fold value. While Class-2 (COVID-19/viral pneumonia classes) are evaluated, 99.4% was reached in the detection of COVID-19, it is seen the training performances of the models given in Figs. 7 and that 99.5% was reached in the detection of viral pneumonia. 8 are quite high. It can be said that the accuracy values and In the last study, the detection success of Binary Class-3 loss values of the ResNet50 and ResNet101 models perform (COVID-19/bacterial pneumonia classes) was investigated. better than the other models. Performance values obtained The performances of 5 different models on both training and through test data are shown in Fig.  9. Here, the models’ test data are given in Figs. 10, 11 and 12. As in other studies, results on the test data are generally more stable. There is no it is clearly seen that the ResNet50 model exhibits higher oscillation except when there is excessive oscillation only in training performance. InceptionV3 model is seen to exhibit 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1215 0.9 0.8 0.95 0.995 0.7 0.9 0.6 0.99 20 22 24 26 28 30 InceptionV3 InceptionV3 0.5 ResNet50 ResNet50 0.85 ResNet101 ResNet101 0.4 ResNet152 ResNet152 InceptionResNetV2 InceptionResNetV2 0.3 0.8 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 9 Binary Class-2: comparison of testing accuracy of 5 different Fig. 7 Binary Class-2: comparison of training accuracy of 5 different models for fold-4 models for fold-4 in the literature. In binary classification, it is common to 0.4 InceptionV3 distinguish COVID-19 positive from COVID-19 negative. In ResNet50 addition, it is very important to distinguish viral and bacte- ResNet101 ResNet152 rial pneumonia patients, which are other types of diseases 0.3 InceptionResNetV2 0.04 affecting the lung, from COVID-19 positive patients. There are a limited number of studies in the literature that work with multiple classes. Narayan Das et al. conducted stud- 0.2 0.02 ies for 3 different classes (COVID-19 positive, pneumonia and other infection). The researchers used 70% of the data for the training, the remaining 10% for validation and 20% 0.1 20 25 30 for the test. As a result, they obtained 97.40% accuracy over test data with extreme version of Inception (Xception) CNN model [9]. Singh et al. proposed a two-class study 51015202530 using limited data. They reported their performances by Epoch dividing the dataset at different training and testing rates. They achieved the highest accuracy of 94.65 ∓ 2.1 at 70% training–30% testing rates. In their study, they set the CNN Fig. 8 Binary Class-2: comparison of training loss values of 5 differ - ent models for fold-4 hyper-parameters using multi-objective adaptive differential evolution (MADE) [52]. Afshar et al. conducted their studies using a method called COVID-CAPS with multi-class (nor- increasing performance toward the end of the epoch number. mal, bacterial pneumonia, non-COVID viral pneumonia and COVID-19) studies. They achieved 95.7% accuracy with the When the detailed results given in Table 4 are evaluated, it can be said that the InceptionV3 model has a performance approach without pre-training and 98.3% accuracy with pre- trained COVID-CAPS. However, although their sensitivity of 100% in the detection of COVID-19, while the overall performance is also said to be the ResNet50 model which values are not as high as general accuracy, they detected the sensitivity without using pre-training and with using pre- has a high success. trained COVID-CAPS as 90% and 80%, respectively [30]. Ucar and Korkmaz carried out multi-class (normal, 5 Discussion pneumonia and COVID-19 cases) work with deep Bayes- SqueezeNet. They obtained the average accuracy value of The use of artificial intelligence-based systems is very com - 98.26%. They worked with 76 COVID-19 data [24]. Sahi- nbas and Catak worked with 5 different pre-trained models mon in detecting those caught in the COVID-19 epidemic. As given in Table 5, there are many studies on this subject (VGG16, VGG19, ResNet, DenseNet and InceptionV3). 1 3 Accuracy values Loss Accuracy values 1216 Pattern Analysis and Applications (2021) 24:1207–1220 Table 3 All performances of 5 Models/fold Confusion matrix and Performance results (%) different models on each fold for COVID-19/viral pneumonia TP TN FP FN ACC REC SPE PRE F1 binary classification InceptionV3 Fold-1 68 292 6 0 98.4 100 98.0 91.9 95.8 Fold-2 68 292 6 0 98.4 100 98.0 91.9 95.8 Fold-3 67 295 4 1 98.6 98.5 98.7 94.4 96.4 Fold-4 68 290 9 0 97.5 100 97.0 88.3 93.8 Fold-5 69 299 0 0 100 100 100 100 100 Total/average 340 1468 25 1 98.6 99.7 98.3 93.2 96.3 ResNet50 Fold-1 68 297 1 0 99.7 100 99.7 98.6 99.3 Fold-2 68 293 5 0 98.6 100 98.3 93.2 96.5 Fold-3 68 298 1 0 99.7 100 99.7 98.6 99.3 Fold-4 66 299 0 2 99.5 97.1 100 100 98.5 Fold-5 69 299 0 0 100 100 100 100 100 Total/average 339 1486 7 2 99.5 99.4 99.5 98.0 98.7 ResNet101 Fold-1 61 294 4 7 97.0 89.7 98.7 93.8 91.7 Fold-2 62 293 5 6 97.0 91.2 98.3 92.5 91.9 Fold-3 68 295 4 0 98.9 100 98.7 94.4 97.1 Fold-4 65 298 1 3 98.9 95.6 99.7 98.5 97.0 Fold-5 45 299 0 24 93.5 65.2 100 100 78.9 Total/average 301 1479 14 40 97.1 88.3 99.1 95.6 91.8 ResNet152 Fold-1 63 291 7 5 96.7 92.6 97.7 90.0 91.3 Fold-2 67 293 5 1 98.4 98.5 98.3 93.1 95.7 Fold-3 66 298 1 2 99.2 97.1 99.7 98.5 97.8 Fold-4 56 299 0 13 96.5 81.2 100 100 89.6 Fold-5 58 298 1 10 97.0 85.3 99.7 98.3 91.3 Total/average 310 1479 14 31 97.5 90.9 99.1 95.7 93.2 Inception-ResNetV2 Fold-1 68 283 15 0 95.9 100 95.0 81.9 90.1 Fold-2 68 267 31 0 91.5 100 89.6 68.7 81.4 Fold-3 68 278 21 0 94.3 100 93.0 76.4 86.6 Fold-4 41 296 3 27 91.8 60.3 99.0 93.2 73.2 Fold-5 69 293 6 0 98.4 100 98.0 92.0 95.8 Total/average 314 1417 76 27 94.4 92.1 94.9 80.5 85.9 They achieved 80% accuracy with VGG16 as their binary COVID-19 data. They found the detection accuracy classifier performances. They worked with 70 COVID of 95.18% with the confidence-aware anomaly detec- positives and 70 COVID negatives in total [26]. Khan et al. tion (CAAD) model [16]. Apostopolus et  al. obtained worked with normal, pneumonia-bacterial, pneumonia- an accuracy of 93.48% using a total of 224 COVID-19 viral and COVID-19 chest X-ray images. As a result, they data with the VGG-19 CNN model for their 3 classes achieved 89.6% overall performance with the model they (COVID-19–bacterial–normal) study [25]. Narin et  al. named CoroNet. They used 290 COVID-19 data. They used 50 COVID-19/50 normal data in their study, where worked with more COVID-19 data than many studies [21]. they achieved 98% accuracy with ResNet50 [31]. In many Medhi et al. achieved 93% overall performance value in their studies in the literature, researchers have studied a limited study using deep CNN. They worked with 150 pieces of number of COVID-19 data. In this study, the differentia- COVID-19 data [27]. tion performance of 341 COVID-19 data from each other In another study, Zhang and his colleagues performed was investigated with 3 different studies. In the study, 5 binary and multi-class classifications containing 106 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1217 0.98 0.9 0.96 0.998 0.8 0.94 0.996 20 22 24 26 28 30 InceptionV3 0.92 InceptionV3 ResNet50 0.7 ResNet50 ResNet101 ResNet101 ResNet152 0.9 ResNet152 InceptionResNetV2 InceptionResNetV2 0.6 0.88 05 10 15 20 25 30 05 10 15 20 25 30 Epoch Epoch Fig. 12 Binary Class-3: comparison of testing accuracy of 5 different Fig. 10 Binary Class-3: comparison of training accuracy of 5 differ - models for fold-4 ent models for fold-4 increase in the work density of radiologists. In these manual 0.4 InceptionV3 diagnoses and determinations, the expert’s tiredness may ResNet50 increase the error rate. It is clear that decision support sys- ResNet101 tems will be needed in order to eliminate this problem. Thus, 0.3 ResNet152 a more effective diagnosis can be made. The most impor - InceptionResNetV2 0.03 tant issue that restricts this study is to work with limited data. Increasing the data, testing it with the data in many 0.2 0.02 different centers will enable the creation of more stable sys- 0.01 tems. In future studies, the features will be extracted using image processing methods on X-ray and CT images. From 0.1 20 25 30 these extracted features, the features that provide the best separation between classes will be determined and perfor- mance values will be measured with different classification 51015202530 algorithms. In addition, the results will be compared with Epoch deep learning models. Apart from this, the results of the study will be tested with data from many different centers. In a future study, studies will be conducted to determine the Fig. 11 Binary Class-3: comparison of training loss values of 5 dif- ferent models for fold-4 demographic characteristics of patients and COVID-19 pos- sibilities with artificial intelligence-based systems. different CNN models were compared. The most important points in the study can be expressed as follows: ∙6 Conclusion There are no manual feature extraction, feature selection and classification in this method. It was realized end to end ∙ Early prediction of COVID-19 patients is vital to prevent directly with raw data. The performances of the COVID-19 data across normal, viral pneumonia and bacterial pneumo- the spread of the disease to other people. In this study, we ∙ proposed a deep transfer learning-based approach using nia classes were significantly higher. It has been studied with more data than many studies in the literature. ∙ It has chest X-ray images obtained from normal, COVID-19, ∙ bacterial and viral pneumonia patients to predict COVID- been studied and compared with 5 different CNN models. A high-accuracy decision support system has been proposed 19 patients automatically. Performance results show that to radiologists for the automatic diagnosis and detection of patients with suspected COVID-19 and follow-up. From another point of view, considering that this pan- demic period affects the whole world, there is a serious 1 3 Accuracy values Loss Accuracy values 1218 Pattern Analysis and Applications (2021) 24:1207–1220 Table 4 All performances of Models/fold Confusion matrix and Performance results (%) 5 different models on each fold for COVID-19/bacterial TP TN FP FN ACC REC SPE PRE F1 pneumonia binary classification InceptionV3 Fold-1 68 554 0 0 100 100 100 100 100 Fold-2 68 551 3 0 99.5 100 99.5 95.8 97.8 Fold-3 68 541 13 0 97.9 100 97.7 84.0 91.3 Fold-4 68 532 23 0 96.3 100 95.9 74.7 85.5 Fold-5 69 521 34 0 94.6 100 93.9 67.0 80.2 Total/average 341 2699 73 0 97.7 100 97.4 82.4 90.3 ResNet50 Fold-1 68 554 0 0 100 100 100 100 100 Fold-2 67 551 3 1 99.4 98.5 99.5 95.7 97.1 Fold-3 68 554 0 0 100 100 100 100 100 Fold-4 65 555 0 3 99.5 95.6 100 100 97.7 Fold-5 69 552 3 0 99.5 100 99.5 95.8 97.9 Total/average 337 2766 6 4 99.7 98.8 99.8 98.3 98.5 ResNet101 Fold-1 42 554 0 26 95.8 61.8 100 100 76.4 Fold-2 33 553 1 35 94.2 48.5 99.8 97.1 64.7 Fold-3 68 554 0 0 100 100 100 100 100 Fold-4 14 554 1 54 91.2 20.6 99.8 93.3 33.7 Fold-5 22 555 0 47 92.5 31.9 100 100 48.4 Total/average 179 2770 2 162 94.7 52.5 99.9 98.9 68.6 ResNet152 Fold-1 9 554 0 59 90.5 13.2 100 100 23.4 Fold-2 64 552 2 4 99.0 94.1 99.6 97.0 95.5 Fold-3 26 554 0 42 93.2 38.2 100 100 55.3 Fold-4 28 554 1 40 93.4 41.2 99.8 96.6 57.7 Fold-5 47 502 53 22 88.0 68.1 90.5 47.0 55.6 Total/average 174 2716 56 167 92.8 51.0 98.0 75.7 60.9 Inception-ResNetV2 Fold-1 60 540 14 8 96.5 88.2 97.5 81.1 84.5 Fold-2 39 552 2 29 95.0 57.4 99.6 95.1 71.6 Fold-3 66 547 7 2 98.6 97.1 98.7 90.4 93.6 Fold-4 34 551 4 34 93.9 50.0 99.3 89.5 64.2 Fold-5 42 536 19 27 92.6 60.9 96.6 68.9 64.6 Total/average 241 2726 46 100 95.3 70.7 98.3 84.0 76.8 ResNet50 pre-trained model yielded the highest accuracy early stage, this study gives insight on how deep transfer among five models for used three different datasets (Data- learning methods can be used. In subsequent studies, the set-1: 96.1%, Dataset-2: 99.5% and Dataset-3: 99.7%). In classification performance of different CNN models can be the light of our findings, it is believed that it will help tested by increasing the number of COVID-19 chest X-ray radiologists to make decisions in clinical practice due to images in the dataset. the higher performance. In order to detect COVID-19 at an 1 3 Pattern Analysis and Applications (2021) 24:1207–1220 1219 Table 5 The performance comparison literature about COVID-19 diagnostic methods using chest X-ray images Previous study Data type Methods/classifier Number of classes Accuracy (%) Narayan Das et al. [9] X-ray Xception 3 97.40 Singh et. al. [52] X-ray MADE-based CNN 2 94.65 ∓ 2.1 Afshar et al. [30] X-ray Capsule Networks 4 95.7 Ucar and Korkmaz [24] X-ray Bayes-SqueezeNet 3 98.26 Khan et al. [21] X-ray CoroNet 4 89.60 Sahinbas and Catak [26] X-ray VGG16, VGG19, ResNet 2 80 DenseNet and InceptionV3 Medhi et al. [27] X-ray Deep CNN 2 93 Zhang et al. [16] X-ray CAAD 2 95.18 Apostopolus et al. [25] X-ray VGG-19 3 93.48 Narin et al. [31] X-ray InceptionV3, ResNet50, Inception-ResNetV2 2 98 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Normal) 96.1 ResNet152, Inception-ResNetV2 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Viral Pne.) 99.5 ResNet152, Inception-ResNetV2 This study X-ray InceptionV3, ResNet50, ResNet101 2 (COVID-19/Bacterial Pne.) 99.7 ResNet152, Inception-ResNetV2 Author’s contribution AN and CK were involved in the study concep- 7. Jaiswal AK, Tiwari P, Kumar S, Gupta D, Khanna A, Rodrigues tion and acquisition of dataset. AN conducted the experiments and JJ (2019) Identifying pneumonia in chest X-rays: a deep learning analyzed the results. AN, CK and ZP wrote the manuscript and revised approach. Measurement 145:511–518 the draft critically. All authors reviewed the manuscript. 8. Antin B, Kravitz J, Martayan E (2017) Detecting pneumonia in chest X-rays with supervised learning. http:// cs229. stanf ord. edu/ proj2 017/ final- repor ts/ 52312 21. pdf Availability of data and materials Correspondence and requests for data 9. Narayan Das N, Kumar N, Kaur M, Kumar V, Singh D (2020) and materials should be addressed to C.K. Automated deep transfer learning-based approach for detection of COVID-19 infection in chest X-rays. IRBM. https:// doi. org/ Declaration 10. 1016/j. irbm. 2020. 07. 001 10. Ayan E, Ünver HM (2019) Diagnosis of pneumonia from chest Conflicts of interest The authors declare that they have no conflict of X-ray images using deep learning. In: Scientific meeting on elec- interest. trical-electronics and biomedical engineering and computer sci- ence (EBBT), Istanbul, Turkey, pp 1–5. https:// doi. org/ 10. 1109/ EBBT. 2019. 87415 82 11. Gaál G, Maga B, Lukács A (2020) Attention U-Net based adversarial architectures for chest X-ray lung segmentation. References arXiv:2003.10304 12. https:// tr. euron ews. com. Accessed 12 Dec 2020 1. Roosa K, Lee Y, Luo R, Kirpich A, Rothenberg R, Hyman JM et al 13. Liang G, Zheng L (2020) A transfer learning method with deep (2020) Real-time forecasts of the COVID-19 epidemic in China residual network for pediatric pneumonia diagnosis. Comput from February 5th to February 24th, 2020. Infect Dis Model Methods Programs Biomed 187:104964 5:256–263 14. Jaiswal A, Gianchandani N, Singh D, Kumar V, Kaur M 2. Yan L, Zhang H-T, Xiao Y, Wang M, Sun C, Liang J et  al (2020) Classification of the COVID-19 infected patients using (2020) Prediction of criticality in patients with severe COVID- DenseNet201 based deep transfer learning. J Biomol Struct 19 infection using three clinical features: a machine learning- Dyn. https:// doi. org/ 10. 1080/ 07391 102. 2020. 17886 42 based prognostic model with clinical data in Wuhan. medRxiv 15. Apostolopoulos ID, Mpesiana TA (2020) Covid-19: automatic 2020.02.27.20028027 detection from x-ray images utilizing transfer learning with con- 3. Stoecklin SB, Rolland P, Silue Y, Mailles A, Campese C, Simon- volutional neural networks. Phys Eng Sci Med 43:635–640 don A et  al (2020) First cases of coronavirus disease 2019 16. Zhang J, Xie Y, Li Y, Shen C, Xia Y (2020) COVID-19 screen- (COVID-19) in France: surveillance, investigations and control ing on chest X-ray images using deep learning based anomaly measures, January 2020. Eurosurveillance 25(6):2000094 detection. arXiv:2003.12338v1 4. Coronavirus (2020) World Health Organization. https://www .who. 17. Singh D, Kumar V, Kaur M (2020) Classification of COVID-19 int/ health- topics/ coron avirus patients from chest CT images using multi-objective differen- 5. McKeever A (2020) Here’s what coronavirus does to the body. tial evolution-based convolutional neural networks. Eur J Clin National geographic. https:// www. natio nalge ograp hic. com/ scien Microbiol Infect Dis 39:1379–1389. https:// doi. org/ 10. 1007/ ce/ 2020/ 02/ here- is- what- coron avirus- does- to- the- body/ s10096- 020- 03901-z 6. COVID-19 Coronavirus Pandemic (2020) Worldometer. https:// www. world omete rs. info/ coron avirus/ 1 3 1220 Pattern Analysis and Applications (2021) 24:1207–1220 18. Chen X, Yao L, Zhang Y (2020) Residual attention U-Net for 36. Saba T, Mohamed AS, El-Affendi M, Amin J, Sharif M (2020) automated multi-class segmentation of COVID-19 chest CT Brain tumor detection using fusion of hand crafted and deep learn- images. arXiv:2004.05645v1 ing features. Cogn Syst Res 59:221–230 19. Adhikari NCD (2020) Infection severity detection of CoVID19 37. Dorj UO, Lee KK, Choi JY, Lee M (2018) The skin cancer classi- from X-rays and CT scans using artificial intelligence. Int J fication using deep convolutional neural network. Multimed Tools Comput (IJC) 38(1):73–92 Appl 77(8):9909–9924 20. Alqudah AM, Qazan S, Alqudah A (2020) Automated systems 38. Kassani SH, Kassani PH (2019) A comparative study of deep for detection of COVID-19 using chest X-ray images and light- learning architectures on melanoma detection. Tissue Cell weight convolutional neural networks. https://doi. or g/10. 21203/ 58:76–83 rs.3. rs- 24305/ v1 39. Ribli D, Horváth A, Unger Z, Pollner P, Csabai I (2018) Detecting 21. Khan AI, Shah JL, Bhat MM (2020) Coronet: a deep neural and classifying lesions in mammograms with deep learning. Sci network for detection and diagnosis of COVID-19 from chest Rep 8:4165 X-ray images. Comput Methods Programs Biomed 196:105581 40. Celik Y, Talo M, Yildirim O, Karabatak M, Acharya UR (2020) 22. Ghoshal B, Tucker A (2020) Estimating uncertainty and inter- Automated invasive ductal carcinoma detection based using deep pretability in deep learning for coronavirus (COVID-19) detec- transfer learning with whole-slide images. Pattern Recogn Lett tion. arXiv:2003.10769v1 133:232–239 23. Hemdan EED, Shouman MA, Karar ME (2020) COVIDX-Net: 41. Yu X, Kang C, Guttery DS, Kadry S, Chen Y, Zhang Y-D (2020) a framework of deep learning classifiers to diagnose COVID-19 ResNet-SCDA-50 for breast abnormality classification. In: IEEE/ in X-ray images. arXiv:2003.11055 ACM transactions on computational biology and bioinformatics, 24. Ucar F, Korkmaz D (2020) COVIDiagnosis-Net: deep Bayes- pp 1–8. https:// doi. org/ 10. 1109/ TCBB. 2020. 29865 44 SqueezeNet based diagnosis of the coronavirus disease 2019 42. Jmour N, Zayen S, Abdelkrim A (2018) Convolutional neural (COVID-19) from X-ray images. Med Hypotheses 140:109761 networks for image classification. In: International conference on 25. Apostolopoulos ID, Aznaouridis SI, Tzani MA (2020) Extract- advanced systems and electric technologies (IC\_ASET), Ham- ing possibly representative COVID-19 Biomarkers from X-ray mamet, Tunisia, pp 397–402 images with deep learning approach and image data related to 43. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature pulmonary diseases. J Med Biol Eng 40:462–469 521:436–444. https://www .natio nalg eog raphic. com/ scien ce/ 2020/ 26. Sahinbas K, Catak FO (2020) Transfer learning based convolu-02/ here- is- what- coron avirus- does- to- the- body/2 tional neural network for COVID-19 detection with X-ray images. 44. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov https:// www. ozgur catak. org/ files/ papers/ covid 19- deep- learn ing. R (2014) Dropout: a simple way to prevent neural networks from pdf overfitting. J Mach Learn Res 15:1929–1958 27. Jamil M, Hussain I (2020) Automatic detection of COVID-19 45. Pan SJ, Yang Q (2009) A survey on transfer learning. IEEE Trans infection from chest X-ray using deep learning. medRxiv. https:// Knowl Data Eng 22(10):1345–1359 doi. org/ 10. 1101/ 2020. 05. 10. 20097 063 46. Lu J, Behbood V, Hao P, Zuo H, Xue S, Zhang G (2015) Transfer 28. Barstugan M, Ozkaya U, Ozturk S (2020) Coronavirus (COVID- learning using computational intelligence: a survey. Knowl Based 19) classification using CT images by machine learning methods. Syst 80:14–23 arXiv:2003.09424 47. Huang Z, Pan Z, Lei B (2017) Transfer learning with deep con- 29. Punn NS, Agarwal S (2020) Automated diagnosis of COVID-19 volutional neural network for sar target classification with limited with limited posteroanterior chest X-ray images using fine-tuned labeled data. Remote Sens 9(9):907 deep neural networks. https:// www. who. int/ health- topics/ coron 48. Wu Z, Shen C, Van Den Hengel A (2019) Wider or deeper: revis- avirus6 iting the ResNet model for visual recognition. Pattern Recogn 30. Afshar P, Heidarian S, Naderkhani F, Oikonomou A, Plataniotis 90:119–133 KN, Mohammadi A (2020) COVID-CAPS: a capsule network- 49. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S et al based framework for identification of COVID-19 cases from X-ray (2015) ImageNet large scale visual recognition challenge. Int J images. arXiv:2004.02696v2 Comput Vis 115:211–252 31. Narin A, Kaya C, Pamuk Z (2020) Automatic detection of coro- 50. Ahn JM, Kim S, Ahn KS, Cho SH, Lee KB, Kim US (2018) A navirus disease (COVID-19) using X-ray images and deep con- deep learning model for the detection of both advanced and early volutional neural networks. arXiv:2003.10849v1 glaucoma using fundus photography. PLoS ONE 13(11):e0207982 32. Cohen JP, Morrison P, Dao L (2020) COVID-19 image data col- 51. Byra M, Styczynski G, Szmigielski C, Kalinowski P, Michalowski lection. arXiv:2003.11597 L, Paluszkiewicz R et al (2018) Transfer learning with deep con- 33. Wang X, Peng Y, Lu L, Lu Z, Bagheri M, Summers RM (2017) volutional neural network for liver steatosis assessment in ultra- ChestX-Ray8: hospital-scale chest X-ray database and bench- sound images. Int J Comput Assist Radiol Surg 13(12):1895–1903 marks on weakly-supervised classification and localization of 52. Singh D, Kumar V, Yadav V, Kaur M (2020) Deep convolutional common thorax diseases. In: IEEE conference on computer vision neural networks based classification model for COVID-19 infected and pattern recognition (CVPR), Honolulu, pp 3462–3471. https:// patients using chest X-ray images. Int J Pattern Recognit Artif doi. org/ 10. 1109/ CVPR. 2017. 369 Intell. https:// doi. org/ 10. 1142/ S0218 00142 15100 46 34. Mooney P (2018) Chest X-ray images (Pneumonia). Kaggle Repository. https:// www. kaggle. com/ pault imoth ymoon ey/ chest- Publisher’s Note Springer Nature remains neutral with regard to xray- pnemo nia jurisdictional claims in published maps and institutional affiliations. 35. Yildirim O, Talo M, Ay B, Baloglu UB, Aydin G, Acharya UR (2019) Automated detection of diabetic subject using pre-trained 2D-CNN models with frequency spectrum images extracted from heart rate signals. Comput Biol Med 113:103387 1 3

Journal

Pattern Analysis and Applications – Pubmed Central

Published: May 9, 2021

There are no references for this article.