Detection of COVID-19 Infection in CT and X-ray images using transfer learning approach

Tiwari, Alok; Tripathi, Sumit; Pandey, Dinesh Chandra; Sharma, Neeraj; Sharma, Shiru

doi:10.3233/THC-220114

Detection of COVID-19 Infection in CT and X-ray images using transfer learning approach

Article type: Research Article

Authors: Tiwari, Alok^{a; 1} | Tripathi, Sumit^{b; 1; *} | Pandey, Dinesh Chandra^c | Sharma, Neeraj^a | Sharma, Shiru^a

Affiliations: [a] School of Biomedical Engineering, Indian Institute of Technology (Banaras Hindu University), Varanasi, India | [b] Department of Electronics and Communication Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India | [c] Department of Management Studies, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India

Correspondence: [*] Corresponding author: Sumit Tripathi, Department of Electronics and Communication Engineering, Graphic Era Deemed to be University, Dehradun, Uttarakhand, India. E-mail: sumitt.rs.bme17@itbhu.ac.in.

Note: [1] Sumit Tripathi and Alok Tiwari contributed equally to this paper.

Keywords: Transfer learning, COVID-19, MobileNet V2, deep learning

DOI: 10.3233/THC-220114

Journal: Technology and Health Care, vol. 30, no. 6, pp. 1273-1286, 2022

Received 11 March 2022

Accepted 2 August 2022

Published: 12 November 2022

Get PDF

Abstract

BACKGROUND:

The infection caused by the SARS-CoV-2 (COVID-19) pandemic is a threat to human lives. An early and accurate diagnosis is necessary for treatment.

OBJECTIVE:

The study presents an efficient classification methodology for precise identification of infection caused by COVID-19 using CT and X-ray images.

METHODS:

The depthwise separable convolution-based model of MobileNet V2 was exploited for feature extraction. The features of infection were supplied to the SVM classifier for training which produced accurate classification results.

RESULT:

The accuracies for CT and X-ray images are 99.42% and 98.54% respectively. The MCC score was used to avoid any mislead caused by accuracy and F1 score as it is more mathematically balanced metric. The MCC scores obtained for CT and X-ray were 0.9852 and 0.9657, respectively. The Youden’s index showed a significant improvement of more than 2% for both imaging techniques.

CONCLUSION:

The proposed transfer learning-based approach obtained the best results for all evaluation metrics and produced reliable results for the accurate identification of COVID-19 symptoms. This study can help in reducing the time in diagnosis of the infection.

1.Introduction

The World Health Organisation (WHO) declared the infection caused by severe acute respiratory syndrome coronavirus (SARS-CoV-2) a global pandemic as the infection spread rapidly throughout the globe. SARS-CoV-2 was renamed COVID-19 by the WHO on Feb 11, 2020 [1]. The presence or absence of protective immunity due to infection or vaccination will affect future transmission and illness severity [2]. The issues caused by COVID-19 are highly complex, which can lead to the severe infections of the lungs and can spread to other organs. The symptoms of COVID-19 take 11–12 days to develop after infection [3, 4]. The timely identification of COVID-19 is much necessary else; it can cause severe issues to humans. The radiographic imaging tools for detection of COVID-19 are Computed Tomography (CT) and X-ray [5]. The computer-based analysis is quite demanding for the analysis of this infection as there is a lot of burdens nowadays on the medical system. In such a situation, computer analysis can save the time and burden of the doctors. The researchers are working across the globe for finding the best feasible solution to accurately detect the COVID-19 infection using computer-based analysis. The infections caused by viral pneumonia infections and COVID-19 are almost comparable [6]. Thus the detection of COVID-19 becomes a tedious task as features of COVID-19 overlaps with the other inflammatory diseases of the lung, including different types of pneumonia. The incorrect detection of COVID-19 may lead to the danger of life. Machine Learning-based methods are detecting the infection quite accurately as proposed in the available literature, but these approaches need handcrafted features for training [7, 8]. The features are extracted using some predefined methods and are then supplied to the classifier for learning [9].

Deep learning-based approaches [10, 11, 12] are gaining considerable attention for computer-based detection of COVID-19 as these approaches are capable of extracting the features on their own and produce more reliable result [13, 14]. Performing the specific task in one domain by using the knowledge acquired from other domain is termed transfer learning [15]. In transfer learning-based approaches, the knowledge is transferred from a source to target for improving the task-specific efficiency of the target [17].

The transfer learning-based methods are categorized into three categories, namely 1) Inductive transfer learning, 2) Transductive transfer learning, and 3) Unsupervised transfer learning. The task of source and target are same in inductive transfer learning while in transductive learning, the task of source and target are different. The unsupervised transfer learning is same as inductive learning, but the target tasks are unsupervised. The machine learning and deep learning approaches which utilized transfer learning showed remarkable improvement in the classification task. These methods reveal the features of the infection, which are not clearly visible in the original images as a better understanding of pathophysiology related to COVID-19 is important to improve the treatment process.

The convolutional neural network (CNN) architectures have shown good efficiency in classifying the infected images [18]. CNN-based binary classifier was used by Asnaoui and Chaw [19] to classify the CT images for detecting the infection. RCNN-based classification technique was proposed by Zreik et al. [20] to classify the coronary artery plaque. Li et al. developed the Cov-Net model to extract the features of COVID-19 and showed a remarkable improvement in the classification of infected images. Melanoma dermoscopy images were classified by Hagerty et al. [21] with remarkable accuracy using the deep learning-based method. VGG-16-based model for identification of lung infection was proposed by Li et al. [22]. Identification of thoracic diseases was made with the incorporation of DenseNet model [23]. An Inception Net-based model was used by Wang et al. [24] to identify abnormalities of COVID-19. An accuracy of 85.20% was achieved by classifying 1065 CT images. Resnet-based transfer learning technique was proposed by Ayrton, who achieved validation accuracy of about 96%. U-Net ++-based model was used by Chen et al. [25] for covid-1p classification, and the accuracy achieved was 98.85%. The pretrained U-net architecture for segmentation of 3D Lung images was used by Zheng et al. [26] and then they applied the DL method for determination of COVID-19 infection. The accuracy obtained by them was 95.9%. A CT analysis method for detection of COVID-19 infection using artificial intelligence was proposed by Gozes et al. [27]. Recently, DCNN-based structure was presented by Linda et al. [28] for detection of COVID-19 from chest X-ray images. The proposed model classified X-ray and CT images with significant increment in the results as compared with other methods. The results were compared using various evaluation metrics. To avoid the ambiguity caused by some metrics, additional metrics are used to confirm the effectiveness of results. Accuracy and F score sometimes misleads the classification judgment [29], so Matthews correlation coefficient (MCC) is used for better understanding [30]. The proposed classifier learns the features with the help of transfer learning and produces remarkable results for both CT as well as X-ray images. The main advantage of the proposed technique is the reduction in training time and requirement of fewer data along with remarkable improvement in image evaluation statistics. Finding a faster method to identify COVID-19 infected images is the driving force behind our endeavour. In order to save medical professionals’ time, it can aid in the automatic detection of COVID-19 images.

2.Methodology

The CT and X-Ray images in the presented work are classified using a transfer learning-based method because both image modalities are utilised to identify COVID-19 symptoms. In this work, the SVM classifier [29] is utilised for classification. The features of the images were extracted using MobileNet V2 architecture. The SVM classifier efficiently classified the infected images after receiving the derived features. The feature extraction process also utilised a number of different networks for comparison. As a comparison tool, the SVM classifier also received the characteristics that were retrieved by several network designs. Transfer learning was used to expedite the classification of photos that were important from a medical standpoint. The pipeline of the suggested work is depicted in Fig. 2.

Figure 1.

Graphical abstract.

Figure 2.

Pipeline of the proposed work.

Transfer learning eliminates the requirement to train the classifier from scratch since it makes use of the knowledge amassed by a network on a large training dataset with labels that are readily available [30]. In the field of machine and deep learning, transfer learning is a type of design process. Deep learning models typically extract the features of edges in the earliest layers for applications linked to computer vision. The specific features linked to the particular purpose for which the network is designed are extracted by the intermediary layers. Typically, the last layers in a transfer learning approach are taught. This method has the benefit of saving time while training large networks and removing the requirement for big training datasets. Thus, the expertise acquired by one network to do a particular task is utilised to complete a different task. We trained an SVM classifier for the classification of CT and X-ray images infected by COVID-19 using the knowledge gathered by the MobileNet model and other state of the art models for comparison. The entire transfer learning process is depicted in Fig. 3.

Figure 3.

Feature transfer using transfer learning.

Figure 4.

Depthwise separable convolution block.

Figure 5.

MobileNet V2 architecture for classification.

2.1MobileNet V2

The next-generation architecture for all-purpose computer vision applications is called MobileNet V2 [31] and was created by Google Brain. Low power and low latency models from the MobileNet family are employed to complete a variety of computer vision use cases. The depthwise separable convolution [33] is the fundamental building element for MobileNet V1 [32]. The linear bottlenecks between the layers and the shortcut connections between the bottleneck blocks are included in MobileNet version 2. The in-between inputs and outputs are encoded by the bottlenecks. The middle layers are responsible for converting lower-level components like pixels to higher-level images like images. Figure 5 depicts the MobileNet V2 bottleneck block that is still in place. Expansion Layer, Depth Wise Convolutional Layer, and Projection Layer are the names of the layers. Depth-wise convolution is incorporated, and each input channel receives a single filter. A pointwise 1 × 1 convolution is used to combine the depthwise convolution results. Depthwise separable convolution is the name given to this combination filtering and combining method. Figure 4 illustrates how the depthwise separable convolution divides this into two layers, one for filtering and one for combining.

Depth wise convolution with one filter per input channel (input depth) can be written as:

(1)

G^k,l,m=K^i,j,m.Fk+i-1,l+j-1,m

Where K^ is the depthwise convolutional kernel of size DK×DK×M where the mth filter in K^ is applied to the mth channel in F¯ to produce the mth channel of the filtered output feature map G^. has The convolution cost of depthwise separable convolution is given by the following equation:

(2)

Cost Input=DK.DK.M.DF.⁢DF+M.N.DF.⁢DF

Equation (2) is the sum of depthwise convolution and 1 × 1 pointwise convolution where DF×DF is the pointwise convolution kernel. The depth-wise separable convolution is represented mathematically altogether in this equation. First part of the equation is depth wise convolution and second part is pointwise convolution. The 1 × 1 pointwise convolution layer makes the number of channels smaller and is also known as Projection Layer and Bottleneck. The amount of data flowing over the network is decreased. In each layer, Batch Normalization and activation function Relu6 [31] is incorporated. For low precision applications, RELU6 is more robust than normal RELU. The MobileNet architecture as shown in Fig. 5, consists of 17 layers of building blocks consecutively which are followed by 1 × 1 convolutional layer, global average pooling layer, fully connected layer, SoftMax layer and a classification layer. One feature map is produced by the global average pooling layer for each associated classification category. By calculating the mean of the input’s height and width parameters, it conducts down-sampling. The global average pooling layer is positioned before the fully connected layer for image classification tasks in order to minimise the amount of the activations without compromising classification performance. When the activation size is decreased, the network becomes smaller because the downstream fully linked layers will have less weights. The data are finally divided into several classes by the classification layer.

The SVM classifier receives the MobileNetV2 features that were finely extracted. The prime strength of the SVM is its non-probabilistic nature. It separates data across a decision boundary determined by only a small subset of the data. The subset of data which supports the decision boundary are known as support vectors. The other data does not have any effect in determining the position of the boundary. The SVM problem can be understood as a basic optimization problem in which a decision boundary is evaluated which gives maximum distance between the classes. It first finds the hyperplane that has the maximum margin and this hyperplane is used to predict the class of new data object. The equation of hyperplane is given as:

w.x=0

The polynomial kernel function was used to train the SVM. The data was split in to the ratio of 80:20 for training and testing. The MobileNet V2 extracted the features of COVID-19 from CT as well as X-Ray images. Both types of image features extracted by MobileNet V2 were fed to the SVM classifier for training. The SVM effectively classified the COVID-19 infected images for both modalities, i.e. CT and X-ray. Other five networks were also used to supply the extracted features to the SVM classifier for identification of images. These networks are Shufflenet, Alexnet, Darknet, Resnet and Inception Net V3. The presented work clearly finds the better method among the compared methods for better identification of COVID-19 infected images. SVM was used as a common platform for all networks. Thus better network can be judged based on the results provided by the SVM. According to the pathological studies of the lung on the autoptic tissues for COVID-19 and viral pneumonia, the widespread thrombosis with microangiopathy and vascular angiogenesis are the main discriminators for texture variations which are efficiently captured by the proposed method [22, 27].

3.Dataset used

3.1X-ray dataset

The dataset used in this paper is obtained from the COVID-19 radiography database of Kaggle [32] which consists of 219 chest X-ray images of COVID-19 positive patients, 1341 images of normal chest X-rays and 1345 images of viral pneumonia infection. In addition to this, 301 images were taken from the GitHub repository of Cohen [33]. To increase the data for training, we augmented the data using data augmentation techniques as the deep learning networks need a large amount of dataset to be trained. The data was rotated at the angles of 300, 600, 900, 1200, 2700 and 3000. In total, we created 3120 COVID-19 infected images, 8046 normal images and 8070 viral pneumonia infected images. This augmented data were divided into training and testing.

3.2CT dataset

The CT dataset is acquired from CT scans for COVID-19 classification database of Kaggle [34]. The database consists of images collected from Union Hospital (HUST-UH) and Liyuan hospital (HUST-LH). The database consists of 5705 non-informative CT (NiCT) images, 4001 positive CT (pCT) images and 9979 negative CT (nCT). Figure 6 shows the sample images of COVID-19 infection.

Figure 6.

Sample images of COVID (+) infection. First row shows X-ray images and second row shows CT images.

The dataset was resized according to the input requirement of each network. For Mobilenet V2, Shufflenet and Resnet the images were resized to 224 × 224. For Alexnet, Darknet and Inception V3 the images were resized to 227 × 227, 256 × 256, 229 × 229 respectively.

4.Results

Table 1

Evaluation metric comparison of CT images

Networks	Accuracy	F1	F2	MCC
Mobilenet V2 [45]	0.9942	0.99	0.9890	0.9852
Alexnet [46]	0.9611	0.9391	0.9643	0.9125
Darknet [47]	0.9805	0.9716	0.9834	0.9606
InceptionV3 [48]	0.9761	0.9635	0.9740	0.9461
Shufflenet [49]	0.9817	0.9702	0.9777	0.9587
Resnet [50]	0.9756	0.9629	0.9707	0.9448

Table 2

Evaluation metric comparison of X-Ray images

Networks	Accuracy	F1	F2	MCC
Mobilenet V2	0.9854	0.9770	0.9815	0.9657
Alexnet	0.9556	0.9298	0.9601	0.9001
Darknet	0.9705	0.9587	0.9540	0.9379
InceptionV3	0.9667	0.9483	0.9683	0.9249
Shufflenet	0.9702	0.9588	0.9542	0.9380
Resnet	0.9611	0.9391	0.9643	0.9125

Figure 7.

Evaluation metrics comparison of CT images.

Figure 8.

Evaluation metrics comparison of X-Ray images.

This section presents the analysis of the comparative results obtained by the proposed and other existing classifiers models. The models used for comparison are Alexnet, Resnet101, Inception V3, Darknet and ShuffleNet. The results obtained are shown in three categories 1) The comparison of accuracy, F scores and MCC, 2) The parameters whose value should be minimum for achieving good performance and, 3) other parameters for firm confirmation of results obtained. The results are categorised into the above-mentioned categories to make them more suitable for analysis. The accuracy, F scores and MCC are evaluated together because the literature review suggests that accuracy and F scores can sometimes mislead the evaluation of classification results. So MCC is used to confirm the reliability of the results as mathematically it takes all four parameters of confusion matrix in a more balanced manner. The second category is of those parameters which should attain minimum value to confirm the effectiveness of classification. Six more evaluation metrics are used for the firm affirmation of results. Tables 1 and 2 show the accuracy, F scores and MCC values obtained by the classifiers for CT and X-Ray images, respectively. Figures 7 and 8 show the results obtained by other parameters which confirm the robustness of results obtained by the proposed method for both modalities.

5.Discussion

The prime focus of this research is to detect the COVID-19 infection through computer-aided analysis. The accurate analysis and detection of COVID-19 infection are quite necessary in the current scenario as the pandemic has placed the whole world in an unprecedented situation by its life-threatening effects. The classifiers segregate the data into COVID (+) and COVID (-) classes where COVID (+) is the patient infected by COVID-19 and COVID (-) is the patient not having the symptoms of infection. The prediction of infection is made on the basis of the four-parameter values of the confusion matrix, i.e. True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN). The TP and TN values identify COVID (+) and COVID (-) patients. FP represents the outcomes which are incorrectly recognised as COVID (+) while FN depicts the outcome which is incorrectly classified as COVID (-).

5.1Quantitative analysis

Accuracy is a measure of accurate identification of the class of a pixel. The evaluation of correctly identified pixels is made using accuracy. Accuracy is computed using the following equation:

(3)

𝐴𝑐𝑐𝑢𝑟𝑎𝑐𝑦=T⁢P+T⁢NT⁢P+T⁢N+F⁢P+F⁢N

The accuracy for both modalities shows an improvement of more than 1.5%. The distribution of data is not taken into account by the accuracy so the F-score [35] is a better parameter for judging the classification efficiency. It is given by the following equation:

(4)

F⁢1⁢ Score=2⁢P⁢RP+R

and

(5)

F⁢0.5⁢ Score=1.25⁢P⁢R0.25⁢P+R

Where P and R are the Precision and Recall values. The F1-score is defined as the harmonic mean of the precision and recall values with a distance error tolerance. It is a measure of a model’s accuracy on a dataset. F0.5 score has the effect of raising the importance of precision and lowering the importance of recall. The F1 score maintains the balance of precision and recall values has shown an improvement of 1.8% for CT images and 1.9% for X-ray images. The f 0.5 score, which gives more importance to precision as desired for medical analysis is also improved by 1.25 % and 1.4% for CT and X-ray images respectively. Accuracy and F1 measure can sometimes mislead the result interpretation because they fail to consider the ratio of positive and negative elements. The MCC is used to firmly quantitative the analysis of classification as the mathematical properties of MCC handles both dataset imbalance and their invariants effectively. It gives correct predictions for both majorities of the negative cases, and positive cases, independently of their ratios in the overall dataset. The following equation describes MCC mathematically [36]:

(6)

𝑀𝐶𝐶=T⁢P∗T⁢N-F⁢P∗F⁢N(T⁢P+F⁢P)⁢(T⁢P+F⁢N)⁢(T⁢N+F⁢P)⁢(T⁢N+F⁢N)

MCC has shown a remarkable improvement of respectively 2.6% and 3.8% for CT and X-ray images. This signifies the better classification ability of the proposed model as the MCC has achieved the highest score along with the accuracy and F-scores.

For more analysis and a better understanding of results, we used six additional evaluation metrics for examining the obtained results. Precision, Recall, Specificity, FM index, Geometric mean and Youden’s index values are used to compare the results of classification by various methods. Precision [37] which is also known as Positive prediction value (PPV) is the ratio of positive samples which were correctly classified to the total number of samples predicted as positive. It is given by the following equation:

(7)

𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛=T⁢PT⁢P+F⁢P

Recall or sensitivity is given as the ratio of positive correctly classified samples to the total number of positive samples. It is also known as true positive rate (TPR) and hit rate. The following equation represents recall:

(8)

𝑅𝑒𝑐𝑎𝑙𝑙=T⁢PT⁢P+F⁢N

The recall is the ability of a model to find all relevant cases within a dataset while the ability of a classification model to identify only the relevant data points is termed precision. A good margin improves the values of precision and recall in both image modalities. Specificity, which is also termed inverse recall or true negative rate (TNR) is expressed as the fraction of correctly classified negative samples to the total count of negative samples. Specificity is expressed mathematically as:

(9)

𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦=T⁢NF⁢P+T⁢N

Fowlkes-Mallows Index (FM Index) [38] gives a more accurate representation of unrelated data. The higher value of FM Index gives more significant similarity between classified and ground truth data. It is given by the following equation:

(10)

F⁢M=T⁢PT⁢P+F⁢P∗T⁢PT⁢P+F⁢N

The prime goal of the classification process is to improve sensitivity without sacrificing the specificity. This is quite a tough task for imbalanced datasets. Thus Geometric Mean (GM) [39] represents the aggregation of both metrics using the following equation:

(11)

G⁢M=T⁢PT⁢P+F⁢N∗T⁢NT⁢N+F⁢P

The values obtained by the Specificity, FM and GM has shown considerable improvement shown in the figure. The higher values of sensitivity and specificity indicate that the true values of positive and negative classified samples are greater in the results obtained by the classifier. Thus the classifier is said to have predicted the COVID (+) and COVID (-) samples more accurately than others if these two scores have larger values than other methods in comparison. Fowlkes-Mallows Index (FM), which gives a more accurate description of unrelated data shows a good degree of improvement for classification of COVID-19. The last evaluation metric Youden’s ndex or Bookmaker Informedness is an important metrics for medical image analysis as it evaluates the discriminative power of the diagnostic test. It evaluates the probability of informed decision and is more suitable for datasets with imbalanced classes. Youden’s index [40] shows an increment of more than 2% for both CT and X-ray images. Mathematically it is represented by the following equation:

(12)

J=T⁢PT⁢P+F⁢N+T⁢NT⁢N+F⁢P-1

Where J represents Youden’s index. The proposed classifier performs best, as suggested by each evaluation metric. Moreover, it works equally well for both types of medical imaging modality, i.e. CT and X-ray. It takes advantage of transfer learning for training and is more immune to false results predictions.

The proposed method classified the COVID-19 infected images with good accuracy which is supported by several other evaluation metrics [41, 42, 43, 44]. This method is time efficient and accurate as compared with the several methods. Moreover, the computation cost is less as compared to other methods because of depth wise separable convolution. Lesser number of convolution multiplications tends to save the spatial resolution of the patterns in the images.

6.Conclusion

The COVID-19 pandemic has caused a large number of deaths across the globe. The early and accurate diagnosis is necessary for the treatment of this infection. The current need in the diagnosis of COVID-19 is to save the precious time of doctors and to reduce the burden on the present medical system. In conclusion, a transfer learning-based approach is presented in this work for accurate identification of COVID-19 in CT and X-ray images. This work proposes a computer-aided analysis of accurate detection of COVID-19 infected images. The features finely extracted by MobileNet V2 model paved the way for this research. The obtained results were evaluated using the well-known evaluation metrics, and to the best of our knowledge, no study till now has not compared their method using such a large number of metrics. The advantage of this comparison is that we can be quite sure about the reliability of our results. MCC was used because sometimes accuracy and F score misleads the prediction of classification as mathematically it is a perfect balance between all four parameters of the confusion matrix. Youden’s Index, which is quite a trustworthy statistical measure for the medical image analysis, shows outstanding improvement, which proves the medical effectiveness of the results obtained by the proposed model. The CT scan and X-ray facilities are common in most medical organisations. Thus the proposed method can help for initial identification of the The infections caused by viral pneumonia infections and COVID-19 are almost comparable disease. We believe that with the increase in the dataset, the effectiveness of the classification process can be further improved.

Funding

The authors did not receive funding to carry out this research.

Availability of data and materials

Not applicable.

Code availability