MCSC-Net: COVID-19 detection using deep-Q-neural network classification with RFNN-based hybrid whale optimization

2.1Survey on CXR segmentation

The COVID-19 region can be localized using the CXR image segmentation techniques described in this section, which also aids in classifying COVID-19. L. O. Teixeira et al. [27] developed CNN architectures for segmentation and classification. The UNet is utilized to perform segmentation, and the VGG16, ResNet50V2, and InceptionV3 are used as classifiers. However, there are computational challenges with this approach. L. Zhang et al. [28] implemented a modified U-Net model with dual encoder fusion, namely DEFU-Net, for CXR image segmentation with accuracy of 98.04%. The computational complexity increases as the network’s depth increase due to the densely connected recurrent CNN blocks. The FractalCovNet, the combination of fractal blocks and the U-Net, is developed by H. Munusamy et al. [23] for the localization of the lesion regions in the CXR and CT images. Further, the FractalCovNet with transfer learning approach is presented to classify the CXR images and suffers from the complexities. The pneumothorax boundary segmentation is perceived effectively by the Deep Signed Distance Map (DeepSDM) proposed by Y. Wang et al. [50]. The classification of CXRs using DeepSDM resulted in missed diagnosis and lower accuracy. S.Tabik et al. [40] presented a COVID-SDNet to improve detection accuracy by combining segmentation, data transformation, and augmentation. Class inherent-based methods are used to differentiate the classification capability of the classifiers. However, these inherence approaches yield lower classification accuracy. S. Motamed et al. [41] implemented a transfer learning (TL) approach for segmenting the lungs in the CXRs and presented randomized GAN (RANDGAN) to identify the images of the unknown class in CXRs. But these methods exhibit low accuracy with high computational complexity.

A variational auto-encoder (VAE) is introduced in the encoder-decoder of UNet to perform the lung lobes segmentation in CXR images by F. Cao et al. [21]. The extracted features are recognized highly due to the attention mechanism. The opacities are affecting the accuracy of segmenting in the majority of the lung regions. M. Kim et al. [32] developed a deep-learning neural network with a self-attention (SADNN) concept to perform automatic lung segmentation in CXR images by modifying the U-Net architecture. However, the segmentation performance depends on the dataset, and this method fails in achieving the segmentation if the dataset contains deformed shapes or lesions in the lungs. The fast and efficient multi-task DL (MTDL) approaches with COVID_MTNet is developed to diagnose COVID-19 in CT and CXRs by M. Z. Alom et al. [33]. These approaches utilize the NABLA-N model to perform the segmentation of both images. But the classification accuracy of the CXR dataset is as low as 84.67%. A Structure Correcting (SA) Network based on the GAN is presented to perform segmentation of lungs in CXR images [22]. This SA network has a similar function to the general adversarial network (GAN). Attention-based-UNet is the backbone network for SAGAN. Three classes are employed to categorize the CXR images using a fully connected network. But the classification accuracy and the dice scores could be higher. The detection of COVID-19 in CXRs using deep TL networks such as ResNet, InceptionV, and the combination of these networks are developed by A. Narin et al. [7]. Three different datasets were analyzed to know the capacity of the five pre-trained models: transfer-learned CNNs (TL-CNNs). Still, the detection accuracy is 96.1% on two class classification datasets. A novel DenseCapsNet is developed by fusing the Dense and CapsNet to classify the CXR images [24]. Before classification, the CXR images are segmented for the lung lobes by the ultimately selected TernausNet. The 4-class classification accuracy is low in the fused network.

2.2Survey on optimization with classification methods

The many feature selection techniques that use evolutionary and natural selection-based features are described in detail in this section. Compared to deep learning feature selections, these strategies produced better features. The classification of different classes in CXR images is proposed in two stages by the advanced squirrel search optimization (ASSOA). The ResNet-50 in the first stage extracts the features, and the CNN learns the features. In the multi-layer perceptron (MLP) neural network, the ASSOA chooses the features and optimizes the weights. The presented algorithm is computationally complex [17]. The ability of imbalanced exploitation and poor diversity with local optima is the significant problems associated with a meta-heuristic slime mold algorithm (SMA). The quasi-reflection with SMA (QRSMA) is developed to improve the performance of the SMA [42]. This combination increases the segmentation problems in CXR images. However, this method fails at the multiple objective optimization problems. The QRSMA performs the classification of CXR images with CNN. The projected algorithm is developed by introducing the concept of population reduction in modified whale optimization (mWOAPR), which performs CXR image segmentation to detect COVID-19 [43]. In classifying CXRs, the population reduction method with a support vector machine (SVM) produces higher accuracy. P. Bhowal et al. [38] presented VGG-16, Xception-Net, and InceptionV3-Net to detect COVID-19 by feature extractions in the CXR images. A two-tier process selects the features in CXRs. With a lower accuracy of 93.33%, the InceptionV3-Net extracts features. A. T. Sahlol et al. [8] used fractional-order, and marine predators swarm optimization techniques (FO-MPA) to identify COVID-19 CXR image features. The CNN classifies the selected features. The Aquila is a swarm optimization algorithm that selects the best features from the features extracted by the MobileNet-V3 in CXRs [34]. This method effectively reduces the dimensionality of the image representation while improving classification accuracy.

To find COVID-19 present in the CXR images, an optimized CNN is developed by optimizing the CNN with the grey wolf optimization (GWO) technique. The OptCoNet is the network designed to identify the CXRs for COVID-19 [46]. The GWO and the hyperparameters select the optimized features to train the CNN are optimized by the same GWO. However, the classification accuracy could be higher on the dataset used by this study. The XGBoost utilizes particle swarm optimization (PSO) to extract the deep features in the X-ray images to predict COVID-19 in CXRs. Deep feature extraction uses VGG-19, InceptionV3, and ResNet-50 DL networks [15]. The PSO-optimized XGBoost selects the best deep-optimized features. However, these methods of deep feature extraction are computationally complex. To detect COVID-19, the computer tomographic image (CT) features are retrieved and used with the CNN. The retrieved features are selected by the genetic algorithm (GA), and four different classifiers are presented to classify the best features [18]. E.-S. M. El-Kenawy et al. [19] presented a feature selection and classification algorithm to classify CT images for COVID-19. The AlexNet-CNN model extracts the features, and the guided GWO algorithm selects optimal features. Furthermore, a voting classifier is developed to classify the selected features and diagnose COVID-19. A. Al-zubidi et al. [9] proposed fuzzy c-mean (FCM) and back propagation-CNN (BP-CNN) to classify COVID-19. The features extracted affect the classification performance of the FCM and BPA. The information gain (IG) mechanism is introduced to improve the FCM and BP-CNN classification accuracy. T. Goel et al. [47] implemented a GAN network to generate sufficient images in the dataset. WOA is used to tune the hyperparameters of GAN to enhance classification performance. A CNN model optimized to detect COVID-19 from CXR images was given by S. Pathan et al. [44]. The GWO, whale, and BAT algorithms carry out the optimization of the CNN model. To optimize the classification results, the hyperparameters are modified automatically. In [10], SegNet, U-Net, Hybrid CCN, and their optimization using grey wolf optimization (GWO) are provided for semantic segmentation of CXR images. However, because of a lack of ground truth images, networks may need to be able to localize COVID-19 infection. In [11], a CXGNet is present to detect the COVID-19, but the classification accuracy is low. One major drawback of earlier research utilizing COVID-19 detection from CXR and CT images is their low detection accuracy. However, prior studies did not differentiate the distinct characteristics of the several classes of CXRs. This study aims to localize the specific features in the CXR classes and make the DL network suitable for the multi-class classification of diseases in the CXR images. The classification accuracy can be improved with the proposed network of this work due to the Disease-Specific Feature Separate Attention Mechanism (DSFSAM) to extract the separate features of each class.

3.1Preprocessing

Noise removal is a significant process in CXR recognition to improve classification accuracy. MCSC-Net employs the Hybrid Median Bilateral Filter (HMBF) algorithm to remove noise in the given image. The input image dimension is 256×256. The proposed HMBF algorithm removes noise and sharpens the image effectually. In addition, it also maintains the fine details of the image. The proposed HMBF algorithm removes the universal noises from the given image, such as impulse and Gaussian. In the HMBF approach, the noisy pixel is identified using the Sorted Quadrant Median Vector (SQMV) algorithm. It preserves the important features of the CXR image, which are edges and texture information.

The functional blocks of the HMBF-based noise removal are depicted in Fig. 4. The four sequential blocks in HMBF are Adaptive Median Filter (AMF), Edge detector, Noise detector, and Switching Bilinear Filter (SBF). The operation of each sequential block present in the proposed HMBF is demonstrated in the following subsections.

Fig. 4

Proposed HMBF approach.

3.2Segmentation

The segmentation is the process of localizing the COVID-19 region from CXR images, and it also identifies the spatial coordinates of COVID-19. Here, the segmentation operation is accomplished to increase the image quality and reduce the CXR image’s adverse effects. The segmentation operation is performed by enhancing the COVID-19 region illumination, which is normalized using the SC-ResNet50-based transfer learning model. Figure 5 represents the proposed SC-ResNet50 model’s block diagram. With the help of the residual convolution module (RCM) and identity mapping module (IMM), the SC-ResNet50 effectively addresses the issue of fading gradients. Each layer of the IMM and RCM contributes to mapping the residual model, which is used in this module to connect the input and output.

Fig. 5

Proposed SC-ResNet50 (a) segmentation process, (b) IMM, (c) RCM.

3.3Feature extraction

Multiple features are extracted from the segmented region using the Robust Feature Neural Network (RFNN) approach, and these particular disease-specific features are then extracted. The COVID-19, pneumonia bacterial, viral, and normal types play a vital role in classification operation because features of each disease type are dissimilar to others. Thus, the proposed deep learning-based RFNN model can extract the disease-specific and dependent features from segmented CXR images. The RFNN is capable of texture, shape, and spatial features. The RFNN descriptor is one of the feature descriptors in computer vision technology, which is partially inspired by the CNN descriptor. The deep learning models are popular for their computational speed and robustness to the illumination, scale, and rotation variations. Figure 6 shows the layer-wise structure of the RFNN model, which contains the nine series bocks for extracting the deep features. Further, RFNN encompasses four sequential layers in each block, such as feature extraction using CONV2D, DSFSAM, MaxPooling of features, and feature generation. Features can occasionally be single values that store the information in a pixel and are statical attributes in a vector format.

Fig. 6

RFNN feature extraction.

3.4Feature selection using HWOA

The Whale Optimization technique developed by S. Mirjalili and A. Lewis [45], it is an efficient evolutionary optimization approach that may be used to find the best solution. Whales are highly intelligent creatures that prefer to live in groups rather than alone. This is the most interesting section of the whales’ bodies. The seeking properties of whales allow for the most optimum solutions to be found using the WOA approach. Walruses have a unique hunting technique in that they like to catch little fish that are near to the surface of the water. The hybrid whale optimization algorithm (HWOA) is constructed by combining the WOA with opposition-based learning (OBL) [25]. The proposed HWOA flowchart is shown in Fig. 7, which is a new and efficient optimization technique. The steps involved in the proposed HWOA are presented in Table 2. It helps to increase the convergence speed and the accuracy of solutions. The OBL approach selects an opposite number and places it at the candidate solution’s mirror location. Because the opposite number is extremely close to the random number that leads to a solution, algorithms utilize less space while converging the answer to the problem. The opposite population is more likely to come up with a global solution when compared to a random population. In the following mathematical explanations, the HWOA may be expressed in various phases such as encircling, hunting, targeting prey, and exploration.

Fig. 7

Proposed HWOA flowchart.

3.5Classification

The HWOA-based optimal features are applied to the Deep-Q-Neural Network (DQNN) for classification. The proposed DQNN classifies texture features by implementing the Q-descriptor. The reason for selecting DQNN for texture feature extraction is that it performs better in extracting features from each image using different layers. The DQNN descriptor comprises three significant layers: DNN, Environment, and state layer, as depicted in Fig. 8. The Deep Neural Network (DNN) layer collects input from the input layer to derive the input image features. The convolution layer utilizes a number of filters to extract high-level features from the given image. The convolutional layer uses a collection of trainable filters to create the feature map. A total of six filters are used to create six feature maps from the input image. The feature map acquired from the individual filter is convolved through the whole image. Each feature map obtained from the filter signifies the precise features of the image. The DQNN operation is performed, which combines the two different functions to generate a third function. The DQNN operation is illustrated with Equation (26).

Fig. 8

DQNN descriptor.

where a_f designates the activation function, j signifies the specific convolution feature map, l exemplifies the layer in the CNN, f_ij represents the filter, b_j is referred to as the feature map bias, and M_l is defined as the selection of feature map.

The environment layer is utilized to accomplish down sampling operation in the DQNN algorithm. The pooling operation is accomplished to diminish the spatial size representation and the volume of parameters and computations in the network. It functions on each feature map individually. The pooling environment operation is expressed with Equation (27).

where pjl denotes the pooling region result applied on the jth region in the input image, pjl-1 refers to the jth region of interest taken by the pooling mask in the preceding layer, and Cjl represents the trainable co-efficient. The fully connected layer is used to derive features that are extracted from the previous layer. It provides the extracted features as output to the upcoming processes, and it is the final layer in the DQNN-based feature extraction that obtains results from the preceding layers in order to give extracted features. Finally, this layer contains the softmax classifier that classifies the 4-class, 3-class, and 2-class classes. The proposed MCSC-Net for multi-class classification of CXR images utilizes stacked ensemble learning to stack each layer.

4.1Dataset

The dataset used in this work is collected from COVIDx CRX-2 [29], which is a publicly available kaggle repository. A total of 5371 posterior-to-anterior CXR scans from the thorax to the abdomen are taken from the repository. All of these CXR scans are a composite of an upgraded version of multiple previous datasets combined into a single final image. COVID-19 is a relatively new illness; as a result, the number of images belonging to this class is limited, which may lead to overfitting in prediction models. The collected dataset in this work contains 1500 images from the COVID-19 affected, 1500 of normal lungs and 1100 of pneumonia bacterial, and 1271 of pneumonia viral. In order to minimize overfitting, data augmentation was avoided. Data augmentation to produce more CXRs from 1500 existing CXR images of the COVID19 class could lead to a loss of generalization because of the large difference in data. As a result, class weights have been developed to address the class imbalance. The dataset is divided into 80% for training set and 20% for testing set. The dataset is split into two sets randomly by the Monte Carlo cross-validation. It was decided to use an intuitive strategy to determine the class weights to strongly penalize the loss that comes with the inaccurate categorization of the COVID-19 image. The detailed organization of the dataset used in this work for multiple segmentation and classification models is given in Table 3.

4.2Subjective evaluation

The subjective evaluation gives the visual performance of the proposed method concerning preprocessing and segmentation methods used in this work. The preprocessing outputs of various methods such as Gabor filter, Wiener filter, histogram equalization, and proposed HMBF are shown in Fig. 9.

Fig. 9

Preprocessing performance evaluation (a) Input (b) Gabor filter (c) Wiener filter (d) Histogram Equalization (e) Proposed HMBF.

The input image in the first row is the COVID-19, the input image in the second row is pneumonia bacterial, and the input image in the third row is pneumonia viral. The proposed HMBF method perfectly enhances the input image with effective localization of disease-affected region visibility compared to the conventional approaches. The performance of SC-ResNet50 Segmentation with HMBF in preprocessing is shown in Fig. 10. Initially, the disease-affected CXR input images are applied to HMBF preprocessing, which enhances the disease-affected region. The SC-ResNet50 is developed to use the binary mask multiplied with the input image to generate the segmented region.

Fig. 10

SC-ResNet50 segmented outputs.

4.3Objective evaluation

Utilizing three separate sets of objective metrics, the objective evaluation compares the proposed method’s preprocessing, segmentation, and classification performance with that of existing conventional approaches.