Parametric optimization and comparative study of machine learning and deep learning algorithms for breast cancer diagnosis

1.Introduction

The increase in occurrences of breast cancer worldwide owing to many factors has instilled a natural fear of the disease in many women. Education about the onset and prognosis of disease is thus crucial as much as the developments in the technological advancement of diagnostic science of the disease. In certain parts of the world such as, India and USA, breast cancer is the most common type occurring in women, accounting for nearly 2,87,000 new cases alone in the USA in the year 2022 and 1,78,000 in India in 2020 [1]. These cases also increase the number of deaths. 43,000 women have died in the USA in 2022 according to the National Breast Cancer Foundation.

An unnatural growth of cells is a tumor. It develops when cells don’t die on schedule or multiply more than they need to [2]. A tumor may be benign or malignant (cancerous) (not cancerous). Unless it pushes on a neighboring structure or produces other symptoms, a benign tumor normally does not pose a major threat. Neoplasm is another synonym for tumor. The five-year survival rate for Stage I is reported 95%, for Stage II 92% and 70% for stage III and merely 21% for patients belonging to the Stage IV [3]. As the disease progresses, the chances of survival significantly drop. Hence modern technological solutions have been developing for better diagnosis of the malignancies found in the breast. These include data mining and machine learning algorithms. Consequently, picking the most effective algorithm out of all the available is most necessary part of the problem.

In business and healthcare applications, data mining serves the purpose of analysing vast datasets to uncover meaningful patterns and insights [29,30]. These valuable data patterns aid in predicting business trends and making informed decisions. In the healthcare sector, data mining plays a crucial role in optimizing efficiency, elevating patient quality of life, and, most importantly, saving more lives by identifying potential improvements and cost-saving opportunities. Techniques such as ML, database, statistics, data warehouse, deep learning neural network assist in the detection of different types of cancers [4].

Three tests make up the traditional approach: a clinical examination, radiological imaging, and a pathology test [5]. Modern machine learning focuses on model creation [37], while conventional methods use regression to detect cancer [33]. The model shows promising results in training, testing, and predicting unseen data. The three key tactics in machine learning are pre-processing, feature selection, and classification. Feature extraction, a core component of machine learning, aids in diagnosing and prognosing cancer and distinguishing benign from malignant tumours. The following research issues are addressed, and the deep learning models are highlighted in this study along with their effectiveness, datasets used, and potential for breast cancer detection and classification:

The remainder of this paper is organized as follows: Related research including surveys on the subject of breast cancer is included in Section 2. The approach for this study is described in Section 3. The collected results and pertinent considerations are presented in Section 4. Section 5 mentions the clinical implications of the study. Section 6 comes to a conclusion and offers ideas for other study possibilities.

2.Related works

There have been many researches on breast cancer detection through machine learning [39] and data mining techniques, in this section we will go through previous work on multiple datasets including Wisconsin Breast Cancer Dataset (Original), Wisconsin Breast Cancer Dataset (Diagnostic) and others as shown in Table 1.

In this [6] study, Ahmed et al. used a dataset of 1189 records and 22 predictor variables to create prediction models for breast cancer recurrence in patients. Artificial neural networks (ANN), Support vector machines (SVM), and decision trees (C4.5) were used, and their sensitivity, specificity, and accuracy were evaluated. The outcomes demonstrated that the SVM model, which had an accuracy of 0.957, was the most accurate, followed by the DT model (0.936), and the ANN model (0.947). This [6] study shows the promise of data mining methods, such machine learning algorithms, for creating recurrence prediction models for breast cancer. To confirm these results and examine their use in different medical settings, more study is required.

In this [7] paper, Chandra P. Utomo et al. used the Breast Cancer Wisconsin Dataset to develop an ANN using an extreme learning machine approach for breast cancer diagnosis, outperforming the traditional Back Propagation ANN in terms of performance. In the training phase, K-fold cross validation with 5 folds was performed, and performance was assessed using specificity and sensitivity as additional precision metrics. The findings demonstrated that ELM ANN [8] was a superior generalization model for the Breast Cancer Wisconsin Dataset, outperforming Back Propagation ANN in terms of identifying breast cancer.

Jyotismita and Sanjib in their work [9], used data mining techniques to identify the early signs of breast cancer. They used the WEKA data mining programme to carry out their experiment in order to distinguish between benign and malignant breast tumors using the ZeroR and J48 classification algorithms. When compared to J48, which delivered 95.37% accuracy, the ZeroR algorithm did very badly, achieving just approximately 63.86%.

Ebru Aydındag and Pınar Kırcı, conducted their [10] research on the Wisconsin Breast Cancer Dataset (original) using SVM and ANN techniques for the prediction and classification of breast cancer. They also used the WEKA tool for visualization, analysis and classification. The novelty in their work includes SMO (Sequential Minimal Optimization) [11] and LibSVM [12] usage as the classification algorithm in SVM in WEKA software. They also included the AUC value of their experiments. SMO produced an accuracy of 96.69% while LibSVM produced nearly 95.53% accuracy with 0.968 and 0.964 as AUCs respectively. They used a Multi-Layer Perceptron [13] and Voted perceptron [14] as their ANN model, producing 95.42% and 90.98% accuracy respectively and 0.988 and 0.929 as their respective AUC scores. Their study showed that ANNs do not generally outperform conventional ML techniques on the Wisconsin Breast Cancer Dataset (original).

Omar et al. conducted a study [15] comparing three machine-learning classifiers (Support Vector Machine, K-nearest Neighbors, and Decision Tree) to identify the best classifier for breast cancer classification. The research utilized the Wisconsin Breast Cancer Dataset (Diagnostic) with the main goal of determining the most accurate classifier. The results revealed that quadratic support vector machines achieved the highest accuracy (98.1%) and the lowest false discovery rates. The experiments were conducted using Matlab, which has a specialized toolbox for machine learning methods.

Ferroni et al. [16] developed a machine learning-based decision support system (DSS) for breast cancer prediction, demonstrating impressive outcomes with a test set of 136 patients (C-index of 0.84 and accuracy of 86%) and a training set of 318 patients. The model was evaluated using a 3-fold cross-validation method and was based on multiple kernel learning and random optimization models. Based on the area under the curve, the researchers determined that ML-RO-4 performed the best (AUC). A promising method for predicting the prognosis of breast cancer, the DSS model shown great performance in the testing set and the capacity to divide patients into low- and high-risk categories.

Using machine learning techniques, Shravya Ch. et al. [17] describe a condensed method for classifying benign and malignant tumors. A 32-attribute dataset from the UCI repository was utilized in the study along with Logistic Regression, SVM, and KNN. SVM produced the best results with an accuracy of 92.7% using PCA for feature extraction. The findings suggest that multidimensional data and various feature selection, classification, and dimensionality reduction strategies can offer practical inference-supporting tools [36]. The goal of future study should be to enhance performance by investigating more models and ensemble-based models.

A machine learning-based Internet of Things diagnostic system for identifying breast cancer is presented in this paper [18]. Wisconsin Diagnostic Breast Cancer has 569 data records, 30 real-valued characteristics, 357 benign cases, and 212 malignant cases in the dataset. The best features were extracted using a Modified Recursive Feature Elimination Algorithm [19] following data pre-processing. Following that, an SVM model with four kernel values—linear, RBF, polynomial, and sigmoid—was applied to the feature set. Metrics including accuracy, specificity, precision, F1-score, MCC, classification error, and execution time were used to gauge the model’s performance. Overall, the suggested system correctly distinguished between breast cancer patients and healthy individuals, and the results demonstrated that the SVM model performed well for all four kernel values.

Three predictive classification methods for identifying breast cancer from mammogram images were given by Padmarpriay B. and Velmurugan T. [20] J48, CART, and ADTree were the three algorithms. These models’ effectiveness was precisely assessed using their specificity, sensitivity, and Kappa statistics. 250 instances of individuals between the ages of 20 and 72 were collected in the DICOM image format from the Cancer Institute, Adyar, Chennai, Tamilnadu, India and used in the tests. Pre-processing was categorized based on each of the nine selected characteristics’ values. The findings revealed that the accuracy of the CART algorithm was 98.50%, while that of ADTree [21] and J48 was 97.70% and 98.10%, respectively.

Fig. 1.

Experiment workflow with Wisconsin dataset.

3.Experimental methods

To carry out a complete data preparation and feature engineering approach, we have principally used 10 supervised machine learning classification algorithms, including specialized boosting-based ensemble algorithms like XgBoost, AdaBoost, and LGBM classifier and proposed a single ANN based custom classifier to cater our solution to the classification problem. Data collection, pre-processing, feature engineering, data exploration, model training, and model performance assessment are the primary processes in this suggested technique. The feature selection, outlier elimination, and skewness correction of the characteristics are all included in the data pre-processing approach. In addition, a thorough data exploration approach is performed to extract meaning from the data and spot trends. The chosen methods are then used to train the model after that. Finally, a variety of performance indicators are used to assess the performance of the model. The challenge of categorization is effectively and efficiently solved by this thorough procedure. Flow diagram for the proposed method is shown in Fig. 1.

3.1.Data pre-processing

The researchers obtained the Wisconsin Breast Cancer Dataset (Original) from the UCI Machine Learning Repository for analysis. The dataset consists of 683 instances classified as benign and malignant, with 11 integer-valued attributes. The dataset comprises of 444 classes of benign class and 239 samples of malignant samples.

In addition to the approaches employed, the performance and precision of the prediction model are also affected by the quality of the dataset and the pre-processing procedures. Preprocessing refers to the activities performed on a dataset prior to the deployment of machine learning techniques. At this stage, the dataset must be prepared and modified so that the algorithm can perceive it. Datasets may have errors, missing data, redundancy, noise, and other issues that hinder the machine learning algorithm from directly utilizing them. Another difficulty is the size of the dataset. When a dataset has a large number of characteristics, it may be more difficult for the algorithm to analyse the data, identify trends, and make precise predictions. Such difficulties can be resolved by evaluating the dataset and employing the right data pre-treatment processes. Depending on the nature of the dataset, the data pre-processing approach may additionally include feature selection, data normalization, missing value imputation, and data cleansing.

Fig. 2.

Correlation matrix.

Fig. 3.

Data distribution of features.

We removed the ‘sample code number’ since it merely served the purpose of identifying the data and was not necessary for our purposes. We also used the StandardScaler [32] method for normalizing data by subtraction of the mean and scaling to unit variance. It is used to alter features by scaling each attribute to a defined range. This is crucial for the proper operation of many machine learning algorithms. It is also used to minimize the impact of outliers and make the dataset more normally distributed, hence enhancing the accuracy of certain models. We applied this scaler on our dataset. We also studied the inter-dependency of every feature with each other which is then used to select the most important features to feed in the machine learning models, the values of correlation of features is visualised in Fig. 2 in the form of correlation matrix. The data distribution of important features is also visualized in Fig. 3 since it is an important part of building machine learning models for classification purposes. Analysing the distribution of data helps in gaining insights into the various properties of the dataset and aids in making informed decisions during the feature selection and model building phases.

The Pearson correlation matrix assesses the linear relationship between variables. It employs the correlation coefficient (r) to quantify the strength and direction of this relationship. x_i and y_i are the individual values of the x and y variables, respectively, while x and y represent their respective means. The matrix utilizes the correlation coefficient (r) to measure the linear relationship between variables. By comparing individual values (x_i, y_i) to their respective means (x, y), it identifies the strength and direction of the association. This matrix facilitates insights into dependencies and interactions within datasets. The equation corresponding to this is given in Eq. (1)

(1)

r=∑(xi−x¯)(yi−y¯)∑(xi−x¯)2∑(yi−y¯)2.

3.3.Model description

A summary of the machine learning models utilized in the analysis of the UCI breast cancer dataset has been provided in the Table 2. The models examined include Logistic Regression, Decision Tree Classifier, Random Forest Classifier, Support Vector Classifier, Naïve Bayes, KNN, AdaBoost, Gradient Boosting, XgBoost, and LightGBM. The parameters employed in training these models are also listed. Logistic Regression employed a maximum iteration of 5000 and an intercept scaling of 0.5. Decision Tree Classifier utilized a Gini criterion, a maximum depth of 5, a minimum number of samples per leaf of 1, and a minimum number of samples per split of 3. Random Forest Classifier employed bootstrap sampling, a maximum depth of 8, a maximum number of features of auto, a minimum number of samples per leaf of 1, a minimum number of samples per split of 2, and 48 estimators. Support Vector Classifier utilized a linear kernel, a probability setting of True, and verbose set to False. Naïve Bayes utilized a variance smoothing factor of 2e-9. KNN employed a nearest neighbour parameter of 5, with uniform weights, and the Minkowski distance metric. AdaBoost utilized the SAMME.R algorithm, a learning rate of 0.1, and 200 estimators. Gradient Boosting utilized a learning rate of 0.5 and 100 estimators. XgBoost employed a learning rate of 0.4, a maximum depth of 3, and 175 estimators. Finally, LightGBM employed a learning rate of 0.05, 200 estimators, and 50 leaves. Overall, these models and their associated parameters provide a comprehensive analysis of the UCI breast cancer dataset.

Fig. 4.

ANN classifier.

4.Performance evaluation

Table 4 reveals an important finding that hyperparameter tuned models using Grid Search CV [28] outperformed non tuned models in terms of performance. As an illustration, the accuracy of the Random Forest Classifier with Hyperparameter tuning was 98.43% for class 2 and 98% for class 4, while the accuracy of the RFC without tuning was 96.48% for class 2 and 96% for class 4 in line with other studies [34]. This emphasises how crucial hyperparameter adjustment is to improve the performance of ML models.

Another observation is that boosting algorithms such as AdaBoost, Gradient Boost, and XGBoost performed very consistently across both benign and malignant tumours, these methods produced results that were comparable, with precision, F1 score, and accuracy all falling within the range of 0.97 and 0.98. This suggests that boosting algorithms are robust and effective in handling the complexity and non-linearity of the data [35], and it can be observed from the given ROC curve in Fig. 5.

The accuracy of the custom classifier was 96.54% for class 2 and 96% for class 4, respectively. The custom classifier’s identity cannot be determined from the Table 4 preventing a comprehensive evaluation of its performance in comparison to other models, despite the fact that it did well.

The majority of research has employed the Support Vector Machine (SVM) algorithm for breast cancer detection, with the maximum accuracy reached between 98 and 99 percent. In specifically, the study [18] employing SVM with Modified Recursive Feature Elimination Algorithm and linear kernel reached 99% accuracy. In addition, research [15,17] have utilized SVM to achieve the greatest accuracy of 98.10% and 92.70%, respectively.

In addition to SVM, additional researches have utilized Decision Tree and CART, with the maximum accuracy achieved by CART in the study [20]. Some research also employed Artificial Neural Networks (ANN), Extreme Learning Machine Artificial Neural Network (ELM ANN), and Backpropagation Artificial Neural Network (BP ANN) with 96.4% accuracy. In other research, other algorithms such as Logistic Regression (LR), K-Nearest Neighbors (KNN), J48, ZeroR, and Multiple Kernel Learning DSS Model were utilized with varied degrees of precision. Next section elaborate the clinical implications of the study.

Fig. 5.

ROC curves.

5.Clinical implications

In the past five years, there has been a significant increase in the integration of AI in medicine [40], with a particular emphasis on developing AI-assisted solutions for treating cancer, especially breast cancer. Breast cancer remains a critical public health challenge, and its early detection is crucial for improving patient outcomes and reducing mortality rates. However, before implementing any automated solution in healthcare, it is essential to thoroughly evaluate its clinical implications, including verifying its safety, advantages, any possible disadvantages, and ensuring it meets regulatory compliance standards. One of the primary clinical implications of our study is the potential for improved diagnostic accuracy in breast cancer detection. Traditional diagnostic methods, such as mammography and biopsy, are effective but are subject to human error and variability. The application of ML and DL models can supplement these traditional methods by providing consistent and nuanced analysis. As per our experiments, high-performing boosting models can easily detect and differentiate between benign and malignant tumors. Such high accuracies can reduce false positives and false negatives, minimizing unnecessary biopsies and surgeries while ensuring that malignant cases are promptly treated.

The implementation of automated diagnostic systems can also lead to cost savings and improved accessibility. In resource-limited settings, where access to specialized healthcare professionals and diagnostic facilities may be scarce, these algorithms can serve as a supplementary diagnostic tool, providing valuable insights and potentially reducing the need for expensive diagnostic tests. We highlight the practicality of using widely available datasets and ML/DL models, suggesting that these technologies can be adapted and utilized in various healthcare settings, regardless of the availability of advanced medical infrastructure.

The use of ML and DL algorithms can also aid in the development of personalized treatment plans. By analysing a patient’s specific data, these models can predict the likely progression of the disease and suggest the most effective treatment options. For example, the use of ANNs and Extreme Learning Machines (ELM ANN) demonstrates the ability to analyze complex patterns within the data, which could be utilized to tailor treatment plans to individual patient profiles. This capability is particularly valuable in cases where the standard protocols may not be optimal for every patient, thereby paving the way for more personalized and targeted therapies. The integration of these algorithms into clinical workflows can streamline the diagnostic process. For instance, an automated system that pre-screens mammograms or biopsy results using ML/DL models could prioritize more intricate cases for radiologists and pathologists, allowing them to focus on the most critical cases. Our study also emphasizes the importance of hyperparameter tuning and boosting algorithms, suggesting that these optimizations can enhance the models’ performance, making them more practical for real-world clinical applications. Early detection of breast cancer is important for increasing survival rates. Our findings indicate that ML and DL algorithms can potentially identify malignancies at an early stage, even when clinical symptoms are not yet apparent. The ability to predict the disease’s progression and recurrence can also be invaluable for prognosis. For instance, models like SVM with various kernel functions and ensemble methods showed promising results in distinguishing early-stage cancers. Early detection through these models can lead to less aggressive treatments and better patient outcomes, including higher survival rates and improved quality of life.

Like any other research on AI-assisted healthcare, our study of ML and DL models in breast cancer diagnosis also raises important ethical and practical considerations. The reliance on data-driven models necessitates careful consideration of data privacy and security. Moreover, the interpretability of these models remains a challenge; clinicians must understand and trust the model outputs to integrate them into their decision-making process effectively. Future research should focus on developing explainable AI models and ensuring that these technologies are used responsibly in regular clinical practice.

6.Conclusion

The study examines the performance of several machine learning models on the original Wisconsin Breast Cancer dataset. Several models, including Logistic Regression, Decision Tree, Decision Tree with Hyperparameter tuning, Random Forest Classifier, Random Forest Classifier with Hyperparameter tuning, Support Vector Classifier, Naive Bayes, KNN, AdaBoost, Gradient Boost, XGBoost, LGBM Classifier with Hyperparameter tuning, and one MLP based Custom Classifier, achieved high accuracy, precision, and F1 scores for benign and malignant tumors.

The research determined that models with hyperparameter tuning outperformed those without. This emphasizes the significance of optimizing model parameters for maximum performance. In addition, boosting techniques like as XGBoost, AdaBoost, and Gradient Boost consistently performed well across both benign and malignant tumors, indicating the efficacy of these algorithms in managing the complexity and nonlinearity of the data.

Comparing the findings of this experiment with those of earlier research, as summarized in the Table 1, it can be seen that the experiment’s greatest level of accuracy, around 98%, is equivalent to the range of 98–99% recorded in previous investigations. Moreover, a number of models utilized in the experiment are consistent with the findings reported in previous studies which also highlighted the efficacy of SVM, Decision Tree, CART, ANN, ELM ANN. Majority of the previous research have utilized some variation of the SVM and it has consistently performed well across all of the breast cancer datasets that are discussed in this study. Besides that, it has been observed that linear kernel performs the best across Wisconsin Breast Cancer Dataset.

In conclusion, the current study illustrates the efficacy of multiple machine learning models in obtaining high diagnostic accuracy for breast cancer, and it can be stated that the Wisconsin Breast Cancer (original) dataset may be utilized successfully to evaluate the performance of these models. This study’s findings also emphasize the significance of hyper parameter tweaking and the efficacy of boosting algorithms in managing the complexity and nonlinearity of data.