Metaheruistic optimization based machine-learning approach for analysis of academic students’ strategies

1.1Related works

A increasing body of research has focused on incorporating machine learning methods into meta-heuristics to address combinatorial optimisation issues. This integration intends to enhance the performance of the me-ta heuristics in terms of the quality of the solutions, the convergence rate, and the robustness of the search [21].

According to [20], forecasting student retention offers information on possibilities for deliberate student counselling. The review offers a research viewpoint on predicting student retention using machine learning through a number of significant discoveries, including the identification of the variables employed in prior studies and the prediction approaches Educational data mining is a group of data mining applications in the field of education, according to [22]. The analysis of student and teacher data is a focus of these programmes. The analysis could be put to use for classification or forecasting. Examined are machine learning techniques including Nave Bayes, ID3, C4.5, and SVM. [23] suggests a new machine learning-based model to forecast undergraduate students’ final test marks using the results of their midterm exam grades as the source data. An educational process and data mining plus machine learning (EPDM + ML) model was introduced in [24] and used to contextually analyse the performances of the teachers and provide recommendations based on information obtained from students’ evaluations of instruction (SET). Higher education institutions have begun integrating technology into their conventional teaching methods in [25] in an effort to improve learning and teaching. In this study, five machine learning algorithms were used to analyse two datasets in order to predict and categorise student performance.

1.2Shortcomings of the current techniques

Because the traditional machine-learning algorithms like SVM, ANN, and K-means techniques were applied to the educational dataset to observe the pattern in the set, and it helps to estimate the possible future outcomes. Even though the existing techniques can estimate the student performance but its performance was lagging in the following process:

The attribute analysis that affects the student performance was performed through the unsupervised learning or correlation analysis. Some of the models utilized thresholds for predicting the performance.

1.3Motivation of study

Demographic elements including a parent’s educational level, environment, and one’s own desire in education are among the many aspects that influence an individual’s education. On the datasets, a variety of artificial intelligence algorithms have been used to analyze these variables and improve the teaching strategy. These models were not enough for analysing every element that has an impact on education. In this paper, a single, optimized machine-learning strategy is suggested.

3.2Dataset

In this paper, the three types of student performance data were collected from the kaggle websites [16–18]. The three datasets describe the following information:

The data distribution in this three datasets is presented in Table 1.

The above information in the dataset were either in categorical or numerical formats. But for an efficient data mining process, the data should be in a single format. In particular, the numeric data format helps to analyze the information in faster and efficiently. Hence, in this, the datasets were subjected to a preprocessing step

3.3Preprocessing

For an efficient and faster analysis, the datasets were subjected to a data transformation process.

In data transformation process, the binary variables like true or false, yes or no, or any two class variables like gender, college, or address nature are transformed to 1 and 0

In categorical variables like educational stage, student nationality, and other categories are transformed into 1 to n categories based on number of categorical options.

Once the datasets are preprocessed using the above Equations 2 and 3, the remaining data also combined together in each set to form a complete numerical dataset. Then, the transformed data was subjected to Z-score normalization to normalize the values between 0 and 1 in all categories. The Z-score normalization calculates the new values for all the variables using Equation 4.

Then, the normalized values are subjected to the data exploration process for the first stage of data reduction.

3.4Data exploration

In this stage, the data distribution individually was analyzed which least impact on the target variable is called output variables in each set. The sample data exploration of the set in terms of gender and school is shown in Figs. 1 and 2.

Fig. 1

Dataset 1 exploration.

Fig. 2

Dataset 2 exploration.

In the above Fig. 1, the demographic variables like gender, subjects, guardian of the student were presented in figure format for dataset 1 with gender as base variable.

In Fig. 2, the demographic variables like gender, nationality, and lunch activities of the student were presented in figure format for dataset with gender as the base variable.

In Fig. 3, the demographic variables like gender, locality, guardian of the student were presented in figure format for a dataset with gender as base variable.

Fig. 3

Dataset 3 exploration.

In the data exploration part, the individual variable analysis was only carried out which gives an idea of the data distribution in the set. Based on this, the general variables like gender, nationality, and place of birth are removed from the sets. Then, the remaining variables were subjected to attribute selection using the proposed hybrid CSO-PSO.

3.5HCSO-PSO-based attribute selection

In this, the preprocessed variable from the set was subjected to optimization process to select the optimal attributes that are responsible for the student performance was determined by the proposed HCSO-PSO process. Both processes utilize the same objective function mentioned in Equation 2 was used. The boundary variables were only modified in the first stage of process using cuckoo search optimization. While in the second stage, the boundary variables are common to find the four optimal variables. The proposed method process is represented in the flow chart in Fig. 4.

Fig. 4

HCSO-PSO processes.

3.6Regression neural network

The reduce attributes from the hybrid CSO-PSO algorithm, a new dataset was framed with the reduced attribute set for analysis. Then, the new set is divided into training and testing using hold-out approach with a 0.3% value. This helps to split the dataset into 70% as training and 30% as testing.

The optimal four attributes will act as the first layer or input to the regression neural networks. In the corresponding test score, categorical variables were given as the outputs to the network.

The kernel function was multiplied with the inputs to determine the neuron weights using Equations 7 and 8.

In this layer, the kernel function in the step 2 is the denominator and the corresponding output is the numerator.

The inputs and outputs are related by dividing the numerator and denominator parts.

Then, the transformed dataset was trained using a regression neural network, and its performance was measured in terms of accuracy, precision, recall, and F1-score.

4.2Accuracy

Equation 10 is used for calculating the accuracy. This value evaluates the model in terms of the correct estimation of student performance.

The accuracy indicates the model types like underfitting, overfitting, and correct model. This accuracy will be calculated for both the training and testing to determine the model types.

4.2.3Accurate model

When the model accuracy is the same and higher for both training and testing means, it is a correct model for future estimation.

From Fig. 5, the following points were observed.

Fig. 5

Model types based on error.

Table 4

Model types based on error

Model	Error nature
Underfitting	Training and testing error is high
Overfitting	Testing error >  training error
Correct model	Training and testing error is optimal and somewhat equal

4.3Precision

Equation 11 is used for calculating the model precision. This value is used to identify the model performance in estimating the values accurately.

4.4Recall

Equation 12 is used for calculating the model precision. This value is used to identify the model performance in estimating the total number of times the model can find the values accurately.

4.5F1-score

F1 score is calculated from precision and recall and is given in Equation 13.

Using Equations 10 to 13, the proposed HCSO-PSO-based regression neural network model is used in estimating the student performance was evaluated and the corresponding results are given below.

5.1Dataset

In this, the sample screenshots of the three datasets are shown in Figs. 6–8.

Fig. 6

Dataset 1 sample.

Fig. 7

Dataset 2 sample.

Fig. 8

Dataset 3 sample.

5.2Preprocessing and exploration

In the preprocessing step, the data were transformed and normalized the values 0 and 1 as shown in the sample Fig. 9 for set 2.

Fig. 9

Preprocessed and normalized set.

5.3Data exploration

The results of data exploration are presented in Figs. 1–3 in the proposed work section.

5.4Attribute selection using HCSO-PSO

The optimization process performance will be evaluated using the convergence curve. The convergence curve is the graph drawn between the iterations and fitness function. This shows the optimization performance speed in finding the optimal attribute selection.

5.5First stage attribute selection using CSO

Figure 10 shows that the CSO finds the six optimal attributes after 30 iterations by maintaining the error rate as minimum below 0.2 for all the sets. Apart from dataset 2, in datasets 1 and 3, the optimal attributes were selected from the set. In set 2, only six attributes are present; hence, all the six attributes were selected for the second stage process. In set 1, the student behavior and education were selected. In set 3, the student behavior like absences, study time, romantic, internet, and education g1, g2 was selected for set 3. These reduced sets were subjected to the second stage of processing using particle swarm optimization.

Fig. 10

CSO convergence.

Figure 11 shows that the CSO find the six optimal attributes after 30 iterations by maintaining the error rate as minimum below 0.2 for all the sets. In set 1, the student performance like discussion, view, raising hand, and response was selected as optimal attributes. In set 2, apart from lunch and place of birth, all other attributes are selected. In set 3, the grade performance, study time, and romantic were selected. Then, the selected attributes were split as training and testing and then subjected to regression neural network modeling.

Fig. 11

PSO convergence.

Around 50 times of the iterations were performed, and various fitness function were used as inputs. The accuracy, precision and recall were then assessed as shown in Fig. 12. As a result, the existing methods achieves 95.65% and 77.98% of accuracy, where the proposed achieves 98.9% which is3.35% and 11.08% better than aforementioned methods. Then, the proposed modeled network performance was evaluated using evaluation metrics and the corresponding values are tabulated in Table 5.

Fig. 12

Comparison for various parameters.

From Table 5, it was observed that the proposed HCSO-PSO-based RNN is the best model by having higher accuracy in training and testing. This proves that the proposed student’s behavioral and past performance is best for predicting their academic performance.

6.1Implications and applications

Data mining methods are effective at locating hidden knowledge in huge datasets. When properly implemented in the educational sector, they can have significant positive effects on higher education institutions, students, education policymakers, and the prosperity of the nation. This work made a beneficial contribution to the field of educational data mining by finding the elements linked to academic probation. Policymakers, professionals, and educators can benefit from the findings by having a better understanding of the causes of the academic failure that can result in probation status. By taking into consideration certain other aspects besides high school performance, this can assist management in making better strategic decisions about the admission standards to universities. The study can also aid in determining the best strategy for pre-major and college admission requirements. The results of this study may be useful to academic advisors as well. According to substantial data, academic supervisors can, for instance, become more aware of the elements that have a negative impact on students’ performance and adjust their advising approaches. Students can also comprehend and steer clear of the issues that contribute to subpar academic achievement. Additionally, after a student enrols in an institution, the proposed model can be used to forecast their academic performance there. The connected educational stakeholders can take the necessary actions in accordance with the model’s predicted results. To avoid any barriers to academic growth, faculty should keep track of their advisees’ academic performance, meet with them twice a semester, and develop appropriate action plans. Students who are on supervision or who are at risk of being placed on probation should get ongoing follow-up sessions from them. Students will successfully exit the probationary period thanks to this. Additionally, parents should, whenever possible, be involved in their child’s academic circumstances. Parents who are well-informed could work together to create a much better learning environment for their children.