Discrete Wavelet Transform based ECG classification using gcForest: A deep ensemble method

3.1Discrete Wavelet Transform

Wavelet transform can comprehensively extract both the time-domain and frequency-domain characteristics of a signal, enabling a holistic representation of the original signal. Contrary to the Short Time Fourier Transform (STFT) [7] which analyzes signals at a fixed resolution, the wavelet transform is more adept at analyzing time-varying and non-stationary signals such as ECG. Consequently, numerous studies utilize wavelet transform techniques for feature extraction from ECG or denoising ECG signals.

Wavelet transform is categorized into Continuous Wavelet Transform (CWT) [8] and Discrete Wavelet Transform (DWT), serving as a fundamental tool in signal processing. In real-world EEG signal acquisition, signals are typically discrete. Moreover, the computational overhead of utilizing all wavelet coefficients by CWT is considerable. Thus, it is adequate to select a discrete subset of child wavelets to effectively reconstruct a signal from its corresponding wavelet coefficients. In this paper, we utilize the DWT method for feature extraction of ECG signals.

The wavelet transform theory is grounded on a function known as the mother wavelet ψ⁢(t), and it satisfies the property below:

Child wavelets are dilated by a factor of a and shifted by a factor of b. In the discrete wavelet transform, we ensure that the pair (a, b) consists of discrete values, hence the corresponding wavelet is represented as:

Where a0 denotes the scale parameter, b0 denotes the translation parameter. Additionally, both m and n are integers belonging to the set Z.

Given the discrete ECG signal f⁢(k), the general equation for the DWT signal is define as:

In practical applications, the fast Discrete Wavelet Transform (DWT) algorithm proposed by Mallat [9] is frequently utilized. This algorithm effectively decomposes an ECG signal into two distinct sub-bands: the low-frequency sub-bands, known as approximation coefficients (cA), and the high-frequency sub-bands, known as detail coefficients (cD). Moreover, fast DWT contains down-sampling by a factor of 2. When an origin signal passes through two filters, the signal is decomposed into a high-frequency component cD1 and a low-frequency component cA1. This process is termed as a decomposition of level 1. During the decomposition of level 2, cA1 is further decomposed into low-frequency component cA2 and high-frequency component cD2. Consequently, the outputs are cD1, cD2, and cA2. The decomposition formula is defined as follows:

Where g⁢[⋅] represents high-pass filter and h⁢[⋅] represents low-pass filter. Based on the theory, a schematic illustration of the feature extraction for ECG signals utilizing the fast DWT is illustrated in Fig. 2.

Figure 2.

Illustration of feature extraction structure.

3.2GcForest

The gcForest model was proposed by Zhi-Hua Zhou et al. [4] in 2017 as a collection of non-differentiable modules, characterizing an ensemble of ensembles approach that necessitates fewer hyper-parameters compared to deep neural networks. This implies a significant reduction in model complexity, which in turn lowers computational demands. Notably, the gcForest demonstrates commendable robustness concerning hyper-parameters. Moreover, the gcForest architecture incorporates a Cascade Forest structure for deriving the final classification and employs a procedure known as Multi-Grained Scanning to enhance the representation of data [10]. In this section, we will delineate both the Cascade Forest structure and the Multi-Grained Scanning mechanism.

3.2.1Cascade forest structure

Inspired by the processing of raw features in Deep Neural Networks, the gcForest model adopts a cascade structure to enhance its analytical capabilities. This structure is comprised of numerous layers, each of which constitutes an ensemble of decision tree forests. Since the gcForest model represents an ensemble of these layers, it is also referred to as an ensemble of ensembles. The structure is shown in Fig. 3.

Figure 3.

Structure of Cascade Forest (Suppose there are 5 classes).

Each layer within this structure accepts data from the preceding layer and concatenates them with the original features before passing them on to the subsequent layer. In the construction of each layer, different types of forests are employed to amplify the diversity of the ensembles, which can contribute positively to the model’s performance. Typically, each layer contains two varieties of forests: random forest [11] and completely-random forest [12]. In the random forest, trees make splitting decisions by choosing the optimal feature with the best Gini index from a randomly selected subset of features. In contrast, trees within the completely-random forest select a feature at random to determine the splitting criterion. Lastly, it is important to note that each layer includes p forests with q trees in each, where p and q are hyper-parameters.

In the gcForest framework, each forest evaluates an instance by calculating the class distribution. This is done by determining the proportion of training examples from different classes at the leaf node where the instance falls [4]. This method helps in estimating the likelihood of the instance belonging to each class based on the data observed in the training phase. Consequently, the class distribution vector is obtained by averaging the results across all trees within the same forest. Following the acquisition of the class distribution vector, it is concatenated with the original feature set to serve as the input for the subsequent level of the cascade. Assuming that the length of the input feature is N, the task is an n-class classification, and each layer of the gcForest consists of p forests. Consequently, each layer will yield p n-dimensional vectors representing the class distribution. For instance, the feature length outputted by the first level would be N+p×n, which represents the original feature length combined with the product of the number of forests per layer and the number of classes.

3.2.2Multi-Grained Scanning structure

To enhance the representation of the original data, a structure known as Multi-Grained Scanning is introduced to process the raw data before feeding it into the cascade forest structure [4]. The raw features are scanned with sliding windows of size t, creating a set of t-dimensional vectors. For instance, if the length of the original data is N, then after scanning with a window size t, stride of 1, and padding of 0, we obtain N-t+1 t-dimensional vectors. These vectors are taken as instances to be input into a layer of the cascade forest structure, where they are trained through random forests and completely-random forests within that layer, producing a set of class distribution vectors. Continuing with the assumption from the previous section, where the task is an n-class classification, and each layer includes p forests, the N-t+1 n-dimensional vectors pass through p forests and eventually yield N-t+1 n-dimensional class distribution vectors. These vectors are then concatenated to form a vector of length 2×(N-t+1)×n, which replaces the original features as the input for the cascade forest. The scanning structure is shown in Fig. 4.

Figure 4.

Structure of scanning (the original input length is N).

To enhance the randomness of data input, the scanning procedure employs multiple scans with different window sizes, a method known as Multi-Grained Scanning. This technique inputs features scanned with various window sizes into different cascade forests. The final results obtained from these forests are then aggregated. This multi-faceted approach allows for a more robust and comprehensive feature representation by capturing the inherent patterns in the data at different scales, ultimately contributing to the versatility of the model.

3.3DWT-cascade forest architecture

The structure for feature extraction, employing Discrete Wavelet Transform (DWT), and ECG multi-classification, using gcForest, is detailed in Fig. 5. During the DWT-based feature extraction, we utilize a Daubechies-4 (db4) [13] wavelet filter bank and set the decomposition level to four. Each level of decomposition yields both approximation coefficients (cA) and detail coefficients (cD), with the decomposition input for the next level being the cA from the previous one. The length of cA and cD obtained at each level is the same, equal to (N+F-1)/2, where N is the length of the input data and F is the length of the db4 wavelet filter. Following a four-level decomposition, the resultant coefficients cD1, cD2, cD3, cD4, and cA4 are concatenated to represent the original features, with a resultant length of N⁢p.

Figure 5.

DWT-based gcforest structure.

Subsequently, the features of length N⁢p undergo Multi-Grained scanning structure, where they are scanned with several different window sizes before being input into different cascade forest structures. Assuming there are three different window sizes for scanning t⁢1, t⁢2, and t⁢3, the raw features will then be transformed into representations of length (N-t⁢1+1)×n, (N-t⁢2+1)×n, and (N-t⁢3+1)×n respectively, with n denoting the number of classes for classification. These vectors are separately fed into three corresponding cascade forests. Following this, the final prediction is determined by aggregating the class vectors from the last level of each forest. The classification is then decided based on which one has the highest cumulative value.

4.1Experimental setup

In order to evaluate our proposed model, we have partitioned the dataset into a training set and a test set, allocating 80% for training and 20% for testing. Meanwhile, we evaluate the performance of classification with the following metrics, which are defined by the following formula:

where FP denotes the False Positive, FN denotes the False Negative, TP denotes the True Positive and the TN denotes the True Negative.

In each layer of the gcForest, there are two hyper-parameters subject to fine-tuning: the number of forests (denoted as p) within each layer and the number of trees (denoted as q) within each forest. It’s important to note that the forests in each layer are divided into two types: random forests and completely random forests. Consequently, in each layer, half of the forests, or p/2, are random forests, and the other half are completely random forests. To understand the gcForest’s sensitivity to these hyper-parameters, we conduct experiments by varying both the number of forests p in each layer and the number of trees q within each forest.

4.2Result and discussion

The classification performance on the test set with the change of the number of forests within each layer and the number of trees within each forest is shown in Figs 7 and 7, respectively.

From Fig. 7, it can be discerned that variations in the number of forests in each layer yield disparities in the maximal and minimal values of accuracy, recall, precision, and F1, with differences approximately around 0.07%, 0.08%, 0.08%, and 0.08%, respectively. Similarly, Fig. 7 shows that by varying the number of trees in each forest, the difference between maximal and minimal values of accuracy, recall, precision, and F1 is approximately 0.06%, 0.06%, 0.09%, and 0.10%, respectively. Hence, to balance computational expenditure with model performance, we elect to set and as the final hyper-parameter configuration. In this setting, the model performance is characterized by an accuracy of 98.55%, recall of 98.48%, precision of 98.44%, and an F1 of 98.46%.

To appraise the performance of our proposed model more thoroughly, we have conducted a comparative analysis with existing methods. This includes traditional machine learning algorithms including LDA-MLP [14], SVM [15], XGBoost [16], and RandomForest [17], as well as deep learning methods comprising CNN [18], ResNet [19], DCNN [20], CNN-BiLSTM [21], and Deep Attention BiLSTM [22]. The outcomes of this comparison are delineated in Table 2.

In traditional machine learning algorithms, the LDA-MLP method [14] employs the forward floating search (SFFS) algorithm to derive a feature set, achieving an accuracy of 89.02%, a recall of 69.53%, a precision of 56.27%, and an F1-score of 62.20% on the dataset. The SVM classifier with Morphological and Dynamic Features [15] reached an accuracy of 86.55%. An ECG classification approach based on XGBoost [16] attained an accuracy of 91.87%. Finally, the Random Forest classifier with features utilizing wavelet packet decomposition [17] achieved the highest accuracy among machine learning algorithms, namely 94.60%.

In the domain of deep learning algorithms, a CNN [18] that directly uses raw signals as input achieved an accuracy of 94.03%, a recall of 73.48%, a precision of 57.00%, and an F1-score of 64.20%. A transfer learning method employing ResNet [19] reached an accuracy of 98.00%. A DCNN with raw ECG signals for inputs [20] obtained an accuracy of 91.33%. The CNN-BiLSTM [21] and Deep Attention BiLSTM [22], both ECG classification methods based on CNN and LSTM, differ in that the latter incorporates an attention mechanism. They achieved accuracies of 96.77% and 96.72%, respectively. Lastly, the deep ensemble method proposed in this paper demonstrated optimal performance, achieving an accuracy of 98.55%, a recall of 98.48%, a precision of 98.44%, and an F1-score of 98.46%.

4.3Ablation study

To evaluate the contribution of the DWT-based feature extraction module, we conduct an ablation study on the proposed model. We only remove the feature extraction module from the proposed model, while the overall hyper-parameter settings remain the same. As shown in Table 3, while the multi-grained scanning structure is capable of enhancing the representation of the original data, the DWT-based feature extraction module improves the performance of the proposed model.