The proposed multiscale DSC-SPP architecture is shown in Fig. 1. First, multiscale feature extraction is performed on the preprocessed EEG signals through 0.5 s, 0.75 s and 1 s elastic time windows, and the signals are subsequently input into a depthwise separable convolution based on spatial pyramid pooling to obtain the depth features of signals of different scales. The fused features generated by the concatenate layer are subsequently fed to the sigmoid layer for spindle detection. In Depthwise Separable Net (DSNet), we incorporate dense connections, which help to minimize the gap between the input and output layers. This approach facilitates smoother propagation of gradients and feature information throughout the network. In addition, when there is less training data, dense connections also play a role in regularizing the model and reducing the risk of overfitting. The following sections demonstrate the proposed fusion architecture and its modules in detail.

Fig. 1.

The framework of the proposed multiscale DSC-SPP model. The network first consists of three different DSC networks, running different context input sizes at the same time. The output of each part is then passed through the maximum pooling layer and regularization. The SPP layer is installed between the convolutional layer and the fully connected layer. Then, the three different feature outputs are combined at the end of the network to obtain rich spindle features. Finally, the sigmoid function is installed in the last fully connected layer to determine whether spindles exist in the feature. DSC, depthwise separable convolutional; SPP, spatial pyramid pooling.

The sliding window is a common data processing technique that is easy to implement and understand. By defining the window size and sliding step, the spindle signal can be segmented into multiple windows for processing. Feature extraction algorithms can be applied to each window to extract spindle-related features such as frequency, amplitude, and duration. However, when using sliding windows for spindle detection, it is important to select an appropriate window size and sliding step, as the duration of spindles can vary, typically ranging from 0.5 seconds to 3 seconds. This ensures that the windows adequately cover the time period of the spindle. The computational complexity of sliding windows increases with increasing window size.

In time series data, the elastic time window [35, 36] can be viewed as a sliding window of variable length. Unlike traditional fixed-size windows, elastic time windows can adaptively adjust their length according to the characteristics of the data, thereby better capturing key information in the data. This approach allows the use of different strides to split consecutive time windows in the same dataset, thus creating multiple scale datasets. The calculation formula for the elastic time window is as follows:

(1)$$window\mathrm{\_}size=M*K$$

(2)$$stride=S*K$$

where M and S are the fixed window size and step size, respectively, and K is the coefficient adjusted according to different scales.

Multiple datasets can be generated by slicing and segmenting the raw signal and labels using different step sizes and window sizes. Specifically, for the i-th scale, the window size is M * i, and the step size is S * i.

In the process of data slicing and scaling, the time interval between windows is controlled by adjusting the step size, thereby achieving elastic partitioning of the dataset. Different step sizes will affect the degree of overlap between windows and thus affect the training effect of the model. The elastic time window can help the model capture more time series features and improve its generalizability.

We validated our approach on the DREAMS dataset (https://zenodo.org/records/2650142#.YRtw6o4zY2w) from the Sleep Laboratory of Andr vsamsale Hospital in Belgium [51]. The DREAMS dataset comprises 30-minute polysomnography (PSG) excerpts obtained from eight subjects, including four males and four females aged 45.88 $\pm$ 7.87 years. These individuals exhibit various sleep pathologies, such as dysomnia, restless syndrome, insomnia, and apnea/hypopnea syndrome. The PSG excerpts in the DREAMS dataset consisted of three EEG channels (FP1-A1, C3-A1, O1-A1), two electro-oculogram (EOG) channels, and one electromyography (EMG) channel. Two sleep experts independently scored the excerpts based on the C3 channel. The sleep spindles were manually tagged by two experts on either the C3-A1 or CZ-A1 channel. The combined annotations, based on the “OR” criterion, were considered accurate references for determining the start and end points of the spindles. Further details can be found in Table 1.

The table shows that each excerpt in the DREAMS dataset is 1800 * 200. In our work, we resampled excerpt 1 and excerpt 3 to 200 Hz using a configurable sampling algorithm for t-wise interaction sampling [52]. For training and validation, we utilized the first six excerpts from the dataset.

In the suggested framework, the Adam optimizer is used to optimize the model by starting with a learning rate of 0.0001 and a batch size of 64 [53]. Additionally, dropout and early stopping techniques are implemented to prevent overfitting. The dropout strategy has demonstrated superior performance compared to other regularization methods [54]. A specified percentage of the input units in each layer (excluding the first layer) is randomly set to zero. Throughout our experiments, we empirically established a dropout rate of 0.2.

In our approach, we adopt a subject-independent methodology for training and testing the model. This implies that the data in the training set and the data in the test set are sourced from different subjects. To ensure nonoverlapping testing, we employed a 6-fold cross-validation scheme in which each fold involved the use of data from different subjects for testing, while the remaining subjects’ data were used for training. This process allowed us to thoroughly evaluate the model’s performance across all subjects in the dataset. As part of the 6-fold cross-validation, we divided the dataset such that 70% of all the samples were used for training and the remaining 30% were used for validation in each fold. During training, we also included 30% of the data as independent validation samples for early stopping purposes. We then averaged the results obtained from each fold to evaluate the overall performance of the model. This approach is described in detail in third order cumulant function (ToC) [55].

The model utilizes the sigmoid function to produce a probability ranging from 0 to 1, indicating whether a given EEG epoch belongs to the main axis. During the training phase, the probability obtained from the model is applied to each data point within the epoch. In the testing phase, the EEG signal is divided into segments three times, with each segment having a different time window. The time windows for segmentation wesssre 0.5 seconds, 0.75 seconds, and 1 second. As a result, each data point will have three distinct probabilities. For a given excerpt x consisting of n data points, the final output for each data point is determined by selecting the highest probability value out of the three distinct probabilities associated with that data point:

(4)$$\mathrm{P}\left({\mathrm{x}}_{\mathrm{i}}\right)=\max ({\mathrm{P}}_{0.5}\left({\mathrm{x}}_{\mathrm{i}}\right),{\mathrm{P}}_{0.75}\left({\mathrm{x}}_{\mathrm{i}}\right),{\mathrm{P}}_1\left({\mathrm{x}}_{\mathrm{i}}\right))(1\le \mathrm{i}\le \mathrm{n})$$

where $P_{0.5}(x_i)$, $P_{0.75}(x_i)$, and $P_1(x_i)$ represent the probability values assigned to the i-th data point within their respective elastic time windows. By comparing these probabilities to a predetermined threshold value, continuous data points that exceed the threshold can be merged to create an estimated spindle wave. Thresholds were obtained through web searches.

The occurrence of spindles in the DREAMS dataset was distributed across stage 2 or stage 3 NREM (N2, N3) sleep. The density and duration of these spindles varied depending on the subject and their respective sleep pathology. Due to the different lengths of the spindles, the predicted spindles and the annotated spindles are not accurately matched. When the overlap between the predicted spindles and the annotated spindles significantly exceeded a specific threshold, the study treated them as spindles. Intersection-over-union (IoU) is utilized to quantify the overlap between predicted and annotated spindles, providing a measure of their similarity.

(5)$$IoU=\frac{N_{x_{\text{predicted}}}\cap x_{\text{annotated}}}{N_{x_{\text{predicted}}\cup x_{\text{annotated}}}}$$

where $N_{x_{\text{predicted}}\cap x_{\text{annotated}}}$ and $N_{x_{\text{predicted}}\cup x_{\text{annotated}}}$ represent the number of data points that cross and merge, respectively, between the predicted spindles and labeled spindles, as shown in Fig. 5. Predicted spindles with IoUs greater than the threshold $\delta$ are marked as true positives (TPs). In this study, $\delta$ defaults to 0.25, following the common definition from previous work [56].

Fig. 5.

A simplified diagram illustrating the ground truth and detection process. The “ground truth” corresponds to the “annotated”, while “detection” corresponds to the “predicted”. The intersection of the ground truth and detection data is referred to as an “union”, while the union of the two is called an “intersection”. Predicted spindle waves with an IoU greater than the threshold are labeled “TP”. TP, true positive; IoU, intersection-over-union.

We marked true positives by calculating the IoU overlap between the predicted spindle waves and expert-labeled spindle waves. Therefore, our model outputs label the predicted spindle waves and outputs the number of predicted spindle waves rather than a spindle length identifier.

When working with imbalanced datasets, a high accuracy may not necessarily indicate superior performance of the spindle detection model due to the larger proportion of negative samples. We calculate six commonly used evaluation indicators, accuracy, precision, F1-score and negative predictive value (NPV), expressed as follows:

(6)$$\text{Accuracy}=\frac{\mathrm{TP}+\mathrm{TN}}{\mathrm{N}},\mathrm{N}=\mathrm{TP}+\mathrm{TN}+\mathrm{FP}+\mathrm{FN}$$

(7)$$F1-\text{score}=2\cdot \frac{\text{Sensitivity}\cdot \text{Precision}}{\text{Sensitivity}+\text{Precision}}$$

(8)$$Precison=\frac{TP}{TP+FP}$$

(9)$$NPV=\frac{TN}{TN+FN}$$

(1) Advanced Performance Solutions: In recent years, many scholars have proposed effective spindle detection algorithms based on the DREAMS dataset. Likewise, we further propose a new spindle detection method for multiscale feature extraction. Compared with the solutions of other scholars, our proposed solution has the best overall performance.

(a) Sun et al. [57] proposed a multi-instance learning framework based on convolutional neural networks, CNN- Multiple Instance Learning (MIL). This framework assumes that only a portion of each labeled spindle segment contains real spindle patterns and learns spindle-related features by distinguishing informative instances in the feature learning stage.

(b) Jiang et al. [58] proposed a two-stage method. In the predetection stage, the Teager energy operator with adaptive parameters is used to discover as many candidate sleep spindles as possible, and a classifier is used to further identify the true sleep spindles among all the candidate spindles.

(c) Kulkarni et al. [25] proposed a deep learning strategy, SpindleNet. The fixed-scale features learned by the CNN are further passed to the RNN, and the RNN output (from 50 time steps) of subnet 1 is combined with the RNN output feature of subnet 2 to achieve spindle multiscale feature fusion.

(d) Chen et al. [36] proposed a method of mixing EEG signal features using an elastic time window to adapt to significant changes in spindle duration and mixing depth features with the information entropy of EEG signals for spindle detection.

(2) Baseline models: For more effective verification, we implemented three baseline models to further compare with published solutions and to benchmark our proposed solutions.

(a) In the pre-detection stage, we use 0.5 s, 0.75 s and 1 s elastic time windows to segment the EEG signals to obtain EEG signals of different scales, which will be used to obtain multiscale characteristic spindles.

(b) We add the SPP layer in the feature fusion stage to fuse inputs of different scales into a fixed-size feature matrix. Specifically, three spatial bins of different sizes (1 * 1, 2 * 2, and 4 * 4) are set in the SPP layer, generating a 21 * N output.

(c) We use the DSC variant as the basic model of the CNN. The network consists of three repeated DSC layers, a concatenate layer, an SPP layer, and a fully connected layer. Specifically, for each DSC, 3 * 3 kernels are used, ReLU activation is used with a stride of 1, and the output range is set between 0 and 1 through the sigmoid function.

To gain a better understanding of the contributions of each component in our framework, we conducted a series of ablation and analysis experiments. These experiments aimed to determine the impact of different modules on the performance of the final model. Overall, our research findings provide evidence for the effectiveness of the proposed framework and emphasize the importance of considering multiple scales in spindle wave detection.

3.5.1 Multiscale Feature Extraction

The performance of a model utilizing an elastic time window design is evaluated and compared to that of a model with a fixed-length window (0.5 s) in this series of experiments. As shown in Table 2, compared to the method using a fixed window, the model using an “elastic” time window has higher accuracy.

Table 2.Results of the ablation experiment based on the DREAMS dataset.

	Accuracy (%)	Sensitivity (%)	Precision (%)	F1-score (%)	p value
DSC	92.84	49.90	48.46	37.17	0.1093
DSC+Multiscale	95.28	49.61	49.08	49.31	0.0352
DSC+SPP	94.87	49.99	49.85	49.46	0.0462
DSC+Multiscale+SPP	95.75	50.02	50.14	49.47	-

Moreover, to observe the impact of the IoU threshold on performance, we used fixed windows and elastic windows to test the spindle proportion detected under different thresholds. As shown in Fig. 6, when the threshold is greater than 0.3, most spindles can be predicted, and compared with that of the fixed window in the elastic time window, the proportion of spindles with IoU $>$0.5 is 11% greater. The results showed that the elastic time window design has advantages in detecting the onset and location of spindle waves, as it can better adapt to different temporal and spectral features as well as duration variations. This approach significantly improves the detection performance of spindle waves.

Fig. 6.

Distribution of IoU on the predicted spindle epochs. The pink bar represents the elastic window, while the yellow bar represents the fixed window. The proportions of detected spindle waves were tested under different IoU ranges.

3.5.3 Detection Results with Annotations

Fig. 7 illustrates the outcomes of the detection approach by employing expert-annotated events as the ground truth in the DREAMS dataset. Fig. 7a shows that within the 20-second raw EEG segment, the experts identified a total of 3 spindles. On the other hand, Fig. 7b shows the spindle candidates detected using the elastic time window after data filtering (the brown area represents the detected spindles). Compared with the spindle waves (red rectangular area segments) marked by experts, 5 spindle waves were detected in this step. Fig. 7c shows the final recognition result after the SPP combines the spindle wave features of each candidate. The final detection results are approximately consistent with the spindles annotated by experts. A comparison of the results in Fig. 7b,c reveals that multiscale feature extraction in elastic time windows can be used to search for ambiguous candidate spindles, whereas SPP performs multiscale feature fusion to identify accurate results.

Fig. 7.

The detection results of each step are analyzed using the proposed method. (a) The blue data represent a sample of unfiltered 20-second raw EEG segments, where the red rectangular window refers to the expert-annotated spindles. (b) The blue data represent filtered EEG data, the brown rectangle represents a large number of sleep spindle candidate data points searched in the elastic time window, and the red rectangle represents the real spindle. (c) The brown rectangle represents the sleep spindle identified by the SPP, and the red rectangle represents the real situation. EEG, electroencephalogram.

To assess the performance of our proposed method, we compare its performance with those of four state-of-the-art methods that were previously mentioned. The compared methods include the multi-instance learning framework (CNN-MIL), a deep learning strategy (SpindleNet), a two-stage method (two-stage), and label noise identification (Labelwix).

When using other machine learning-based methods, all training, validation, and test sets in each experiment were the same as those used by this model to ensure a fair comparison. Table 3 (Ref. [25, 57, 58, 60]) shows the evaluation results of our proposed method and the abovementioned methods on the DREAMS dataset.

Fig. 8 shows several examples of detection results from different methods. In each subfigure, the first row is a 20 s EEG signal, and the shaded rectangle represents the real situation. As shown in Fig. 8a,b, the predictions in this article include all true spindles, and the polarization positioning of spindle waves is more consistent with the real situation than is that of other methods.

Fig. 8.

Visual comparison of the spindle detection results with existing methods. The shaded rectangle and the first black line represent the adopted ground truth (GT), which comes from expert annotations. The colored lines correspond to the occurrence of spindles predicted by the automatic method. (a) The EEG signal between 7935s and 7955s. (b) The EEG signal between 13,560s and 13,580s.