Research On Self-supervised Time Series Representation Learning For Sleep Staging

Sleep staging is an essential approach to study human sleep and can assist experts in the diagnosis of sleep-related diseases,which is of great significance in clinical research.While automatic sleep staging based on the deep learning obtained fast development,this kind of supervised learning models on the one hand,heavily relies on large scale labeled data.On the other hand,it is difficult to represent the internal rich semantic information data with information of labels,which is easy to cause overfitting and poor performance across individual.Recently,self-supervised learning has achieved outstanding performance in the field of representation learning,which can effectively work on target tasks with few labels.In order to solve the above problems,this paper studies self-supervised learning of physiological time series for sleep staging task.Firstly,Sleep DPC is proposed to learn the semantic representations of sequential signals based on context contrast predictive coding.The whole self-supervised classification process includes two stages: pre-training and fine-tuning.The model mainly builds auxiliary tasks for contrastive pre-training,and designs positive and negative samples,which including basic representation module and autoregressive prediction module to model the context of EEG signals in the time domain.The representation network is optimized by two contrast learning objective based on predictive and discriminative coding.Finally,in the fine-tuning stage,the parameters of pre-trained network are frozen,and a small amount of labeled data is used to classify through multi-layer perceptron.Experiments on sleep-EDF and ISRUC datasets show that Sleep DPC achieves good performance of temporal representation learning.Secondly,we construct a sleep staging framework(Co Sleep)based on time-frequency multiview of EEG signal for contrast learning.Based on Sleep DPC,we introduced complementary view(frequency domain)to mine more positive samples,and proposed a memory queue for dynamically updating negative sample representations.Thus overcoming the shortcoming of Sleep DPC,which lacking of sufficient positive and negative samples and further enhance the quality of representation learning.Co Sleep uses a twostage pre-training method.First,the two views are trained from scratch independently to obtain the initial representations,and then more semantically consistent positive samples are searched from the complementary view by multiview co-training.Experiments on extended datasets of sleep-EDF and ISRUC verified the effectiveness and superiority of the Co Sleep model.Meanwhile,ablation experiments demonstrated the contributions of different modules and proved the practical significance of the learned representations by visualization.Finally,we propose a method MMSleep,which combining multi-modal physiological signals to solve the problem of insufficient information only with EEG signal in self-supervised learning.For the physiological signals in the same modal,a contrastive learning framework based on data augumentation pretraining method is applied to construct robust representations;by introducing an autoregressive model based on attention mechanism,the model training realizes efficiently parallel computation.After the independent training process of two modals completed,the cross-training is applied to further mine more semantically rich samples.Experiments on two multimodal datasets,MASSSS3 and ISRUC,demonstrated that MMSleep achieved competitive performance with cross-modal co-training and got excellent performance on sleep staging task.