Research On Attention Mechanism Based Sleep Staging With Deep Learning

Sleep is a complex and essential physiological process in human life activities.With the acceleration of social rhythm,people’s sleep quality is affected by various factors.Sleep disorder has become a recognized social and livelihood problem in the world.In order to evaluate sleep quality more effectively,a variety of methods have been proposed for sleep staging tasks.At present,the sleep staging task has some pain points in terms of data and models.In the terms of data,because sleep EEG is usually unbalanced data,there are problems with popular tendencies in the training process and staging results.In addition,a variety of internal and external interference sources in EEG also seriously affect the validity of data.Among them,the effect of eye artifact on EEG is the most significant.In the aspect of model,the early expert artificial sleep staging method has some problems such as expert experience and artificial misjudgment.Traditional machine learning based sleep staging method is complicated in feature extraction.At present,automatic sleep staging based on deep learning is a hot topic,but there are some problems such as the slow improvement of accuracy of staging results,the complexity of models and the unapplicability of portable sleep monitoring equipment.In this paper,combining event-related potential technology and deep neural network technology,the following work has been done to address the above-mentioned pain points.In the data preprocessing part,this paper proposes a relative voltage threshold method to remove bad values to assist in eliminating electrooculogram artifacts.This method can reduce the artifacts during awake period and retain the information of REM period to provide more effective information to assist sleep staging.In addition,focal loss is introduced as the loss function in the model training to improve the imbalance of sample categories.In the aspect of model,this paper proposes an automatic sleep staging method based on attention mechanism,and constructs four automatic sleep staging models based on soft attention mechanism.In order to improve the staging effect of the existing automatic sleep staging model based on deep learning,the Deep Sleep Net-SE model and Deep Sleep Net-CBAM model are based on the model structure of Deep Sleep Net,and the soft attention mechanism SE module and CBAM module are respectively introduced.In the attention mechanism part,the bottleneck layer is used to replace the original full connection layer to reduce the frequent dimension transformation between layers,save the amount of calculation and increase the nonlinear expression ability of the model.The quick connection operation on the original Bi-directional LSTM is removed,and the residual structure is used to improve the attention mechanism,so that the network pays more attention to the learning of the attention mechanism.In order to explore the lightweight model for portable sleep monitoring devices,Attention Sleep Net-SE model and Attention Sleep Net-CBAM model are proposed.The model uses convolutional neural network,attention mechanism and dilated convolutional neural network to simplify the network effectively.It reduces the complexity of the model from the parameters of the model,the space occupation and the running speed of the model,so that the automatic sleep staging model based on deep learning can be used in portable sleep monitoring system and equipment.This experiment uses the sleep data of the top 20 individuals in the public data set Sleep-EDFx.The experimental results show that most of the four automatic sleep staging models based on attention mechanism achieved more than 85% overall accuracy,more than78% MF1 value and more than 85% WF1 value.Among them,the overall accuracy of the best model Deep Sleep Net-CBAM model is 86.84%,the MF1 value is 79.82%,and the WF1 value is 86.77%.The Attention Sleep Net-SE model and the Attention Sleep Net-CBAM model prove the effective simplification of the network with a model parameter volume of about0.3M,a space occupation of about 1.4MB,and an average prediction time of about 0.15 milliseconds per sample.The automatic sleep staging method based on attention mechanism proposed in this paper not only improves the performance of the model,but also makes the model lighter,which provides a new idea for the research of automatic sleep staging method based on deep learning.