A Study On Deep Learning For Motor Imagery Electroencephalography Decoding

Brain-computer interface(BCI)system overcomes the dependence of peripheral nerves and muscles to establish a direct communication pathway between the brain and external devices,providing a new research approach to explore the brain cognitive mechanis.As a kind of typical spontaneous BCI system,motor imagery(MI)based BCI does not require the external stimulation,and records the EEG signals during imagining movements.Among the traditional MI decoding algorithms,common spatial pattern(CSP)based method and its extensions are representative.The idea of CSP-based methods is to spatially filter the EEG signals by solving the spatial filtering matrix,and then extract the energy features of each EEG channel for subsequent classification.However,these energy features no longer keeps the temporal representation,and thus ignored the further exploration of temporal information of EEG signals.Therefore,how to further exploit the temporal information of EEG signal is a challenge in MI decoding research.In addition,how to design an effective learning framework which enables the labelled and unlabelled trials are trained collaboratively,is also another challenge in current MI decoding research.To solve the problems and challenges in the existing MI decoding methods,in this thesis we introduce the deep learning method to improve the MI decoding performance,the main contents and results are as follows:1.In order to exploit the temporal and spatial information hidden in the EEG signals,a deep learning approach termed FBSF-TSCNN was proposed for MI decoding in this thesis.Firstly,the FBSF block spatially filtered the raw EEG signals,and the processed EEG signals still remain temporal representation.Then the TSCNN block extracted the features of the transformed EEG signals,classified these features,and returned the final decoding results.Especially,the FBSF-TSCNN is an effective tool to analyze the discriminative temporal and spatial information of EEG signals,which provides new technique for the research and application in MI-based BCI system.2.In order to mitigate the optimization difficulty of the deep model with random initialization in the case of insufficient training samples,a novel stage-wise training strategy is proposed for the TSCNN block in this thesis.Firstly,the feature extraction layers are trained by optimization of the triplet loss.Then,the classification layers are trained by optimization of the cross-entropy loss.Finally,the entire network(TSCNN)is fine-tuned by the back-propagation(BP)algorithm.Experimental evaluations on two public datasets reveal that the proposed stagewise training strategy helps the TSCNN block to yield significant performance improvement compared with the conventional end-to-end training strategy;the proposed approach FBSFTSCNN achieves significantly better decoding performance than other competing methods such as FBCSP-NBPW,and is competitive as the state-of-the-art method.3.In order to achieve collaborative training of labeled and unlabeled EEG trials,a semisupervised deep model termed SVAE based on the stacked variational autoencoder is proposed in this thesis.Firstly,the spatially filtered EEG signals with temporal representation can be obtained by applying the filter bank spatial filtering(FBSF)algorithm.Secondly,the envelope representation of these spatially filtered EEG signals is extracted via the Hilbert transform.Finally,the EEG envelope was inputted to the proposed model.Experimental results reveal that the proposed semi-supervised deep model achieves significantly better classification performance than the competing supervised models.Moreover,the proposed semi-supervised deep model is also scalable.Therefore,the proposed SVAE has a high value in the application of MI-based BCI system.