Research On Classification Of Motor Imagery Based On Two-stream Convolutional Network

Motor imagery(MI),an important application of brain-computer interface(BCI),is essential for motor rehabilitation training.MI have successfully applied in many fields,including military,transportation,entertainment,and etc.Therefore,accurate MI classification is of importance.The Electroencephalogram(EEG)signals of different motor imagery have significant differences in temporal domain,spectral domain,and spatial domain.However,most researches only consider information of a single dimension or two dimensions,and cannot fully capture the internal features of EEG signals.Besides,the information carried by EEG signals is not equally important.Existing methods cannot adaptively capture the most valuable information in all dimensions.Furthermore,deep network-based classification models require plenty of computation capabilities,which is infeasible to be deployed on mobile devices.To solve the above problems,this paper proposes a two-stream framework to simultaneously represent the time-frequency-space information of EEG signals,and designs two motor imagery classification algorithms and one model compression method.The main research contents are as follows:(1)A classification model of MI-based on two stream graph representations is proposed.According to the non-Euclidean characteristics of the EEG electrode distribution,the model uses a graph structure to represent the EEG space,and maps the time domain information and frequency domain information of the EEG signal to the graph,forming two stream representations of space-time stream and space-frequency stream.A time-frequency-space attention mechanism is designed to adaptively capture the most valuable information in the time,frequency,and space dimensions.Introducing graph convolution to obtain a two-stream attention based graph convolutional network(TAGCN),and extract spatial correlation,time dependence,and frequency dependence from two-stream graph representations.A large number of experiments are conducted on three public BCI datasets.The experimental results show that TAGCN is superior to the existing classification methods,which verifies the effectiveness of TAGCN in the classification of motor imagery.(2)A classification model of MI-based on two stream three-dimensional(3D)representations is proposed.According to the relative distribution between the electrodes,the model uses a matrix structure to represent the brain electrical space,and obtains two stream 3D representations of space-time stream and space-frequency stream.A timefrequency-space cascade attention mechanism and parallel attention mechanism are designed for 3D data,which can adaptively capture the most valuable information in all dimensions.Combining the attention mechanism and 3D convolution,a two-stream attention based 3D convolutional network(TA3D)is proposed to recognize motor imagery.A large number of experiments have been conducted on three public BCI datasets.The experimental results show that TA3 D is significantly better than existing classification methods.Besides,the performance of the parallel attention mechanism is better than the cascade attention mechanism.In addition,through the visual analysis of the internal structure of the model,the effectiveness of TA3 D model is further explained.(3)A classification model compression method based on knowledge distillation is proposed.This paper designs a lightweight network based on the idea of knowledge distillation to make the lightweight network critically learn the probability distribution of TA3 D output,and realize knowledge transfer and compression.Experiments show that the model after knowledge compression not only have higher classification performance but also have a smaller memory,calculation,and time overhead.This method promotes the deployment of deep networks in wearable medical devices and provides new ideas for future practical applications.