Research On Coding-Decoding Of Visual EEG Based On Deep Learning

Vision is the perception that human beings deeply rely on,and one of the most significant research materials in neuroscience.In Brain-Computer Interface(BCI)researches,the BCIs based on visual EEGs are considered to be the most promising way for scalable application.However,most of the visual BCI systems are implemented by some encoded external stimulation,as humans can differentiate millions of scene information by vision without any coding in real life.Decoding and reconstructing the process of visual image coding in brain through EEGs and then developing new BCI methodologies,is an efficacious way for above problems.Based on the above issues,we proposed a new visual-based BCI paradigm.Unlike any other paradigms,the only differences between visual stimuli is about stimulus images,theoretically the diversity of evoked EEGs will entirely depend on the spatial distribution of stimulus images.In order to recognize and decode EEGs,a classifier based on LSTM was designed,with effective regularization methods,successfully solve the over fitting problem of EEG recognition.The model was also compared with other machine learning methods which commonly used in BCI system.Finally,the deep learning model constructed in this research have the highest accuracy of 88.6% in verification set,which is significant higher than other traditional methods.In addition,this study first employed Layer-wise Relevance Propagation method to interpret the deep neural network used in EEG signal processing,interpret the discriminant basis of network from the input;further correspond to the brain structure,explaining the results from physiology intuitively.This research has also realized decode human vision by taking LSTM as encoder,and Generative Adversarial Network(GAN)as decoder.By employing conditional GAN and other technique to optimize GAN,a coding-decoding system that can restore and reconstruct subject’s visual image was obtained.By human evaluation,the average similarity between real and fake image was over 77%.This study also try to decode mixed patterns in the last step,and analyze the decoding results.All the work proved that graphics can be used as an effective EEG stimulation,and the advantage of deep learning in EEG recognition.At the same time,the feasibility of spatial dynamics among EEG channels as EEG features was also clarified and visual image reconstruction was finally accomplished.These are expected to provide a new way to expand instruction set and new data analysis methods for visual BCI.