Research On EEG-Based Semantic Recognition And Visualization

Electroencephalogram(EEG)signals contain rich semantic information that objectively reflects neuronal activity in the brain.Effectively extracting the semantic features contained in EEG and visualizing them as the corresponding images,also known as Reconstruction from EEG into Image(RE2I),can provide an important basis for accurately understanding and evaluating people’s mental activities and cognitive states.However,due to the low signal-to-noise ratio and significant individual differences of EEG signals,there is still a lack of effective methods to extract semantic information contained in EEG effectively.This thesis first proposes a multimodal semantic recognition method based on EEG and the corresponding images to verify the feasibility of establishing a matching relationship between EEG and the corresponding images.Then,an EEG unimodal semantic feature extraction is studied to be closer to the practical application.Finally,a visualization system based on EEG semantic features is designed to verify the effectiveness of the proposed EEG unimodal semantic feature extraction method in this thesis.The main research contents are as follows:(1)Research on the feasibility of constructing the relationship between EEG and the corresponding images,this thesis proposes a multimodal semantic feature extraction framework(EDNet).EDNet includes the EEG semantic feature extractor EEG-VisualNet(EVNet)and the image feature extractor Dense Net.In addition,when exploring the above problems,a loss function applied to multimodal training is also designed,which provides a simple and effective anti-overfitting scheme for multimodal models.(2)Research on EEG unimodal semantic feature extraction,this thesis proposes an EEG unimodal semantic feature extractor EEG-Visual-Residual-Net(EVRNet).The EVRNet includes EVNet and Multi-Kernel Residual Block(MKRB),inheriting the advantages of EVNet compatibility with different shapes of EEG data;In addition,the MKRB designed in this thesis effectively alleviates the gradient explosion or disappearance problem,and enhances the semantic feature extraction ability and generalization of the model.(3)Research on EEG semantic feature visualization,this thesis proposes a RE2 I framework based on the diffusion model,namely EEG-Guided Diffusion Model(EGDM).EGDM consists of two parts: EEG Semantic Feature Extractor(EVRNet)and Image Generator.In the image generator,two EEG-Guided image generation algorithms(EG-DDPM algorithm and EG-DDIM algorithm)are designed by mathematical derivation,which solves the problem of insufficient generalization of existing EEG semantic visualization methods.