Research On Image Oriented Brain Activity Reconstruction Under Visual Stimulation

Research on image oriented brain activity reconstruction under visual stimulation is of great significance to patients who have lost the ability to speak and move,and has a wide range of applications in the fields of diagnosis and rehabilitation of neurological and psychiatric diseases and military fields because of its fast communication speed,unnecessary training of subjects and low intersubject variability.However,existing methods for brain activity reconstruction under visual stimulation still suffer from problems such as loss of partial feature information due to vector dimensionality reduction during feature extraction,and mismatch between the amount of image and EEG signal data.In this thesis,the end-to-end brain activity reconstruction in response to visual stimulation is focused on.The research includes model design,algorithmic network improvement and experimental paradigm optimization,and the specific research content is as follows.(1)An end-to-end EEG-image mapping model under visual stimulation is built.Aiming at the commonality problem of vector dimensionality reduction and thus losing feature information in the research of visual evoked Brain-Computer Interface,the end-to-end mapping model for direct reconstruction of EEG signals into images is designed in this thesis to establish the mapping relationship between two signals belonging to different signal domains,which avoids the information loss problem in the decoding process by using the cross-layer connection property.It is verified that this model can reconstruct the characterization of different types of visual stimulation.(2)Quad-GAN network algorithm for image reconstruction is proposedTo address the problem that image and EEG signals are in different signal domains and have no common features in terms of amplitude and frequency and cannot be mapped directly to each other,a connection signal is designed in this thesis as a bridge between the two signals,making it possible to reconstruct the cross-domain mapping of the two signals.For the problem of mismatch between image and EEG signal data volume,the method of expanding EEG signal volume by nearest neighbor interpolation is adopted in this thesis through theoretical analysis and experimental verification,which makes the two data volumes match.By introducing a multi-scale discriminator,the receptive field is increased by discriminating at different scales at the discriminant end,and features are extracted from the receptive field size of different scales to enhance the discriminator’s ability,thereby improving the quality of the reconstructed images.In order to solve the problems of frame chaos,frame error,and non-fluidity of reconstructed video,we draw on the optical flow method in the field of video processing to constrain the front-to-back relationship of the reconstructed video frame by extracting the relationship between the front and back positions of the video frames.After combining the original image and the optical flow,the video frames generated through the end-to-end mapping network have stronger constraints,effectively improving the fluency of the reconstructed video.(3)Image oriented brain activity reconstruction system under visual stimulation is implementedTo solve the critical problem that the existing EEG acquisition paradigm does not take into account the subject’s concentration and affects the quality of EEG signals,based on the actual study,the corresponding experimental paradigm is developed in this thesis and the experimental device used to monitor the subject’s concentration is fabricated to enhance the quality of the acquired EEG signals.Image-oriented brain activity reconstruction under visual stimulation implementation system is built,and experiments are conducted based on the professional EEG acquisition platform of our group to train,validate,and evaluate the proposed algorithm to realize the decoding and reconstruction of visually evoked brain activity for images.