Development Of Man-machine Interaction System Based On EEG-EMG And Virtual Reality

EEG signals and EMG signals are important physiological signals of the human body,they come from the brain and nervous system.EEG and EMG contain a wealth of human body intention information.Decoding and understanding the in-depth information of EEG and EMG is the basis for the realization of a human-computer interaction system based on EEG and EMG.This technology will have the potential to change the way people interact.The research and application of EEG and EMG system is very wide.With the convenience of EEG and EMG collection equipment in the future,the interaction method of EEG and EMG may provide drivers with a new interactive experience.Therefore,this paper has completed the research and development of a human-computer interaction system based on EEG-EMG and virtual reality.The specific work is as follows:(1)HTC Vive and Unity3 D are used as the software and hardware developed for virtual reality to build a VR visual system,which provides a realistic virtual scene and immersive interactive experience for the human-computer interaction system.The HTC Vive head-mounted display and its positioning base station can locate and obtain head movement information,providing a full range of visual effects for the human eye.(2)Improved the traditional visual guidance method in the process of collecting EEG and EMG data,and built an EEG and EMG data collection system based on virtual reality.The collection process of motor imagination EEG data provides visual cues and guides the participants’ movement imagination through virtual human movements.This method changes the monotonous and tedious data collection process and makes the experimental process more vivid and interesting.The EMG data collection process combines motion capture gloves to capture gestures into the VR scene and give the participants visual feedback,making virtual reality-based EMG data collection possible and enriching the EMG data collection methods.(3)The EEG data and EMG data are processed separately based on a one-dimensional convolutional neural network,and the classification results are analyzed.The average classification accuracy of EEG data of motor imaging is 72.7%,and the average classification accuracy of EEG data of different gestures is 84%.Then according to the classification effect of EEG and EMG and combined with the function of humancomputer interaction system,this paper has done the information fusion of EEG and EMG at the decision-making level.(4)First,build a virtual cockpit scene based on the driving simulation system through Unity3 D,and then combine the brain-computer interface system,the surface electromyography system and the virtual reality system to build a human-computer interaction platform.The interactive way based on EEG and EMG realizes the interaction between people and the car and increases the immersion of the interaction through virtual reality.