Research And Implementation Of Audio And Video Communication System Based On Hybrid Brain-Computer Interface

In modern society,audio and video communication has been widely used in daily communication,business talks,online education and medical services,and has become an indispensable part of people's life.The existing audio and video communication software can only be used with the help of traditional human-computer interaction devices such as keyboard,mouse or touch screen.But millions of people around the world suffer from severe motor function diseases,such as cerebral palsy,stroke and spinal cord injury.Because they can not move their hands,they can not use the existing audio and video communication software independently.This brings great inconvenience to their lives.By reading brain activity information,Brain-Computer Interface(BCI)can realize direct communication between brain and machine without relying on peripheral nerve and normal muscle conduction channels.Based on this,the main research content of this thesis is the audio-video communication system based on the hybrid brain-computer interface,which aims to help patients with motor disorders communicate with others better and improve their quality of life.Firstly,we propose a hybrid BCI paradigm by studying three single modality BCI paradigms of SSVEP,P300 and EOG.The asynchronous performance of SSVEP-BCI is not good.P300-BCI need to repeat many times.EOG signal strength is high,and is easy to detect.But unconscious blink will bring a certain false trigger rate to the system.Three kinds of signals are combined to improve the performance of BCI.Specifically,SSVEP(steady state visual evolved potential)and P300 potential are detected simultaneously to realize the state control of brain switch.And EOG signal generated by blinking is detected to send out specific control instructions.Secondly,we designed an audio and video communication system based on hybrid BCI.The system consists of two sub modules,BCI module and audio-video communication module.In the BCI module,the main tasks include designing the multi-level menu of the system,designing the asynchronous detection algorithm of SSVEP and P300 mixed paradigm,and designing the online detection algorithm of EOG signal.In the audio and video communication module,the main work includes the design of the software architecture of the audio and video communication module,the design of the signaling protocol,and the design of the real-time processing program of audio and video data.Finally,we test the system in detail to verify the reliability and stability.The test is divided into two aspects,audio-video communication quality test and brain computer interaction performance test.We invited eight healthy students to participate in BCI experiments to test the actual performance of the system.The experimental results show that the system has high practical value.