A Study On Hybrid Human-computer Interface For Virtual Reality And Its Application

Virtual reality(VR)is a dynamic and interactive three-dimensional(3D)virtual world,which can provide a strong sense of presence for users.Therefore,the human-computer interaction without breaking immersion is the difficulty of VR.Non-manual human-computer interface(HCI)is an interactive method that enables computer to perform some operations without upper or lower limbs.The advantage of non-manual HCI is natural and barrier free interaction,compared with traditional VR interactive tools such as handle,ray projector,etc.This thesis mainly studies the human-computer interaction method based on non-manual signals(EEG,EOG)and the practical application in VR interaction of hybrid human-computer interface(h HCI).Firstly,to address the problem of existing brain-computer interface(BCI)devices,such as bulky and few types of signals that can be collected,this thesis proposes an open modular acquisition architecture of hybrid brain-computer interface(h BCI)signals for VR.The architecture adopts a reconfigurable method,which can be compatible with other VR devices,directly realize the end-to-end operation,and support developers to expand functions for cooperation and comparative research.We solve three difficult problems: sensor electrode design in moving state,high common mode rejection circuit in complex environment,and general data marking method of multiple brain and bioelectrical signals.Based on this architecture,two prototype systems are designed,and the complete performance tests of gain,common mode rejection ratio,noise,power consumption and human body signal are carried out.The data analysis results prove the effectiveness of the proposed architecture.Secondly,a new HCI based on EOG for VR is proposed.The interaction method is designed based on the correlation between blink and button flicker in user graphical interface.This method combines the traditional EOG detection algorithm with flashing button stimulation similar to P300.The user only needs to blink twice synchronously when the target button is flashing to complete the selection.Using this method,the number of instructions is increased greatly,and the accuracy of EOG instructions is improved.Based on this method,a music on demand system with multi-layer GUI structure is developed in virtual environment.Eight healthy subjects and five patients with spinal cord injury participated in two online experiments.The results show that the system is simple,excellent performance,and suitable for VR interaction.Thirdly,a new immersive brain-computer interaction method for P300 is proposed.This method uses three-dimensional(3D)objects in the environment as stimuli to improve the naturalness and directness of VR interaction.Furthermore,the self-adaptive bayesian linear discriminant analysis(SA-BLDA)algorithm is used to improve the efficiency of target recognition.We carried out a comparative experiment between this method and the traditional two-dimensional(2D)paradigm based on flashing button.The experimental data show that the proposed new paradigm can elicit higher amplitudes of P300 waveforms than the 2D paradigm,without breaking immersion of VR.Finally,a h HCI based on EOG and voice is proposed to solve the problems of non-manual control wheelchair,including low accuracy,high error rate and long response time of command.In this method,there is only one button in the user interface.the user interacts by blinking synchronously with the flashing button and issuing voice commands at the same time.In the virtual park scene,we conducted a comparative experiment between this HCI with other methods to control the wheelchair.Compared with the single round EOG control system and the wheelchair control method designed by previous researchers,the h HCI based on EOG and voice has a greater improvement in the accuracy and false alarm rate,and therefore the asynchronous performance of the system is more excellent.