Humanoid Robot Control System Based On Augmented Reality Brain-computer Interface

Brain computer interface(BCI)is a kind of communication or control system that allows human brain to interact with external devices in real time,which helps people with disabilities regain the ability to communicate and control the outside world.In brain computer interface,steady state visual evoked potential(SSVEP)is a common input signal of human-computer interaction system.However it needs a fixed computer screen as a visual stimulator,which limits its application flexibility.Robot is usually the executor of external command,but the traditional single human-computer interaction can not meet the needs of people.Therefore,people need to seek a portable intelligent system.In this study,a brain computer interface experimental control platform based on augmented reality(AR)is built.The platform uses holographic hololens as a visual stimulator to induce EEG signals.The user does not need to carry out visual stimulation in a fixed position which can enhance the applicability in complex environment and achieve more natural human-computer interaction.In order to make the external equipment highly intelligent,a multi-sensor fusion humanoid robot is introduced based on the platform.System one can control the human robot Nao to grasp and place objects.System two can quickly control Nao to complete the "maze" walking.In system one,twelve healthy subjects were tested off-line.When the stimulation time is 2 s,the recognition accuracy of EEG signal is 94.05%,and the information transfer rate(ITR)is 30.32 bits / min.According to the change of information transmission rate with time window,the ITR of the system is the highest when the stimulation time is 2 s.Subsequently,this study uses augmented reality glasses to carry out online experiments to control the humanoid robot.The online experiment includes two tasks: random prompt and independent choice.Twelve healthy subjects participated in the online experiment.The average accuracy of random prompt is 95.83%,and the information transmission rate is 32.32 bits / min.In the autonomous selection,the subjects can complete the task,and the robot can recognize and grasp the target in the area.A monocular ranging model is established based on monocular vision imaging.Through the measurement,the robot can reach the destination accurately.At the same time,the forward and inverse kinematics model of the arm is established,and the robot can accurately grasp the target.The experimental results verify the feasibility of the system.The online experiment of system two is carried out to verify again.Twelve healthy subjects were tested off-line.The average accuracy of random prompt is 98.15%,and the information transmission rate is 34.03 bits / min.The subjects controlled the robot to complete the walking with mark in the "maze".The experimental results show that the system is feasible,which is helpful to promote the application of augmented reality brain computer interface in robot field.