Research On Hybrid Continuous Control System Of Robotic Arm Based On Non-invasive Motor Imagery Eeg

As a new type of multidisciplinary technology,brain computer interface(BCI)has been applied in many fields and brought hope to these disabled persons with physical.The controlling of robotic arm based on BCI can revolutionize the quality of life and living conditions for individuals with physical disabilities.For a long time,researchers have been trying to make life easier for patients by using brain-computer interface technology to make them control a robotic arm flexibly by sending commands directly from their brain.Invasive electroencephalography(EEG)based BCI has been able to control multi-degree of freedom(DOF)robotic arms in three dimensions.However,these implants increase clinical risks of infection and brain tissue damage,so the invasive BCI is mostly applied to animal research.It is still hard to control a multi-DOF robotic arm to reach and grasp the desired target accurately in complex three-dimensional(3D)space by the non-invasive system mainly due to the limitation of EEG decoding performance.Motor imagery EEG signals are generated in the sensorimotor areas of the brain by imagining the movement of users' limbs.Motor imagery EEG related to motion directly and do not depend on visual or auditory stimulation,so it is a good method to control the robotic arm based on it.In the current researches on robotic arm control based on motor imagery,and most of them were discrete control systems.There were few researches could control the robotic arm moved continuously.If the participants want to control the movement of the robotic arm,he/she must switch the motor imagery task constantly,which makes user exhausted and the movement of the robotic arm is very unnatural.For online experiments,most studies did not consider the complex situation of multiple targets with obstacles.In order to solve the above problem,we proposed a non-invasive EEG-based BCI for a robotic arm control system that enables users to complete the multi-target reach and grasp tasks and avoid obstacles by hybrid control.The entire process can be roughly divided into three steps: firstly,the user's motor imagery based EEG was decoded into continuous commands to control the end-effector of the robotic arm to the surrounding area above the target.Then,the target that the user desired was captured by the eye tracker and sent to the Robot Operating System(ROS).Finally,the robotic arm would grab the target automatically when it get the position of the target.Meanwhile,the obstacle in the workspace was detected by a depth camera,so the robotic arm could intelligently avoid the obstacle during motion planning.The results obtained from the seven subjects demonstrated that motor imagery training could modulate brain rhythms and all of them completed the online tasks using the hybrid control based robotic arm system.The proposed system combining BCI with computer vision,gaze detection,and semi-autonomous guidance,drastically improve the accuracy of reach and grasp tasks and reduce the brain burden caused by long-term mental activities in the traditional control system.