Research On Human Computer Interaction Technology Of Upper Limb Rehabilitation Based On Gesture Recognition And Speech Control

As the population ages,there are more and more patients with upper limb motor dysfunction caused by cardiovascular and cerebrovascular diseases.Exercise therapy can effectively help patients with limb rehabilitation,but the number of existing rehabilitation physicians is insufficient and the ability level is uneven.With the rapid development of robot technology,upper rehabilitation robot emerges as the times require.The application of upper limb rehabilitation robot can reduce the burden of rehabilitation physicians with lower cost.It can effectively alleviate the lack of rehabilitation physicians and help patients achieve the purpose of rehabilitation training.However,after the research and analysis of the existing upper limb rehabilitation robot,there are also some shortcomings,such as poor versatility and human-computer integration.At present,how to achieve better human-computer interaction between patients and rehabilitation robot has become a research hotspot.According to the current development status of the upper limb rehabilitation robot,we design the upper limb rehabilitation robot independently,build the experimental platform,and realize the human-computer interaction mode based on the combination of gesture recognition and speech control.The specific research contents are as follows:Firstly,according to the characteristics of human upper limb movement,a two limb universal upper limb rehabilitation robot is designed to meet the needs of left and right limbs.In order to verify the rationality,the statics,kinematics and dynamics simulation analysis are carried out respectively.The software and hardware system development based on CAN bus and the upper computer software development based on CANopen communication protocol are studied.Secondly,a non-contact human-computer interaction mode combining gesture recognition with speech control is proposed.This paper studies gesture recognition technology based on Kinect.Pixel filter and improved median filter are used to repair Kinect depth image.Kinect skeleton acquisition technology is used to track and locate left hand and right hand respectively.Hand segmentation is performed based on depth threshold,and HOG features of hand region of interest are extracted to realize real-time static gesture recognition based on SVM.In order to judge the results of gesture recognition and realize the motion control of the upper limb rehabilitation robot,the speech recognition technology for isolated words is studied.The speech signal is preprocessed successively,including sampling quantization and other steps.The MFCC feature parameters are extracted to obtain the static and dynamic feature information of the speech signal.The DTW algorithm and the improved DTW algorithm are used for recognition and comparison,Finally,the improved DTW algorithm is selected for speech recognition.Finally,in order to verify the rationality of the design,a rehabilitation training experiment was carried out on the upper limb rehabilitation robot.Based on the completion of gesture recognition and speech recognition,it was integrated with the upper limb rehabilitation robot to build a human-computer interaction.The experimental platform is used to test the realization of multiple rehabilitation modes for the training of the left and right limbs.