Hybrid Control Of Bionic Prosthetic Hand Based On SEMG And Computer Vision

The research of the bionic prosthetic hand and its control system is of great significance for improving the lives of those patients with upper limb amputation.At present,the research on the bionic prosthetic hand is mainly divided into two directions.One is bionic prosthetic hand based on sEMG signal,that is,the electromyographic signals generated by the residual muscles of amputees are taken as the source of control signals of prosthetic hands.Because of the non-linear,non-stationary and easily interfered characteristics of sEMG signals,furthermore,different amputees own different degrees of amputation and different residual muscles,it is difficult to develop a stable sEMG prosthetic system only depending on the sEMG signals.The other is to fuse the information of the vision sensor based on sEMG signals to make a joint decision to stabilize and control the prosthetic hand.Due to the fusion information of various sensors,this kind of research is more complex,but the control robustness of the prosthetic hand is better,which is one of the research directions in the field of medical rehabilitation.The prosthetic hand control system that integrates sEMG and visual information usually needs to solve three problems:(1)Classification of sEMG signals in amputees.Recognizing the movement intention through the sEMG signal is a key link to realize prosthetic hand control.The classification accuracy of the sEMG signal determines the accuracy of the prosthetic hand control.(2)Compliant grasp of the bionic prosthetic hand.Bionic prosthetic hand,as an auxiliary grasping tool for amputees,needs to meet the grasping function of a human hand,and it needs to independently control the grasping strength according to the stiffness of the object to be grasped to meet certain compliance.(3)Processing of visual feedback information,and integration of sEMG signals and visual information.In this thesis,a multi-sensor bionic prosthetic hand is studied.Based on the sEMG signal,the information of the vision sensor is fused to make a joint decision to steadily control the grasp of objects by the prosthetic hand,which mainly includes the following parts:(1)For the classification of sEMG signals in amputated patients,BP neural network was firstly built to analyze the myoelectric dataset of Ninapro DB3 amputation patients,and the effect of amputation degree on the results of sEMG classification was studied,and compared with the classification effect of machine learning methods.Secondly,the convolutional neural network CNN and the long short-term memory network LSTM are combined to obtain C-ConvLSTM and P-ConvLSTM networks,so that the convolutional network can learn the spatial characteristics of the sEMG signal,and the long short-term memory network can learn the timing excitation of the sEMG signal.With the Ninapro DB3 amputation patient sEMG dataset,the performances of the four networks such as CNN,LSTM,C-ConvLSTM and P-ConvLSTM in this dataset were compared,and the effectiveness and performance of the proposed method were verified.Finally,the influence of the selection of the time window size on the classification performance is analyzed.(2)Aiming at the problem of compliant grasping of the prosthetic hand,the finger dynamics model of the bionic prosthetic hand was firstly established and the Simulink simulation block diagram was built.Such influences as the target inertia coefficient,target damping coefficient,target stiffness coefficient and object stiffness of the admittance control algorithm on the control performance of the finger terminal force for the prosthetic hand were analyzed.Finally,the admittance control algorithm is tested on the hardware platform of the prosthetic hand.(3)In order to verify the grasping effect of hybrid decision which is based on sEMG and visual sense,firstly,the hardware platform for the control system of the multi-sensor bionic prosthetic hand is built.Secondly,the sEMG signal is not only online collected but also online applied.For the visual part,the MobileNet-SSD object detection algorithm is used to obtain the label of the grasping object.Finally,the hybrid grasping process and scheme of multi-sensor bionic prosthetic hand were physically designed to simulate the object grasping experiment of amputees,and the results of grasping based on sEMG and mixed sEMG as well as image information were compared accordingly.The results show that the hybrid grab strategy resembles the natural grab method more closely,and the robustness and stability of the hybrid grab strategy are also verified.