Study On Human Motion Perception And Recognition Technology Based On Monocular Video For Training-evaluation Of Upper Limb Motor Function Rehabilitation

The number of disabled persons in China is about82.96million, and physicallydisabled patients are with the highest proportion of29%in all patients withdisabilities. Especially in recent years, the number of upper limb motor functiondisorder because of the accidents as well as stroke and other diseases is significantincrease in the trend. Upper limb motor dysfunction impact on the patient’s lifeseriously and reduce the patient’s quality of life. In addition to the basic surgicaltreatment, rehabilitation is an effective and absolutely necessary means to restorepatient’s motor function. Traditional rehabilitation is that patients do some repetitivetask training in the help of physical therapist with high cost, poor autonomy, at thesame time, and no real-time and accurate evaluation of the rehabilitation.This thesis proposed a new technology about human motion perception andrecognition based on monocular video for training-evaluation of upper limb motorfunction rehabilitation. Human-machine interaction control was achievedautomatically through movement tracking and static gesture recognition during therehabilitation process and the upper joint angles was acquired online forrehabilitation evaluation. In this study, monocular camera was used to acquire thevideo images from the subject with the glove of special color in three kind of typicalupper limbs motion mode (shoulder joint coronal movement, shoulder joint sagittalplane motion, elbow joint sagittal plane motion), and the video image of ninegestures command. The height of the camera and its distance from the human bodywas determined according to the human anatomy and camera transmission model.The Camshift algorithm was used for subsequent movement hand target tracking,which had better robustness and lower time complexity. Tthe coordinates of thereal-time location of the hand portion were obtained by tracking. Static gesturerecognition process included pretreatment (gesture segmentation, edge detection, anddown-sampling), extracting three characteristics of boundary contour, includingFourier descriptors/boundary orientation histogram and boundary invariant moment,and then, pattern recognition with template matching for270samples of gestureimages for training and540samples of gesture images for test. The results ofrecognition were compared and after the characteristics fusion at feature layer,effectively improve the whole recognition effect (the average recognition rate is 91.85%and the highest recognition rate is100%) to make up for the lack of singlefeature. Two methods were tested to extract joint angles including the line detectionbased on the Hough transform and the direction of the vector angle based on trackingresults, and then the results of these two methods were compared with the resultsfrom VICON motion capture system. With800samples of images for each motionmode, average angle error of two methods were16.34°and3.34°, respectively, andthe correlation were0.833and0.885, respectively. Results showed the advantages ofthe direction of the vector angle based on tracking results for joint angles extraction.The achievements in this thesis may further applied in the research and developmentof practical clinical system for the rehabilitation of upper limbs combing machinevision and virtual reality technique.