Research On Upper Limb Rehabilitation Training System With Kinect

In the medical rehabilitation field, there are two kinds of traditional rehabilitation training methods, the human assisted rehabilitation and the robot aided rehabilitation, but they have some disadvantages, such as lacking of medical resources, expensive equipment and maintenance costs. Human-computer interaction refers to that people convey information to computer through body posture (including gesture, posture, facial expression, etc), the computer identify human’s action and then make feedback, it can solve the problems above efficiently. It is a hot spot to provide patients with a better, cheap and convenient rehabilitation training system.This thesis uses game mode based on virtual role to get human’s depth information and skeleton with Kinect, we identify the body posture and reconstruct the patient’s action flow, by matching with standard action flow, the quality of patient’s movement is assessed, through this way we can do upper limb rehabilitation training. The main research content is as follow:1. Doing research on the structure, system architecture, technical specifications of Kinect, we get access to the working principle of the sensor for obtaining human’s depth image and motion skeleton, build the experimental environment at the same time, then we analysis the feasibility of upper limb rehabilitation training.2. Studying the rehabilitation training method, we use the method of gesture recognition based on the Mean Hausdorff Distance and Hidden Markov, the method of posture matching based on DTW-RANSAC algorithm, by analyzing the flow of human’s action during rehabilitation activities, we can match it with the standard action flow in the database, so the problem of patient training effect evaluation can be solved. To solve the problem of data accuracy, we propose a method to reconstruct human’s skeleton based on reliability assessment. First we evaluate the reliability of human’s behavior, skeleton and the tracking condition, then we use motion database to reconstruct the flawed human skeleton, so the problem of sheltered posture which is not steady can be solved effectively.3. Doing research on 3D modeling system. Using Unity3D to create virtual characters and virtual scene, we use the patient’s data movement to drive virtual character. So we can develop the upper limb rehabilitation training system combined with OpenNI and Kinect SDK.The experimental results show that using this system for upper limb rehabilitation exercise can achieve the rehabilitation goal, it can also increase patient’s interest in game forms.