The Hand Rehabilitation Of Natural Interaction In Augmented Reality Based On Kinect

Augmented Reality (AR), is a new developed technology based on virtual Reality. It overlays virtual object, scene or message generated by the computer to the real scene, so as to realize augment of reality. In recent years, AR technology has been applied gradually in the rehabilitation training. Application of AR technology to hand rehabilitation also has perceived advantages besides the advantages of Virtual Reality (VR) rehabilitation technology. Users in it are not isolated with the real environment, they can see their real hands in the scene and have a real-time interaction with the real scene and virtual objects in a more natural way which improves the factuality and practicability of the interaction.Based on the national natural science Fund project as the background, the paper introduces the EMD algorithm into complex gesture recognition such as syndactylia, and develops a natural interaction system which is to augment and realize hand rehabilitation based on Kinect. In this paper, the main research contents include the following:First this paper, with the Kinect equipment and under the static recognition, puts forward to identify gestures accurately using the EMD algorithm for complex problems such as fingers adhesion or syndactylia in rehabilitation process, makes a comparison between EMD algorithm and traditional recognition algorithm such as module-based or vector machine, analyzes in detail the advantages and disadvantages of various algorithms and proves superiority of gesture recognition algorithm based on EMD.In view of dynamic hand gestures AR, this paper builds AR environment in the selected frame of the Open frameworks in which AR interaction module are divided into 2D interaction and 3D interaction.2D interaction is mainly to overlay virtual objects into the scene directly according to the 2D coordinate system and interact, while 3D interaction is mainly to get internal and external parameters first, then create a 3D coordinate system and finally load 3D model to realize interaction. Under the guidance of related doctors and according to hand rehabilitation treatment and training method, the paper has designed several training programs as grasping, holding and pinching training, track training, single hand interaction training and both hands interaction training. There is a progressive relationship among these training programs on the difficulty level and at last, through questionnaire assessment and analysis on parameter value index, the experimental results validate the effectiveness and feasibility of this system.