Design Of Hand Natural Interactive Rehabilitation System Based On Augmented Reality

Augmented reality technology provides an interactive experience between the real world and the virtual world.It augments the real world by superimposing computer-generated sensory information or virtual objects generated onto the real world,and enriches the way users experience the real world.In recent years,hand function rehabilitation training system based on augmented reality has become a research hotspot in China’s medical field.With a certain social value,it can not only stimulate the enthusiasm of patients to participate in the rehabilitation training,but also alleviate the current shortage of occupational therapists and professional nursing staff.In the context of National Natural Science Foundation of China,this paper designs and develops a natural gesture interactive rehabilitation training system based on augmented reality by combining static gesture recognition and fingertip detection algorithms.The main research contents of this article are as follows:Firstly,based on the research of static gesture recognition algorithm based on geometric features,this paper designs a set of new feature descriptors for the effective representation of hand shape with good scale invariance and rotation without deformation,and constructs a gesture classifier,which is proved to have excellent robustness and practicability,by combining SVM.Secondly,a fingertip detection algorithm based on deep convolutional neural network is designed in this paper,which consists of two parts: hand area detection algorithm and fingertip detection algorithm.First,the YOLOv2 network is used to locate the coordinate information of the 2D hand gesture,and then the detected gesture area is cropped out as the input of the fingertip recognition part.Finally,a UGF-Net network is designed for accurate fingertip detection.Through the experimental analysis,the algorithm can effectively solve the problem that human fingertips cannot be accurately identified due to their small size.Finally,this paper designs and develops a natural gesture interactive rehabilitation training system based on augmented reality by combining the static gesture recognition and fingertip detection algorithm.At the same time,four rehabilitation training projects with different difficulty are designed: brick training,gripping pinch training,trajectory training and shooting training.These items show an increasing relationship in the difficulty level,mainly used to improve the users’ movement disorders of fingers,wrists,elbows and shoulder joints.In the end,the feasibility of the system is verified by testing different users.