SLAM-based Augmented Reality Registration Method And Its Application In Hand Rehabilitation Trainin

Augmented reality is a computer technology that superimposes virtual information onto real-world scenes,enabling users to perceive and interact with virtual information through inputs such as images and sounds.In recent years,this novel and interactive technology has garnered increasing attention and application in areas such as rehabilitation training.In the context of hand function rehabilitation,the application of augmented reality can reduce the tedium experienced by patients during repetitive movements by presenting a novel and multisensory approach,thereby achieving efficient rehabilitation outcomes.However,challenges remain in improving the stability of augmented reality registration and the authenticity of occlusion between virtual and real objects.In this study,we developed a hand rehabilitation training system based on SLAMenhanced augmented reality technology,improving the robustness of AR registration and the authenticity of virtual-real occlusion.The main contents of this paper are as follows:To address the issue of unstable registration and tracking in augmented reality,we propose a 3D object detection method based on point clouds.Firstly,we introduce a point cloud generation method that combines YOLOv5 s and VSLAM for dynamic removal.Subsequently,we integrate the Point-RCNN algorithm,utilizing a detection approach that combines 2D images with 3D point clouds to achieve superior tracking capability in translation and rotation for 3D object detection and augmented reality overlay.Finally,we employ this method to design an item placement training program within the hand rehabilitation system,collecting and analyzing patients’ hand motion data.In response to the problem of rough edge processing in existing virtual-real occlusion methods,we proposes a three-dimensional object dense reconstruction and segmentation method based on SLAM.By optimizing the constraints of virtual-real edges and increasing the fusion degree between virtual objects and the real environment,the virtual-real occlusion is made more realistic.Firstly,we obtain dense point cloud data through SLAM and voxelize the point clouds,using OPTICS clustering for voxel edge constraint and segmenting the corresponding objects.Subsequently,we open a second thread,taking the point cloud generated by the SLAM system as input,and use the required depth shape as a prior.By comparing edge SDFs,we perform dense point cloud prediction and reconstruction for the input objects.We then combine the predicted point clouds with the voxel segmentation edges,update edge points,and fit new edge surfaces,making the object edges after 3D segmentation more accurate and shape-preserving.Finally,we apply this method to design a table tennis training program within the hand rehabilitation system,efficiently assisting patients in their training.In conclusion,integrating the aforementioned methods,we have designed an augmented reality-based hand rehabilitation training system.By employing a VSLAM system with dynamic object removal and point cloud 3D tracking technology,we have improved the training system for patients in phases two and three.Building on the foundation of the first two proposed algorithms,we have refined the training system for patients in phases four and five.We have designed a data recording system and an evaluation system that includes assessment of training task completion,task completion time,and hand movement trajectory records.Moreover,by periodically evaluating the patients’ hand function,we assess the overall effectiveness of the training.