Research On Key Technologies Of Augmented Reality For Spinal Navigation System

The traditional computer-assisted minimally invasive spine surgery navigation system mainly registers the pre-operative 3D medical image information with the intraoperative patient spatial information,and accurately locates the lesion to guide the doctor to perform the surgical puncture operation.However,when the doctor performs the corresponding puncture according to the registration results,the movement of the patient and the effect of breathing may affect the results of the registration,leading to inaccurate puncture.Therefore,doctors can not fully trust the registration results during the operation.They also need to observe or master the whole puncture process in real time,and check the accuracy of puncture in real time,so as to avoid the harm caused by the deviation of puncture to patients.Usually doctors take a large number of X-rays regularly in minimally invasive spine surgery to grasp the effect of puncture surgery,which will not only reduce the efficiency of surgery,but also cause excessive radiation.Consequently,in this paper,augmented reality navigation system for spinal surgery is developed by using augmented reality technology to help doctors realize real-time monitoring of the whole operation puncture process.Firstly,the hardware and software components of the augmented reality spinal surgery navigation system were introduced.On the hardware platform,the high-definition camera was used to capture the surgical scene in real time and transmit it to the computer.The video signal with virtual and real registration by the computer was output by 3D display.During surgery NDI was used to track the surgical tools and X-ray machine was used to check the accuracy of registration.On the software platform,modules such as video capture,camera calibration,and three-dimensional registration were included to realize real-time monitoring of the whole operation puncture process.In addition,the flow of the entire augmented reality navigation system was introduced,and the coordinate conversion relationship between the software and hardware modules was explained.In this paper,the virtual-real registration is realized in the real-time video captured by the camera,and the imaging model of the camera is analyzed.Based on the imaging principle of the camera,the calibration algorithm of the camera was researched.Then the algorithm principle and implementation process of the two-step calibration method were deeply analyzed.Finally,the camera was calibrated by two-step calibration method,and its internal parameters,external parameters and distortion parameters were calculated.In order to get the required virtual model,the three-dimensional reconstruction technology was studied,and the principle of MC algorithm was analyzed.Then,based on the MC algorithm,the CT images of the spine were reconstructed.The parameters of material,illumination and transparency of the reconstructed three-dimensional model were optimized and adjusted to meet the interactive needs.Finally,the three-dimensional registration algorithm was studied,and the transformation relationship between the coordinate systems in the virtual-real registration algorithm was analyzed,and the registration matrix of each frame image was calculated,so that the qualified virtual spine model could be registered with the surgical scene.In this paper,the marker points were fixed on the spine model,and the registration method based on the marker points was adopted.The ARToolKit development kit was used to integrate the 3D navigation system and the augmented reality module in the VS2010 environment,and the identification method was used for registration and complete the registration experiments.In view of the shortcomings of the identification method in the experimental process,the development kit was improved and tested.The improved identification method improved the operability and accuracy of the experiment.In view of the inadequate registration accuracy under the condition of a single camera,this paper added the function of multi-sight registration.Another camera was added to the original hardware.The two cameras acquired the surgical scene at different sights and simultaneously performed virtual and real registration in the respective video signals.After experimental verification,the multi-sight registration method could effectively improve the registration accuracy.Finally,a spinal surgery navigation platform based on augmented reality technology was built,and all the research methods in this paper were experimentally verified.In order to restore the real surgical situation,this paper used silica gel instead of skin and muscle tissue to wrap the spine model,so that the experiment could be completed without seeing the spine model in the real scene.Firstly,the three-dimensional navigation module was used to register the CT images of the spine model with the spine model before operation,and the registration parameters were obtained.Then simulated the operation.During surgery,the augmented reality module was used to present the spine morphology and its relative position to the surgical tool in real time,monitoring the entire surgical procedure.By the error calculation,the error of virtual and real registration was about 2.5mm,which could meet the accuracy requirement of the restoration of the spine morphology under the patient’s skin in real time during surgery,so as to guide doctors to operate and improve the success rate of surgery.