The Research On Key Technology Of Digital Surgery System Based On Augmented Reality

With the continuous development of computer technology and digital medicine,computer-assisted surgery has brought great progress to clinical diagnosis and surgical treatment.Its surgical mode based on image guidance and surgical planning has effectively solved the problem of locating lesions in traditional surgery,which has promoted the development of surgery towards precision and minimally invasiveness.In clinical practice,because the three-dimensional structure of the lesion is not visually visible,the doctor needs to switch the field of view between the screen and the lesion,which not only affects the consistency of the operation and the efficiency of the operation,but also increases the risk of surgery.To this end,this paper uses Augmented Reality(AR),a cutting-edge technology,to achieve the registration of the preoperative planning model and the lesion site,providing doctors with intuitive three-dimensional structural information of the patient’s lesions.The main research contents of this paper are as follows:An augmented reality digital surgery system based on SLAM(Simultaneous localization and mapping)is developed by using monocular vision and RGBD algorithm.The system realizes the accurate positioning of the pre-planned model in the space reconstructed with SLAM and realizes the virtual model,achieving a stable static registration effect.Using the image processing algorithm such as the OTSU and contour extraction,an augmented reality digital surgery system based on two-dimensional identifier tracking registration virtual model is developed.The system implements dynamic identification to track the QRCode and accurately locate the virtual model at the QRCode.Based on the SLAM algorithm,in order to realize the real-time dynamic registration fusion between the virtual model and the real model,this paper uses the infrared positioning and tracking device to locate and track the real model and the motion of the HoloLens helmet in real time,and proposes the hand-eye calibration algorithm used to calibrate coordinate system of the Camera coordinate system to the HoloLens reference frame.A point registration algorithm based on quaternion is proposed to solve the registration problem between virtual model and real model in augmented reality surgery system.The quaternion interpolation algorithm is used to smooth pose of the virtual model.The position of the cusp of the surgical instrument is calibrated based on the quaternion.Based on the research of positioning registration algorithm,this paper develops an augmented reality digital surgery system with optical positioning and tracking based on open source SDK and development tools such as MixedRealityToolkit,IGSDK,Unity and QT.The model test was implemented and the accuracy test was designed to test the error of the system.In the coordinate system of the optical positioning tracker,the average error in the x direction is 17.26 mm,in the y direction is 13.70 mm,in the z direction is 20.71 mm.Stability of Our system is superior to similar systems.In summary,this paper studies the algorithm based on SLAM positioning registration,realizes stable and accurate static positioning registration,and develops a system using dynamic positioning registration of QRCode to achieve relatively stable dynamic tracking;The augmented reality surgical system was studied,and the stable and reliable dynamic tracking registration of the target model was realized.The research in this paper promotes the development of medical digitalization and has very important clinical application value and potential.