Research On Multi-level Interactive Visualization Of Medical Images For Surgical Navigation

With the rapid development of computer graphics technology,3D visualization technology is increasingly integrated with the medical science.Modern medicine,with the help of the development of computer-aided intelligent systems,comes to the stage of digital medicine and precision medicine,forming a new field in the meanwhile-computer-aided surgery.Preoperative planning and intraoperative navigation are two of the most important parts of computer-aided surgery.However,most of the existing visualization platforms are limited to the PC(Personal Computer),which often limits the time and space for doctors to make preoperative diagnosis and planning.Besides,existing visualization platforms are unable to provide intraoperative navigation for healthcare workers in intraoperative scenarios,and when multiple tissue models are all applied in the same scene,the view would be blocked.To address those problems,this paper proposes an approach of cross-platform medical image examining,an approach of AR-aided navigation,and an approach of hierarchical display of AR scenarios,and presents a multilayer interactive visualization system of medical images for surgical navigation developed by the author.The main problems discussed in this thesis is divided into the following three parts:(1)The equipment and platform used by the existing visualization platforms have limited functions.This thesis proposes a cross-platform/device approach to examine medical image Slice View,which is web-based and enables cross-platform operation.All 2D and 3D image interaction is done on the PC,making this approach deviceindependent.Considering the differences between different display devices,the interface is designed to adapt to different terminals itself.(2)The existing visualization platforms only allow medical workers to examine images before the operation,and have limitations in pragmatic scenarios.This thesis proposes an AR-assisted visualization approach.This approach,by combining AR technology with medical image visualization,can overlap 3D models reconstructed based on preoperatively acquired medical data such as CT and MRI,with information like preoperative surgical paths.And then apply these models and information to real surgical scenarios in real time during surgery.In this way,doctors can then see the overlap of the virtual model and the surgical path with the patient in real time,thus helping them in the surgical operations and risk judgments.(3)AR-assisted visualization using a mobile terminal or AR device can provide intuitive surgical navigation,but the simultaneous display of a large number of models still limits device performance,the view of operation team and physicians' operation.Therefore,this thesis proposes a hierarchical display method,which includes a regional hierarchical display and a hierarchical display of auxiliary information.A dynamic hierarchical display of areas is achieved by predetermining the areas of interest and tracking the current view of the surgical operation in real time.At the same time,a series of distances are calculated,such as the distances from the tip of the surgical instrument to the target tumor,the offset value of the tip of the surgical instrument from the surgical path and so on,to obtain effective AR information,which is provided to the physician simultaneously,thus achieving the multi-level hierarchical display of auxiliary information.