Design And Implementation Of Medical Image Assisted Therapy System Based On HoloLens2

Mixed reality technology is a new type of digital holographic imaging technology after virtual reality and augmented reality,which can bring a good immersive interactive experience to users.The Ministry of Industry and Information Technology of the People’s Republic of China pointed out in the "Guiding Opinions on Accelerating the Development of the Virtual Reality Industry" that China should promote the innovation of virtual reality applications including mixed reality technology,and cultivate new momentum for the information industry."VR + Health" is one of the key industry applications it promotes.Statistics from the World Health Organization show that stroke has become the third leading cause of death in the world.Carotid artery stenosis due to causes such as atherosclerosis can lead to insufficient blood flow from the heart to the brain,which is one of the main causes of ischemic stroke.For carotid artery stenosis,the key part of treatment is to find the lesion site of stenosis and determine the surgical opening for minimally invasive surgery.However,with two-dimensional imaging examination images such as CT of patients,doctors often need to rely on years of clinical experience and high spatial imagination for diagnosis and treatment.And mixed reality technology can greatly reduce the requirements of surgery on these two abilities of doctors.In this paper,aiming at the research of carotid endarterectomy assisted by mixed reality technology,a set of medical imaging-assisted treatment system based on HoloLens2 is developed combined with Web service technology.The main research work of this paper includes the following aspects:(1)Research on the 3D reconstruction technology of medical images.In this paper,an automatic 3D reconstruction method of carotid artery is proposed by combining the region growing method with the moving cube algorithm using the similarity of adjacent tomographic images.The method requires only one human interaction and has the advantage of saving a lot of human intervention.(2)Study the display and interaction technology of mixed reality.By dynamically loading local STL model files,the bone,artery and vein models can be rendered into different colors according to user’s settings and display in the scene,and the whole composed of them can be interacted with.(3)Research on data service technology for medical imaging.Combined with web service technology and mixed reality technology,this paper constructs a medical image assisted treatment prototype system with the functions of three-dimensional reconstruction of medical image,medical image data management,three-dimensional medical model display and interaction.The three-dimensional reconstruction algorithm is integrated into the system.The web terminal is responsible for the three-dimensional reconstruction and data management of medical images,and the HoloLens2 terminal is responsible for the display and interaction of three-dimensional medical models.(4)The system is applied to clinical trials to assist in carotid endarterectomy.The feedback results show that the medical image assisted therapy system based on HoloLens2 can help inexperienced doctors quickly locate the location of vascular stenosis,determine the surgical opening,and reduce the patient’s trauma area,which is of great significance for disease diagnosis,preoperative planning,and intraoperative navigation.