The Research On Key Technology Of A Patient-specific Surgical Simulation And Training System Based On Force Feedback And Virtual Reality

With the development of information technology,Virtual Reality(VR)is becoming an important part of computer-assisted surgery,which brings great changes to the traditional medical training.Virtual reality surgical simulation systems integrate the disciplines of biomedicine,computer graphics and image science,mathematics,mechanical dynamics and so on.With using force touch technology and visual immersion technology,the VR systems can easily create a visual and tactile fusion virtual environment for users.It can be carried out on the patient-specific models,which is conducive to the doctor to observe the lesion,to develop a reasonable surgical approach to facilitate preoperative simulation and training.It is also conducive for medical students or novice surgeons to simulate the surgical operation,and the VR systems can help users understand the surgical procedures and surgical methods and enhance the skill of surgical operation of the users.Virtual Reality Surgical Training Simulator(VRSTS)consists of hardware system and software system.The hardware which includes force feedback instrument Omega.6,immersion platform and high-performance graphics workstations,is mainly responsible for providing users with visual and tactile interaction tools and software platform.The software includes computer vision and tactile development library CHAI3 D,graphic application function library VTK and graphical user interface development package Qt.The system uses separated tactile and visual multi-threaded programming technology which can solve the problem that visual and tactile processes require different refresh rate.We have studied the visual simulation algorithm in virtual reality environment,analyzed the working flow of visual rendering engine OpenGL,and studied the transformation matrix of translation,scaling and rotation of OpenGL.The AABB bounding box was used as the collision detection algorithm between the target models and the surgical tool,and a model multi-level bounding tree was constructed.Aiming at improving effects the rough or non-smooth mesh of the target models,the spatial Hermite interpolation and the Loop mesh subdivision algorithm are proposed.After the mesh subdivision,the mesh of the surgical model is smoother,which is conducive to obtain good vision results of the mesh deformation.In this paper,the constitutive equation of tactile feedback force is constructed based on the density of tissue,the thickness of tissue and the speed and depth of surgical operation.The calculation method of reaction force is based on a virtual spring algorithm which complies with the actual operation.This VRSTS system has carried the research on the simulation of two kinds of representative operation: drilling and cutting,which are the most common operation of real surgery.By using a constraint algorithm of complex surgical tools,it is useful to restrict the free-moving surgical tools to the specified direction of the surgical operation,and to prevent disorderly beating of surgical instruments due to the collision detection feedback force.In this paper,a new mesh deformation control algorithm is proposed,in which different deformation factors will produce different deformation effects.According to the needs of actual surgery,a step-by-step reaming algorithm is proposed,which can simulate a variety of step-by-step reaming operation and can realize multiple drilling with different aperture and different rotational speed.In this paper,the mesh segmentation and reconstruction process of the patch subdivision method and the vertex moving method are studied.The cutting operation simulation is realized based on the patch subdivision method and can give users good vision feedback of cutting operation.Volume rendering technology is a key method of 3D medical image reconstruction.This paper also studied the operation simulation based on volume rendering.First we introduced the technique of volume rendering reconstruction,then use the light projection method to reconstruct the 3D visualization of the model.The simulation of the drilling and cutting operation based on the volume rendering models was carried out by using the cell removal method.In this paper,a Virtual Reality Surgical Training Simulator(VRSTS)is implemented,which has a good practicality in the aspects of the user-friendly interface,the convenience of operation and the fidelity of visual and tactile feedback.Students and experts have tried out the VRSTS,and the results of the trial report show that the system has good performance in generalization,practicability,usability,visual and tactile authenticity.It can be used as a method for the training of medical student,and it is also helpful for the development of medical information technique,which will have great value of application.