Research On Surgical Assistant System Based On Vision And Impedance

The Surgical Navigation System can assist the surgeon to precisely locate the exact position of the lesion during the operation and assist the surgeon to remove the lesion with visual guidance.As one of the core systems,the spatial positioning system can provide real time feedback on the spatial positioning information of the surgical instrument tips,assisting the surgeon to complete the surgical operation more accurately.However,intraoperative operations usually cause soft tissue deformation or displacement,which makes preoperative planning and intraoperative registration invalid,and it is difficult to determine whether the diseased tissue is completely removed.Relevant studies have used animal experiments to verify the differences in impedance values of different biological tissues and achieve accurate classification,which indicates that intraoperative bioelectrical impedance technology has great potential to distinguish healthy and diseased tissues.Therefore,this thesis mainly studies the positioning system of surgical instruments,supplemented by a tissue identification instrument based on bio-reactance to build a surgical assistant system based on vision and impedance.While giving the surgeon a three-dimensional perception of space,it also feeds back the category information of the surgical instrument tip in contact with the tissue in real time,providing effective assistance to the surgeon during surgery.The specific work of this thesis is as follows:Research on the overall process of positioning surgical instruments.In this thesis,the types of auxiliary localizers are firstly compared and analyzed,and the colored spheres are selected as the marker.Besides,the center of mass of the marker point is extracted by using image processing technology.Then,combined with the experimental scene,the spatial position constraint is selected as the matching principle to complete the matching of the markers.Finally,the spatial transformation relationship is used to locate the tip of the instrument.For the problem of conflicting colors between background and marker and other interference problems in the actual scenes,two solutions based on detection and tracking are proposed.In the detection scheme,the YOLOv5 algorithm is used to detect the markers,and the background and other interference factors are excluded.In addition,ECA attention and AlphaIo U are used to improve the detection performance of YOLOv5.The experiments show that the improved model has a maximum relative improvement of 0.87% and 3.5%,respectively,compared with the original model on m AP@0.5 and m AP@0.5:0.95.Based on the detection scheme,the tracking scheme uses Kalman filtering and Hungarian matching to track the markers,and matches the markers according to the tracking frame ID,which avoids the calculation of spatial constraints.Finally,the detection algorithm and the tracking algorithm are integrated into the instrument positioning process respectively,and the design of the instrument tip positioning method is completed.Finally,the experimental analysis and system construction are carried out.First,complete the camera calibration experiment to obtain the camera parameters for positioning? Secondly,the detection and tracking experiments of surgical instruments are compared,and the detection scheme with better effect is selected.Then,the instrument tip positioning and biological tissue electrical impedance measurement experiments are carried out respectively.The tip positioning error is 2.23 mm,which meets the system requirements.And the measurement experiments verify the feasibility of tissue identification using bioelectrical impedance.Finally,the visual positioning system and the tissue identification instrument are combined to build a surgical assistant system.Meanwhile,the basic functions of the system are verified and its limitations and subsequent optimization directions are pointed out.