Research On Computer-assisted Surgical Instrument Tracking Based On Deep Learning

With the continuous improvement of medical level and people’s living standards,modern surgery are developing in the direction of minimally invasive,accurate,less contact,and faster recovery.To achieve this goal,the concept of Computer Assisted Surgery(CAS)was proposed.CAS is an emerging field that is different from the traditional surgery.It is a combination of multiple disciplines and a number of modern advanced technologies.It can complete surgical operations with a minimum of surgical personnel efficiently and accurately,while reducing patient pain,blood loss and shortening the patient’s recovery time from surgery.The increasing maturity of CAS and the promotion of sophisticated technologies like the DaVinci robotic surgery system have brought good news to patients who need surgical operations.One of the core technologies of CAS is to use the images obtained through the endoscope during the surgery to track and locate the surgical tools in real time,so as to provide the attending physician and the robot with the precise position of the tools,which is convenient for the doctor and the surgical robot to carry out the next decision.This paper provides a general overview of CAS and surgical tool tracking algorithms.In order to track multiple surgical tools accurately through real-time detection in two-dimensional space,we compare multiple current state-of-the-art target detection algorithms on real datasets in terms of accuracy and speed.After combining the advantages of many current excellent target detection networks,we propose an efficient and accurate one-stage surgical tool detection network structure.In other word,we track surgical tools by detection.The main work of this paper is as follows:(1)we build a convolutional neural network(CNN)that can accurately detect surgical tools in real time,combining novel backbone feature extraction network,feature fusion,detection head part and loss function,introducing two attention mechanisms and our method showed good performance on our labeled dataset-Cholc80-locations.The novel network structure proposed in this paper is inspired by the current classic object detection networks,such as YOLOv3,YOLOv4,DetNet.The proposed network combines their advantages.(2)At the same time,we have done a lot of comparative experiments with two different specifications of SSD,SSD variants RFBNet and RefineDet,Faster-RCNN,YOLOv4,and YOLOv4 introduced SENet,anchor-free FCOS.We trained these network with same epochs on the same real dataset and the same device,then tested the above-mentioned surgical tool detection network in terms of detection speed(FPS)and accuracy(mAP and center point distance threshold evaluation),the proposed network structure proposed in this paper is better than the above-mentioned classic object detection network.(3)In order to enable the network structure to be deployed in embedded devices,we also optimize the backbone for extracting features,and replace it with the latest lightweight network structure GhostNet to speed up the network detection speed.The detection speed of the network is improved without a significant drop in accuracy.At the same time,we compared the proposed network with MobileNetv2 version of SSD,ShuffleNetv2 version of SSD.Experiments show that the surgical tool detection network proposed in this paper that can be deployed on embedded devices is superior to the above-mentioned lightweight detection networks.