Research On Laparoscopic Surgery Video Semantic Segmentation Algorithm Based On Deep Learning

As an intelligent medical device,the laparoscopic surgery robot has now become a current research hotspot in the field of medical robotics.Semantic segmentation of laparoscopic video is the basis for the research of visual perception and control system of laparoscopic surgery robot.Real-time semantic segmentation of surgical instruments can accurately locate surgical instruments and estimate their postures,real-time semantic segmentation of targets such as blood vessels and ureters can predict dangerous operations and reduce clinical surgery risks,real-time semantic segmentation of surgical anatomical structures can provide guidance information for building surgical navigation system.Due to the dynamic unstructured surgical environment,there are some difficult problems in the semantic segmentation of laparoscopic surgery video scene tasks,such as real-time deformation and displacement of soft tissues such as blood vessels,low image contrast and blurred edges.To solve the above problems,aiming at the accuracy and real-time of laparoscopic surgery video semantic segmentation algorithm,this paper constructs a pixel level laparoscopic surgery video semantic segmentation dataset,and proposes a semantic segmentation algorithm.Finally,the effectiveness of the algorithm is verified by experiments.The specific research contents are as follows:(1)152100 video images of laparoscopic surgery were acquired and collated in this paper,and 5070 of these video images were finely annotated at the pixel level.As one of the contributions of this paper,the dataset constructed in this paper can not only serve as a benchmark to evaluate the performance of semantic segmentation algorithms of laparoscopic surgery video,but also provide a reference for subsequent research.(2)This paper proposes a Swin Transformer Feature Propagation Network(STFP-Net)based on deep learning for real-time semantic segmentation of laparoscopic surgery video.Specifically,we use the Swin Transformer neural network as the backbone network of the model to efficiently extract the spatial features of the segmented target.In order to improve the segmentation accuracy by using the timing information of video data,this paper improves it and generates three feature vectors for the adaptive feature propagation module.To take into account the segmentation speed,the window size of the Swin Transformer is adjusted in this paper.At the same time,to improve the accuracy of video semantics segmentation for laparoscopic surgery,a lightweight adaptive feature propagation module is designed to achieve spatial correlation of features.The module combines the semantic features of video sequences from different time steps to achieve the effect of deep network feature extraction.Meanwhile,it uses the timing information between video frames to compensate for the spatial dislocation caused by inter-frame motion,and reduces the impact of segmentation target movement and deformation on segmentation performance.Finally,in order to ensure the performance of semantic segmentation,this paper designs the corresponding feature graph decoder and Loss function.(3)The visual comparison and quantitative analysis experiments are performed on 9050 images on the public dataset of laparoscopic surgery video and 5070 images on our dataset.The experimental results confirmed that the STFP-Net proposed in this paper can achieve excellent performance in segmentation accuracy and computational efficiency.STFP-Net can achieve83.52% and 78.18% MIo U accuracy metrics and 50.34 and 47.02 FPS speed metrics on the public dataset and the dataset constructed in this paper,respectively.The accuracy and real-time semantic segmentation of laparoscopic surgery video are realized.