A Real-Time System Using Deep Learning For Detecting And Tracking Ureteral Orifices During Urinary Endoscopy

In urology endoscopic procedures,the Ureteral Orifice(UO)finding is extremely crucial but may be challenging for inexperienced doctors.Because the shapes of UOs vary from people to people and they deform with various positions and timing,which could make UO difficult to identify.To automatically identify various types of UOs in the videos,we developed a real-time computer-aided UO detection and tracking system based on deep learning algorithm.This system is substantially constructed from three modules: the preprocessing module,the UO detection module and the tracking module.As for the preprocessing module,we applied both general and specific data augmentation strategies to obtain a large number of training samples.For the detection module,we proposed a novel UO detection network(Refined-SSD),which was an improved version of the Single Shot Multibox Detector(SSD).Finally,a UO detection and tracking system is proposed which is the combination of Refined-SSD and CSRT tracking algorithm.For the training steps,we only utilize resectoscopy images which have more complex background information,and then,we use ureteroscopy images for testing.Simultaneously,we demonstrate that the detection model trained with resetcoscopy images can be successfully applied in the other type of urinary endoscopy images with all evaluation metrics around 0.9.We further evaluate this detection model on resectoscopy video datasets and ureteroscopy video datasets,which demonstrate that our deep-learning based UO detection module can precisely identify and locate UOs of two different urinary endoscopes in real time.In addition,we compared the performance of our proposed detection and tracking system(RefinedSSD+CSRT)not only with the detection module(Refined-SSD)but also with the detection module which integrated with different tracking algorithms.Experiments shows that our proposed UO detection and tracking system performs the best overall with average processing time equal to 20 ms per frame.Therefore,this system can detect and locate UOs in real time and simultaneously maintain a high accuracy in two different types of urinary endoscopes.