Deep Learning Based Endoscopic Image Super-resolution Algorithm Research

The rapid development of medical imaging technology makes medical imaging equipment gradually become an important auxiliary tool for medical institutions to carry out clinical treatment and pathology research,of which endoscopy is one of the most widely used medical imaging equipment today.Due to the limited size of endoscopes and the intricate environment in the human body,the images captured by endoscopes are usually not of high quality due to hardware conditions,noise and light.This thesis focuses on the field of endoscopic images and super-resolution reconstruction of endoscopic images based on deep learning techniques,aiming to improve the image quality of endoscopic acquisitions and assist doctors in diagnosis and surgery.Image super-resolution is a fundamental computer vision task that enhances the target image quality by increasing the resolution of the input image.Since endoscopic data are mostly video data,special attention to the time-domain information between multiple frames is needed in the process of endoscopic image super-resolution in addition to the spatial domain information.In order to better combine the space domain and time domain information of endoscopic images,this thesis proposes two endoscopic image super-resolution networks based on the recurrent structure and Transformer,respectively.In this thesis,endoscopic superresolution dataset is produced for the study of image super-resolution algorithm in endoscopic surgery scenes by using the acquired endoscopic surgery videos to simulate the low-resolution images in real scenes on the basis of the original high-resolution images.Recurrent structure is a common network structure in image super-resolution algorithms.In this thesis,based on the recurrent network structure,a novel non-local deformable recurrent neural network for endoscopic image super-resolution is proposed.The deformable convolution-based inter-frame alignment module is embedded in the recurrent neural network for capturing time-domain information,and the non-local residual module is used to capture the global spatial domain information of multi-frame low-resolution images,combining the time-domain and spatial domain information in multi-frame low-resolution images to achieve better super-resolution of endoscopic images.In response to the difficulties of parallelization in recurrent neural networks and considering the excellent performance of Transformer structure in the task of processing sequential data,this thesis proposes a novel deformable Transformer super-resolution network.This thesis proposes a bidirectional deformable convolutional neural network for capturing time-domain information between multiple frames using a deformable convolution-based interframe alignment module and a bidirectional mechanism,and capturing global spatial information of multi-frame low-resolution images using an improved self-attention layer.Combining the bidirectional deformable convolutional neural network with the improved selfattention layer,a new deformable Transformer network is proposed in this thesis,which achieves better super-resolution results and less number of network parameters.By training and testing different networks in the endoscopic super-resolution dataset and comparing the performance of other existing image super-resolution methods in the endoscopic super-resolution dataset,after sufficient experiments and analysis,it is proved that the two endoscopic image super-resolution networks proposed in this thesis outperform other superresolution networks in both quantitative metric comparison and qualitative image detail comparison,In this thesis,several sets of ablation experiments are designed to demonstrate the effectiveness of the proposed multiple super-resolution modules.