Deep Learning Fusion Method For Colon Polyp Segmentation

Early detection and diagnosis of colorectal cancer is gradually becoming a key concern worldwide.However,the formation of colorectal cancer is mainly related to lesions of colonic polyp tissue in the inner layer of the intestinal mucosa.At present,colon polyp segmentation is mainly divided into traditional segmentation methods and deep learning segmentation methods.Conventional segmentation methods are mainly based on low-level features,but colonic polyps are variable in shape,so the improvement of segmentation accuracy is limited.Deep learning methods show better performance than traditional segmentation methods.Among them,such methods are represented by convolutional neural network-based and Transformer-based methods.However,the segmentation method based on convolutional neural networks has inherent structural limitations,i.e.,it cannot establish the association between arbitrary pixels and thus achieve global feature learning.Another class of deep learning methods Transformer can establish long-range dependencies between pixels to achieve global information acquisition,but for low-level information acquisition is limited.In response to the above challenges,this paper presents an in-depth study of new deep learning methods applied to colon polyp segmentation,which mainly consists of the following work:First,to address the problem of low segmentation accuracy of many segmentation methods on small target colon polyp dataset,this paper proposes a multi-view collaborative learning enhanced segmentation method Fu-Trans HNet(Fusion-Transformer-Hard Net MSEG).The model is mainly based on the advantages of global feature learning of Transformer and local feature learning of convolutional neural network in vision field,and a novel fusion module is designed to fully fuse the feature information extracted from two branches in order to minimize the loss of small target information.The Transformer branch,CNN branch and fusion branch are considered as three perspectives in Fu-Trans HNet,and the training process is further optimized using a multi-view collaborative strategy.The weights are dynamically assigned according to the importance of each perspective during the training process.The output prediction maps of each perspective are multiplied by their respective weights during testing and then summed up as the final prediction result.Numerous experimental results have shown that Fu-Trans HNet achieves superior performance compared to other advanced methods in datasets containing more small diameter colon polyps,and good results have been obtained in other publicly available datasets.It is effectively verified that Fu-Trans HNet is helpful for the improvement of colon polyp segmentation accuracy,especially in the small target segmentation scenario.Further,in order to further improve the segmentation accuracy on the more general colon polyp data set,this paper proposes a colon polyp segmentation method Mix Form Net based on lightweight model and multi-scale feature fusion.The method uses the multiscale feature output Mix Transformer as an encoder,and then designs a novel decoder structure based on the output.In the decoder,the different scale feature maps output by Mix Transformer are processed separately: For low-level feature maps containing rich geometric spatial information they are fed into Pre Net and more detailed local detail information is extracted using convolution operations;For the other three layers of feature maps are fused using the multi-scale feature fusion module,with the aim of obtaining richer feature information and enhancing the characterization of information.The experimental data show that Mix Form Net achieves better metric results on several publicly available datasets,has better generalizability,and appropriately reduces computational cost.Finally,based on the above work,a prototype system of intelligent colorectal cancer online consultation platform based on colonoscopy images was designed and developed.The platform is first analyzed in terms of problems and feasibility,the overall architecture of the platform is determined,and then the detailed design of each module is carried out.The whole platform is developed in the way of front-end and back-end separation,in which the front-end uses VUE framework,the back-end uses Django framework,the data storage uses a combination of local service and cloud service for storage,and the segmentation function is implemented using the segmentation algorithm proposed in this paper.The platform is designed primarily to serve patients with colon disease and gastroenterologists.Patients can use the platform’s shared online medical resources to simplify the offline consultation process;Doctors can use the platform’s segmentation algorithms to automatically segment colonoscopy images to assist in the diagnosis process.The design and development of the consultation platform will help improve the consultation environment and approach for colorectal-related diseases and,to some extent,alleviate the imbalance in the allocation of medical resources.