Research On Medical Image Segmentation Based On Transformer

As people become more and more concerned about their health conditions,the analysis of medical images for diagnosis has become more and more important.However,the analysis of medical images is highly professional,and relying on doctors to analyze and diagnose them manually will increase their workload and may lead to misdiagnosis and omission.Therefore,the application of automated medical image analysis system is of great importance as a tool to assist doctors in diagnosis and treatment.And the medical image segmentation technology,which is the core of this system,has also attracted much attention.Thanks to the rapid development of computer hardware,deep learning models can be used in the field of medical image segmentation.The main work done in this thesis for medical image segmentation problem is as follows:(1)In medical image segmentation task,convolutional neural networks(CNN)are difficult to capture long-range dependencies,but transformers can model the long-range dependencies effectively.However,transformers have a flexible structure and seldom assume the structural bias of input data,so it is difficult for transformers to learn positional encoding of the images when using fewer medical images for training.To solve these problems,a dual branch structure TC-Fuse is proposed.In one branch,Mix-Feed-Forward Network(Mix-FFN)and axial attention are adopted to capture long-range dependencies and keep the translation invariance of the model.Mix-FFN whose depth-wise convolutions can provide position information is better than ordinary positional encoding.In the other branch,traditional convolutional neural networks(CNN)are used to extract different features of fewer medical images.In addition,the attention fusion module Bi Fusion is used to effectively integrate the information from the CNN branch and Transformer branch,and the fused features can effectively capture the global and local context of the current spatial resolution.The experimental results show that TC-Fuse has better segmentation effect compared with the comparison methods.(2)For the problems of irregular shape,different size and blurred boundary between skin lesions and normal tissues in skin lesion segmentation task,the model TFC-Net with two parallel branches of Transformer and CNN is proposed.The Transformer branch of TFC-Net adopts the design of outputting multi-scale feature maps directly at encoding,instead of outputting single-resolution feature maps and then gradually upsampling them,to avoid spatial information loss and better mitigate the problem of varying shape sizes of skin lesion.In addition,CNN blocks,overlapping patch embedding,improved self-attention and improved Mix-FFN form the Transformer branch,and their combination can effectively capture long-range dependencies and global contextual information to alleviate the boundary blurring problem.The CNN branch of TFC-Net uses Res Net-101 with pre-trained weights to better extract detailed information.In addition,the use of Bi Fusion module effectively fuses the features of both branches,and the use of hybrid loss function and deep supervision also has an enhancing effect on segmentation results and generalization performance.Experimental results on the ISIC 2016 and ISIC 2017 datasets demonstrate the validity of each major building block of TFC-Net and the excellent performance for skin lesion segmentation.(3)A prototype medical image segmentation system is designed and implemented for the application scenario of automatic medical image segmentation.The system prototype integrates TC-Fuse and TFC-Net models using Python language and Py Qt5 framework,and mainly includes a login module,an image import module,a gland segmentation module,a skin lesion segmentation module,and a help module.After importing histopathological images of colon adenocarcinoma or skin lesion images,the prototype system can realize automatic segmentation of gland or skin lesion,and the segmentation results displayed can assist doctors in diagnosis and reduce their workload.