Research On Multi-modal Head And Neck Tumor Segmentation Based On Deep Learning

Semantic image segmentation classifies each pixel point in an image to determine the class of each point and thus the region.It is an integral part of image processing and image understanding,and is a core problem in computer vision.Medical image segmentation is an interesting application area of semantic image segmentation,which mainly assigns labels to medical images with special semantic information(e.g tumors,organs,etc.),with fewer categories and higher accuracy requirements compared to conventional semantic image segmentation tasks.Head and neck tumors are among the most common cancers in the world.This region is anatomically complex,with a high density of tissues and organs,and therefore has a complex pathology.Its diagnosis and treatment rely on several combined imaging modalities,among which Positron Emission Tomography/Computerized Tomography(PET/CT)is the recognized standard imaging protocol,but there is a lack of relevant automatic segmentation algorithm research.Therefore,the segmentation of head and neck tumors in PET/CT modality has become one of the current research difficulties and priorities in the field of medical imaging.Currently,there are three main options for head and neck tumor segmentation: manual segmentation,semi-automatic segmentation and fully automated segmentation.Manual segmentation is time-costly and error-prone,while semi-automatic segmentation improves the segmentation speed but still relies highly on manual intervention.Due to the rapid development of artificial intelligence methods,deep learning has been widely used in the field of medical image segmentation.This paper focuses on the end-to-end automatic head and neck tumor segmentation based on deep learning methods,and the main contents are as follows:(1)Tumor segmentation algorithm based on 3D simple attention module is proposed.Head and neck tumors are mainly imaged in 3D implementation,and in order to obtain information between image layers and ensure a change continuity between interlayer image labels,the algorithm uses 3D UNet as the base network and adjusts the width and depth of the model encoding and decoding layers.The modified 3D UNet can learn from sparse annotations in medical images and provide dense 3D segmentation,which is suitable for dealing with shortaxis slice-rich problems such as head and neck tumors.In addition,a parameter-free attention module is introduced to learn image feature information from both spatial and channel perspectives,and to perform feature collaboration and complementation.The module supports pluggable approach can ensure the simplicity of the network model architecture,but also help to improve the generalization and expansion capability of the module.(2)A multi-branch and multi-scale 3D segmentation network based on the concept of hierarchical fusion is proposed.The algorithm focuses on feature complementation between different modalities of medical images,and addresses the phenomenon that input-level fusion is prone to information loss and conflict,choosing to encode PET and CT images separately,allowing the network to refine learning from both tumor location and tumor boundary directions.Subsequently,a feature fusion scheme with CT features as the main feature and PET feature information as a supplement was used to effectively fuse tumor location coding information and tumor boundary refinement information to suppress conflicting or confusing features.Finally,the improved Ghost convolution replaces the traditional 3D convolution,and the Ghost scheme that actively generates redundant feature maps at the cost of linear transformation can more effectively improve the inference speed and segmentation efficiency of the model and ensure the generalization ability of the model.(3)A tumor segmentation algorithm based on structural reparameterization is proposed.The algorithm revolves around equivalent transformations in the training phase and inference phase of the network model.Firstly,a multi-scale large kernel convolution layer is introduced to obtain feature maps of different sizes using convolutional kernels of different sizes,which enriches the features of the image and can work on encoding and decoding the feature information of the image from a global perspective to improve the segmentation performance of the image.On this structure,the inverted residual structure is introduced to enhance the dimensionality using point-by-point convolution,and then depth-separable convolution is performed.Under the premise of reducing the number of parameters and computation,using point-by-point convolution to increase the dimensionality of the feature map first can ensure that more semantic information can be extracted by subsequent deep convolution.Finally,with the purpose of improving the inference speed of the model,the parameters in the multi-scale convolutional layers are fused and transformed into another series of parameters of simple structures by using the structural reparameterization method,which can effectively improve the inference speed of the model and achieve the goal of lossless compression of the model without changing the inference accuracy.