Research On Deep-Learning-Based Lung Tumor Segmentation Methods For Multi-Modal Mri Images

Lung cancer has become the cancer with the highest incidence and mortality in the world in recent years.Accurate segmentation of lung tumors in images is of great significance for the precise implementation of lung cancer treatment plans.Compared with Computed Tomography(CT)imaging,Magnetic Resonance Imaging(MRI)has better soft tissue contrast,and has a multi-modal imaging method,which can reflect the physiological function changes of soft tissue,and MRI has no radiation invasion.So multi-modal MRI images have gradually attracted the attention of researchers in the detection and diagnosis of lung cancer.However,doctors’ manual segmentation of lung tumors in MRI images is highly subjective,and has a large workload and low efficiency.Therefore,the research on automatic and accurate segmentation of lung tumors is of great significance.However,there are still many challenges in the segmentation of lung MRI images:(1)The lung tumors are adjacent to the mediastinum,and the signal contrast is low in a single-modality lung MRI image,which makes it difficult to accurately identify the boundary between the tumors and the mediastinum;(2)There is semantic content shift between multi-modal lung MRI images,and the spatial alignment is difficult,which causes the subsequent multi-modal image fusion and segmentation low accuracy;(3)Due to the long scanning time of multi-modal MRI images,the acquisition of paired multimodal images are expensive.In response to the above challenges,this paper uses deep learning methods to conduct in-depth research on the multi-modal lung MRI image tumor segmentation method,as follows:(1)Aiming at the problem that the edge of the tumors adjacent to the lung mediastinum is not clear,and the false positive results of the lung tumor segmentation,combined with the characteristics of high contrast between the lung tumors and the mediastinal signal in DWI MRI images,a new lung tumor segmentation method based on adaptive multi-modal image feature fusion weighting was proposed.In order to fuse the strong contrast information of DWI MRI images with the rich tumor texture information of T2 W MRI modal images,a dual-path U-net encoder with hyper-dense connection of multi-modal feature maps is constructed,which combines all the pre-order features of the dual-channel U-net.The feature maps are stacked in the channel dimension to fuse multi-modal features of different scales and improve the fusion effect of complementary multi-modal features.At the same time,in view of the difference in the importance of T2 W and DWI image features for varying cases and on the varying network layers,an adaptive multi-modal fusion weight module is designed,which uses modal feature stack descriptors to compress the multi-modal fusion feature stack of the network coding layer.The compression coefficient is used as the fusion weight,and the multimodal fusion features of all network layers are re-calibrated to improve the multi-modal feature fusion effect of different feature layers.The experimental results on the T2 WDWI multi-modal MRI dataset show that the proposed segmentation method achieves better accuracy than the existing multi-modal image fusion segmentation methods,and effectively solves the problems of false segmentation of lung tumors close to the mediastinum.(2)In view of the semantic content shift in multi-modal MRI images,the traditional two-stage registration and segmentation algorithm has a low spatial alignment degree in the tumor region of the multi-modal image and a high time cost in clinic,hence an endto-end jointly training model of multi-modal image spatial alignment on tumor region and the tumor segmentation is proposed.In order to learn the nonlinear deformation field of the tumor region between the DWI image and the T2 W image,a parameter-learnable spatial aligning network module was used to spatially aligning the pre-segmentation mask of the multi-modal image of the lung tumors,and the tumor region obtained by training was used for spatial aligning.The deformation field acts on the DWI modal feature map.In order to simultaneously improve the performance of tumor region spatial aligning and tumor segmentation,the multi-modal pre-segmentation loss,the lung tumor segmentation loss and the multi-modal image tumor region spatial alignment loss are jointly trained with constraints.In addition,in order to improve the training effect of different subtasks,a multi-stage training strategy is adopted,and different loss function weights are used for different training stages.The experimental results show that the proposed method can effectively overcome the problem of semantic content shift between DWI images and T2 W images,and improve the performance of lung tumor segmentation with multi-modal feature fusion and the engineering efficiency of the multi-modal feature registration and segmentation.(3)In the multi-modal MRI image lung tumors segmentation task,the acquisition cost of multi-modal image data is high,and there is a large semantic content shift between modal images,a method based on multi-modal image generation is proposed.The method of lung tumor segmentation generates a mixed-modality image with high contrast and rich texture from T2 W images to assist in the segmentation of lung tumors on T2 W images.In this method,an image conditional variational autoencoder based on semantic cycle consistency is designed.After learning the modal distribution characteristics of DWI,only the T2 W image is used as the semantic condition of the generative model to achieve mixed-modal image generation.To train the generator synthesizing the auxiliary modal images with DWI modal characteristics and consistent with T2 W structural information,the T2 W image is used as the generation condition of the variational autoencoder,and the T2 W image reconstruction error is used as the generator’s semantic cycle consistency loss,Together with the generative loss of mixed-modality images and the generative-modality-aided segmentation loss,the jointly constrained model generates a mixed-modality image that matches the spatial structure of the T2 W image and that the tumor region has high signal contrast features of the DWI image.The experimental results show that compared with the existing multi-modal image segmentation methods,the proposed method achieves better lung tumor segmentation performance,and effectively solves the problem of high cost of paired multi-modal image acquisition.