Automatic Segmentation Of Brain Tumor Mr Images Based On Missing Modalities

Brain tumor is a highly malignant disease,which often causes complex physical and cognitive damage to patients.Effective segmentation of brain MR images in patients with brain tumors often helps doctors make correct diagnosis and treatment of patients.Usually,brain MR images have four modalities: T1,T2,T1 C and FLAIR.However,in actual clinical practice,due to the limitations of equipment and other conditions,patients often only have part of the modalities,resulting in the problem of inaccurate segmentation of brain tumors.Therefore,automatic segmentation of brain tumor MR images based on missing modalities has important research value and significance for diagnosis and treatment of brain tumor patients.In this paper,an improved segmentation algorithm based on full convolutional neural network is proposed to solve the problems of modal loss,dependence on key modalities and unstable segmentation accuracy of brain tumor MR images.The main research contents of this paper are as follows:（1）For the characteristic that MR images of multimodal brain tumor have similar features,based on U-NET neural network model,we propose the AMM U-NET（Adapted Missing Modalities）segmentation algorithm.Based on U-NET,the algorithm combines the idea of feature sharing and reconstructs the missing features through lightweight cascade feature generation modules to supplement visual information.In addition,a multi-modal fusion module based on channel attention is proposed to select and fuse the generated feature images,which ensures the effective use of generated features and suppress invalid features.（2）The segmentation accuracy of AMM U-NET algorithm is limited by the number of model parameters,the size of data set and the fusion modalities.This paper proposes a model structure of gaussian multiplication fusion modal feature distribution based on variational autoencoder.By combining DRM（Dual Residual Multi-VAE）structure of global residual module and local residual module,the latent distribution extracted by encoder is enhanced and the latent variables generated by decoder are provided with more semantic information and detail correction,so that the features of input decoder are more accurate.This model can decompose difficult learning tasks into subtasks,improve the efficiency of back propagation,and further improve the segmentation accuracy of brain tumors.（3）In view of AMM U-Net’s over-dependence on key modalities,this paper designed AEO（Approximate to Each Other）loss function based on KL divergence.The loss function makes the non-critical modalities have partial distribution characteristics of the critical modalities by calculating the latent distribution of all sub-modes fusion and expecting these distributions to be the same,thus alleviating the dependence of the shallow visual feature algorithm on the critical modalities.In this paper,the relevant algorithms are verified by using the Brain Tumor Segmentation（Bra TS）Challenge dataset.The experimental results showed that the AMM U-NET proposed in this paper achieved the average Dice score of 84.92%,75.20% and 71.89% in edema area,core area and enhancement area respectively.The improved DRM-VAE segmentation algorithm further improved the Dice score to 86.55%,75.53% and 72.10%,surpassing the current SOTA（State-Of-The-Art）model.In addition,the AEO loss function proposed in this paper can effectively alleviate the dependence on key modes,and greatly improve the accuracy of the model in the absence of key modes.