Research On Glioma Molecular Classification Based On Multimodal Fusion

The origin of glioma is closely related to the molecular phenotype of glial cells,and the corresponding pathological state and treatment sensitivity of different molecular phenotypes are significantly different.IDH mutation and lp/19q codeletion status are two typical molecular phenotypes,which have important clinical significance for the individualized treatment of patients with glioma.As a common imaging technique in intracranial examination,the imaging features of magnetic eesonance imaging can reflect the information of molecular phenotype.Compared with single modal imaging,multimodal imaging can observe the anatomical information and characteristic performance of the nidus from multiple angles.Therefore,how to use multimodal fusion technology to realize the effective complementarity of cross-modal information,and capture the potential relationship between modalities is of great significance to the development of clinical medicine.Based on the above analysis,from the perspective of multimodal magnetic resonance imaging,this thesis studies the prediction of glioma molecular phenotype based on evidential deep learning and mixture model networks.The main contributions and research contents of this thesis are as follows:(1)This thesis introduces the multimodal magnetic mesonance imaging technology in detail,and summarizes the advantages and disadvantages of the general multimodal imaging fusion scheme.In view of the characteristics of glioma molecular phenotype data,According to the characteristics of data on molecular phenotype of glioma,this thesis selects decisionlevel fusion scheme and feature-level fusion scheme for prediction.(2)Aiming at the shortcomings of large information loss and low reliability in traditional decision-level fusion scheme,this thesis proposes a multimodal imaging classification framework based on evidential deep learning.Based on the idea of evidential deep learning,this thesis estimates the evidence and subjective opinions for each modality in one-dimensional convolutional neural networks,and realizes the uncertainty estimation for single modal decision.Then,this thesis uses Dempster-Shafer algorithm to generate a multimodal fusion opinion,which increases the complementarity between modalities.Taking the cross entropy function with penalty term as loss measure,the NovoGrad optimizer is used to complete multimodal imaging classification.Finally,by measuring the decision reliability of the modalities,the optimal modal fusion scheme is selected.The experimental results show that the framework proposed in this thesis has better classification performance,and provides a more reliable basis for accurate diagnosis of glioma.(3)Aiming at the shortcoming of traditional feature-level fusion schemes which ignore the potential relationship between multimodal imaging features,this thesis proposes a multimodal imaging classification framework based on mixture model networks.Multimodal features are used to construct a feature graph,and Graclus algorithm is used to coarse the feature graph for several times to achieve feature dimension reduction.Then,based on mixture model networks,the graph convolution operation is performed on the feature graph,and the edge weights are continuously updated adaptively to fully exploit the potential relationship between the multimodal features.Finally,compared with the traditional feature-level fusion scheme and the graph convolutional network scheme with fixed edge weights.The experimental results show that the framework in this thesis has better classification performance,and the potential correlation between multimodal features is verified to have a more positive effect on the diagnosis of glioma.