Automatic Glioma Classification Using Multimodal Medical Images

Glioma is the most common central nervous system malignant tumor,which seriously endangers human life and health.The accurate molecular classification of gliomas has important clinical significance because it helps patients and doctors choose appropriate treatment options.As two different modal medical images,pathological images and MR images reflect the tumor characteristics of glioma at organ level and cell level,respectively.However,the existing computer-aided algorithms still have many challenges in MR images and pathological image analysis.Pathological image is the gold standard for the diagnosis of gliomas.However,the strain variation,inaccurate nuclei segmentation,and huge resolution harm computer-aided algorithms.For MR images,existing algorithms need to perform segmentation operations and use a single sequence of MR images.Combining pathology and MR images can make a more comprehensive diagnosis of the patient.However,the image features,resolution,and tumor information reflected in these two modal medical images are very different,which makes their fusion particularly difficult.To this end,this article has done the following:(1)To solve the problem of pathological image staining differences,this paper designs a self-supervised learning network for stain normalization The network uses a restain mechanism to protect the texture structure of the image.The network can be deployed in any scenario because it does not need paired training images.The network outperforms the stateof-the-art method on a public dataset.(2)To solve the problem of inaccurate nucleus segmentation,this paper designs a multitask learning nuclei segmentation network.The network does not require stain normalization and is sensitive to cell nucleus boundaries.Besides,this paper designs a progressive feature fusion module to fuse features from different components.The network achieves the state-ofthe-art segmentation performance on several public datasets(3)To fuse the multi-modal medical image better,this paper designs an automatic glioma classification model based on multi-modal medical images.The model automatically fuses the pathology images and MR image and then predicts the molecular classification of glioma.The model is evaluated in the CPM-Rad Path 2020 MICCAI International Challenge and achieved fifth place.