Weakly-Semi-supervised Glioma Segmentation And Survival Prediction Based On Magnetic Resonance Imaging

With the development of computer science in recent years,especially the emergence of deep learning technology,making it became a reality to fully automatic segment medical image.In addition,the survival analysis of patients can be more comprehensive and thorough with the help of deep learning.Primary glioma is the most common type of malignant tumor.It has the characteristics of high incidence and low cure rate.Surgical resection is the only effective treatment.Therefore,it is important to segment the tumor area in the MRI image accurately,and the result of segmentation determines the quality of the treatment.However,surgical resection is not suitable for all patients,and treatment should vary from person to person.The patient should be comprehensively evaluated first,weighting the pros and cons,and then deciding whether to resection.In this article,we propose a multi-task model that simultaneously performs image segmentation and survival prediction.For tumor segmentation,traditional methods require pixel-level annotation.However,the pixel-level annotation of medical images is very time-consuming and requires experts in the field to complete.Therefore,exploring a semi-supervised or weakly supervised algorithm is significant.In our experiments,a collaborative learning method is proposed to improve the performance of survival prediction and tumor segmentation through weakly supervised and semi-supervised with attention mechanism.For semi-supervised,unlabeled data is mainly used in Teacher-Student model,pseudo-labels derived from unlabeled data inference are further added to training,and the segmentation performance is gradually improved.As for weak supervision,we use survival days as the weak supervision source,and the role of weak supervision is confirmed through CAM.For survival prediction,the features learned from the segmentation task are fused with the survival prediction task,then calculate the risk value.In addition,we introduced the Transformer attention module in segmentation task,which can make the model focus on the tumor area.The results show that our multitask model is better than the single-task model,segmentation and survival prediction can promote each other.In order to have a more comprehensive survival analysis of patients,we believe that survival prediction based on multiple time series will be better than single time series because it provides more information.Therefore,we collected MRI images of different patients in multiple hospitals at multiple time points,and designed a multitime series model based on CNN+LSTM to verify our assume.In this article,we designed the comparative experiments,the experimental groups are mainly divided according to different time series,including one time point,two time points,three time points,and four time points of multiple time series data.The experimental results of hospital data further prove that the prediction effect of the multiple time series model is better than that of the single time series,and the more information provided,the better the result.For the first time,the experiment started from the perspective of time and space,and used the patient’s MRI changes to make survival predictions more accurately.And by calculating the risk ratio to analyze the impact of different factors on the survival of patients,the data sets of the three institutions including Southern Medical University,Henan Provincial People’s Hospital,and the First Affiliated Hospital of Zhengzhou University were calculated separately.In addition,a Nomogram was drawn to visualize the results.