Study Of Breast Cancer Survival Prediction With Multimodal Data Fusion

With the increasing incidence of breast cancer,accurately prognosis prediction for breast cancer patients is a key issue in the current cancer research.And survival prediction is one of the important research contents in predicting the prognosis of breast cancer.Accurately predicting the survival of breast cancer patients has important significance for the psychological rehabilitation of cancer patients and assisting with clinical decision making.Recently,with the advance of high-throughput sequencing technology and the pathological image production technology,a large amount of multimodal data including multi-omics data and pathological images are accumulated.The introduction of these multimodal data can greatly help to improve the predictive performance of breast cancer survival prediction.Therefore,how to effectively fuse the above multimodal data to help predict the survival time of breast cancer is an urgent problem in the field of cancer survival prediction research.In this paper,a study is proposed for predicting breast cancer survival by multimodal data fusion.The main contributions of this paper are as follows:(1)The recent advances of breast cancer survival prediction research are systematicly investigated and the details of breast cancer-related databases are understood.According to the investigation results,multimodal data from TCGA is extracted to build benchmark dataset for breast cancer survival prediction research.The dataset includes a variety of modal data information such as gene expression,copy number alteration,DNA methylation,protein expression,pathological images and so on.(2)To explore the validity of the fusion of different multi-omics data and pathological images in breast cancer survival prediction,in this study,a multiple kernel learning algorithm is used to fuse gene expression,copy number alteration,DNA methylation,protein expression and pathological images,significantly improves breast cancer survival prediction performance.In addition,by analyzing the effect of different multimodal data on the predictive performance of breast cancer survival,it is further shown that multi-omics data and pathological images can effectively improve the predictive performance of breast cancer.(3)Considering the diversity of different modal data,a deep neural network based method named MDNNMD by integrating multi-omics data is proposed.As a first attempt of multi-data fusion technology based on deep learning in the breast cancer related multi-omic data,the method designs different deep neural network models for each modal data,respectively,and then fuses differet models by late-fusion to promote the performance of predicting breast cancer survival.The systematic analysis of the results shows that MDNNMD can accurately predict the survival time of breast cancer patients.(4)On the basis of above research,a hybrid deep neural network based method named MHDNNGP by integrating multi-omics data and pathological images is further proposed.This method employs a novel DNN-CNN hybrid network structure,and constructs deep neural network and convolutional neural network for multi-omics data and pathological images,respectively.This method not only fully extracts the rich correlation features of survival time from multi-omics data and pathological images,but also considers inherent characteristics implied by both of them.Through performance evaluation on the test sets,it shows that MHDNNGP is more effective in predicting breast cancer survival.