A Study Of Lymph Node Metastasis Prediction Algorithm Based On Multi-modal Multi-instance Learning

Breast cancer is the most common cancer among women and the second most common cause of cancer death.Malignant tumors can metastasize,and the most common site of metastasis is the axillary lymph nodes on the same side as the breast cancer lesion.In order to avoid lymph node dissection for patients without lymph node metastasis,preoperative prediction of lymph node status is an important scientific exploration process.In this thesis,we propose three methods for predicting lymph node metastasis in breast cancer based on deep learning related techniques that are directly oriented to clinical data.1.In this thesis,we propose a deep learning based table learning model,CliTabNet,to predict lymph node metastasis status based on patient’s clinical information.CliTabNet introduces a sparse attention mechanism for feature selection and uses gated linear units to reduce the effect of gradient disappearance while maintaining the nonlinear abstraction capability of the model.The method is able to give the importance ranking of the features,which is important for the interpretability of the model.Experimental results show that the model improves the AUC by 6.31%compared to the basic fully connected network.2.In this thesis,we propose a dynamic thresholding method for tissue region detection to screen the foreground regions of tissues;we use a color-texture-based image block scoring method to rank the images and filter out the high quality image blocks,which can effectively solve the modeling problem caused by the huge size of pathology images.A multi-scale multi-instance fusion model(MSMI)is proposed to predict the lymph node metastasis status by combining the information at multiple field of view magnifications.The final AUC value of 0.7569 was achieved on the test set.3.In this thesis,we propose a multimodal multi-instance fusion model that combines data from different modalities to jointly predict lymph node metastasis status.In this model,the multimodal multi-instance(M3IF)module is constructed for learning cross-modal feature representations from different modalities,and guides the feature learning process for multiple modalities to improve the consistency and expression of features in multiple modalities,and finally the fusion of features.An AUC value of 0.8844 was achieved on the test set,which significantly outperforms existing methods.The algorithms proposed in this thesis for a series of analyses provide a benchmark for deep learning on breast cancer lymph node metastasis prediction tasks with some clinical value.