Research On Diagnosis And Classification Algorithms Of Multimodal Medical Image With Few Samples

Hepatocellular carcinoma（HCC）is a highly prevalent and deadly tumour and the third most common cancer disease worldwide.Generally,the differentiation grade of hepatocellular carcinoma is divided into well-differentiated,moderately-differentiated,and poorly differentiated.A lower grade of differentiation represents a higher grade of tumour malignancy.From a clinical perspective,precise discrimination of the HCC differentiation grade preoperatively is crucial for treatment regimen formulation and prognostic effect estimation.Radiology technology,especially the emergence of intelligent imaging technology,provides possible solutions for the non-invasive quantitative analysis of HCC.The combination of radiology and machine learning methods has potential application prospects and important research implications and is currently an essential aspect of intelligent medical treatment.At present,research on intelligent analysis and diagnosis of medical images has achieved excellent scientific research results on various organs such as the chest,brain,and abdomen,which is attributable to the good characterization ability of deep learning on medical imaging.However,Due to the limitation of the number of clinical cases,the quality of clinical images,and the cost of data annotation,the current public medical imaging datasets,especially the well-labelled datasets,are much smaller than the traditional natural image datasets.These factors limit the learning ability of deep learning models.Therefore,improving the diagnostic ability of the deep learning model with limited training samples has become a significant issue at the moment.Besides,the multimodal imaging characteristics of the MRI imaging method helps to reflect the information of the lesion from different perspectives,which can improve the classification performance of the model.However,it is difficult for the deep learning model to use multimodal data with significant differences effectively.Therefore,it is necessary to design a method that can guide the model to extract the information that best reflects the lesion’s characteristics from the complex multimodal data.In summary,this paper takes the automated and accurate identification of HCC lesion differentiation as the research goal and proposes using metric learning and attention mechanism methods to construct a diagnosis and classification model of HCC differentiation to alleviate the impact of insufficient training samples and complex input data.The main contributions of this dissertation are as follows:1.In view of the lack of labeled data in the medical dataset,this paper proposes a Multimodal Contribution-Aware Trip Net（MCAT）network based on metric learning and attention aware weighted multimodal fusion,referring to the clinical diagnosis experience of radiologists.The innovation of MCAT is to propose a self-attention mechanism for calculating and using the contribution of 2D slices to classification tasks in multimodal fusion data and using the metric learning method to improve the classification performance of the model in the small-scale multimodal HCC dataset.The experimental results on the clinical data set show that MCAT proposed in this paper is better than the previous models in the histological classification tasks of HCC.The accuracy,sensitivity,and precision are 84%,87%,and 89% respectively.2.For the research of the non-invasive quantitative estimation method of the differentiation grading of hepatocellular carcinoma,this paper proposes a selfattentional guidance based histological differentiation discrimination model combined with multi-modality fusion and attention weight calculation scheme for DCE-MRI sequences of hepatocellular carcinoma.The model combines the spatiotemporal information in the enhancement sequence,learns the importance of each sequence and each slice in the sequence for the classification task,and effectively uses the feature information in the enhancement sequence in the temporal and spatial dimensions to improve the classification effect.During the experiment,the model is trained and tested on the top three hospitals’ clinical data set.The experimental results show that the selfattention-guided model proposed in this paper achieves the highest classification performance compared with several benchmark and mainstream models.The results show that our proposed self-attention model can achieve acceptable quantitative measuring of HCC histologic grading based on the MRI sequences.In the WHO histological classification task,the model’s accuracy,sensitivity,and precision reach80%,82% and 82%.