| Classification of medical images using deep learning technologies such as convolutional neural networks is one of the important applications of deep learning technology in the medical field.Alzheimer’s Disease(AD)is a neurodegenerative disease that causes cognitive impairment and has a high fatality rate.HIV-associated Nurocognitive Disorders(HAND)is an HIV infection Early(1 week)manifestations,accurate prediction diagnosis and targeted prevention and intervention in the early stages of AD or HAND through convolutional neural networks may delay or reverse the pathophysiological process of the disorder,which will improve the prognosis and improve the patient quality of life,which has important scientific significance and clinical value.This paper builds an MRI medical image classification model based on a 3D convolutional neural network based on the HAND dataset provided by Beijing You’an Hospital affiliated to Capital Medical University and the ADNI-1 dataset of Alzheimer’s disease open dataset,exploring the use of multimodal and multiscale ideas to further optimizes the performance of the classification model.Aiming at the classification task of the ADNI-1 dataset,a multimodal classification algorithm based on graph convolutional neural networks is proposed,which not only uses 3D convolutional neural networks to obtain image features,but also makes full use of the subject’s age,gender,clinical diagnosis,Cognitive function test results and other electronic medical record information are used to construct a multimodal medical image classification model.Using the subjects as nodes,the medical record information as edges,and the acquired image features as feature vectors,a topological graph is constructed.While making full use of the medical image image features,it also uses the subject data extracted from the electronic medical record data.Features such as similarity and interaction,integrate multi-modal features to build a classification model.Experiments show that the multi-modal classification model based on graph convolutional neural network can significantly improve the classification performance compared with the single-modal classification model.However,ADNI-1,as a public dataset of Alzheimer’s disease,has the characteristics of high quality,well-labeled and sufficient,and real clinical medical image data such as HAND is difficult to meet this standard.Experimental results also show that multi-modal model based on GCN does not achieve good classification performance in HAND dataset.Therefore,for the classification task of HAND dataset,this paper adopts the following strategies to improve the performance of the model:(1)Solve the problem of fewer samples through data augmentation and transfer learning.First,based on the existing original HAND dataset,a variety of data augmentation methods such as flip and shear are used to amplify valid samples.The second is to transfer the knowledge learne from large-scale medical image datasets to the HAND classification model through transfer learning,which effectively reduces model overfitting.(2)A multi-scale 3D convolutional neural network and a multi-scale 3D convolutional auto-encoder are proposed to extract more levels of low-level and high-level image features from medical images of different scales.Image samples of different scales are obtained by downsampling method,and features of larger neighborhoods centered on voxels can be captured by multiscale models.Experimental results show that the multiscale classification algorithm proposed in this paper can improve the performance of classification model based on HAND dataset.Based on the ADNI-1 and HAND datasets,this paper proposes an algorithm to build a medical image classification model using multi-modal and multi-scale ideas.The results show that the multi-modal classification model based on GCN can significantly improve the performance of the three-class classification of ADNI-1,but the performance is poor on the HAND dataset.In order to further optimize the classification performance based on the HAND dataset,data augmentation,transfer learning,and multi-scale classification algorithms are used to improve the performance of the classification of the HAND dataset.Experimental results show that it can achieve a better classification performence. |