Research On Medical Image Models Compression Methods Based On Knowledge Distillation

In recent years,deep learning methods represented by convolutional neural networks have achieved achievements beyond traditional methods in tasks such as classification and segmentation of medical images,and have been more and more widely studied and applied.However,on the one hand,due to the special characteristics of medical images,the model accuracy of deep learning models applicable to natural images degrad when used on medical images? on the other hand,the existing deep learning models are generally computationally intensive and have too many parameters,which are difficult when applied to devices with limited hardware resources,and large models also suffer from overfitting problems when the training set data is small.In order to improve and solve the above-mentioned problems,this thesis revolves around the model itself and the knowledge distillation method,with the following main research and contributions:(1)Aiming at the the accuracy decrease problem of the model obtained by training the traditional knowledge distillation framework in medical image classification model compression,the theoretical principle of the traditional knowledge distillation framework is analyzed,the compression method of medical image classification model based on knowledge distillation is studied,and the hybrid knowledge distillation method based on hierarchical connection is proposed.The effectiveness of the proposed algorithm is verified by experimentally evaluating distillation algorithm on a dataset of ten image categories(2)To address the problems of incomplete region of interest segmentation and imprecise edge segmentation in the existing methods for fundus image optic disc segmentation,structural optimization of UNet,which is the commonly used medical image segmentation network,is investigated.Multi-Scale Connection block is proposed to enhance the cross-layer connectivity of the network.The Edge Aware Loss function is proposed to enhance the model’s ability to perceive the edges of the region of interest.Experiments are conducted on several datasets to verify the performance improvement of the proposed method on the model.(3)To address the problem of accuracy degradation of traditional knowledge distillation algorithms on the compression task of medical image semantic segmentation models,a knowledge distillation-based compression method for medical image semantic segmentation models is investigated,comparing traditional knowledge distillation algorithms,proposing knowledge distillation of multilayer intermediate features on the model encoder and decoder paths,and jointly training student networks using teacher network prediction soft labels and true value hard labels,both the previously proposed hybrid knowledge distillation method based on hierarchical connectivity and the multi-scale connection block are applied to the model.The advantages of the proposed algorithm and the traditional algorithm are experimentally evaluated on multiple data sets to verify the improvement of the proposed algorithm on the model accuracy.Experiments show that the method proposed in this thesis achieves good results on medical image model compression tasks,with some performance improvement over traditional methods.