Research On Medical Image Fusion Method Based On Convolutionl Neural Network

Image fusion is the process of combining useful or complementary information from multiple images into one image.Although the traditional medical image fusion research has made some achievements,the shortcomings are gradually exposed.For example,the medical image fusion method based on wavelet transform generate the fused image with artifacts,the fusion method based on sparse representation has poor performance in detail preservation,and the fusion method based on nonsubsampled contourlet transform(NSCT)has the limitation of a specific number of directional components.With the advent of the upsurge of deep learning,deep learning has been widely used in medical image processing field.However,the basic advantage of deep learning in medical imaging is based on the foundation which using large scale annotated datasets.The datasets are usually difficult to obtain in reality,it needs to be manually annotated by medical experts so that is extremely expensive and difficult to realize.This paper proposes a medical image fusion method based on Zero-shot Learning in order to solve the problem which is difficult to obtain manually labeled medical image data.This method combines nonsubsampled contourlet transform(NSCT)with convolutional neural network(CNN).Firstly,NSCT is used to decompose the source image into low-frequency subband and highfrequency subbands;secondly,the high-frequency subbands and low-frequency subband are fused by using the CNN model pre-trained by Image Net and the maximum selection fusion rule(MAX);finally,the fused image is obtained by inverse NSCT transformation.The experimental results show that,compared with five common image fusion methods,such as DWT,NSCT＿SF＿PCNN,NSCT＿PCPD,U-Net and CNN.The method proposed in this chapter has some advantages in detail preservation and visual effect.The high-frequency features are obtained through the CNN model pretrained by Image Net in the previous work.In order to better accommodate the characteristics of medical images,there also proposes an unsupervised medical image fusion method.Firstly,the fusion image is decomposed into high-frequency subbands and low-frequency subband.For the high-frequency subbands,it will first introduces the perceptual loss function,and then obtains it by training the unsupervised CNN model.The process is divided into two parts: the first part is pre-training,which use the dataset of which more than 80000 images came from MS-COCO;the second part is fine tuning which using the dataset composed of240 medical images from Harvard Medical School and the enhanced images.For the low-frequency subband,the weighted local energy is used as the lowfrequency fusion strategy.Finally,the final fusion image is obtained by inverse NSCT operation on high-frequency subbands and low-frequency subband.The experimental results show that,compared with seven common fusion methods,such as DWT,NSCT＿SF＿PCNN,NSCT＿PCPD,U-Net,CNN,GF＿WC and NSCT＿PC＿LE.The unsupervised medical fusion method proposed in this chapter is more effective.