Medical Image Fusion Method Research Based On Deep Neural Networks

Image fusion is an important branch and research hotspot in the field of information fusion.Its purpose is to make the fusion images have comprehensive and rich multimodal information by fusing multiple multimodal images,so as to realize deep image analysis and understanding.With the development of sensor and computer technology,the number and types of multimodal images are increasing day by day.After the fusion process of multimodal images,the output single fusion image can significantly improve the information deficiencies of singlemodal images,and it is helpful for the computer to further analyze and process subsequent image processing tasks,such as image recognition,image retrieval,image enhancement,and medical diagnosis,etc.Presently,medical image fusion has become one of the research hotspots in the field of image fusion,and it is widely used in the fields of surgical planning,condition analysis,lesion location and other fields.Medical image fusion realizes the information complementarity of multimodal images and can provide reliable reference information for clinical diagnosis and accurate treatment.Therefore,it has important research significance and practical application value.Commonly used multimodal medical images include MRI(Magnetic Resonance Imaging),CT(Computed Tomography),PET(Positive Electron Tomography)and SPECT(Single Photon Emission Computed Tomography).Due to the different imaging principles of different devices,medical images of different modalities provide different characteristic information of human tissues and organs.For example,MRI images have high resolution and display abundant soft tissue physiological and anatomical information;CT images display organ,bone and high-density tissue information;PET and SPECT images provide metabolic information of lesion tissue.Single-modality medical image contains limited physiological or pathological information,and it is difficult to be used alone for accurate medical diagnosis and analysis.To overcome this shortcoming,researchers have proposed a large number of multimodal medical image fusion methods.However,existing multimodal medical image fusion methods generally suffer from edge degradation,detail loss,color distortion and slow fusion speed.Aiming at these shortcomings of existing medical image fusion methods,this thesis proposes four medical image fusion methods based on deep neural networks.Aiming at the problem of edge degradation,a fusion method based on Laplacian pyramid and deep convolutional neural network is proposed.The proposed method first reconstructs input images using a deep convolutional neural network.Since the input images have undergone the residual learning of different convolutional layers in the image reconstruction process,the reconstructed images obtain residual information,thus enhancing the image edge strength.Then,the reconstructed images are decomposed and fused using Laplacian pyramid.To reduce the information lost in the process of pyramid decomposition and fusion,a gradient energy fusion strategy is proposed.When the Laplacian pyramid image fusion process is over,the gradient energy fusion strategy is used to fuse the pyramid fusion image and the Gaussian fusion image again,thereby enhancing the fusion image details.Experiments show that the proposed method significantly enhances the edge strength and details of fusion images.Aiming at the problem of serious noise,a fusion method based on rolling guided filter and deep convolutional neural network is proposed.The method first utilizes a rolling guided filter to decompose the input image into base image and detail image.To retain enough information in the base image and detail image at the same time,the research determines the parameters of the rolling guided filter through experiments,and then uses a deep convolutional neural network to extract the perceptual image of the input image.Since the perceptual image is more in line with the human visual perception mechanism,the perceptual information of the fusion image can be increased during the fusion process.To reduce the feature difference and noise in the perceptual image,the research normalizes the feature maps in different convolutional layers.Finally,the basic fusion image,the detail fusion image and the perceptual fusion image are added to obtain the brightness image,and the final fusion image is obtained after color transformation.Experiments show that the proposed method can effectively reduce the noise of fusion images.Aiming at the problem of color distortion,a fusion method based on multiscale residual pyramid attention network is proposed.The proposed method uses the residual attention network and the pyramid attention network to construct the multiscale residual pyramid attention network,which combines the advantages of the fast training speed of the residual attention network and the strong feature expression ability of the pyramid attention network.The proposed method consists of three parts: feature extractor,feature fuser and feature reconstructor.The function of the feature extractor is to extract the deep features of the input image,the function of the feature fuser is to achieve deep feature fusion,and the function of the feature reconstructor is to reconstruct the fusion features into a fusion image.During the training process,the feature fuser does not participate in the training.When the training process is over,the deep features obtained from the feature extractor are fused by the feature fuser,and then the fusion image is obtained by the feature reconstructor.Experiments show that the proposed method reduces the color distortion of fusion images,but brings a certain loss of details.Aiming at the problems of detail loss and slow fusion speed,a fusion method based on dual-stream attention mechanism generative adversarial network is proposed.The proposed method includes two parts: generator and discriminator.The research uses a dual-stream attention mechanism and three attention modules to build a generator to improve the image fusion quality.The function of the discriminator is to discriminate the authenticity of the input image.The generator and discriminator compete with each other and learn alternately.In the discriminator update stage,the input image is judged to be true,and the fusion image is judged to be false;in the generator update stage,the fusion image is judged to be true,thus forming an adversarial process with the discriminator.To further improve the fusion quality,the loss function of the generative adversarial network combines content loss,adversarial loss and perceptual loss.During the fusion process,the content loss preserves the basic content information,while the adversarial loss and perceptual loss enhance the detail information.Experiments show that the proposed method preserves rich details and improves the fusion speed.