Deep Fusion Of Medical Multi-modal Images Based On Multi-level Features

Image fusion is becoming increasingly important for various clinical applications such as diagnosis,treatment planning,medical assessment and surgical navigation.It can reduce the redundancy between data,integrate the advantages of different modal images,and fuse multiple medical image information.The fused medical images will help doctors observe the lesion more intuitively and comprehensively,and improve the accuracy of diagnosis.The main contributions are as follows:1.In this paper,a deep multi-fusion framework with classifier-based feature synthesis is proposed to automatically fuse multi-modal medical images.It con-sists of a pre-trained autoencoder based on dense connections,a feature classifier and a multi-cascade fusion decoder with separately fusing high-frequency and low-frequency.Specifically,in proposed feature fusion block,parameter-adaptive pulse coupled neural network andl₁-weighted are employed to fuse high-frequency and low-frequency,respectively.Finally,we design a novel multi-cascade fusion decoder on total decoding feature stage to selectively fuse useful information from differen-t modalities.Through fusion experiments and clinical classification experiments,the proposed method has good fusion performance and classification effect,and the statistical significance test shows that the results are statistically significant.2.A nested fusion network based on non-subsampled shear domain transform（NSST）is proposed for medical image fusion.The framework consists of three layers of NSST nesting,which decomposes the source image into 1 low-frequency image and 1 batch of high-frequency images,and then use DMC-Fusion and PA-PCNN to fuse low-frequency and high-frequency images respectively,and then pass three NSST inverse transformations to obtain the final fusion image.Experiments show that this method has good performance in visualization and objective evaluation indicators.