Research On Medical Image Fusion Algorithm Based On Multimodal Feature Learning

Medical imaging plays an irreplaceable and critical role in the clinical diagnosis of brain diseases,and there are many imaging methods available to detect the state of tissues in the brain.Although these imaging methods have certain advantages,switching between images of different modalities may reduce the diagnostic efficiency and increase the bias of physicians in inferring clinical diagnoses because of the diverse imaging mechanisms and representations of medical images.The purpose of medical image fusion technology is to obtain comprehensive salient features and detailed information from medical images of different modalities,which can further improve the accuracy and time efficiency of clinical diagnosis.At present,the fusion performance of multimodal medical image fusion algorithms has achieved great improvement,but there is still room for improvement in the feature extraction of source images and the utilization of network intermediate layer information.To address the above problems,this paper is of great value to improve the multimodal medical image fusion algorithm by studying the multi-level feature extraction of source images and the fine fusion of features.The research in this paper includes:(1)A medical image fusion algorithm based on trident dilated perception and hyperdense connectivity is proposed.To address the problems that existing image fusion models do not fully utilize the useful information in the middle layer of the network,the algorithm constructs a dual-residual hyperdense module to realize the interaction of different path information and fully utilize the information in the network.In addition,a trident dilated perception module is proposed to obtain accurate location information and improve the feature representation capability of the network.Moreover,a new content-aware loss is designed to enable the algorithm to produce fused images more stably.Among them,the SSIM_f loss function makes the fused images have better consistency in structure with the source images,and ensures the generation of more edge information through gradient loss.Experiments show that the method can achieve good fusion quality in both subjective and objective evaluations.(2)A Transformer multi-task learning based medical image fusion algorithm is proposed.To address the problems that the fusion network may lead to the loss of some global and local information in the source images,the algorithm first trains an encoder-decoder network by multi-task learning,which makes the network learn medical image features accurately.In addition,a feature extraction network by combining an adaptive feature extraction module and a Transformer module is proposed to fully extract global and local features.And the global feature enhancement attention module is designed,to improve the performance of the deep neural network by reducing the loss of information and improving the interaction of global features to amplify the global interaction information.Experiments show that the method can improve the performance of medical image fusion in both subjective and objective evaluations.(3)A medical image fusion algorithm based on dual cascade attention is proposed.To address the problems such as the loss of detailed texture information of the source images by the fusion network,the algorithm first performs multi-scale feature extraction on the source images.In addition,a dual cascade attention module is proposed to retain more detailed information from the source images.And the fusion model is learned by a two-stage training strategy to ensure that the expected performance can be achieved in feature extraction and image reconstruction.It is experimentally demonstrated that the fusion results obtained by this method improve the multimodal medical image fusion in both subjective and objective evaluations.