Research On Infrared And Visible Image Fusion Algorithm Based On Deep Learning

The key to image fusion is to extract important information from the source images and integrate them to generate a complete image with high quality and rich detail,enhancing people’s interpretation of the scene.Infrared images can capture hidden information by receiving thermal radiation and ignoring the influence and interference of the external environment,but the clarity of the obtained images is insufficient.Visible images can capture more details and texture information in the image,but they are easily affected by the external environment.Therefore,in order to integrate the advantages of both,infrared and visible images are fused to provide a theoretical basis for resource detection,security protection,target detection,etc.Based on deep learning,this paper proposes two infrared and visible image fusion algorithms,aiming at the problems of noise,blur,distortion,artifact,and loss of detail in existing research algorithms.(1)A dual-branch fusion algorithm based on semantic features and detail information.In response to the issues of information loss and fuzzy details in existing fusion algorithms,this paper proposes a dual-branch fusion network based on semantic and detail information(DBSD),which utilizes a dual-branch structure to more fully capture the structural and textural features of multimodal images,resulting in fusion images with complete texture information and clear edge information.The network adds multi-scale receptive blocks and large kernel blocks in the semantic branch,and adopts full-scale skip connections to achieve multi-scale feature extraction.In the detail branch,dense connection blocks are used to improve feature reusability.The dual-branch fusion block,which is based on channel and spatial attention mechanisms,integrates multi-scale semantic and detail features to enhance the key features of the fusion results.A series of experiments were conducted on three public datasets,TNO,MSRS,and Road Scene,and the results demonstrate that the proposed algorithm can effectively enhance the clarity of the fused images,exhibiting superior performance in both subjective visual and objective evaluation.(2)A dual-branch fusion algorithm based on pyramid attention and cross-convolution.The problem of how to fully extract and preserve the structural and textural details in the source images is still a pressing issue.In order to integrate effective information from different sensors and enhance the completeness of image information,this paper constructs a dual-branch U-Net fusion network based on pyramid attention and cross convolution(PACCDU)to obtain fused images with high contrast,rich information,and clear contours.The encoder of this network adopts cross-encoding blocks and pyramid attention blocks to extract context features in different directions and cross-scale correlation features.The fusion block uses parallel spatial attention and channel attention to merge features of different scales.The decoder uses large kernel convolution blocks and pyramid attention to reconstruct the fusion features.A series of experiments were conducted on infrared and visible datasets,as well as medical image datasets,and the results showed that the proposed algorithm can effectively extract and preserve the details and background information of the source images,achieving good fusion performance in all aspects and demonstrating strong generalization ability.