Research On Image Fusion Method Based On Disentangled Representation

Image fusion technology is an image processing technique that combines two or more source images into a single fused image,and its fused image has higher quality compared with the source image,which is more beneficial to people’s subsequent visual tasks.Infrared and visible image fusion,as an important research branch of image fusion,has received attention from many scholars at home and abroad,and the research results have also produced greater value in practical applications.In recent years,with the new theories and methods in deep learning proposed one after another,image fusion technology is also developing rapidly.This paper focuses on a self-coding infrared and visible image fusion method based on decoupled representations,explores the complementary redundancy relationship between infrared and visible images from the goal of maximizing the retention of effective information in the source images,and proposes three innovative methods that can effectively improve the quality of fused images:(1)To address the problems of insufficient information extraction and feature decoupling and low interpretability in current fusion methods,an infrared and visible image fusion method(DIBF)based on information bottleneck twin self-coding network is firstly proposed in this paper in order to fully extract and fuse the effective information of source images.This method decouples the complementary features from the redundant features by constructing an information bottleneck module on the twin branch,and then corresponds the expression process of complementary information to the feature fitting process in the first half of the information bottleneck and the compression process of redundant features to the feature compression process in the second half of the information bottleneck,which cleverly expresses the information extraction and fusion in image fusion as an information bottleneck trade-off problem,and achieves fusion by finding the optimal information The fusion is achieved by finding the optimal expression of information.In the information bottleneck module,the network is trained to obtain the information weight map of the features,and based on the information weight map,the redundant features are compressed using the mean features,while the expression of the complementary information is facilitated by the loss function,and the optimization of the tradeoff between compression and expression is carried out simultaneously,and the redundant and complementary information are decoupled in this process.In the fusion stage,the information weight map is applied to the fusion rules,which improves the information richness of the fused images.Through subjective and objective experiments on standard TNO and Rosd Scene datasets and comparative analysis with traditional and recent fusion methods,the results show that the DIBF method can effectively fuse useful information in IR and visible images,and achieve better results in both visual perception and quantitative metrics.(2)An infrared and visible image fusion method(HRIBF)based on hierarchical representation self-encoding network of information bottlenecks is further proposed on the basis of DIBF.The method constructs a hierarchical representation coding network and an information bottleneck fusion network based on the self-coding network,which can reconstruct the fused image while preserving the complementary information of the source image and compressing the redundant information of the source image.The method firstly constructs a hierarchical representation network by repeatedly cascading adjacent features,encodes and decouples the source image into the hidden space,and obtains the hidden space features containing background branches and detail branches;secondly,in the fusion stage,a fusion module is constructed from information bottleneck units in series based on the idea of optimization of information bottleneck expression and compression tradeoff,and the input features and output features are constrained by information bottleneck fusion loss between The expression of complementary information is constrained by the information bottleneck fusion loss between the input features and output features,and the compression loss is used to constrain the compression of redundant information between the internal features to ensure that the fusion network has a better feature selection and fusion capability;finally,the decoding stage splices the fused background features and detail features in the channel dimension and decodes them by the decoder to obtain the fused image.In the reconstruction process of the overall network,the information-weighting weights are used to enhance the adaptive ability of the reconstructed network.Through experimental verification,the HRIBF method achieves better results in both subjective perception and objective evaluation compared with the current mainstream methods.(3)Both DIBF and HRIBF methods explore the complementary redundancy characteristics of source images at the feature level,and also propose an infrared and visible image fusion method(HFIAF)based on hierarchical feature injection self-encoding network to jointly process the complementary redundancy information at the pixel and feature levels.First,the method inputs infrared images,visible images,visible difference images and infrared difference images into the network simultaneously,and implements feature coding and fusion by means of hierarchical feature injection in the coding stage,and for each layer of features obtained from the coding of infrared images,the corresponding layer of features of visible difference images is injected into them,and the injection process is implemented by the feature injection module,and the obtained features are used as the input of the next layer of the network The visible image encoding process is the same as the infrared image.Then,the feature maps obtained from both infrared and visible branches are fed into the decoder to obtain two fused images,and finally the two fused images are fused into a single image through a mean value strategy.To improve the adaptive capability of the network,the method sets the adaptive weights of the infrared and visible parts of the loss function based on the amount of information at both pixel and feature levels.Experiments on TNO and Rosd Scene datasets show that HFIAF achieves better fusion results in terms of both subjective effects and objective evaluation.