Research Of Multimodal Image Registration And Color Fusion Technique

Since application environment becomes more and more complex,the photoelectronic imaging system using single sensor has been difficult to meet the real requirement.In the imaging system with mutiple sensors,the technique of multimodal image fusion can combine the information from different image sensors,so it has been applied in the photoelectronic imaging and detection domain.Actually,we must apply image registration technique to align these images in space,if we want to finish high-quality image fusion.However,due to texture difference,scene depth and other factors,many problems still exist in the implementation of accurate multimodal image registration.Besides,there are also some drawbacks in traditional color fusion methods,such as the lack of texture information and the poor significance of targets.To address these problems and realize the fusion of infrared and visible videos or images,this paper is devoted to study the registration technique and color fusion technique for multimodal images.The main work and achievements of this paper are as follows:1.It is hard to accurately extract and match features in multimodal images,so this paper proposes a novel multimodal registration method with hierarchically combining motion and feature information.The method first estimates the motion vectors of targets using homologous feature point matching,and then computes a coarse registration matrix.The targets and features are re-located using the matrix.Based on the re-location,the normalized location and the histogram of edge orientation are used to describe the feature points,and a strict matching strategy is established.The mismatches are eliminated according to the matching orientation,and the point pairs from different frames are saved in a dynamic reservoir.Finally,a global registration matrix is precisely determined using all correspondences in the reservoir.The experimental results show that the average value of standard overlap errors of our method on all test videos is only 18.4%,which outperforms the state-of-the-art registration methods.2.To overcome the influence of depth difference between multiple targets in non-planar scenes,this paper proposes a novel multimodal registration method with independently analyzing multi-target.The targets are first matched using their shape and relative distribution information,and the matching results are used to simplify the feature matching step.In order to separately save the matches from different targets,the reservoirs based on Gaussian criterion are created,and a KCF multi-target tracking method is used to allot a reservoir for each target.Finally,a precise registration matrix is determined for each target with all correspondences in corresponding reservoir.The experimental results show that the foreground overlap errors of our method on each test videos reduce 10%-35%comparing with global registration methods,which is more robust and accurate.3.To improve the contrast ratio of texture and the significance of targets in the fused image,this paper proposes a novel color fusion method with multi-resolution analysis and target enhancement.The multimodal images are first decomposed into multi-resolution using the wavelet transform,and a gray fused image is obtained by applying different rules to fuse the high and low frequency components.The gray fused image is used to produce the false color fused image,and the moving object detection technology is used to split the target and the background.The color transfer is performed between the reference image and the false color image,and a transfer weight is used to protrude the target.The experimental results show that our method improve the significance of texture and target,which has a better fusion performance.4.The publicly universal LITIV and OTCBVS infrared-visible video datasets are used to test and verify the methods proposed in our paper.Comparing with some state-of-the-art registration algorithms and evaluating registration quality,the experimental results of image registration are given.The results show the proposed registration algorithms have good robustness and accuracy for the texture difference in multimodal image and the varying target depth.Comparing with typical color transfer algorithm,the experimental results of image fusion are given.The results show that the proposed fusion algorithm obviously enhances the texture and target information in the fused image.At last,the conclusion and prospect are given.