Research On Infrared And Visible Image Matching Technology Based On Deep Learning

Infrared and visible image matching is a topic with high application value and theoretical difficulty,and occupies an extremely important position in aircraft navigation,remote sensing reconnaissance and other fields.With the development of related fields,the matching technology not only needs to adapt to the image matching under different time and view points,but also faces the imaging characteristics difference between infrared and visible images,which makes the traditional matching algorithm difficult to cope with more and more complex application scenarios.How to use neural networks to effectively extract the common features between infrared and visible images,achieve faster,more accurate,more stable infrared and visible image matching algorithm,and apply it to the actual scene is the main research content.For the infrared and visible image matching problem,the existing domain adaptive siamese network uses a domain assignment unit to reduce the difference of heterospectral images,but the unit only works at the head,resulting in weak scale adaptability.A multiscale domain adaptive siamese network MS-DASiam Net is designed,which based on adjusted CIRes Net22,can effectively extract the common features of infrared and visible images.MS-DASiam Net introduces pyramid segmentation attention to fuse the multiscale features,and conditional domain normalization is used to reduce the feature differences at the backend of each layer.By connecting the RPN network to the back-end of MS-DASiam Net,a multiscale domain adaptive matching network MS-DASiam RPN based on anchor is proposed to realize the cross-domain matching of infrared and visible images.In order to further compress the parameters of the network model and reduce the complexity of the algorithm,the bounding adaptive network BAN is used to replace the RPN network.A matching network MS-DASiam BAN without anchor is proposed,which introduced a multi-level prediction mechanism,both the low-level features and high-level semantics of the network is used to accurately locate the matching target.The comparison experiment was carried out using the public dataset and self-labeled dataset of infrared and visible images,the results show that MS-DASiam Net is more suitable for infrared and visible image matching than the existing siamese networks.MSDASiam RPN and MS-DASiam BAN can effectively cope with the application scenarios of target deformation,scale and perspective changes,partial occlusion and phase changes,and the matching accuracy of MS-DASiam BAN is better.Finally,MS-DASiam BAN is trimmed and transplanted on Jetson TX2 embedded platform.The method has the prospect of engineering application.