Research On Target Recognition Algorithm Based On Multimodal Image Fusion

In recent years,with the rapid development of modern society,information has presented diverse and complex characteristics.Target recognition technology is gaining popularity as an important research area in the field of computer vision.And it is greatly developed and applied in many fields,such as intelligent driving,video surveillance,and remote sensing.Among them,because of the complexity of the urban road environment in the intelligent driving,effective identification of pedestrian targets is an important part of ensuring intelligent driving safety.Traditional pedestrian target recognition methods solely rely on visible unimodal image features,resulting in low target recognition performance in all-weather conditions.Because the infrared and visible modes have complementary characteristics,a target recognition algorithm with the idea of multimodal image fusion is currently proposed.However,existing algorithms have problems,such as not considering different lighting conditions and the differences between different modes.So how to effectively combine the feature information between these two modes to improve the accuracy of target recognition has emerged as a hotspot and the focus of current research.In this paper,the target recognition algorithm based on multimodal image fusion is mainly researched for visible mode and infrared mode.The following are the main research findings:Aiming at the problem that the pedestrian target detection algorithm with a single mode has poor detection performance in all-weather scenarios.And existing multimodal image fusion methods only fuse at one stage,resulting in wasted features at subsequent stages.Therefore,this paper proposes a pedestrian detection algorithm based on multimodal and multi-stage image fusion.This algorithm takes the SSD algorithm as the basic detection framework and extends it to dual-stream.It adopts the fusion strategy of direct stacking to fuse the two modal features.Meanwhile,compare and analyze two different feature fusion approaches,single-stage and multi-stage.The multi-stage feature fusion method has been experimentally proven to be the best fusion method.In addition,the multimodal pedestrian target recognition performance is more advantageous compared to the unimodal condition.Aiming at the problem that existing multimodal image fusion-based target recognition does not account for the different proportion of different modalities to the generation of fused features under different lighting conditions.This paper proposes a illumination perception weight fusion based multimodal pedestrian target recognition algorithm.In this algorithm,firstly,visible and infrared features are passed through ECA attention mechanism module to enhance feature representation.Then,on this basis,the features are sent to the light perception weight fusion module designed based on small neural network to learn the weights corresponding to different modes.This method solves the problem that the existing light weights stack the features of different modes in a way of 1:1.Finally,the obtained weights are weighted and fused with their corresponding features to generate fused features.And these features are fed into the pedestrian detection and recognition network to complete the recognition.It has been experimentally demonstrated that the fusion strategy can improve pedestrian target recognition performance to some extent.Aiming at the existing multi-modal feature fusion module is directly used the generated fusion module in the pedestrian detection network.This has the problems of large differences among different modes and insufficient interaction.Therefore,this paper proposes a pedestrian detection algorithm based on the mutual guidance of multimodal features.Firstly,by sending the generated visible and infrared features into the differential modality aware fusion(DMAF)module.It can reduce the difference between the two different modes and generate the fusion features.Then,the fused features are returned to the visible and infrared feature streams,providing richer information to the visible and infrared features generated in the next stage.Additionally,the fused features are acted on in the next stage to improve the characterization ability of the fused features.In the end,visible light features,infrared features,and fusion features from the last stage are sent into the detection network for pedestrian target detection.It has been experimentally demonstrated that the algorithm improves target recognition performance and model robustness.In this paper,based on feature-level fusion method,this paper studies the target recognition algorithm under visible light and infrared mode fusion.While the performance of the algorithm is verified by the KAIST dataset,the generalization performance is also verified on the LLVIP dataset and the M~3FD dataset.It shows to some extent the feasibility of the algorithm in practical scenarios and has some theoretical research significance.