Brain-Computer Fusion-Based Target Recognition Technology For UAV Onboard Images

The complexity of the modern war environment has led to an increasingly higher risk factor for battlefield reconnaissance.In order to reduce the consumption of personnel and equipment,and reduce combat costs,the use of drones to conduct long-term reconnaissance,surprise attacks,and attacks on enemy targets has become an effective means of combat.Among them,the massive amount of UAV onboard data is an important source of reconnaissance intelligence and battlefield data.However,the movement of the target,the change of lighting conditions and the large amount of noise in the process of data transmission have brought huge challenges to the unmanned onboard image target recognition based on computer vision.The human brain’s cognitive visual system is the result of long-term evolution in nature,and has the advantages of fast,robust,and antinoise in image retrieval.Therefore,this article is oriented to the task of UAV onboard image analysis,and fully considers the problems of UAV onboard image background clutter,lighting changes,noise interference,etc.,and combines the advanced cognitive functions of the human brain with the powerful information processing capabilities of computer vision to design The UAV onboard image target recognition system based on the brain-computer combination improves the accuracy of UAV onboard image target recognition in complex environments,and provides a new technical approach for manmachine intelligent UAV military target monitoring.The main content and innovations of this article are as follows:(1)Aiming at the problem of UAV sensor image target recognition,RSVP experimental paradigm was designed and the experimental data analysis was completed.First,aiming at the problem of UAV onboard image target recognition,a Rapid Serial Visual Presentation(RSVP)experimental paradigm is designed,and the EEG signals of10 subjects are collected during the task of performing UAV onboard image target retrieval.In order to improve the signal-to-noise ratio of the EEG signal,the original EEG signal is preprocessed to remove noise and artifacts.The ERP component analysis of the preprocessed EEG signals shows that the subjects are sensitive to the target image during image retrieval and there will be obvious ERP components in the EEG signal,which can be used for the target image retrieval task.(2)Combining the temporal and spatial distribution characteristics of ERP signals,a Multi-Attention-based Convolutional Recurrent Model(MACRO)is proposed.The model uses the Convolution Neural Network(CNN)and the Long Short-Term Memory Network(LSTM)cascade framework as the basic framework,and the Channel-wise Attention Mechanism,Selective Kernel Networks and the Self-Attention Mechanism is also integrated into the cascade framework to extract more discriminative spatiotemporal features of EEG signals.Experimental results show that the attention mechanism in the model helps to capture important features in the space and time domains,and the average AUC detected by ERP is as high as 0.9310,which is significantly better than the selected comparison method.(3)Aiming at the limitations of the single-modal target recognition algorithm,an image target recognition algorithm of UAV onboard image driven by brain-computer signal feature fusion is proposed.On the basis of Res Net-18 extracting UAV onboard image features and MACRO extracting EEG signal features,a Human Guided Adaptive Instance Normalization(HGAda IN)algorithm is proposed for multimodal feature fusion.The algorithm uses two fully connected layer to learn affine parameters from the EEG signal characteristics to transform the image features,so as to improve the target recognition performance of the model.The experimental results of UAV reconnaissance images show that the proposed brain-machine signal feature fusion-driven target recognition algorithm can reach 95.91% accuracy,which is 12.4% and 14.83% higher than the pure computer vision method and the pure EEG method,respectively.Compared with the single-mode algorithm,it can adapt to the actual mission environment of the UAV more effectively.