Research On Complex Target Retrieval Online System Based On Brain-machine Collaborative Intelligence

Target image retrieval based on computer vision is extensively applied in fields such as video surveillance and medical image analysis.However,accurate recognition of targets becomes challenging due to some targets possessing characteristics such as camouflage,obstruction,abstraction,and uncertainty.Researchers have proposed target image retrieval based on the Rapid Serial Visual Presentation(RSVP)paradigm.The RSVP paradigm relies on Event-related Potentials(ERP)generated by the human brain when observing target images to accomplish complex target image retrieval.Nevertheless,the practicality of the RSVP paradigm is limited when there are numerous videos and images in the database,as human energy and cognitive abilities are finite.Given the limitations of the RSVP paradigm,it is crucial to combine human cognitive abilities with computer high-throughput to implement target image retrieval in the form of a closed-loop retrieval system.This paper conducts research on the complex target retrieval online system based on brain-machine collaborative intelligence,addressing issues such as difficult singleERP detection,poor cross-subject stability,and massive data processing.To tackle the difficulty of single-ERP detection and the susceptibility of single subjects to interference,this paper introduces a dual-subject RSVP paradigm and designs a Hyperscan network to recognize dual-subject EEG signals.The two modules of this network fuse the data and features of the two subjects at the data and feature layers,achieving multi-level integration of EEG signals.A chunked Long Short-Term Memory(LSTM)artificial neural network is used in the time dimension to extract features at different periods separately,completing fine-grained underlying feature extraction.While fusing the feature layer,some simple operations are employed to complete the fusion of the data layer,ensuring important information is not missed.Using five-fold cross-validation,experimental results show that this method improves the F1-score by at least 5% compared to traditional single-subject RSVP and has better stability,precision,and recall.To address the poor cross-subject detection stability,this paper proposes a multifeature low-dimensional subspace embedding ERP detection method for cross-subject RSVP.Previous evaluation methods mostly use a single EEG feature to detect ERP,which has stability issues when applied across subjects.This paper’s method first uses the transfer learning method of Euclidean space alignment to align data from different subjects,then supervises dimensionality reduction and reconstruction of features from different spaces separately.Ultimately,adopting the leave-one-subject-out method as the test method and balanced accuracy as the evaluation metric,12 out of 14 length segments achieve optimal classification results in both the Physio Net RSVP dataset and the Tsinghua RSVP dataset.The results demonstrate that the proposed multi-feature low-dimensional subspace embedding method effectively improves the stability of ERP detection.To address the issue of massive data retrieval,this paper designs a complex target retrieval online system based on brain-machine collaboration.The EEG discrimination module guides target detection in computer vision,and the pedestrian relocation module searches for targets,while computer vision modules assist human cognition and retrieval of targets.To tackle the non-uniform and non-portable access to different EEG acquisition devices,an abstract brain-computer interface module is designed to unify signal access interfaces for various EEG acquisition devices.Lastly,a thread pool is introduced to enhance the response speed of real-time EEG signal discrimination in EEG target detection tasks.Online experiments verify that the proposed complex target retrieval online system exhibits high responsiveness,customizability,and efficiency.Guided by the human brain and assisted by computer vision,this paper investigates a complex target retrieval online system based on brain-machine collaborative intelligence.These research findings not only provide an effective supplement to the current brainmachine collaborative intelligence technology but also offer more ideas and solutions for complex target retrieval.