Research On Working Memory Analysis Based On EEG-fNIRS Bimodality

Working memory is a system with limited resources for information processing and short-term storage.Working memory plays an important role in advanced cognitive tasks.People's daily life,study and work are inseparable from the influence of working memory.Therefore,the study of working memory is of great significance to the development of human society.At present,researchers are still not clear about the neurovascular coupling mechanism of the brain under working memory,and the accuracy of working memory load classification is not high.Electroencephalogram(EEG)records the brain waves on the scalp with high time resolution.Functional nearinfrared spectroscopy(f NIRS)records the hemoglobin changes of cerebral blood vessels with high spatial resolution.Therefore,the bimodal fusion of EEG and f NIRS can complement the advantages and disadvantages of the two technologies,which is conducive to a more comprehensive study of the brain's neurovascular coupling mechanism and working memory load classification in working memory.In this paper,we collected the synchronization data of EEG and f NIRS of 23 subjects under the nback working memory task.The specific research is as follows:(1)In order to study the neurovascular coupling mechanism of working memory,the synchronously collected EEG signal and f NIRS signal were preprocessed and feature extracted respectively.Then based on the fusion theory,the Canonical correlation analysis(CCA)fusion algorithm is used to extract the maximum correlation matrix of the EEG and f NIRS feature matrices.To understand the neurovascular coupling mechanism of the brain under the working memory task by analyzing the correlation matrix.After analyzing the results of working memory,we found that each group of related EEG and f NIRS components have a common trend of change.We conclude that the rhythms of delta,theta and alpha of EEG are related to the frontal polar region and dorsolateral prefrontal region of f NIRS.(2)In order to carry out the research of working memory load classification,select SVM,decision tree and BP neural network classifier to classify,and classify EEG feature matrix,f NIRS feature matrix,CCA feature fusion,direct feature fusion and PCA feature fusion respectively.Then,we compare the classification performance of CCA feature fusion with EEG feature matrix,f NIRS feature matrix,direct feature fusion and PCA feature fusion respectively.The results show that,compared with single-modal classification,CCA feature fusion can indeed improve the classification accuracy,and realize the complementary advantages of EEG signal and f NIRS signal.Compared with direct fusion and PCA feature fusion,we find that CCA feature fusion and SVM classifier have the highest classification accuracy.We confirmed that CCA feature fusion has excellent classification effect.