EEG And FNIRS Based Multimodal BCI: Feature Extracrion And Classification

Brain-computer interface(BCI),a novel technology that directly extracts the information from the central nervous system and manipulates the peripheral devices by human’s thought,is considered as the highest form of future human-computer interaction and has been not only extensively applied in medical rehabilitation area,but also exhibits a high potential in the fields of military,entertainment,psychological and artificial intelligence.This paper begins with the background introduction of BCI.Then with the aim of overcoming the disadvantages of traditional electroencephalogram(EEG)-based BCI system,such as noise-sensitive and poor classification accuracy,the EEG-BCI is introduced into the function near-infrared spectroscopy(fNIRS).A EEG-fNIRS multi-modality BCI study focuses on feature extraction and classification,which are considered as the core of BCI,is carried out using a new and simplified designed hand grasping experimental paradigm.In this study,3 healthy subjects participated in the experiment using a concurrent EEG-fNIRS measurement configuration.The preprocessing of the raw data consists of filtering and baseline correction.Then the band power,autoregressive(AR)model coefficients and wavelet coefficients associated with the phenomenon of event-related desynchronization,event-related synchronization and time-frequency characteristic caused by hand grasping task is extracted as features.In addition,the mean value and slope of change of the oxygenated hemoglobin(HbO)corresponding to the motion-evoked hemodynamic response is also extracted.After the normalization of the feature vector,linear differential analysis(LDA)and support vector machine(SVM)are used for the classification by using different feature vectors with 8 times 5-fold cross validation.The result indicates that the wavelet coefficients shows better performance than combination of the band power and AR coefficients,while the slope of HbO change works better than the mean value.Moreover,the highest accuracy of slope is observed 5-7 seconds after the task onset and the accuracy of LDA is higher than that of SVM.Then,according to the classification results of the single-mode feature,a fusion feature based on the combination of EEG wavelet coefficients and fNIRS slope is proposed,and principal component analysis(PCA)is used for the fusion feature.LDA and SVM are again used for the classification with 8 times 5-fold cross validation.Then we compare and evaluate the classification accuracies of multi-modality and single modality.The present experimental result demonstrates that the complement of EEG and fNIRS can significantly improve the classification accuracy with 3~9% on average.Based on the above result,we are able to conclude that fNIRS-derived feature significantly increases the performance of the EEG-based BCI system and multi-modality,which can be used to optimize the traditional BCI system and would help for the application of EEG/fNIRS-based BCI in the future.