Research On EEG Emotion Recognition Based On Cognition Inspiration

As a common mental phenomenon,emotion is closely related to human's cognition and behavior.It is a hot topic in the field of artificial intelligence that machines can understand human emotions by calculating the emotion signals.As one of the most active research topics of affective computing,emotion recognition had received substantial attentions from computer vision and pattern recognition research communities.Basically,the responses of emotion can be roughly divided into the external and the internal responses.The typical external responses include facial expression,gesture or speech of human beings,and the typical internal responses include skin conductance response,heart rate,blood pressure,respiration rate,electroencephalograph(EEG),magnetoencephalogram(MEG).From the neuroscience view of point,there are some major brain cortex regions,e.g.,the orbital frontal cortex,ventral medial prefrontal cortex,and amygdala,are closely related with emotions,which provides us a potential way to decode emotion by recording human's brain signals over these brain regions.For example,by placing the EEG electrodes on the scalp,we can record the neural activities of the brain,which can be used to recognize human's emotions.In this dissertation,we focus on two challenging problems in EEG emotion recognition,i.e.,(1)how to extract more emotion discriminative features and(2)how to handle the problem that the training and testing EEG data are from different subjects hence there exists a potential data distribution shift between them,and propose some effective and well-performing deep neural network methods.In detail,the research mainly includes the following aspects:1.We propose a novel neural network model,called bi-hemisphere domain adversarial neural network(BiDANN)model,for electroencephalograph(EEG)emotion recognition.The BiDANN model is inspired by the neuroscience findings that the left and right hemispheres of human's brain are asymmetric to the emotional response.It contains a global and two local domain discriminators that work adversarially with a classifier to learn discriminative emotional features for each hemisphere.At the same time,it tries to reduce the possible domain differences in each hemisphere between source and target domains so as to improve the generality of the recognition model.2.We also propose an improved version of BiDANN,denoted by BiDANN-S.Compare with BiDANN,BiDANN-S includes an additional subject discriminator,which can lower the influences of the personal information of subjects to the EEG emotion recognition for subject-independent EEG emotion recognition problem.3.We propose a novel bi-hemispheric discrepancy model(BiHDM)to learn this discrepancy information between the two hemispheres to improve electroencephalograph(EEG)emotion recognition.Concretely,we first employ four directed recurrent neural networks(RNNs)based on two spatial orientations to traverse electrode signals on two separate brain regions.This enables the proposed model to obtain the deep representations of all the EEG electrodes' signals that keep their intrinsic spatial dependence.Upon this representation,a pairwise subnetwork is designed to explicitly capture the discrepancy information between the two hemispheres and extract higher-level features for final classification.Furthermore,considering the presence of the domain shift between training and testing data,we incorporate a domain discriminator that adversarially induces the overall feature learning module to generate emotion-related but domain invariant feature representation so as to further promote EEG emotion recognition.Experiments are conducted on three public EEG emotional datasets.The result well demonstrates the better performance achieved by the proposed model.4.We propose a novel EEG emotion recognition method inspired by neuroscience with respect to the brain response to different emotions.The proposed method,denoted by R2G-STNN,consists of spatial and temporal neural network modelds with regional to global hierarchical feature learning process to learn discriminative spatial-temporal EEG features.To learn the spatial features,a bidirectional long short term memory(BiLSTM)network is adopted to capture the intrinsic spatial relationships of EEG electrodes within brain region and between brain regions,respectively.Considering that different brain regions play different roles in the EEG emotion recognition,a region-attention layer into the R2G-STNN model is also introduced to learn a set of weights to strengthen or weaken the contributions of brain regions.Based on the spatial feature sequences,BiLSTM is adopted to learn both regional and global spatial-temporal features and the features are fitted into a classifier layer for learning emotiondiscriminative features,in which a domain discriminator working corporately with the classifier is used to decrease the domain shift between training and testing data.5.We propose a transferable attention neural network(TANN)for EEG emotion recognition,which will learn the emotion discriminative information by highlighting the transferable EEG brain areas.The existed methods for EEG emotion recognition always train the models based on all the EEG data indistinguishably.However,some of the training data may lead to a negative influence because they are significant dissimilar with the testing data.Then it is necessary to give more weights to the EEG data with strong transferability rather than forcefully training a classification model by all the data.Furthermore,motivated by work 4,namely,for an EEG data,not all the brain areas have the equal contribution for emotion expression,TANN will learn the emotion discriminative information by highlighting the transferable EEG brain areas and samples,namely local and global attention.This can be implemented by measuring the outputs of multiple brain area discriminators and a single EEG sample discriminator.