Research Of Emotion Recognition Based On Multi-channel Brain Neural Signals

Emotion recognition is becoming a practical need in many application areas,such as human-computer interaction and mental health monitoring.Exploring the inner emotional state from various types of human perception data has developed a number of technical directions based on facial expression and speech,etc.However,people habitually wear ‘social masks' to cover up their true emotion,which poses a challenge to the study of emotion recognition based on body's explicit data,such as facial expression and speech.Brain neural signal is the direct reflection of the true psychological activity of the human heart,and it can not be concealed,which makes it possible for reliable emotion recognition.However,as a high-level cognition process,there is a semantic gap between emotions and noise-rich brain neural signals.How to effectively mine emotion related information in brain signals and improve emotion recognition has become a common proposition in computer science,cognitive neuropsychology and other disciplines.To this end,this paper has carried out targeted research on some issues,including:1.Aiming at the problem of insufficient generalization of the features in the designed feature-based recognition method,this paper systematically studies the robustness of 18 signal features in emotion recognition on two benchmark EEG datasets,and analyzes the brain signal variables most related to emotion recognition.In this paper,it is found that the features of the temporal lobe electrodes on both sides of the head and the Hjorth mobility parameter in Beta rhythm have excellent generalization ability in emotion recognition.These findings narrow the feature extraction range for designed feature-based emotion recognition methods,which helps to reduce the cost of the required calculations.2.In order to provide the interpretability and cognitive science theoretical basis for the use of these features,we analyze the underlying mechanism of the correlation features that plays a role in emotion recognition from the correlation difference between the head electrode points of the positive-negative emotion group based on the analysis method of brain function network.We find almost for each feature,the average correlation coefficient between electrode points of user with negative emotion is significantly greater than that of user with positive emotion.In the related research,there is a lack of analysis of the feature action mechanism similar to the method of brain cognitive science research included in this paper.3.Aiming at the limitations of traditional brain neural signal features,we study from the perspective of latent factor decoding of brain signal in cognitive research based on the premise that ‘there exists cross-user,default brain hidden source variables to participate in emotions',various unsupervised neural network models are utilized to decode the latent factors from the multi-channel brain neural signals,and the emotional states are inferred above the hidden source sequence.The experimental results show that the unsupervised neural network model is more suitable for the modeling of brain neural signals than the widely used ICA method in the field,and the hidden source component contains the information about the emotional psychological process,which can be used as an effective representation of the emotional information of brain neural signal used for emotion recognition research.4.For the above-mentioned limitations based on traditional features and based on hidden source decoding emotion recognition methods,this paper proposes a complete framework used for ‘end-to-end'emotional state recognition for long-term brain signals from data representation to model construction,combining feature learning and state sequence modeling.The multi-channel signal representation method of ‘scalogram frame cube'proposed in this paper is helpful to mine and fuse emotion-related information between multi-channel signals,and the constructed ‘fusion layer of user emotional experience'conforms to the user decision mechanism in the time-varying phenomenon of emotional psychophysiological process in the mechanism of decision fusion.Experiments show that the framework of the proposed method can effectively improve the emotion recognition effect for multi-channel brain neural signals,and the hybrid model is suitable for application scenarios requiring real-time emotion monitoring.