Research On Emotion Classification And Recognition Based On Multimodal Physiological Signals

Emotion classification recognition is one of the most important aspects in the field of emotion computing.The changes of human emotions are usually accompanied by the changes of physiological signals.Physiological signals can reflect the real emotional state,and the representation of emotions is more delicate,and not easy to disguise,which has been widely concerned by people.At present,the research on emotion mainly focuses on the study of EEG.However,the physiological information contained in different physiological signals can complement each other.The recognition accuracy of different emotions can be improved by making full use of the information represented by multi-modal physiological signals on different emotions.To solve this problem,this study will start from the single mode of electroencephalogram,electromyography and electrodermal signal to classify and identify the three kinds of emotions induced by the experiment:happiness,sadness and fear.This topic selects video materials to induce different emotions through the form of questionnaire,and designs experimental paradigm to induce three different emotions to obtain different physiological signals corresponding to different emotions.The collected physiological signals are analyzed,including signal preprocessing,feature extraction,feature selection,feature fusion and classification.The main research contents of this paper are as follows:(1)Classify and recognize different emotional EEG signals.After the pretreatment of EEG signals,the brain maps were drawn and the power spectral density features of five frequency bands were extracted.Through the energy brain maps,the main leads related to emotion can be obtained:FT7,FT8,FPz,FP1,O1,O2.Support Vector Machine(SVM),Extreme Learning Machine(ELM)and Convolution Neural Network(CNN)were used for classification.It was concluded that CNN algorithm can effectively distinguish three different types of emotion,and CNN has higher recognition accuracy and better stability than SVM and ELM.(2)Classification and recognition of different emotional EMG signals and EDA signals.The time domain features of EMG and EDA signals were extracted,and the features with high differentiation were screened by the R~2separability coefficient.SVM and ELM were respectively used for classification.When the ELM algorithm was used for classification,the optimal parameters were selected by comparing the number of neuron nodes in the ELM algorithm.The classification accuracy obtained under ELM algorithm is compared with that under SVM algorithm,and the conclusion that ELM has higher recognition accuracy and better stability effect than SVM is drawn.(3)Fusion and classification of multimodal physiological signal features under different emotions.A method based on feature splicing and feature fusion under different weights is used to study the method.The classification and recognition of emotions based on feature splicing multi-modal physiological signals were respectively compared under different conditions of single mode,horizontal and vertical splicing based on SVM、ELM and CNN classification algorithms.It is found that under the two different classification algorithms,EEG has the best classification and recognition effect of three single mode physiological signals,and its average classification and recognition accuracy is higher than that of the horizontal and vertical splicing of corresponding algorithms without setting weight.When using the method of feature fusion based on different weights for research,compare the classification accuracy of different weight matrices under the SVM、ELM and CNN algorithms respectively,and get the best classification effect when the weight coefficients are respectively EEG 0.7,EMG 0.15 and EDA 0.15.Compared with a single EEG signal,the classification accuracy of SVM,ELM and CNN algorithms has been improved by 5.81%,2.95%and 10.20%respectively,achieving better classification results.