| Emotion is often a condiment in human life.Effective emotion recognition technology can adjust the influence of one’s body on negative emotions by means of feedback,which is not only helpful to maintain one’s physical and mental health,but also conducive to the realization of human-computer interaction of intelligent emotion calculation.However,there are still many optimization problems in the existing emotion recognition technology and its reliability,for example,According to the authenticity and accuracy of emotional results detected by subjective and singlemodal data,this paper proposes to identify emotions by integrating explicit facial expression and implicit objective physiological signals in response to the above problems.The main research contents are as follows:(1)The characteristics of the Heart Rate Variability(HRV)signals are analyzed.By analyzing the needs of the emotion recognition system based on the integration of the HRV signals and facial expressions,the overall platform is built by combining the required parameters.Including the acquisition end of the emotion recognition system based on the fusion of HRV signal and facial expression,the control end of the ECG signal acquisition system and the emotion recognition system software.Finally,the functional design of each module is completed.(2)Study on facial expression recognition algorithm based on VGG deep neural network,adopt Adaboost-based face detection and normalization processing method for preprocessing,input the deep neural network model optimized in this subject,and obtain the emotion classifier based on facial expression image.The emotion recognition accuracy of 94.69% was verified on the common data set CK+.(3)Research on HRV signal processing and emotion classification methods.Firstly,the ECG signal is de-noised,and the Pan-Tompkins algorithm is used to detect R wave.Secondly,multiple features in the time domain and frequency domain are extracted respectively by using linear analysis method,and finally the SVM emotion classifier based on HRV signal is trained.After analyzing the characteristic parameters of various emotional states,the accuracy of emotion recognition based on ECG is 84.6% under the verification of experimental data set.(4)Research on Emotion recognition method of fusion of HRV signal and facial expression,two decision level-based fusion methods based on feature layer and majority voting method and DS evidence theory were adopted to analyze the twomode data fusion,and improvements were proposed to solve the problem of evidence body conflict with DS evidence.Finally,an emotion assessment report was given for the analysis of the results after fusion within a period of time.(5)50 subjects were invited to participate in this study.The ECG data and facial expression images of the subjects were collected when they watched the seven emotional segments of happiness,sadness,fear,surprise,disgust,anger and neutral by using the emotional induction method of the video clips.The scale verification method was adopted to screen the collected data.The improved DS evidence theory in this paper improves emotion recognition by fusion of HRV signal and facial expression by 7.1% compared with single mode,9.4% compared with feature fusion method,2.3% compared with original DS evidence,and the final emotion recognition rate is 91.7%. |
