Studies On Physiological Signals Based Emotion Recognition

Analysis of variance (ANOVA) and the pattern-classifier based methods are usually adopted to examine the physiological differentiability of the discrete emotions in the fields of psychophysiology and intelligent human-computer interaction. These methods directly depend on the statistic features of the physiological signals to find out whether there are emotion-specific physiological response patterns. However, no confirmation about the existence of affective physiological response in the physiological signals was given. For the first time, this thesis applied Random Matrix Theory (RMT) to study whether a certain kind of physiological signal contained correlated affective physiological response, and what kind of reaction time patterns the affective physiological signals had. To demonstrate the validity of the universal prediction of RMT for the eigenvalue statistics of the signal sequence empirical cross-correlation matrix, we first calculated the distribution of the nearest-neighbor spacings and the spectral rigidity of their eigenvalues, and then compared them with the RMT prediction for real symmetric random matrices. To compare the experimental data with the mutually independent time-series, a null hypothesis R which was the cross-correlation matrix of uncorrelated time-series having the same length and the same amount to the empirical data was constructed. And then the eigenvalue probability density and the eigenvector component distribution of the empirical cross-correlation matrix were computed and compared with the analytical results of the eigenvalue distribution and the eigenvector component distribution of the null hypothesis R, giving information that enabled us to identify which signals were reliable for emotion recognition in the current work. According to the analysis results of RMT, physiological features were extracted from the reliable signals, and the affective physiological signal samples each of which was depicted as a vector of the features were classified by using a Fisher classifier. In order to find out which feature subset could well distinguish the target emotion from the interference emotions, and which feature selection method was the best for the two-class emotion recognition problem, the Genetic Algorithm (GA), the Sequential Backward Selection (SBS), the Ant Colony (AC) algorithm, the Swarm Particle Optimization (SPO) algorithm and the Forward Floating Selection (FFS) were applied to the feature selection process, and the prediction performance, the computational complexity, the ability to avoid over-fitting and the feature reduction ability of these algorithms were compared. Feature combinations having the best recognition performance to the discrete emotions were given by the best feature selection method during the comparison of the above feature-space search strategies.The research and results of this thesis are as follows(1) An affective physiological signal database of 300 subjects was established, including six kinds of emotions (i.e. joy, surprise, disgust, grief, anger and fear) and eight kinds of physiological signals (i.e. GSR, HR, BVP, ECG, Rsp., facial EMG and two EEGs form the frontal lobe). The user-independent property given by the large number of subjects enabled the research results to be generalized well. Some of the eight kinds of physiological signals could be the reliable affective physiological feature extraction source.(2) By means of the analysis of RMT, it was found that the above 8 kinds of signals had the universal properties predicted by RMT. However, when the slow variation of the above signals was explored, marked deviations of the largest eigenvalue and the corresponding eigenvector component distribution from the RMT prediction were shown in the empirical data of GSR, HR, ECG and Rsp.; when the fast variation of the signals was explored, the above-mentioned marked deviations were only displayed in the empirical data of GSR and HR. Two conclusions can be drawn from the above two results:the signals of GSR and HR can be the reliable sources for feature extraction of emotion recognition; only the fast variation of GSR and HR can quickly respond to the emotional feeling, revealing the relationship between the affective physiological response and the time.(3) By comparing five kinds of feature selection methods, the Sequential Backward Selection (SBS) was found the best for the feature selection of two-class emotion recognition problems. And such comparison revealed the difference of the feature selection problem in emotion recognition from the other combinatorial optimization problem such as Traveling Salesman Problem (TSP).(4) The two-class emotion recognition systems established on the basis of RMT data analysis and the SBS feature selection method had good prediction performance, and the true positive rates of these systems were all 20% larger than their false positive rates. At the same time, the number of the selected features of each emotion recognition systems was less than 10, revealing the key features to distinguish a certain discrete emotion from the others. The database established on the affective physiological responses of 300 subjects ensured the user-independent property of the emotion recognition systems; the data analysis of RMT eliminated the disturbance of non-affective specific physiological responses to the affective specific physiological responses and increased the generalization performance of the research results; by comparing several searching algorithms, the best feature selection method was found; the best physiological feature subset of each emotion recognition system revealed the affective information encoding in the physiological signals.