| Automatically detecting human affective states is fundamental to the all-round development of artificial intelligence. The affective detection is to infer individual affective states using direct observations, such as neural activations, facial videos,voice recordings, body gestures, and physiological signals, etc. It includes affective recognition and estimation. In affective recognition researches, the state-of-art machine learning algorithms are applied to classify and recognize feature vectors, which are extracted from observation signals, into basic emotions(e.g., ’happy’, ’anger’,’sad’, and ’fear’, etc.). The affective recognition is suitable for detecting individual extreme affective states. Compared with the affective recognition, the affective estimation adopts function mappings by which one can estimate individual’s continuous affective states from the features of observed signals, and can supply more quantitative results than what classification procedures can supply. These continuous affective states are commonly represented by valence and arousal scores. The affective estimation is suitable for detecting individual affective states in daily life. For now, more and more studies indicate that human affective states can be estimated from observation signals.Methods of estimating human affective states, such as the multivariate linearregression analysis(MLR), partial least-square estimation(PLS), genetic algorithm optimized support vector regression(GA-SVR), artificial neural network(ANN),and bidirectional long short-term memory neural networks(LSTM-NNs) have been proposed in the past decade. Through computational models obtained by these methods, affective valence and arousal are regressed to the feature vectors, which are extracted from electroencephalogram(EEG), facial videos, voice recordings, body gestures, and physiological signals, etc. The nonlinearities of psychophysiological systems were confirmed by studies, in which the MLR and PLSR were adopted. Researchers obtained relatively accurate models to some extent by using ANN methods.They supply us a good prospect in solving affective estimation in a better manner.However, linear methods are lack of precision due to ignoring intrinsic nonlinearities of complex psychophysiological systems. The SVR method has its limitations in processing nonlinearities by adopting several kinds of kernel functions. The rest methods commonly adopt depth mathematics and complicated algorithms that are not easily understood and convenient to practically implement affective estimation.There has been no single method that is not only intuitive but also accurate up to now. Hence, developing a method, by which one can obtain a intuitive and accurate computational model for affective estimation, is important for replying the affective estimation problem that how the physiological organs determine the human continuous emotion dimensions, improving the affective detection accuracy of machines,and enriching the methodology of affective estimation.To further improve estimation accuracy, simplify the form of a computational model, and enhance models’ practical utilities, the study specially introduced the higher-order multivariable polynomial regression(HMPR) method to find a higherorder multivariable polynomial model for estimating affective dimensions. The proposed method will be an important supplement to the affective estimation methodology. Since the affective motivational brain and the central nervous system mechanisms of skin conductance signals are overlapped to some degree, the thesis took a pure affective skin conductance response as the input variables to predict corresponding affective valence and arousal, illuminate the theory, algorithm, results and psychophysiological significance of the HMPR method, and state the advantages of the HMPR method in solving the affective estimation problem. For all these, works that have been made include as follows:(1) We designed two affective psychophysiological experiments that are affective induction and affective evaluation, and built the integrated data acquisition system. During the experiments, we adopted the event related experimental paradigm, used the standard pictures, which were chose from the International Affective Picture System(IAPS), to induce college female subjects’ affective states. Synchronizing with the presentation of the picture stimuli, we recorded the affective physiological signals that contain pulse, electrocardiogram(ECG),and skin conductance(SC) signals for each subject. In the affective evaluation experiment, we used the Self-Assessment Manikin(SAM) scales to assess the subject’s valence and arousal scores. Through running the integrated data acquisition system and experiments, we collected valence-arousal scores and physiological signals for each subject;(2) We extracted pure affective psychophysiological features. Applying the Lim’s nonlinear curve fitting to averaging 10-second SC segments, we extracted affective pure skin conductance responses(SCRs). These pure SCRs were represented by four dimensional feature vectors that contains the response onset time, rise time, rise time, and decay time constant parameters. These feature vectors and average valence-arousal scores compose the fundamental data sets.The effective experimental operations were illustrated by applying mono factor analysis of variance onto each column of the fundamental data sets. Extracting the gain, decay time constant, valence, and arousal columns of the fundamental data sets, we defined the experimental data sets. The simulated data sets were constructed from the experimental data sets by using a statistical formula that proposed by us to solve the problem of requiring a relative large training data in executing the HMPR method. The simulated data sets remain the statistical characteristics of the experimental data sets and consist of new data which is distinct with the experimental data;(3) From the perspective of modeling methods, we discussed the problem that how to obtain an intuitive and accurate computational model by the proposed HMPR method. Given a system with multiple input variables and a single output variable, the HMPR method is on the base of the Taylor theorem that any complex smooth function can be approximated by its Taylor polynomial(a higher-order multivariable polynomial) at any precision in the convergent domain. Based on the observed data, the coefficients of a polynomial function can be estimated by using the least square regression, and the significance of the obtained polynomial model function were also tested. The point and interval prediction for a new input point can be realized through the optimal significant polynomial model. For choosing the optimal model, we proposed the Index that is used to evaluate quantitatively the performance of a computational model. The Index of a computational model is the ratio of its mean squared error(MES) to its Pearson’s correlation coefficient. The bigger the Index is, the better the model is. For obtaining the stable and actual performance of a computational model, we proposed a new technical procedure that is to establish the best computational model on the simulated data sets and test the obtained model on the entire experimental data sets. Under the theorems of the HMPR method, we designed an executable algorithm. On the simulated data sets, we regarded gain and decay time constant as input variables to predict valence and arousal and implemented the HMPR algorithm. As a result, we obtained the affective higher-order multivariable polynomial model(AHMPM) that translates the pure skin conductance response(SCR) signals into their corresponding affective valence and arousal. On the experimental data sets, the AHMPM is able to obtain significant correlation coefficients of0.98 and 0.96 for estimating affective valence and arousal scores, respectively.Hence, the AHMPM can be used as the computational model to enhance the smart equipments’ affective detection level, such as smart watches, Mi Band,and Google Glass, etc. Analyzing the gradient fields of the AHMPM, we found that the valence factor is sensitive to the gain and decay time constant dimensions and the arousal factor is mainly determined by the gain. Hence, we can provide certain indirect evidences that the valence factor is related with the selected activation of appetitive or defensive subcircuits, and the arousal factor may only reflect activation intensity;(4) We compared the proposed HMPR method with the existing methods. Based on the simulated and experimental data sets, different methods will obtain different model functions with different performances. We also adopted the proposed technical procedure that is to establish the best computational model on the simulated data sets and test the obtained model on the entire experimental data sets for other methods. The result is that the HMPR method currently is superior to other methods. Further, We proposed the method spectrum that is a plain graph whose abscissa axis represents the method intuition, and whose vertical axis represents the method accuracy. In the valence and arousal method spectrums, the actual positions of these five methods form practical spectrum lines whose shape are inverted U-shape, and the HMPR method are at the top of the practical spectrum lines. This means that the HMPR method is a good one for the SCRAS because it is not only relatively intuitive but also can obtain an accurate computational model. The differences between the actual spectrum lines and the theoretical expectations, that the more complex a method is, the more accurate the results, indicated that the methodological level for affective estimation is still at an elementary stage. More efforts are necessary to solve theoretical and technical problems in complex methods.In conclusion, taking modeling the system that translates the pure skin conductance response(SCR) signals into their corresponding affective valence and arousal as an example, the study specially introduced the higher-order multivariable polynomial regression method as an important supplement to emotional estimation methodology. We found that one can obtain a higher-order multivariable polynomial model for a particular affective estimation problem using the HMPR method and the obtained model can perform well. This is because the HMPR method is one kind of nonlinear methods and can describe the nonlinearities of a system by its nonlinear polynomial terms. Under the big data era, it is a trend now to detect human affect by multimodal signals. The use of the HPMR on the multimodal signals can bring obvious benefits that human affective states can be effectively detected, at the same time, much more detailed psychophysiological and neural mechanisms can be revealed, if one can effectively solve the three main open problems that include illustrating certain neural mechanisms, obtaining pure affective signal patterns for each modal, and efficiently fusing these multimodal feature vectors. These intuitive and accurate models can provide machines(e.g., wearable affective computer,European robot Nao, and Japanese AIBO dog) more intelligent affective detection abilities and enhance human mental health. |
PDF Full Text Request