Research On Dimensional Emotion Recognition Methods Based On Ranking

In people's daily interactions,emotion plays a very important role.It not only enriches the expression of the expresser,but also helps people understand the state and behavior of others.Therefore,the analysis and understanding of emotion has become an important research topic.In terms of emotion description,the dimensional emotion description model represents different emotional states as different points in the emotional space composed of multiple dimensions,so it can describe complex,subtle and continuous emotional states.At this stage,dimensional emotion recognition has attracted the attention of more and more researchers.In the current dimensional emotion recognition,the machine learning methods used to model and predict emotion are mostly classification and regression.However,the annotation of dimensional emotion usually takes the form of a continuous real value in a limited range,which has an ordinal property.The aforementioned methods do not focus on taking advantage of this property.Therefore,in order to appropriately utilize the ordinal property of dimensional emotion annotation and improve the performance of dimensional emotion recognition,this paper studies the dimensional emotion recognition methods based on ranking,and the specific research content is as follows:1)The dimensional emotion recognition method based on pointwise ordinal regression is proposed.This method transforms emotional rank prediction problem into a series of binary comparison problems.Based on this idea,the first emotional rank ranking framework in Arousal-Valence dimensional emotion space is built.In this framework,we firstly discretize the continuous dimensional emotion annotations to form finite emotional ranks.Then,a series of basic cost-sensitive binary classifiers are trained and each of them is used to determine the ordinal relationship between the emotional rank of a given sample and the emotional rank of itself.After all these binary classifiers are trained,in the testing phase,the given testing sample is classified by all binary classifiers successively.By aggregating all binary classification results,we can get the prediction result of the testing sample finally.The experimental results in AVEC 2015 benchmarking dataset and SEMAINE subset prove that the proposed method performs better than traditional and deep learning based classification and regression methods in mean absolute error and cumulative score evaluation indices.2)The dimensional emotion recognition method based on pairwise ranking and deep learning is proposed.This method directly uses the relative order in ordinal sample pairs and combines the strong learning ability of deep learning to learn a good emotion ranking model.In this method,we firstly propose and use a method for ordinal sample pair selection for continuous dimensional emotion labels,and a controllable sample pair set can be generated to train the ranking model.Then,we build the emotion ranking model based on siamese network.This model includes two deep convolutional neural networks with shared parameters and they transform input to emotional score.During training,two samples from a sample pair are input to the two branches separately and get their corresponding emotional scores.Then,the value of ranking cost function is calculated according to the ordinal relationship between emotional scores of two samples and the sample pair label.Back propagation algorithm is used to optimize the ranking model.In the testing phase,testing samples are input into one of the branches of the model successively and get their corresponding emotional scores.Finally,we can rank the testing samples according to their emotional scores.The experimental results in AVEC 2015 benchmarking dataset and SEMAINE subset show that the proposed approach performs better than traditional pairwise ranking and regression and deep learning based regression in emotion ranking task.3)The prototype system of ranking based dimensional emotion recognition is designed and implemented.The Matlab and C ++ are used to implement this prototype system.It mainly includes dimensional emotion recognition module based on pointwise ordinal regression and dimensional emotion recognition module based on pairwise ranking and deep learning.The prototype system implemented verifies the availability and effectiveness of the proposed methods well.