An Approach On Motion Video-based Emotion Extraction

Emotion recognition is an advanced subject in the field of human-computer interaction. Now, most of the research is based-on facial expression or voice. However, the research based-on body movement is rare. We focus on the action video, study the relationship between the action and emotion, and explore the way to extract the emotion information from the action video.Firstly, we study the behavior psychology theory and the technology in pattern recognition area, and design a set of emotion. It can not only reflect human emotion, but also suit the request of pattern recognition. The emotion set contains four typical emotions, happy, angry, sad and fear. To suit the recognition, we also add neutral and untrustworthiness to the set.Secondly, we use the covariance matrix and the silhouette tunnel shape features to recognize the action in the video, and then analyze the action from static posture feature and dynamic movement feature based on the recognized result. The static posture feature contains the centre of gravity lean, the bending degree of chest, head, upper arm and elbow. The dynamic movement feature contains the speed of centre of gravity and the relative speed of other joints. Then, we construct the mapping model from action to emotion. We combine the static posture feature and dynamic movement feature to get the emotion extract result.Finally, we verify our method with the dataset of Action Perception Lab of York University. The experiment result shows that our method can extract the emotion from action video effectively.