Research On Facial Micro Expression Recognition Based On Video Sequence

Micro expression is a kind of hidden emotion.It is people’s spontaneous emotional expression in the state of depression.Its recognition can help to explore human’s real emotion.It is widely used in criminal investigation,psychology and other fields.However,compared with macro expression,micro expression has the characteristics of low intensity and short duration,which makes it difficult to be detected,and many traditional methods perform poorly in this task.In order to improve the accuracy of micro expression recognition(MER),the following work is carried out in this thesis:1)In order to improve the accuracy of MER algorithm and enhance the processing ability of the model for rich features in video sequences,this thesis designs a MER algorithm based on 3D spatial-temporal attention network(3D CSTN).This method is based on C3 D network,it adds the inception structure with residual connection and extends it to 3D space,and uses convolution kernels of different sizes to broaden the receptive field of the model for features of different sizes.Thus it improves the ability of the model to deal with the associated data and feature changes of the front and back frames in the video sequence.At the same time,a 3D spatial-temporal attention module is introduced,which integrates the attention mechanisms of time sequence,space and channel,and improves the aggregation and screening ability of the model for different dimensional features,so as to obtain more discriminant features.Compared with the original network,this method improves the recognition accuracy by 4.46% on the experimental results of the self built micro expression data set HVMED.2)2)In order to further improve the discrimination ability of the model for the types of easily confused micro expressions and synchronously realize the recognition task of facial local feature AU,this thesis designs an AU collaborative micro expression classification method based on 3D graph convolution.This method takes3 D CSTN as the backbone network to extract AU features and complete its recognition task.At the same time,the relationship map between AU nodes is established,and the node features of the map are enhanced by graph convolution network,The self attention pooling operation is introduced to screen out more representative nodes.Finally,the micro expression classification task is realized through node aggregation.This method integrates AU recognition and micro expression recognition tasks,deeply explores the correlation between AU tags and micro expression tags,and takes them as an important basis for micro expression classification,which effectively improves the recognition ability of the model for easily confused micro expression types.Compared with the original network,this method improves the recognition accuracy by 4.57% on the experimental results of the self built micro expression data set HVMED.