Research On Video-based Micro-expression Recognition

Human expressions play an important role in non-verbal emotional expression,including macro-expression and micro-expression.Macro-expression are more wellknown,and facial muscle movements have a relatively large range,which can clearly identify specific expression.Micro-expression research is relatively new.Different from traditional macro-expression,the duration of micro-expression is generally between 1/25 and 1/3 of a second,and it is difficult to extract the best expression of movement features.Micro-expression analysis has great application potential in social security,criminal investigation and other fields.As the most realistic response to human psychological activities,it has attracted more and more attention.Although microexpression is very short and difficult to detect,they contain rich emotional information.Macro expression can be forged and suppressed,and it is difficult to reflect people’s true emotions.Micro expression cannot be suppressed autonomously and are not controlled by humans.They have higher objectivity.As a spontaneous expression,they reveal the true emotions hidden by human beings.The study of micro-expression has a great impetus for related disciplines such as psychology and clinical medicine.Although the research on the macro-expression of the human face has yielded fruitful results,the research on micro-expression analysis is still very limited and has a lot of room for development.In the early micro-expression research,which heavily relied on expert experience,researchers proposed some auxiliary micro-expression training tools to detect and recognize micro-expression,but the high cost and low recognition rate made it difficult to be practically applied in reality.Later,with the establishment and release of some micro-expression databases in academia,researchers conducted a large number of experimental verifications on public datasets to analyze micro-expression.In recent years,deep learning technology has played a greater advantage than traditional methods in fields such as macro-expression classification.Many end-to-end neural network-based models have been applied in the field of micro-expression recognition.The feature extraction method based on deep learning avoids specialized and tedious manual feature design,can automatically capture and classify microexpression,and can extract the effective features and subtle changes of microexpression.Although deep learning has greatly improved the accuracy of microexpression recognition in recent years,there is still a lot of room for improvement and many open problems.This paper proposes corresponding solutions for the problems existing in the field of micro-expression recognition.The main research work of this paper are as follows:1)A series of preprocessing methods for micro-expression videos are proposed.Most deep learning-based micro-expression recognition algorithms usually directly perform end-to-end network training,ignoring more feature extraction and analysis in the preprocessing process before inputting to the network.In this regard,this paper completes the whole preprocessing from the original micro-expression database image to the network input feature map.This paper applies the latest research work on face alignment,explores video motion magnification techniques,and innovatively combines different optical flow features generated by weak facial movements.2)An improved residual network unit is designed,which integrates the spatial attention module for micro-expression data into the deep network to deal with the challenge of micro-expression recognition.In the recognition model proposed in this paper,the attention mechanism is used to learn more subtle features.Experiments show that the stacking combination of the improved residual units can extract the features of the weak motion area of the face.And each improved residual module is able to focus on the facial region of interest,and has superior performance compared to conventional methods for facial micro expression recognition.3)Aiming at the problem of low accuracy of micro-expression recognition algorithm,a novel classification model combining traditional feature extraction and deep learning is proposed.A dual-branch network architecture based on the superposition of Res Net residual modules is proposed,which can further capture the relevant features of micro-expression sequences for different combination components of the input optical flow features,and obtain better classification results.At the same time,the shallow stack of network units used in this paper not only ensures the good performance of the network model,but also reduces the size of the network model.The validity of the proposed model is verified by both quantitative and qualitative analysis on the public micro-expression dataset.