Research On Micro-Expression Temporal Localization And Recognition Based On Deep Learning

Micro-expressions are extremely brief(usually 1/25s～1/3s)facial expression changes,which are usually caused by individual emotional or cognitive stimuli and are important for interpersonal communication,emotion recognition,and psychological disease diagnosis.In recent years,with the rapid development of artificial intelligence technology,micro-expression analysis has received more and more attention and research,and many breakthroughs have been made,but at the same time,it also faces a difficult problem:the problem of small samples of micro-expression datasets.To address this problem,this thesis proposes a micro-expression temporal localization task to speed up the construction of the dataset by accelerating the encoding speed of micro-expression coders.In addition,this thesis also investigates micro-expression recognition,a popular area in micro-expression analysis,and designs Inception-CBAM~+deep learning neural network to solve its problem of complex features and uneven distribution.The main contributions of this thesis are as follows:(1)A new task is proposed:Micro-expression temporal location,which aims to achieve precise positioning of the start frame,peak frame and end frame of micro-expression clips under the premise of small manual overhead.Firstly,the definition of the task is introduced in detail,and then a solution is designed for this task,namely Micro-Expression Contrastive Identification Annotation(MECIA).The algorithm consists of a deep neural network MECIA-Net and a frame expansion algorithm.Inspired by manual labeling,the MECIA-Net network includes a recognition module,a comparison module and a labeling module,which correspond to different steps of manual labeling.The frame extension algorithm realizes automatic micro-expression/non-micro-expression annotation by comparing network scores between frames.Finally,this thesis designs ablation experiments and psychological experiments,which prove the feasibility of micro-expression timing positioning and the effectiveness of the MECIA algorithm.(2)A micro-expression recognition network Inception-CBAM~+combining Inception and attention mechanism is designed.For the problem that the identity information in the original image is too large and the direct input of the original image will cause the network to learn too much identity information and ignore the action information,the network uses the optical flow features of micro-expression fragments as input;for the problem that it is difficult to extract diverse features in ordinary convolutional networks,the Inception module is used to extract multi-scale features of faces;for the problem that the micro-expression information in faces is not distributed For the problem of uneven distribution of micro-expression information in faces,a parallel channel attention and spatial attention mechanism CBAM~+is introduced to extract features of more interest to the recognition task.The unweighted F1score and unweighted average recall of this algorithm on the hybrid dataset MEGC2019 are0.7420 and 0.7435,respectively.ablation experiments show that Inception-CBAM~+improves by 0.0643 and 0.0686,compared to the network without adding Inception module with CBAM~+module.which shows its effectiveness.