Research On Micro-Expression Recognition Based On Region Adaption

People often unconsciously make subtle facial movements,known as micro-expressions,when trying to hide their true emotions.These movements can reveal a person’s authentic emotional and psychological state,making micro-expression recognition useful in fields like national security,public interrogation,depression treatment,and financial credit assessment.Artificial micro-expression identification presents greater difficulties and unreliability,and micro-expression research aims to create intelligent methods that enable machines or software to identify genuine emotions automatically from facial video sequences.However,it is challenging to extract accurate micro-expression features due to their weak and fleeting nature.Research on micro-expression recognition has greater theoretical significance and application prospects.In the last decade,extensive efforts have been made to advance micro-expression recognition technology;nevertheless,it still faces many challenges such as the lack of microexpression training samples leading to model overfitting,the abundance of redundant information in micro-expression videos resulting in high computational complexity,and difficulty in capturing fine-grained changes in facial expressions leading to suboptimal model performance.Taking into account the above puzzles,this thesis carries out the research on micro-expression recognition based on region adaption and makes the main contributions in three aspects as follows:(1)Haphazard Cuboids Feature Extraction for Micro-Expression Recognition.This approach proposes a haphazard cuboids method for overlappable sampling for microexpression clips by integrating facial landmarks.In contrast to existing methods that use a strict form of non-overlapping blocks and regions of interest,our proposed method resolves the issues of feature redundancy and loss caused by splitting of key regions.Extensive experiments on CASME II and SAMM and MEGC 2019 composite dataset validate the effectiveness and merits of proposed method.Further,by analyzing the temporal segments generation module in the proposed method via the maximum overlap interval,it is found that analogous to the facial regions of interest,the same segments of interest exist in the temporal domain for micro-expressions,which are highly relevant to the apex frame.This demonstrates that the apex frame in micro-expression clip convey substantial emotional information,and the rationality of the apex frame-based method for micro-expression recognition.(2)Shallow Multi-branch Attention Convolutional Neural Network for MicroExpression Recognition.This method first performs a simple region division by incorporating the spatial distribution of action units and pre-extracts micro-expression optical flow features from onset and apex frames,which allows subsequent models to focus on motion information within local regions.A multi-branch network with an attention mechanism is then designed to learn the micro-expression optical flow features in each local region.Finally,dynamic weighting is introduced to globally adapt and integrate the local micro-expression features obtained from different branches for classification.The proposed method effectively enables the model to improve the global micro-expression representations with limited samples via reasonably dividing facial regions in the input phase and dynamically learning the weights of each region in the fusion phase.Extensive experiments on the MEGC 2019 composite dataset demonstrate the effectiveness of our proposed method.The results also show that the method can yield salient and discriminative micro-expression representations and outperform comparable state-of-the-art methods.(3)A Homo-modal Framework Based on Optical Flow and Distance Correlation for Micro-Expression Recognition.Firstly,this method pre-extracts the optical flow information between the onset and apex frames of the micro-expression clips,which helps to represent the micro-expression deformations without bringing in facial identity information.Secondly,an attention mechanism-equipped dual-branch network is used to learn the identical optical flow features parallelly,building a more comprehensive micro-expression representation.Then,a dilated loss is employed to distinguish the dual-branch-learned micro-expression features,improving the network’s overall feature entropy and representation capacity.Finally,the features from both branches are fused for micro-expression classification.In contrast to previous simple single-branch or multi-branch networks,our proposed method increases the amount of information transmitted by two branches’ differentiated features,which effectively elevates the whole model’s representation capacity under limited micro-expression samples.Extensive experiments on the MEGC 2019 composite dataset validate that our proposed method is effective.Furthermore,the results indicate that our proposed method is superior to comparable state-of-the-art methods for micro-expression recognition.