A Micro-expression Recognition Method Based On Multi-angle Feature Fusion

Micro-expressions refer to the rapid and subtle changes in facial muscles that cannot be restrained by the brain in a short period of time when humans are subjected to some external stimuli or under mental stress,which can reflect human emotion fluctuations more accurately and have wide application value in many fields that require high authenticity of human emotions.However,micro-expressions are formed under harsh conditions,making it difficult to produce large-scale high-quality datasets.The lack of datasets and small expression changes make microexpressions difficult to be recognized accurately.Existing micro-expression recognition methods mainly improve the distinguishability of micro-expressions by using multiple dimensional features or improve the recognition accuracy by focusing the model on learning features that are more conducive to differentiation through an attention mechanism.However,these methods often neglect the joint changes that occur in each key local region of the face when microexpressions occur and their gradual change processes in the time domain that make important contributions to the emotion expressions,losing some important micro-expression information and limiting the recognition accuracy of micro-expressions.To this end,the main contributions of this paper are as follows.First,to address the problem that some samples with different labels are difficult for local features to help the model improve the accuracy of micro-expression recognition due to similar variations in the same local region,a micro-expression recognition method that fuses spatial global information and local region linkage information is proposed.The method can effectively recognize micro-expressions under the condition of relying only on the apex frame,and has better recognition results than the existing apex frame-based models,in which the recognition accuracy and F1 score of the model improve about 13.6% and 0.14,respectively,over the LGCcon model on the CASMEII dataset;and about 29.5% and 0.33,respectively,on the SAMM dataset.Second,to address the problem that the existing models ignore the change information of local regions in the time domain when describing the time-domain features of micro-expressions,especially the change trend features formed in the time domain by the joint change information unique to each local at each stage of the change process,this paper proposes a micro-expression recognition method that incorporates the joint change trend features in the local time domain.The method describes a more complete micro-expression feature by combining spatial global,spatial local linkage and local time-domain joint change trend features.And it has a better recognition effect than the existing TSCNN with multi-stream input network,and its recognition accuracy and F1 score are improved by about 1.2% and 0.011 on CASMEII dataset,and about1% and 0.031 on SAMM dataset,respectively.Finally,a financial auditing system based on micro-expression analysis is designed and implemented.The main modules are login and registration module,financial application module,instant video audit module,integrity intelligent evaluation module and staff management module.A complete set of financial application and intelligent auditing process is formed.It realizes the intelligent evaluation of user information authenticity.In summary,this paper mainly addresses the problems of insufficient distinguishability of micro-expression features caused by the lack of micro-expression samples and small variability,and proposes a joint learning model based on the spatial global features of the apex frame and local area linkage features and a joint learning model of multidimensional features incorporating local time-domain change trend information,respectively.And a financial audit system based on micro-expression recognition is implemented.