Few-shot Micro-expressions Recognition Based On AU-GCN

In daily life,people usually express their joy,anger,sadness,fear and other emotions through the combination of facial features and muscles.At the same time,they are also used to observe the emotions of others by exploring the changes of expressions,so as to achieve the purpose of communication.In addition to the common macro-expression,the face also shows another action that is difficult to catch: micro-expression.An expression that persists for a brief period is termed micro-expression,and can express the real thoughts that people try to suppress and hide.Because the micro-expression can ’t be disguised,it can show the real emotion that the individual tries to hide,and it can become an important clue of behavior analysis and lie recognition.It will be widely used in the fields of national security,medicine,criminal investigation and psychology.However,micro-expressions are characterized by small data sets,short duration and low expression amplitude.i There are some difficulties in the recognition of micro-expressions,such as small sample size,large amount of calculation,missing key features and easy to over fit.The aim of this thesis is to improve the precision of micro-expression recognition,and to do some research on it from three aspects:(1)This paper approach for extracting Face Action Units(AU)features in critical areas is proposed.When micro-expressions are present,they are found only in certain areas of the face.Therefore,the whole face of the person is divided into seven local areas,and the sum of the optical flow amplitudes of the AU in each area is calculated and arranged in descending order.The first few regions with the highest amplitude are chosen as the representative features,and the attention mechanism is added to search for the inner connections of the different facial regions.Finally,the weighted average method is adopted to outperform the final result,which is helpful to extract the features of the face video sequence.The experimental results on CASME II and SAMM datasets can prove that compared with the global method,which lists irrelevant changes as feature information,the selection of 5 AU regions has a good extraction effect.(2)In this paper,a method of correlation characteristic based on the Graph Convolution Network is presented.When micro-expressions appear,the connection structure of facial composition cannot be ignored.The application of GCN can effectively find the dependency relationship between each node of AU,extract the spatial characteristics of facial topology,and aggregate them into global features,so as to do feature collection for subsequent micro-expression classification.Because the representative feature area accounts for a small portion of the entire face,a dice loss is selected in extracting the representative feature area to reduce the loss.(3)This paper deals with a method of micro expression recognition based on the prototype network.Compared with deep learning,it has a great deal of data support,but it is short of micro-expression database.Therefore,few-shot learning is combined with micro-expression,and the micro-expression categories identified during training are input into the prototype network as prototypes.Then,the distance between the new sample data and the category prototype was calculated to predict the micro expression.At the same time,the Triplet Loss function is used to optimize the hyper parameters,and the adaptive boundary value is obtained through a series of transformations.By incorporating this parameter into the cross-entropy loss function,we get a new loss algorithm and enhance the generalization capability of the model.