Research On Expression Recognition Based On Image Sequence

Expression recognition is to give some expression images or image sequence training samples,and use these samples to predict the expression category of any unknown image or image sequence.According to the division of research objects,expression recognition can be divided into two types:static expression-based and sequence-based expression.The research based on static expressions has been greatly developed in the past few decades.It has the advantages of small amount of calculation and simple and convenient feature extraction.It also achieves good recognition results in certain occasions,but static expressions ignore the dynamic expressions.The fact of attribute,and although its feature extraction is simple,it is easily disturbed by the external environment and individual differences,and it is not robust.With the deepening of the research,more and more researchers began to study the expression of sequences,because it captures the process of expression from emergence to disappearance.The extracted sequence expression features not only contain facial information but also have a time correlation.Sex,so the recognition rate can reach a higher level,and the research based on sequence expression is more realistic.In this article,the research focuses on the sequence database of two expression images,CK+and Oulu-CASIA.The specific work and main contents are as follows:(1)Give detailed algorithms included in sequence expression recognition,including face detection and positioning,image preprocessing,feature extraction,and recognition classification.This paper uses Viola-Jones face detection and Caffe-based face detection to locate and crop faces.In order to eliminate the influence of external interference factors such as uneven lighting and different image sizes on the expression recognition results,the sequence pictures were processed using preprocessing techniques such as image graying,histogram equalization,and size normalization.(2)A method of facial expression recognition based on feature point tracking and random field with variable state conditions is proposed,which has achieved good results on the CK+database.The feature point tracking part uses the active appearance model.To solve the problem of poor real-time performance caused by the slow fitting speed of the traditional active appearance model regression fitting algorithm,a linear regression parallel incremental cascading fitting strategy is proposed and tested on the LFPW database.The results show that the proposed fitting algorithm has improved accuracy and speed compared with the linear regression algorithm.Then use the latent conditional random field for feature classification.Although the latent conditional random field can encode the facial dynamic features such as facial expression or AU well with latent state,there is a problem that the latent state pattern is fixed,such as in a certain image sequence In the detection of whether the AU is activated or inactive,the ordered latent state can better describe the sequence segment containing the AU activation,but the unordered latent state can better describe the segment where the AU does not appear,and the latent condition random field All latent states are fixed as ordered or disordered.For this,a variable-state conditional random field model is proposed,which can automatically adjust the optimal latent state according to the input data,and test with 327 image sequences in the CK+database.Results The recognition rate on the CK+database of the marked feature point position reached 95.8%,and the recognition rate on the CK+database of the unmarked feature point position reached 95.0%.Compared with other feature extraction methods and classification methods,the results proved that The effectiveness of the algorithm in this chapter.(3)During the deep learning period,the two core steps of facial expression recognition are combined:extracting and classifying facial expression features,and a deep learning model is proposed.This model can automatically learn facial expression-related features and classify them,and achieves a significant recognition effect.This paper proposes a fusion model of convolutional neural network and recurrent neural network.The network model mainly includes two parts:spatial feature extraction network and time series information extraction network.For a given sequence of expression images,the spatial feature extraction network extracts the spatial features of the expression from each image in the sequence.Then use the long-term and short-term memory network to process and count the sequence information of the sequence,mainly to obtain the time context information of the features in the sequence,and finally classify these features that integrate the sequence information.In order to make the proposed deep learning model fully trained,the database was expanded by 14 times by means of angle transformation.Finally,it was tested on the expanded CK+and Oulu-CASIA databases and compared with some algorithms.The test results show that the fusion network model can effectively extract the spatial characteristics and timing information of expressions,and achieves an accuracy rate of 96.4%on the CK+database,which is improved from the previous deep learning algorithm of conditional random field proposed in the previous chapter.It is more effective than some algorithms such as TMS(96.1%)and 3D-CNN(92.39%),and has achieved an accuracy rate of 91.5%on the Oulu-CASIA database,proving the effectiveness of the algorithm.