Research On The Facial Expression Recognition Method By Attention Mechanism And Multi-feature Fusion

Expression is one of the main means for expressing emotions in the human daily communication.In recent years,with the development of artificial intelligence-related technologies,automatic facial expression recognition has been widely used in extensive humancomputer interaction fields including public security,healthcare,commercial promotion,and automatic driving etc.It has become an important research topic,making significant progress.However,the facial expression recognition in real-world application still faces challenges such as complex background interference,uneven distribution of spatial information,and difficulty in extracting local detail features,resulting in that the accuracy needs to be further improved.Based on the convolutional neural network,this thesis improves and optimizes the facial expression recognition algorithm by introducing attention mechanism,designing dual-branch network structure,extracting texture features and optimizing loss function.The proposed methods have been trained and tested on the public datasets.The results show that the proposed method has good accuracy and robustness.In addition,the application of the expression recognition application demonstration interface system is designed and implemented to verify the applicability of the proposed methods.The main contributions are as follows:(1)To suppress background interference and improve unbalanced spatial information distribution,we have proposed a facial expression recognition network improved by the attention mechanism and Involution operator.Using VGG19 as baseline,it introduces the attention mechanism in the front to extract vital features of facial expressions.The joint normalization strategies are employed to balance the distribution of feature data to improve the training quality of the model.In the back end,dense connection has been utilized to strengthen effective feature reuse and extract deeper semantic information.The proposed network has been validated on three datasets CK+,FER2013 and RAF-DB,achieving a significant improvement in the accuracy.In addition,in order to improve the ability of the network to process datasets of complex condition,the Involution operator is introduced at the back end to replace part of convolution operators,which enhances the perception ability of spatial diversity information.Experimental results on complex datasets such as RAF-DB validate the proposed model can effectively improve the accuracy of facial expression recognition.(2)To improve the fusing ability of the global context information and local detailed features in the process of expression recognition,a facial expression recognition convolution neural network with a dual-branch has been proposed.It employs ResNet to extract shallow feature maps in the front,and then sends feature maps into the global and local dual branches to obtain global features and local ones respectively,improving the fusing ability of features.Thereinto,the global branch extracts the context information of facial expression,while the local branch extracts the facial detailed features.In addition,the global branch introduces the multi-head attention mechanism to suppress the invalid information interference,and the attention residual blocks are utilized in the dual branches to acquire the key features.The proposed network has been tested on three datasets CK+,FER2013 and RAF-DB.The results show that the facial expression accuracy of proposed network outperforms many state-of-the-art methods,indicating high accuracy and good stability.(3)In order to extract diverse features and improve the multi-task learning ability,an expression recognition method integrating GLCM texture features has been presented.This method uses ResNet18 as the basic network architecture,fuses image and GLCM texture features,and extracts detailed texture features generated by facial muscle movements.A joint loss function with the cross-entropy,center and triplet loss function is used to supervise the training,expanding the inter-class gap,narrowing the intra-class gap and reducing the impact of abnormal samples on the recognition results.The proposed method is tested on three datasets such as CK+,FER2013 and RAF-DB.The experimental results show that the proposed method has strong competitiveness.