Multi-scale And Multi-channel Facial Expression Recognition Based On Attention Mechanism

Affective computing is a popular research direction in the field of artificial intelligence,along with the development process of deep learning,many applications based on affective computing have emerged in people’s daily life.Facial Expression Recognition(FER),as a significant portion of affective computing,is of great scientific value in human-computer interaction,education and medical fields.Facial expression recognition is vulnerable to occlusion,character posture and illumination change and other factors,which reduces the accuracy of expression recognition.By fusing multiscale expression features,combining multi-channel approach,and adding attention mechanism to the model,we can achieve the improvement of expression recognition accuracy in this paper.The main research contents of this paper are as follows:(1)Because of the problem that a single scale feature cannot describe the rich facial expression information,this paper proposes a multi-scale convolutional neural network,which completes the extraction of multi-scale expression features by designing convolutional modules containing convolutional kernels of different sizes.Meanwhile,since different parts of the face have different effects on expression recognition,the divided images of different parts of the face(left eye,right eye,nose,and mouth)and the complete face image are input into a multi-scale convolutional neural network to construct a multi-scale and multi-channel expression recognition method,and the parameters are not shared among channels.With this method,global expression features and local expression features fused with multiple scales can be extracted.According to the experimental results,the method improved the recognition rate by6.31%,10.31% and 7.68% on the CK+,MMI and RAF＿DB expression datasets,respectively,compared with the baseline(VGGNet)of the experimental setup.(2)Convolutional neural networks are used to extract image features by doing convolutional operations uniformly on the input image,however,the expression features are not uniformly distributed on the face image.To address this problem,this paper designs a multi-scale attention module,introduces the multi-scale attention module into a multi-channel convolutional neural network,and constructs a multi-scale and multi-channel expression recognition method based on the attention mechanism.The method can assign weights to the expression feature maps from both spatial and channel domains,and adaptively learn features that are more important for expression classification.It is shown experimentally that this method further improves the recognition rate on the CK+,MMI and RAF＿DB expression datasets compared with the method in(1)by 3.9%,4% and 4%,respectively.(3)In order to verify and analyze the effectiveness of the expression recognition method proposed,multiple sets of comparison experiments were conducted on the CK+,MMI and RAF＿DB expression datasets,and the experimental results further demonstrate that the method in this paper can effectively improve the correct expression recognition rate.