Multi-task Face Description

With the development of the artificial intelligence,there are more and more applications of face recognition.However,there are still difficulties in facial attribute prediction,facial expression recognition,and micro-expression spotting in the field of face recognition.The location of each facial attribute is different,the number of attributes that need to be recognized at the same time is large,and the accuracy is low.It is difficult to correctly recognize all kinds of facial expressions due to the diversity of facial expressions.The feature representation used in micro-expression research is still poor,which affects the performance of micro-expression spotting.Therefore,this thesis focuses on these three tasks.The main works of the thesis are as follows:1.A multi-task convolutional neural network with a weighted loss penalty is proposed to recognize the facial attributes by groups.In this way,the spatial relationships among attributes are fully leveraged and the problem of unevenness of the positive and negative samples of the attributes and the difficulty of identifying different attributes are solved.The whole networks are trained on the CelebA and LFWA datasets.In addition,an adaptive threshold learning algorithm is designed to improve the performance.2.Based on convolutional neural networks,various modules of a facial expression recognition network are constructed.The network is trained not only on LSEMS W dataset which has many expression categories,but also on two basic expression datasets,JAFFE and CK+.Evaluations under several dataset splitting ways are conducted and analyzed.3.The traditional image feature representation in micro-expression research is improved.This thesis proposes facial muscle structure based ROI(Region of Interest)and dense sampling to extract LBP(Local Binary Pattern)features and MDMO(Main Directional Mean Optical Flow)features.For LBP features,the preprocessing of Gabor filtering is also introduced.4.This thesis constructs the balanced micro-expression image dataset with binary labels and trains the binary classification neural network to extract deep learning features for micro-expression spotting.Experimental performances of traditional image features,deep learning features and the combined features on SMIC-NIR,SMIC-VIS,and SMIC-HS datasets are fully analyzed.