Design And Implementation Of Action Unit Detection System Based On Computer Vision

In recent years,computer images have begun to be widely used to assist to evaluate the severity of patients in the medical field.Intelligent medical diagnosis has become a trend.Early work demonstrated that untrained observers were able to identify depressed individuals from their expressions.In order to study the possibility of the use of computer vision technology in the diagnosis of depression,this thesis focuses on facial action unit detection(also known as AU detection).The physical expression of emotion is systematically classified by extracting face feature effectively.Furthermore,the severity of depression is evaluated with statistics on facial emotion recognition in patients.Firstly,some basic technologies involved in AU detection are summarized in this thesis.After that,this thesis emphasizes the research of convolution neural network design of AU detection algorithm.To reduce the effect of face alignment algorithm error,we propose non regional convolution to enhance the effectiveness of network.At the same time,the hierarchical residual network is introduced to increase the depth of the network.What is more,the attention mechanism is thoroughly studied across two dimensions:space and channel.The effectiveness of attention information on the two dimensions is firstly verified by position weighting matrix and depth attention model respectively.Then,the attention information from the two dimensions is combined for the experiments.Finally,a hierarchical residual network based on multi-dimensional attention mechanism is proposed.It can extract facial AU features effectively.Secondly,to solve the problem of unbalance samples in multi-label classification,we not only balance the samples with data augmentation,over sampling and under sampling,but also introduce the strategy of weighted loss function to solve the problem of uneven distribution of the sample number from a loss function standpoint.Finally,the thesis builds a data set consisting of video frames of depressed outpatients.Through fine tuning with data from depressed outpatients on the pretrained model,the average F1-score of AU detection reaches 59.5%.This result is 12.6%higher than that of a previous study,which is a high level in the field.At the same time,experiments in predicting depression are given to show the possibility of depression screening with facial AU detection.