Unconstrained Facial Beauty Analysis And Prediction

The research of facial beauty belongs to an emerging field about human perception nature and rule. Facial beauty prediction is a challenging task in pattern recognition and biometric recognition as its indefinite evaluation criterion, compared with the other facial analysis task such as emotion recognition and gender classification. There are many methods designed for facial beauty prediction, whereas they have some limitations. Firstly, the results are almost obtained on a small-scale facial beauty database, thus it is difficult to model the structure information for facial beauty. Secondly, most facial beauty prediction algorithm presented previously needs burdensome landmark or expensive optimization procedure. To this end, we establish a larger database and present a novel method to predict facial beauty. The main work of this thesis are as follows:(1) A large-scale database is established whose distribution is more reasonable and utilized in our experiments. Different with other face pattern analysis tasks, there are several central problem in this field. One is that it is difficult for people to apply an objective criterion to estimate facial beauty. The other is that it is difficult to obtain a number of extremes beauty: very attractive and very ugly. Consequently, most experiments are evaluated on relatively small-scale self-established database with different rating criterion and scheme. It is inconvenient for the public to compare their algorithm with the research results that have been published. To address this, we constructed a face image acquisition framework and designed a reasonable artificial rating scheme based on previous methods, then established a large-scale database with 20000 labeled images and 80000 unlabeled images.(2) Both female and male facial beauty are analyzed under unconstrained conditions without landmark. In the unconstrained environment, there are many variations like pose, expression, lighting, background noises, image quality and so on. In previous study, researchers did not analyze these variations, they just analyzed facial beauty in constrained environment. Obviously, it required a lot of manual operation to eliminate those variations and didn’t model the characteristics of facial beauty under the real world. So it is difficult to be applied in engineering. In this thesis, we take image pixel as input of feature learning algorithm without any preprocessing.(3) Eigen faces, CRBM(Convolution restricted Boltzmann machine) and K-means feature learning algorithm are analyzed. Those three algorithms are evaluated on our database and experimental results show that the elaborated methods such as CRBM and K-means are more robust to complex environment. K-means outperforms CRBM since it needs less parameters to adjust and requires less computation expenditure, while CRBM is preferable when labeled sample is limited. A Multi-scale K-means model is presented to extract aesthetics feature. Multi-scale K-means is more preferable for facial beauty prediction, since the local feature and object-part feature learned by it can encode the spatial relationship between adjacent facial structures, which affect facial beauty assessment heavily, and can’t be represented by general appearance feature well. A novel experiment is conducted and the results demonstrate that multi-scale K-means is more expressive to minor change of spatial relationship between facial structures than K-means. Finally, the results of several elaborate experiments validate the efficacy of the presented model from the aspect of accuracy and time expenditure.