Fashion Compatibility Evaluation Based On Deep Learning

Recently,people’s fashion pursuit of dressing has been increasing.Fashion compatibility evaluation method based on deep learning can provide consumers with professional dressing evaluation,and help them make purchase decisions.Thus,the fashion compatibility evaluation method has important application prospects and research significance.Usually,an outfit consists of several fashion items.Fashion compatibility refers to the matching degree of items in material,style,and other aspects.The fashion compatibility evaluation mentioned in this paper means mapping the image and text of items into a compatibility score.The fashion compatibility evaluation includes feature extraction,compatibility calculation and model training.For feature extraction,existing methods extract item features individually,ignoring the influence among items.Existing methods calculate the outfits’ compatibility based on item pairs or based on the context of items.These methods use a equal weight summation to calculate the outfit’s compatibility score,neglecting the different contributions of each item.For model training,most of existing models are trained by triplet loss.However,a triplet only contains a single contrastive relationship.Moreover,in a triplet,the sampling of negative samples is unreliable.To address the problems above,this paper proposes two fashion compatibility evaluation methods.Based on these method,a fashion compatibility evaluation system is designed and implemented.The main work and contributions of this paper are as follows:Firstly,a fashion compatibility evaluation method based on matching template and selfattention is proposed.For feature extraction,a multi-stage embedding network extracts item features group by group.Matching template is defined as a group of item categories,describing the combination of items in an outfit.Thus,the matching template provides the basis for the division of item group.Then,the self-attention mechanism is introduced into the process of compatibility calculation.The self-attention mechanism adaptively allocates attention weights to each item,drawing the contribution of each item to outfit’s compatibility score.Secondly,this paper proposes to evaluate fashion compatibility based on mutual reference momentum contrast with multiple positive-negative samples.This paper first introduces multiple positive-negative samples,forming rich contrastive relationships.In this way,a strong supervisory signal is constructed.Then,this paper proposes a concept of weak-positive samples,which are mined accordding to style consistency and visual similarity.As the supplement of positive samples,the weak-positive sample is compared with the negative ones to form a contrastive relationship.This provides an additional supervisory signal for training.Finally,the single reference mode of momentum contrast method is extended to a mutual reference mode,which further enriches the supervisory signal for training.Experiments on Polyvore data set show that,comparing with the existing advanced method,the fashion compatibility evaluation method based on matching template and self-attention proposed in this paper performs better.This method improves the ACC of fill-in-the-blank questions by 1%and the AUC of compatibility evaluation by 0.01.Moreover,experiments on Fashion VC data set show that,comparing with training driven by triple loss,the mutual reference momentum contrast with multiple positive-negative samples proposed in this paper performs better.This method improves the AUC of compatibility evaluation by 0.03 and the MRR of compatibility ranking by0.06.Thirdly,according to the application demand of offline fashion sales,this paper designs and implements an “Outfit Impression” application system based on the first two parts.The fashion compatibility evaluation method proposed in this paper and a clothing segmentation network are deployed in the system.The system can segment user’s dressing outfit by pixel,and evaluate the fashion compatibility of them.Based on these results,the system can give users dressing suggestions.After verification,the system has marvelous application effect.