Research On Weakly Supervised Animated GIF Classification In Open Environment

Animated Graphics Interchange Format is a silent multi-frame animation image format between still image and video,with the advantages of strong expressiveness,high information content,low volume and easy transmission,which has been widely used in people’s daily online life.In recent years,with the development of the Internet and the popularity of smart devices,the number of animated GIFs on the Web has shown an explosive growth trend.Content-based animated GIF classification is the most basic and efficient way to organize massive animated GIF data,and can be widely used in GIF retrieval,sensitive information screening,and network public opinion prevention and control.Due to the openness of the Internet environment,learning animated GIF classification from Internet data is a highly challenging task.On the one hand,the semantics of the multi-frame images that make up an animated GIF are often discontinuous,and there are a large number of animated GIFs in the Internet that have sparse key frames related to the semantics of the animated GIF as a whole.On the other hand,The annotation of animated GIFs on the Internet lacks a unified specification,and the semantic levels of labels of different samples are inconsistent,that is,there is a affiliation between category labels,and there are very few animated GIFs with fine-grained precise annotation.This paper conducts extensive and in-depth research on the challenges of learning animated GIF content classification task from Internet data.(1)To train and evaluate the GIF content classification model,we first present a benchmark dataset,named WGIF(Web Animated GIF)for network animated GIF classification,which contains coarsegrained and fine-grained annotations from the Internet.(2)In order to solve the problem that key frames are sparse in relation to the overall semantics of the animated GIF,a keyframe attention pooling network based animated GIF content classification method is proposed in this paper.Through the key-frame attention mechanism,the model automatically re-weights the frames in the animated GIF.At the same time,the attention weight entropy regularization constraint is used to avoid the over-smooth problem of attention weight distribution.(3)Further,in order to make full use of the animated GIF data with different granularity annotations on the Internet,this paper proposes a coarse-grained aided fine-grained self-teaching method based on the key-frame attention pooling network for fine-grained animated GIF classification.The model can not only adaptively utilize coarse-grained and fine-grained annotations as supervision,but can also generate fine-grained pseudo-annotations for coarse annotated data to achieve self-teaching.Extensive experiments on the WGIF dataset show that both proposed models can well address the weakly supervised animated GIF classification in the open Internet environment,efficiently solve the challenges brought by the animated GIF itself and the annotation,improve the accuracy of animated GIF classification,and significantly outperform various existing baseline methods.