Research On Hashtag Recommendation Algorithm Based On Attention-based Deep Learning

In the real-world social network,such as Facebook,Instagram,Twitter and so on,users spontaneously generate several hashtags for users' contents(such as tweets,user's personal images),the behavior that assigning some hashtags to the social content is a kind of social behavior,which have random and spontaneous attributes.Hashtag is very common in social network and they are also very useful in many applications.For example,hashtag can be used for classification problems,topic models,sentimental analysis,information retrieval and so on.In research area,most works focus on hashtag recommendation for tweets,few works pay attention to the task of image hashtag recommendation.However,In some photo sharing websites,such as Instagram,Flickr,people tend to share more pictures,so there is a large number of images existing in these social networks.Moreover,the social images contain much semantic information that we can make use of for recommendation tasks.Automatically generating a list of hashtags of the social content is a challenging task,because hashtags are related to user preference,and it is not necessarily related to the content of the images.The relationship between hashtags and social images can be called weak relations.Most approaches mainly focus on text twitter hashtag recommendation,this task can be expended to images.To use deep learning techniques to model hashtag-hashtag relationship,we propose a new method,called A-NIH(Attention-based Neural Image Hashtagging Network),A-NIH model consists of Encoder and Decoder.Encoder use attention mechanism to select important features and filter out noise,it can get a better feature representation;Decoder generates hashtags by applying greedy search on recurrent neural networks,which not only overcomes the problem of repetitive hashtag sequence generation but also model the hashtag relationship.A-NIH model views hashtag recommendation task as a sequence generation task,A series of experiments are conducted on the real world datasets to prove the effectiveness of A-NIH,A-NIH achieves the best results.The Resnet50 based A-NIH achieves 2.34% improvement on Precision@1,3.56% improvement on Recall@5,and 8.4% improvement on Accuracy@5 compared with VGG.Furthermore,In order to prove the effectiveness on large datasets,additional experiments are conducted on NUS-WIDE,Inception V3 based A-NIH get 1.73% improvement on Precision@1,1.92% improvement on Recall@5,and 3.64% improvement on Accuracy@5 compared with VGG.There is a large number of short text posts existing in social networks,in most occasions,text contains much information that can help to improve hashtag recommendation.Therefore,considering the combination of images and tweets,we propose a new deep learning framework to handle these occasions,called MMAN.MMAN consists Encoder and Decoder.Based on A-NIH's Encoder,MMAN's Encoder add the extra text Encoder,Text Encoder can capture the contextual information from both the forward and backward directions by applying Bi-LSTMs,and attention mechanism is used for select important text features.The decoder makes use of Encoder's features to decode Hashtags,different from A-NIH,MMAN both use the feature from both the image Encoder and text Encoder,and it can capture the relationship between the text and hashtags.Experimental results on Custom Instagram datasets show that MMAN(MultiModal Attention-based Neural Network Model)achieves 13.15% improvement on Precision@1,8.2% improvement on Recall@5,and 27.99% improvement on Accuracy@5 compared with CoAttention,MMAN get the fantastic results.