Social Media Popularity Prediction Based On Deep Multi-Modal Fusion Features Of CNN

With the rapid development of social media,social media data are increasingly diversified,which includes video,images,texts,time and geographical information,etc.,making it possible to use multi-modal data to predict the popularity of social media.Traditional methods for predicting the popularity of social media need to manually extract features through prior knowledge,which is difficult to combine feature extractors with regressors,and the process is complex and not applicable to big data feature processing.For the prediction of social media popularity with the aid of more factors,the generalization ability of traditional methods is insufficient.In the actual experiment,we find that the influence of different modal data on the prediction of social media popularity is quite large,and combining with different modal data is not linear superposition on the predictions.Large-scale multi-modal data cannot be used fully and effectively to predict social media popularity.In addition,the problem of serious sample imbalance in social media data also constrains popularity predictionBased on the characteristics of multi-modal data,the main research and contribution of this thesis are as follows(1)For single-scale data such as time and geography,according to the characteristics of these data,time-scale transformation and geographic information conversion are used respectively to transform these data into multi-scale features as much as possible.For text data,it is transformed into semantic related vectors using the Doc2vec algorithm(2)In order to obtain more image modal features,this thesis adopts pre-trained deep learning model Inception v4 and Place2-365-CNN to obtain the category features and scene features of the images respectively,and adopts dominant tone extraction algorithm to obtain dominant tone features of the images(3)In order to explore and utilize multi-modal data to predict the popularity of social media,this thesis proposes to use the deep convolutional neural network superimposed with the full connection layer to extract and integrate the multi-modal features,and finally we use XGBoost to predict the popularity(4)We analyze the effective components of the deep multi-modal fusion features obtained in this thesis,and verify the effectiveness of the proposed CNN-XGBoost algorithm.We investigate the influence of CNN structure on the prediction results.Experiments show that the social media popularity prediction algorithm based on deep multi-modal fusion features effectively extracts and fuses multi-modality,which performs better on SMHP dataset,with MSE of 0.9187,MAE of 0.5592 and SPR of 0.9175.