Multimodal Information Fusion For Fake News Detection On Social Media

The rapid development and popularization of social media platforms have created a direct path from news producers to consumers,making people no longer access news from traditional channels such as well-planned news broadcast and daily news programs.Although social media platforms have accelerated the propagation of news and facilitated human-to-human communication,social media platforms offer a hotbed for fake news propagation due to their low cost,easy access and high anonymity,and lack of effective regulation and factchecking measures.Currently,fake news detection on social media presents challenges such as how to synergistically utilize the semantic relevance information of social media news text and news images,how to model the differences in feature distributions of social media news from different domains,how to model the dynamic evolution propagation of social media news dissemination networks over time,and yet existing fake news detection methods fail to address these problems.Therefore,this thesis conducts research from the perspectives of image information fusion for fake news detection,domain information fusion for fake news detection,news propagation network fusion for fake news detection.The main contributions of the thesis include:In terms of image information fusion for fake news detection,a multimodal fake news detection model based on cross-modal attention residual network and multi-channel convolutional neural network is proposed.First,it is a challenge of fusing the consistent and complementary information between news text and news images,and at the same time take into account the differences in the representation space of news text features and news image features to some extent.To solve the problem,we design a cross-modal information fusion module based on cross-modal attention mechanism and residual network structure which can selectively extract the relevant information related to a target modality from another source modality while maintaining the unique information of the target modality.Second,for some news,semantic information fusion between text and image may affects the model’s performance.Thus,we design a multichannel convolutional neural network to selectively extract textual feature representations from textual features unfused and fused image information.In terms of domain information fusion for fake news detection,a multimodal and multidomain fake news detection model based on knowledge augmented transformer and adversarial learning is proposed.In view of existing studies ignore the differences in feature distributions of social media news from different domains,a domain adversarial learning module based on sharedprivate architecture Transformer network is proposed to model domain-specific information and domain-shared information of news from different domains.Moreover,because social media news entities generally lack sufficient background knowledge,to enrich news with knowledge information in a homogeneous embedding space,we use the knowledge augmented transformer to selectively encode the information of entities from an external knowledge source into the feature representation of news text.In terms of news propagation network fusion for fake news detection,the fake news detection algorithms based on continuous-time dynamic graph neural network and based on discrete-time dynamic graph neural network are proposed,respectively.Existing propagation-based fake news detection methods focus on static networks and assume the whole information propagation network structure is accessible before performing learning algorithms.However,in real-world information diffusion networks,new nodes and edges constantly emerge.A static graph only captures the graph structure without dynamic evolution process over time.To address these limitations,we adopt the idea of continuous-time and discrete-time dynamic graph to model the diffusion process and propagation mode of social media news.Accordingly,we propose a fake news detection algorithm based on continuous time dynamic graph neural network and a fake news detection algorithm based on discrete-time dynamic graph neural network,respectively.