Fragmented Data Analysis Based On Metadata And Knowledge Graph

With the improvement of the level of social informatization,a large number of news events,newspaper conferences or political and business bulletins containing social political and economic information can be easily obtained through open source channels.However,unlike the traditional targeted collection of structured data,these open source data often have the characteristics of massive,multi-source,heterogeneous and presented in the form of fragmented data.How to effectively filter low-quality irrelevant information and deeply mine its internal correlation to obtain high-value information in fragmented data is an important topic of intelligence research in the era of big data.In this context,this paper proposes an analysis and processing framework for fragmented data,including the collection and processing of fragmented data,organization management,and association discovery in open source intelligence analysis.And for the lack of a unified specification to describe the content of multisource heterogeneous data,and the difficulty of finding associated data,etc.The association analysis of fragmented data is realized through three aspects: metadata extraction of fragmented data,enhanced knowledge graph based on entity cooccurrence relationship,and fragmented data representation learning of fusion knowledge.Specifically,firstly,the sequence of entities mentioned in the fragmented data is used as metadata describing its subject and content.Entity is an important element that carries text information content.The content carried in fragmented data text is often composed of entities and their relationships.Extracting entities as metadata can facilitate the program to understand its content.Second,the external knowledge graph is enhanced according to the entity co-occurrence relationship in the fragmented data.The external knowledge graph can introduce prior knowledge in the graph for the analysis process,but its scope is limited to the content contained in the graph and ignores the information carried by the fragmented data itself.Using high-frequency co-entity pairs to enhance the external knowledge graph can realize the introduction of external knowledge.Knowledge utilizes the effect of the information contained in the fragmented data itself.Third,the text and enhanced graphs of the fragmented data are combined to obtain its fused knowledge representation vector.In this paper,the attention mechanism is used to obtain the influence weight of different entities in the fragmented data,and based on this,its knowledge representation vector is obtained.Through the fully connected network,the knowledge representation vector is fused with the text representation vector to obtain its fusion knowledge representation vector,and the associated information discovery is carried out based on the fusion knowledge representation vector.Finally,design and carry out experiments to test the performance and effect of fragmented data analysis technology,and evaluate the feasibility and effectiveness of the algorithm It is verified that the fragmented data analysis method can achieve the effect of fragmented data association discovery.