Research On Big Data Fusion And Storage Management Based On Public Safety

In recent years,various types of public safety incidents have shown a trend of changing from a very normal incident to a normal one,and the safety of people's lives and property is facing increasingly severe challenges.In order to accurately track and monitor the development and evolution of public safety incidents,a large amount of multi-source heterogeneous data needs to be aggregated,fused,and stored.Due to the complex and diverse characteristics of public safety data and difficulty in obtaining it,public safety business scenarios are flexible and changeable,resulting in a large number of deficiencies in public safety data integration and organizational management,which has become a bottleneck restricting the development of public safety data science.Based on the above background,the main research work of this paper is as follows:First of all,based on the three main characteristics of public safety data,combined with the different management needs of public safety data,for the three stages of data access,data processing and data storage,I designed their own technical solutions and proposed public safety big data.The overall system architecture of full chain,multi-level integration and storage management.Secondly,the pretreatment and fusion methods of public security big data are studied.Designed and implemented a typical architecture of public safety big data preprocessing based on Kettle,studied the “dirty data” recognition and data cleaning algorithm based on the binary classification logistic regression model;the Apriori algorithm was used to construct the correlation between the data,to achieve deep data fusion,the effectiveness of the algorithm is verified through the case of earthquake data in Sichuan over the years.Finally,the public safety big data storage management platform and data indexmethod are studied.Through the integrated deployment of the data collection tool Flume in the Hadoop ecosystem,the distributed computing engine Spark and the non-relational database HBase,real-time storage of network public opinion data based on the Hadoop ecosystem is realized.Aiming at the problems of slow public safety data index and low data processing efficiency,a hierarchical index method of public safety data based on information entropy is proposed.Experimental cases show that the real-time storage architecture has good feasibility and reliability.