Research On Hydrological Monitoring Data Sharing Mode Under Cloud Platform

Today,the scale of hydrological big data is getting bigger and far beyond the scope of traditional hydrological data processing.This is mainly manifested in two major aspects:First,the variety of hydrological data produced is very diverse and large,and the traditional relational database can no longer meet its storage and management.Secondly,the traditional single-machine parallelized hydrological data processing method not only has low fault tolerance,but also has very low performance,which is no longer suitable for large-scale hydrological data processing.Therefore,in order to solve these problems,we need to find a better solution.In the hydrological monitoring data sharing mode under the cloud platform,three key issues are mainly solved.The first is the problem of hydrological monitoring data storage.Here,the hydrological monitoring data is divided into two types for storage.The structured hydrological monitoring data is designed and stored in the HBase database.The unstructured hydrological monitoring data is stored in the HDFS and analyzed and managed by the Hive data warehouse.Secondly,the data transmission sharing problem,how to ensure the transmission performance in the data sharing process on the basis of ensuring the security and integrity of massive data,here the Kafka streaming processing model based on message distribution strategy is used for hydrological data sharing;Data analysis mining problem,how to design and implement data directory indexing service,etc.,provides platform foundation for data mining.Here,non-primary key index is established for HBase.This paper uses the most widely used big data related technology to realize the hydrological big data platform based on Hadoop technology.On this basis,it realizes the sharing of hydrological data,and then implements non-primary key indexing and hydrologicalmetadata management for HBase.Thus formed a complete hydrological data directory system,which laid a platform foundation for the mining and analysis of big data.Through experiments and verification,the hydrological monitoring data sharing cloud platform designed and implemented in this paper can achieve the integration and sharing of hydrological monitoring data.