Research And Application Of Inland Waterway Sensing Data Storage Method

In the Internet of Things for inland waterways,the intelligent perception of sensing devices is the key to realize the measurable and controllable smart waterways.and management challenges.Taking the inland waterway in Ankang City,Shaanxi Province as an example,this paper focuses on the attribute characteristics of inland waterway sensing data,proposes a multisource heterogeneous inland waterway sensing data storage framework based on Mongo DB,and focuses on improving the efficiency of mass data query.The shard storage technology designed and implemented an inland waterway sensor data management platform.The main contents include:(1)Research and analysis on the data characteristics of the Internet of Things in inland waterways.Due to the scattered locations of channel sensing nodes,numerous types of sensing devices,and mixed wired and wireless transmissions,there are large differences in communication protocols and data formats for data transmission,and data attributes are characterized by multiple sources,heterogeneity,and massive amounts.Traditional relational databases cannot balance storage capacity and query efficiency.Therefore,it is necessary to design a storage structure based on the non-relational database Mongo DB in view of the characteristics of the waterway data sensing nodes,but few addition,modification and deletion operations.(2)Key technologies for mass data storage based on sharding keys.By analyzing the cardinality and occurrence frequency of Mongo DB’s sharding keys,a sharding cluster is built,so that massive sensory data is evenly distributed to different shards,and the query efficiency of data is improved.And through the shard cluster test,the data update time is compared when the data volume is from 1w to 500 w,and the effectiveness of the scheme is verified.(3)Design and implement the inland waterway sensor data management platform.The data management platform includes equipment management and data management modules.Through the Apache Bench performance test experiment,the test results show that when the number of concurrent users is 1000,the average waiting time of users and the average processing time of servers do not exceed 2s,indicating the availability and stability of the data management platform and proving the massive data based on sharding keys the validity of the storage method.