Real-time Storage And Query Of Urban Residents’ Water Data Based On Cloud Platform

The real-time and intelligent storage and management of all kinds of data related to the water use of urban residents is one of the urgent problems to be solved in the current urban water management system.In this paper,a storage system is designed to gather the water data acquired by the intelligent water meter of the user end(front end)to the data center(cloud)in real time through the network with the help of the cloud computing platform.The system can store the data to the cloud in real time and accurately,solve the massive storage problem of water data,facilitate the later data analysis and processing,provide decision support for the water supply department,and realize the user’s charge management and water supply scheduling.This paper designs a data format for the water data obtained from the water meter,which contains information such as water consumption,location,household and so on.As the information is generated in the form of text,in order to ensure that the generated files can be read across platforms,increase the readability of the data,and obtained water data is not limited by the length of characters,the file storage form with TXT as suffix name in the text file is selected.The storage and retrieval methods of this kind of data are studied,the aim is to store the data in the intelligent water meter in real time and provide the later data analysis requirement.This paper is based on cloud platform,using flume data acquisition technology,unstructured database HBase,data caching technology Kafka and real-time flow processing technology Spark Streaming to design storage system.Aiming at the residential water data from the front end,the real-time storage and query system designed and realized has been gradually optimized.This system not only realizes the mass storage of water data and meets the data storage requirements of this paper,but also further improves the retrieval efficiency of the whole system.