Research On VGI Vector Spatial Data Management Method Based On Hadoop

With the rapidly growing popularity of mobile intelligent terminal, massive geographic data were generated by the mobile client users. Management skills based on the relational database limited the efficiency with the growth of VGI(Volunteered Geographic Information)data, and could not satisfy the high concurrent access needs in the WEB2.0environment. Current research involves less about the growing mass VGI data management and high concurrency. Management and analysis of VGI data and application-oriented structure of the new hardware is becoming one of the hot researching areas in the earth science fields. VGI data’s rapid growth rate led to the need to design simple, efficient, and strong scalability of data management solutions. How to use Hadoop as the representative of the cloud platform to manage growing VGI data is worth studying. This paper is focused on this issue, and the main contents include the following three aspects:1) VGI vector data organization based on HadoopUnder the needs of spatial data organization in VGI applications environment, the paper analyzed the data structure of VGI, including multi-source heterogeneous spatial data, and designed the VGI data storage solutions based on Hadoop and the corresponding spatial vector data model.2) VGI vector data management based on HadoopVGI data management methods include multi-level spatial indexing mechanism, data compression method, system reliability mechanisms and performance optimization strategies. Based on the characteristics of VGI vector data, this paper constructed a Hilbert-R-Tree index, and a multi-level spatial index on this basis. In terms of data compression, this paper used Snappy compression algorithms in HBase itself to achieve good results. In addition, after studying deeply of the reliability and performance of Hadoop strategy optimization strategies, this paper configured parameters recommendations according to the characteristics of the practical application.3) Designing and implementation of VGI vector data management system based on HadoopThe paper analyzed the functional requirements of VGI data management platform, and researched the method of VGI vector data cloud storage platform using the open source cloud storage platform:Hadoop, which includes data management module, incremental data management module and data query management module, deploying Linux PC cluster in distributed environment, completed the development of a prototype system of VGI vector data management based on open source cloud computing platform Hadoop.In summary, the paper designed and implemented the management methods about VGI vector spatial data based on Hadoop, and proposed multi-level spatial index method and data storage models aimed at VGI vector spatial data, which were validated in instances. The results showed that the VGI vector spatial data management method based on Hadoop can meet the needs of the mainstream of VGI data storage, the construction of a multi-level spatial index can meet the demand of the space of the data retrieval.