Research On Vector Big Data Storage And High Performance Computing In A Cloud Environment

Breakthroughs in Earth observation and sensor technology,particularly the emergence of high-resolution satellite remote sensing technology,have resulted in the significant growth of geographic data.The development of the social economy has likewise led to the extensive and profound application of ubiquitous measurement and mobile apps in various industries,thereby resulting in the emergence of massive pan geographic data.Multi-source and heterogeneously massive geographic vector data provide extensive information support for fine management,social governance,and macro decision-making.However,among the popular issues in geographic information science that should be addressed immediately include effectively managing massive geographic vector data;querying,analyzing,and applying efficiently;and maximizing the value of data.In recent years,the rapid development of information technology,as represented by cloud computing and big data,has resulted in the emergence of No SQL database and high-performance computing cluster technology,thereby providing powerful storage and computing support for massive geographic data storage and high-performance computing.Starting from the characteristics and application requirements of massive geographic data,this study focuses on the storage and high-performance computing of geographic vector data.Moreover,the current research explores the storage and high-tech computing architecture of massive geographic vector data from the aspects of geographic information expression model,storage model,index organization,and spatial analysis and calculation.This study aims to establish a set of efficient storage and calculation of geographic vector big data.The proposed model also provides a set of efficient storage and high-performance computing solutions related to the research of geographic vector big data.The specific research contents are as follows.(1)Spatiotemporal storage model and index organization of geo vector big data.Combined with the characteristics of the No SQL database and geographic vector big data,the storage method of geographic vector big data for No SQL database is studied.The HBase database is used as basis to analyze the object storage model,data partition,spatiotemporal index organization,and parallel spatiotemporal query method of massive geographic vector data;and establish the storage and retrieval models of geographic vector data.(2)Geographic vector big data high-performance spatial analysis technology.This research utilizes high-performance computing cluster in a cloud environment as basis for the following objectives: study the influence of the shape and distribution characteristics of vector objects on the performance of spatial analysis algorithm,establish an algorithmoriented vector object shape complexity measurement model,and formulate a parallel computing data partitioning strategy based on shape complexity weighting.Experimental results show that the parallel computing data partition strategy based on shape complexity weighting further improves the parallelism of cluster spatial analysis.(3)A method of building spatiotemporal vector tiles for big vector data.Aiming at the problem of slow loading speed of local vector tiles caused by uneven spatial distribution of vector data,an adaptive size vector tile partition is constructed with tile data volume as the threshold,and an adaptive temporal vector tile partition method and space-time coding of vector tiles are proposed.Then,temporal vector tile storage is designed based on redis memory database and vector tile space-time coding.Then,different automatic update strategies of vector tiles are designed according to the application requirements of different scenes.Finally,the effectiveness of the proposed method is verified by experiments.(4)A prototype system of the geographic vector big data management and visualization prototype system is constructed.The system adopts the key technologies proposed in this paper,and reflects the typical requirements of "storage,management,query,analysis and display" of vector data in big data scenario.By testing the prototype system with the actual data of land survey of Yunnan Province,the key technologies proposed in this paper are proved to be able to be used for large-scale vector data management.Research and engineering applications show that the geographic vector big data storage and high-tech spatial analysis technology proposed in this study effectively combine cloud computing,big data,and other high-tech information technologies and GIS theoretical methods.In addition,the proposed technology realizes the efficient storage and retrieval of massive geographic vector big data and high-performance spatial analysis,and has been fully applied in the data management system of Yunnan Province.This technology provides technical support for the storage and application of large-scale complex geographic vector data,and has important significance in scientific research and engineering practice.