Research Of A Distributed Muliti-Layer R Tree Spatial Index Based On HDFS

With development of Surveying and mapping technology, geographical data which mapping geographic information sector owns have boomed while traditional methods of storaging and processing spatial data can not support the massive geographical data. With the continuous development of computer technology, database technology and file organization has changed greatly.Today, cloud computing, distributed processing and parallel grid computing technology is gradually mature and applied in all walks of life widely. As the wide application of distributed storage and computing framework, HDFS (Hadoop distributed file system) and MapReduce support massive data storage and fast processing using computer clusters, which provides a new way to improve the storage and computation of massive vector data.Therefore,it is one of the important topics in the research of spatial data storage and processing methods to introduce distributed technology into spatial data storage and organization facing the increasing demand of large dataset management.This paper uses traditional spatial data storage, query methods and HDFS distributed storage technology, solved the problem of massive spatial data storage by using a scalable distributed file system, HDFS. Traditional spatial indexes are not well suited for distributed spatial data storage structure, so we design a Distributed Multi-layer R Tree spatial index, i.e., DMLR, combined global index with local index. DSLR spatial index divide spatial data into different sub-regions by the application of the idea of segmentation, which can solve the uneven distribution of data better. Building DMLR index paralleled by using parallel computing framework-Spark can speed up the building efficiency of the index DMLR. In addition, we designed a parallel query method aimed for spatial data that were established by DMLR spatial index. Including the methods of parallel range aggregate query, K adjacent parallel query and parallel spatial join queries.DMLR spatial index provides a new method for distributed spatial data index. Experiments of spatial data parallel query verified the effectiveness of the DMLR index in distributed environment.