| Earth observation technologies make the quantity of Digital Elevation Model (DEM) data with a wide range and high precision explosive growth. Due to storage capacity and computational performance are limited, traditional digital terrain analysis method based on a single computer can’t meet the demands of geoscientific research and production applica-tion. The Research of theory and method of parallel digital terrain analysis based the new hardware architecture has become a hot point for geologists. However, current research mostly concentrate on the parallelization of terrain analysis algorithm, while rarely involv-ing how to manage large amounts of DEM data for parallel digital terrain analysis and re-sult data of digital terrain analysis. The existing DEM data management models can’t meet the needs of parallel digital terrain analysis. Hadoop, a distributed computing platform, is considered to be the edge tool of mass data processing and analysis. How to use Hadoop to manage the irruptive DEM data to meet the needs of the parallel digital terrain analysis is a valuable research topic. This article launches the research around this subject. Therefore, this article mainly includes the following three aspects:In the first place, the research of the DEM organization mode faces parallel digital terrain analysis. Based on the deep analysis of the characteristics of parallel digital terrain analysis and the organization and management demands for DEM data, DEM pyramid structure and its construction method are described in detail in this paper. And the distrib-uted storage model of DEM and incremental data based on this structure is designed. They can achieve the effective storage of mass DEM data and result data of parallel digital ter-rain analysis.In the second, the research of the DEM management methods faces parallel digital terrain analysis. DEM management methods including spatial index mechanism, data compression method, system fault tolerance mechanism and concurrent access strategy, etc. Combining with the storage model of HBase database, three-level spatial index is built and spatial index method based on content is put forward. These indexes enable the system to support characteristics-based query as well as content-based query. In the aspect of data compression, the run-length coding compression algorithm of height increment is designed. And the experiments have been carried out to verify the applicability of the algorithm. This article analyzed in detail fault-tolerant scheme and the high concurrent access strategies that Hadoop took on. And the suggested values of configuration parameters and part of the implementation code combined with the characteristics of DEM data are given in this pa-per.In the third, the design and implementation of DEM cloud storage prototype system. This paper firstly analyses the design requirements of DEM cloud storage system. Then the detailed design scheme of this system is given. A Hadoop cluster is set up to simulate DEM cloud storage environment, and DEM cloud storage prototype system is implement-ed based on this cluster. Four kinds of query methods are supported:one, querying based on file name; two, querying based on geographic range and resolution; three, querying based on geographic range and number of grid; four, querying based on specific content. Global SRTM3data is taken as experiment data to verify the correctness and efficiency of data retrieval. The experimental results show that the search result is completely correct and the time efficiency is satisfying under the condition of the existing cluster environ-ment. |
PDF Full Text Request