| Forest resource data reflects current status and changes of national forest, and it provides important basis for forest department and relevant enterprises. National Forestland “One Map”System has acquired remote sensing images, boundary data, change data and DEM(Digital Elevation Model) data from the beginning of system construction, which at the total amount of33 TB after pre-processing. With further investigating and enriching of types of application,National Forestland “One Map” contains lost more data, and data types are enlarging in the meantime. Current management methods’ problems in efficiency, avalibility and scalability are becoming more and more prominent encountering with huge data amount, and there is no suitable overall solution to solve existing problems. In this context, this paper deeply discusses and studies the organization mode, querying and analyzing method of large scale spatial data in distributed system.This paper deeply analyzed existing problems of traditional GIS architecture and current research of distributed GIS in deploying and running, designed the system architecture suitable for storage, spatial query and spatial analyze of distribute spatial data, described main used techniques, and implement the prototype system to verify relevant technologies. The result shows the prototype system has high performance of spatial query,spatial analysis and concurrency spatial access, and it can meet the time requirement of National Forestland “One Map”. The research work in this paper are as follows:1.Analyzed data connotation and application requirements, established the distributed architecture of National Forestland “One Map” system in theory, and put forward three main key problems: storage model of distributed spatial data, algorithm of distribute spatial query and distribute spatial analysis, task scheduling of distribute spatial computing task. 2.Research of distributed spatial storage model: by designing structure of key-value data in HDFS(Hadoop Distributed File System), distributed spatial database architecture based on memory storage, spatial data structure based ondistributed database, distributed spatial index based on hash code, realized the storage model of spatial data in distributed structure, which avoided the damage of spatial relation of distributed spatial data in current research. Test result shows the storage model speeds 17-70 times more than the traditional way. 3. Distribute spatial query and distribute spatial analysis algorithm:realized the basis logic of spatial analysis based on MapReduce using Hadoop’s MapReduce distribute computing architecture, and implemented some specific algorithm of spatial analyzing. Test result shows this method can reduce the system performance requirement of complex spatial analysis, and greatly enhance the efficiency of spatial analysis of large computing amount. 4. Task scheduling algorithm of distribute computing: designed the task scheduling algorithm of distribute computing based on the idea of users’ minimum quota,guaranteed the basis computing ability of spatial computing task, and assigned as more task to the computing node where the data located. Test result shows, comparing to MapReduce’s default algorithm, this algorithm increases the speed of average response time by 35%-40%,increases the speed of average time consumption by 15%-20% and increases the percentage of task containing local data by 5%-10%.The innovation of this paper are as follows: 1. Designed the system architecture which satisfied the need of distribute storage, distribute spatial query and distribute analyzing. 2.Designed the physical storage model, logic storage model and distributed spatial index of spatial data in distributed file system. 3. Designed the basis logic of spatial query and spatial analysis in distributed computing architecture, and designed some distribute computing algorithm of some typical spatial analysis. 4. Designed the scheduling process of distribute spatial computing task in distributed system architecture. |
PDF Full Text Request