Distributed Preprocessing Technology For Path Analysis And Terrain Analysis Services

With the development of GIS data collection technology,geospatial data has grown rapidly and the data volume has become larger.Spatial analysis services deal with large-scale dataset rather than small-scale dataset.Online analysis service is inefficency on large-scale datasets and it is difficult to respond to user request in a short time.With the development of computer science in recent years,distributed computing technology has emerged.This brings opportunities for the above-mentioned problems.Based on requirements of map service related laboratory projects,this paper provides preprocessing technology for online path analysis services and terrain analysis services,which can improve the efficiency of online services.Due to the large-scale of spatial data in the project,this paper provide distributed preprocessing technology by Hadoop platform.The main contributions of this paper are as follows:(1)Based on the characteristics of road features under large-scale road network,this paper designes and implements a simple and fast method to generate large-scale road network graph by Map Reduce,and uses adjacency list to represent this sparse graph.(2)This paper pre-solves the shortest path of vertex pairs based on the large-scale road network graph.This paper implements MRA* algorithm and applies it to path analysis,and Considering the actual situation,this paper divides the road network into a set of subgraphs of circle-based outer rectangles.The Mapper stage searches for the shortest path of each sub-graph,and the Reducer stage merges subpaths and generates final shortest path.Preprocessing results are stored in HBase for query by online path analysis service.(3)This paper pre-calculates slope and aspect of terrain based on DEM.This paper designs and implements a distributed computing method for slope and aspect based on Map Reduce.The Map stage calculates the slope and aspect in parallel,and cuts into tiles by column.The Reduce phase stores tiles into HBase for query by online path analysis service.Finally,the experiment performed on Hadoop cluster makes a comparative analysis from the aspects of computation time and acceleration ratio.The results of the experiment show that the distributed generation of road network graph,the distributed calculation of distributed shortest path,and the distributed computing of slope and aspect in this paper are more efficient and stable,compared to stans-alone.Moreover,the distributed method is flexible and controllable.When the data scale increases,this method can expand the computing resources by increasing the numbers of server node,and improve the data processing speed.The shortest path calculation in this paper introduces a heuristic function to guide the search,which reduces the search coverage and avoids searching many unnecessary vertices.It takes less time to calculate than the existing Map Reduce-Dijkstra distributed method.