Research On The Key Technologies Of High Performance Computing WebGIS Model

The outline of land and resources medium and long term program for scientific and technological development (2006-2020) promulgated by the ministry of land and resources made geographic information system as a explicit priority themes of earth space observation technology development area, put forward to promote the work of land and resources informatization into a main task, develop the core of modern information technology including the geographic information systems, aeronautics and astronautics remote sensing technology, global position system, improve the ability for real-time, accurate, and economic access to all kinds of space information technology and distributed spatial information integration of the description, organization, storage, processing and publishing, strive to get some breakthrough from the research on several key technology. The outline put forward geographic information system and its key technology research to unprecedented heights on the national macro decision-making level.With the rapid development of social economy, the traditional self-sufficient closed geographic information resources model can’t meet the requirements of social and economic development, geographic information system network already constitute a hot spot in today’s society. Network geographic information system (WebGIS) as an important part of geographic information system, realize network publishing and management of geospatial information through the internet technology, allow users to access, retrieve, edit the geographical data and do online analysis through the network, realize the remote interoperability and sharing of spatial data. As the "digital city","digital China geo-spatial framework" and the construction of surveying and mapping informatization system, WebGIS has been the primary technology for realizing the sharing of geographic information and providing online services.Along with the application and development of the Internet technology, application of GIS has permeated all fields of the national economy by WebGIS, and become a popular tool for information. Along with the development and progress of the earth observation system, geographic space information people can obtained present explosive growth, more industry and department’s space data quantity reach at TB even PB level, this has brought huge challenge to the development of WebGIS. The WebGIS realized preliminary space information service based on network, but due to performance bottleneck in the system, we can’t solve a variety of problems that the massive spatial information services faced fundamentally, this leading to the present many industries and departments have rich spatial data cannot release space information on network and provide to public for use, this constrained these industries and departments in the space information development. Therefore, we need to build a high performance computing WebGIS model(HPCWM), and research all the important problems in the model to solve the problems spatial information sharing service faced in all fields of the economic development of our country.A WebGIS model capable of high performance computing must have the following features:1. The WebGIS model requires high performance computing power, it’s important to support huge amounts of spatial data, and it’s also a premise condition. Due to the quantity of the spatial data on a single server node for management is limited, so we must use multiple spatial database server nodes in the network for distributed storage and management of large-scale spatial data. In practice, we can classified data according to the subject, years, purpose, etc., deploy huge amounts of spatial data on all nodes separately. WebGIS model must have a general and effective node metadata publishing, retrieving, scheduling and synchronization mechanism to implement distributed organization and management for huge amount of spatial data.2. It must have task parallel processing capabilities for large user concurrent access, the task scheduling mechanism for concurrent requests in WebGIS system is the key factor that influence the performance of WebGIS model, the quality of task scheduling algorithm directly affect the response ability of WebGIS service. High-performance computing ability of WebGIS model not only reflected in the parallel computing ability, must also consider the rationality and fairness of the task allocation strategy, and therefore it must have a good task allocation scheduling algorithm, for scheduling concurrent requests task reasonably and efficiently.3. The WebGIS model should have the ability to balance the load on the server, using the GIS server cluster is a simple, quick and effective solution, whether the load is balancing on each server nodes in the cluster is an important index for measuring the cluster performance. A WebGIS model which has high performance computing ability, must be able to balance the load on the server effectively for large users concurrent access.4. It must have the ability to maximize server parallel computing resources, computer hardware upgrade quickly, generally upgrade once a year, at present, the main popular server’s CPU has8,16or more cores, the parallel computation ability of the server enhances unceasingly, it laid the foundation of hardware for the concurrent performance ascension of the WebGIS service. If WebGIS model must have high performance computing ability, it must carry on the parallel computing optimization on multi-core servers, so that it can take full advantage of parallel computing resources of the server, in order to shorten the task average response time.5. It must have intelligent decomposition capability for large spatial data computing tasks, The WebGIS model which has high performance computing ability should have a kind of task decomposition mechanism for intelligent analysis and decomposition of large spatial data calculation tasks, decompose complex GIS functions which need a lot of calculation in WebGIS server cluster, schedule all server nodes processing the same large computing tasks at the same time concurrently, shorten the service response time.This research is is an important part of the "12th Five-Year Plan" national science and technology support program subject "geographic space information toolset service platform development (2011BAH06B04)", the main purpose is to analysis existing WebGIS model, research a high performance computing WebGIS model oriented multi-core cluster environment, the massive spatial data real-time efficient release, task scheduling of cluster, large space computing tasks intelligent decomposition, multi-core parallel computing optimization as the key research, and ultimately improve the concurrent processing performance of the existing WebGIS computing model, solve the problems that server resources low utilization rate, the load imbalance, difficult to support large-scale spatial data released by network.This paper finish the following main research work:1. This paper analyzed a series of technical bottlenecks and challenges that WebGIS is facing now, put forward the research necessity based on high performance computing WebGIS model (HPCWM), both present research situation in terms of high performance computing WebGIS model at home and abroad are analyzed, and analyzes the feasibility of research on HPCWM from four aspects:the network information technology, the WebGIS technology, hardware and software development technology.2. This paper studied the evolution process of the WebGIS software architecture, proposed the problems of traditional WebGIS system, service oriented WebGIS system, WebGIS architecture based on independent GIS server and WebGIS system based on server farms, and analyzes inevitability of the HPCWM architecture, presents a high performance computing WebGIS model framework based on demand for high-performance, and put forward the three key problems which HPCWM needs to solve.3. Summarizes the existing network distribution mode of spatial data and analyzes the defects of various technologies, proposes and studies the organization and scheduling strategy of massive spatial data in HPCWM and expounds its implementation mechanism in detail, distributed deployment of massive spatial data and how to realize the real-time scheduling, etc were studied, and studied how to use the cache to optimize performance of HPCWM, proposed the three layers cache system, to optimize concurrent access for massive spatial data under a large amount of users.4. In order to handle concurrent access requests for a large amount of users, the high performance of WebGIS is mainly reflected in whether it can distribute the concurrent access request effectively and reasonably, make full use of the parallel computing resources of server, this article proposes and studies the task allocation method based on response priority scheduling, calculate task priority according to the waiting time and response time of tasks, schedule tasks according to the priority order, and then propose concurrent task scheduling optimization technology based on multi-core according to the characteristics of the multi-core servers, the purpose is to take full advantage of the resources of servers which have multi-core processors, improve server’s concurrent processing capability, scheduling technique for concurrent tasks was tested and analyzed in this paper, the test results show that the parallel task scheduling method proposed by this paper can improve the concurrent processing capability of WebGIS effectively, it is one of the key technologies of high performance computing WebGIS model.5. According to the process time consuming problem of massive spatial data, this paper propose smart decomposition and load balance method of HPCWM for large-scale spatial calculation, its purpose is to decompose large-scale spatial calculation tasks fairly by a kind of task decomposition algorithm, distribute it to multiple server nodes fairly for parallel processing, give full play to the advantages of server cluster. The task decomposition technique for visualization tasks of massive vector data is mainly studied, using the content grid technology to analyze and recognize real distribution of vector data automatically by the content grid of vector data, through grid scans and aggregation, realize fair division of visualization range and distribute it to multiple server nodes for parallel processing, shorten the average response time of the service manyfold.6. In order to verify the feasibility of the HPCWM proposed by this paper, based on the key technologies of HPCWM model, the author research and develop a WebGIS software which has high performance processing capabilities-IGServer, system architecture, the function of the software are introduced, and describe the system design strategy for high performance computing, and verify the validity and practicability of the research contents of this paper.This research work has gotten the following main results:1.Solve the problem how to release the massive spatial data on network efficientlyIn order to make industries and departments release massive spatial data effectively by network, in this paper, the massive spatial data real-time organization and scheduling strategy has been deeply research, focus on the massive spatial data real-time dynamic generation and rapid display technology, solve the time-consuming problem in traditional WebGIS preliminary cutting tiles and the problem of low efficiency for massive spatial data network show.2.Solve the load imbalance problem in traditional WebGIS modelIn order to improve the WebGIS system to support for large amount of concurrent access, solve the existing disadvantages of the WebGIS system in task allocation and scheduling, and enhance the use experience of terminal user, in this paper, the WebGIS task scheduling mechanism and load balance technology has been deeply studied and resolve load imbalance problem of the server nodes in the clusters.3.Realize the decomposition and scheduling of large-scale spactial data computing taskIn order to adapt to the large scale spatial data computing and to solve the WebGIS is not competent for large space data computing tasks, in this paper, the characteristic of large-scale space data computing tasks has been thorough analysised and researched, focus on the automatic identification technology of massive spatial data visualization content distribution, solve the fair decomposition problem of large scale spatial data computing tasks. At the same time the cluster service node parallel computing technology both has been researched, scheduling all service node for large space data computing tasks in parallel processing, solve weak handling ability problem of the traditional WebGIS for large space computing tasks. 4. Improve the parallel processing ability of the WebGIS modelIn order to make the WebGIS adapt to the development of current multi-core computing technology, to solve the problem that present WebGIS service can not make full use of and play parallel computing ability of multi-core servers, this paper realize parallel computing optimization for WebGIS service engine, make full use of the multi-core computing resources of CPU, improve the parallel processing ability of single service node, so as to improve the response ability of WebGIS cluster for large amount of users concurrent access.To sum up, along with the expanding of application fields of WebGIS and go deep into needs, the WebGIS performance requirements are mainly concentrated in two aspects, one is concurrent access request of a large amount of users, the second is the performance for network publishing of large amount of data. Due to the performance bottlenecks existing in the traditional WebGIS, we cannot solve various problems for the huge amounts of spatial information sharing services fundamentally, it lead to industry and department who has a lot of rich spatial data cannot publish the spatial information by network and supply to departments or public, and make the industry sector severely restricted in the development of the spatial information, directly influence the spatial information development in our country. The high performance computing WebGIS model(HPCWM) built in this paper has carried on the research on several important problems existing in the WebGIS, in order to satisfy network oriented spatial information sharing services needs of all departments and industries in the process of economic development in our country, solve performance bottleneck problems in the process of application and development of geographic information system, it has important scientific significance and practical value.The research on key technologies of HPCWM involves the geographical distribution of many entities, therefore, in order to solve the problem of WebGIS spatial information sharing and performance bottlenecks completely and efficiently, it need to rely on a broad areas of technology. This article will combine geographic information system technology and computer software technology, based on the high performance computing WebGIS model framework, organization and scheduling of huge amounts of spatial data, load balancing of cluster, large-scale spatial computing task decomposition and task parallel processing and other key technologies, build a high performance computing WebGIS model. HPCWM model can organized the computer in different geographical location in the Internet into a "virtual super computer", implement network publishing and sharing of huge amounts of spatial information resources efficiently for each industries and departments across the board.The research of this paper will greatly improve the performance and efficiency of massive spatial data publishing in WebGIS. It can provide the software supporting for building the spatial information sharing platform in different fields and has important academic value and application prospect in theoretical research and practical applications.