Research On Obtaining Geographic Information Based On Web Crawler Theory

In recent years, with the development of computer science, the arrival of the era of big data network, provides a new way of thinking for data acquisition. The traditional GIS information acquisition is mainly used for field measurements and raster map, image acquisition and need a lot of manpower and material resources. Through in-depth network data mining,attribute data and vector data can also be obtained from the network. As a search engine, web crawler provides convenience for GIS information acquisition, it can obtain reliable relative real data from a large number of network platform. The application of web crawler technology can be used to solve the problem of information acquisition in geographic information system.In this paper, it researchs on the principle of web crawler, and the application of web crawler in GIS information retrieval process. It mainly studies the web crawler how to get the attribute information and vector information,and then how to combine the information using in a real estate assessment system. And research on the key technology of web crawler software and based on the key technology of the development of information management system of WebGIS. Through the web crawler to obtain data and it uses the data into the GIS information management system.The main research contents in the paper:(1)Through the study of relevant page structure, and the analysis of static and dynamic web page,research on how to obtain relevant data on the page. Using of related technologies, and the user operation process simulation, to obtain the return data.(2) Through the analysis and research of electronic map tile structure, understand the structure of electronic map. Get the link to download the address tiles and mosaic the titles.(3) Deeply study on the process of ArcGIS automatic vectorization, the grid map obtained two values, and automatic vectorization.(4) It studies a group for JavaScript script provided by ArcGIS in this paper, learn how to map service calls, tried to obtain vector map application in practical work.(5)It studies the technology of JavaWeb, and learn how to use GIS data and spatial data into a estate assessment system.