Web Page Data Mining For Ecological Disaster Elements

The information of ecological disasters on the Internet is extremely rich,mainly including information on five major disaster elements: meteorology,floods and droughts,earthquakes,geology,and oceans.However,because the data attributes such as ecological disaster data storage and data definition of each platform are limited to the internal communication of the platform,effective data sharing cannot be achieved,and the data cannot be connected,which leads to the "information isolation" of the ecological disaster data,which prevents Analysis and mining of its comprehensive information.In response to this phenomenon,this article takes ecological disaster elements as the research content,obtains some public ecological disaster element data from web pages,builds and visualizes the ecological disaster element data sharing database,and provides data sources for ecological disaster research.There are three main problems involved: firstly,the ecological disaster elements in the web page are from a wide range of sources and complex types,so how to effectively obtain the ecological disaster element data from the web page and extract the required data attributes;secondly,the ecological disaster elements data obtained from web pages are multi-source and heterogeneous,which is difficult to be managed uniformly,resulting in the difficulty of data sharing,so how to store and manage the multi-source heterogeneous ecological disaster elements data in a unified manner;thirdly,part of the ecological disaster element data belongs to spatial data with obvious spatial characteristics,so how to visualize the spatial data in the elements of ecological disasters.In response to the above problems,with the support of the project "A new monitoring and early warning method and system for collapsed hills based on multi-source data integration of sky and ground",this paper studies how to use data mining technology to obtain public ecological disaster element data from web pages and build ecological Disaster data sharing library and visualization of spatial data in ecological disaster elements.The main research work of this paper is as follows.1.Aiming at the problem of how to effectively obtain the ecological disaster element data from web pages and extract the required data attributes,this paper studies the ecological disaster element discovery method based on web crawlers,and obtain a part of the original data from the five major ecological disaster elements of meteorology,floods and droughts,earthquakes,geology,and oceans disclosed on the internet,and use data mining technology to extract and clean the data.2.Aiming at the problem of how to uniformly store and manage the multi-source heterogeneous ecological disaster element data obtained from web pages,the distributed file system-based method is currently used to store the heterogeneous data in the web pages,but the problem of data sharing is difficult due to the tight coupling between the file structure and the application.For this reason,this paper studied the method based on database technology,using XML to structure the heterogeneous ecological disaster element data and store it in the My SQL database,completed the construction of the ecological disaster data sharing database,and solved the problem that the multi-source heterogeneous ecological disaster element data is difficult to be uniformly managed.At the same time,based on the constructed data sharing library,the Kriging interpolation algorithm is used to predict and analyze the temperature attributes in the meteorological data.3.Aiming at the problem of how to visualize the spatial data in the ecological disaster elements,commercial GIS software is currently used to visualize the spatial data,but the maintenance cost is high,and the GIS data is not completely convertible and shareable.For this reason,this paper studies the use of the open source project MapServer to visualize spatial data in ecological disaster elements,which can convert and share spatial data,and solve the problem of spatial data visualization in ecological disaster elements.Based on the above research work,the paper collected some public ecological disaster element data from web pages,and constructed an ecological disaster data sharing library.Based on the data sharing library,Kriging interpolation method was used to predict and analyze the temperature attributes in the meteorological data,and using MapServer to visualize the spatial data in the elements of ecological disasters.On this basis,the three modules of ecological disaster element data collection,data sharing library construction and spatial data visualization in the project "A new monitoring and early warning method and system for collapsed hills based on multi-source data integration of sky and ground" have been completed.These research works provide basic data sources for follow-up research on ecological disasters.