Research On Key Technology For Data Resource Catalogue Of Atmospheric Environment Monitoring Data Lake

Atmospheric environmental monitoring data provide an important reference for the comprehensive management of ambient air quality.It has the characteristics of extensive sources,complex structure,large amounts of data,and strong spatiotemporal attributes.The current atmospheric environment monitoring data resource management is scattered and lacks a unified data description,which brings great difficulties to the comprehensive processing and analysis of atmospheric environment monitoring data.The proposal of the concept of data lake provides a new solution for the management of atmospheric environment monitoring data.At the same time,it provides meaningful semantic descriptions for data resources through the directory of atmospheric environment monitoring data lake data resources,and supports semantic retrieval,which can comprehensively and accurately obtain users’ information.Provide users with corresponding data services quickly and efficiently.This thesis analyzes and summarizes the domestic and foreign research status of data lake architecture,semantic data resource catalog,and semantic retrieval model,as well as the development of related technologies,and optimizes and improves the existing problems.The main research contents are as follows:1.The atmospheric environment monitoring data lake architecture and the atmospheric environment monitoring data resource directory architecture are constructed.Apply the idea of semantic technology management data lake to the atmospheric environment monitoring data lake architecture,add a semantic layer to the data lake architecture,this semantic layer runs through the entire life cycle of the atmospheric environment monitoring data,through the atmospheric environment monitoring data resource directory of the semantic layer Architecture to add semantic knowledge and contextual association to atmospheric environment monitoring data.2.A metadata model for atmospheric environment monitoring is proposed,and a knowledge graph of atmospheric environment monitoring data is constructed.It provides a unified metadata model for multi-source heterogeneous atmospheric environment monitoring data,and realizes metadata extraction and fusion of data from different data sources.Based on the atmospheric environment monitoring metadata,a knowledge graph of atmospheric environment monitoring data is established and stored in the graph database to provide support for semantic retrieval of atmospheric environment monitoring data.3.A semantic retrieval model for atmospheric environment monitoring data based on knowledge graph is proposed.Aiming at the problem that keyword retrieval cannot fully obtain the user’s retrieval needs,semantic reasoning is performed on user needs by combining the knowledge graph of atmospheric environment monitoring data.The comparative experiment verifies the feasibility and correctness of the method.4.Design and implement the atmospheric environment monitoring data lake management system.Based on the theory of software engineering,combined with the atmospheric environment monitoring data lake architecture,atmospheric environment monitoring data resource catalog,atmospheric environment monitoring data knowledge graph,and atmospheric environment monitoring data semantic search model,the atmospheric environment monitoring data lake management system is designed and developed.It provides efficient management and comprehensive retrieval for atmospheric environment monitoring data,and at the same time verifies the main research content of this thesis.