Research On The Integration And Cleaning Of Heterogeneous Data Based Of Bridge Preventive Maintenance

As a special architectural model,bridges have their own special health monitoring methods.Bridge workers will calculate observation points related to bridge health based on the structure of the bridge itself,obtain relevant data through advanced measurement equipment,and master the current health status of the bridge based on these data.However,in the bridge data collection process,there are a large amount of data,data repetition and redundancy,a large number of data unit types,and a long data collection cycle.These phenomena will lead to obvious data deviation problems in the process of preventive maintenance analysis of bridges,affecting the final results and causing unnecessary economic losses.The bridge monitoring data used in this paper is heterogeneous data,that is,the data exist in a variety of forms and sources,the structure is also different,but there is a close relationship between the data,each database system itself before joining the heterogeneous database system It already exists,has its own DBMS,and has its own autonomy.This thesis starts from the heterogeneity of data,introduces and analyzes the solution of heterogeneous data,and builds an integration framework based on SpringMVC + MyBatis and Json,aiming at the particularity of the bridge monitoring data.The integrated data does not solve the dirty data phenomenon.Therefore,in order to make the data cleaning more clean,we conduct research on the data quality problems in the integration process and discuss how to formulate cleaning strategies for bridge preventive maintenance data.For the data of bridge preventive maintenance after integration,this thesis selects the em data management tool in the Oracle database to filter out the data quality problems,including missing data,duplicate values,and abnormal values.According to the specific situation of these problem data,the optimization algorithm is designed based on the traditional data cleaning algorithm,which solves the data problems encountered in the data cleaning process.From the result point of view,the integration and cleaning methods of this paper can meet the requirements for integration and cleaning of bridge preventive maintenance heterogeneous data.