Research On Data Quality Management System Of CAN Bus Based On Metadata

In the era of large data,the quality of data has been paid more and more attention with the rapid expansion of information.Especially with the development of mobile Internet,as an important information technology,Internet of things has generated a lot of data and information through RFID,sensors and so on.As a prerequisite for effective processing and analysis of large data,the quality is the basis of ensuring the value and gain of the data.The data of bus CAN are collected through the integrated application platform of the Beijing bus group CAN data.CAN bus,Controller Area Network,is also known as CAN-bus.It is the new generation of vehicle bus developed by Bosch company,which is used to collect data of all core components of vehicle.Now the Beijing bus group has 5000 new cars began to realize the real time data acquisition of engine,vehicle electrical door device.In general,the quality of the bus CAN bus data is not high,and there are some problems such as incomplete,inconsistent and repeated records in the data.The reasons for these problems are manifold.One aspect is that the application of Internet of things technology on public transport vehicles is just in its infancy.The sensor devices of public transport cannot collect all the data correctly.On the other hand,the lack of management of the collected data is also an important factor in the quality of data.Based on the data of bus CAN bus,this paper makes an in-depth analysis and research on the problems in the data quality management of CAN bus,proposing the method of applying metadata to the application of CAN bus data quality management.Based on the business logic of data,the metadata model of CAN bus data is designed and constructed,and a set of constraint rules for data quality management is designed.Through the effective combination of metadata management and data quality management technology,it aims to break the technical barriers between business personnel and data management through the system.Let business personnel participate in the process of data quality management,and improve the efficiency of data quality management.The system mainly realizes the functions of three aspects:rule management,solution to the problem analysis about the quality of the data.Rule management can allow system users to customize the quality constraint rules of data to meet the needs of data management.The data processing function is to complete the cleaning of the data based on the defined data quality rules and provides data support for the analysis and prediction of the health condition of public transport vehicles.The function of analysis provides information about the analysis of single table data,single index trend,problem location,and so on.It is convenient for users to quickly find the index of the problem,which lays the foundation for data management personnel to make data acquisition strategy.