Study Of Marine Data Quality Inspection Model Based On AQL

Data quality guarantees the marine data mining and application. Due to the instability of ocean itself and limitations of human cognitive skills in marine data acquisition, processing and transmission, the abstraction and expression of realistic ocean is not accurate for all use cases, while the incorrect data may directly affect the quality inspection result and the application of marine data. How to design an optimal quality inspection plan rapidly and control the marine data quality efficiently, grows more and more significant.Concerning of characteristics of marine data like large volume and spatial connection, quality inspection plan for marine data cannot be exactly the same as the one for general industrial products, which need to take both random batch lots and specific applications into consideration. Based on existing researches on spatial data quality inspection models, and consideration of the characteristics of marine data, like:massive lot, multi-source, multi-class, an optimal quality inspection plan for marine data is proposed. In comparison of existing spatial data quality standard of GB/T 2828.1(2012) and ISO/TC 211, the validation of proposed inspection plan for marine data is proved as followed. The contents of research are as follow:Firstly, this paper describes the background of the research, raises the potential problems, which the dirty data has brought on in the era of big data, and then summaries the researches on relevant fields like, quality inspection model of spatial data, domestic and international standards and protocols, and the spatial data quality elements. Then it is noted that there’s still a few problems in current theories of quality inspection plan and assessment methods of marine data.Secondly, aiming at the demand-driven marine data with features of multi-dimension, multi-source, multi-scale, an optimal quality inspection plan for marine data is proposed. Based on the spatial data-sampling scheme, a key parameter of acceptance quality level, i.e. AQL, is introduced to balance the needs between data producer and data consumer. Combining with the hyper-geometric distribution scheme, the relation of lot size and the sample size of marine data is built, and then experiments are illustrated to prove the validation of proposed sampling scheme.Considering about the thematic feature and applicability of marine data under the theory of rough set, the research evaluates the quality of marine thematic datasets from the aspects of quality elements in use. By setting the quality elements rough sets of marine data, and discovering the reference dependencies among spatial data quality elements, the attribute sets are reduced and weighted, and based on which, a scheme of marine data quality evaluation with rough decision is developed.Finally, combining the scheme of marine data quality inspection and data evaluation, a marine data quality control model based on AQL is proposed. By applying the model to the actual inspection of marine forecasting data, the flexibility and efficiency of the proposed model are proved to be valid.