Research Of Anomaly Detection On Argo Profile Floats

At present, the Argo profile floats are the only way to provide real-time global views of the upper ocean. The Argo profile floats’ observation data that reflect the three-dimensional distribution of temperature and salinity of the sea and offer a data base for the study of ocean circulation, global climate change, ocean analysis and forecasting system are extremely important significance and scientific interest. Therefore, this paper uses the profile data of Argo as an entry point to discover the information security issues of the ocean big data. In order to solve the anomalies in the profile which caused by uncertainties factors such as environment and equipment, this paper studies how to combine with the characteristics of Argo profile floats’ data such as an amount of data, regionality, non-leaner and discrete distributions well in the anomalies detection of Argo profile floats’ data. This study will provide the theoretical basis and technological means for increasing the accuracy and reliability of Argo profile floats’ data.This paper focus on the two stages of anomaly detection, the main achievements and creative result are as follows:In the training phase, in view of the profile data’s problems that the file format is complex and the data is so large, a new information fusion algorithm(information fusion algorithm for Argo profile base on MapReduce and Principal Curves, AMPC) is proposed. This algorithm utilizes MapReduce to improve the efficiency of execution effectively. It uses latitude and longitude cell as analysis object to enhance the relevance between the profiles and highlights regionality of the profile. In addition, it based on the Kegl’s principal curve theory; the principal profile is generated by the method of adding a fitting profile point constantly. Eventually, achieves the objective of reduces data storage in the next phase and provides reference for point anomalies, contextual anomalies and behavioral anomalies. In the detection phase, learn methods’ advantages of anomaly detection from the anomaly detection method based on three sigma rule and prediction model, an improved method which based on self-adaptive threshold is designed that combining advantages of two method: segment three sigma rule and k-nearest neighbor profile with principal profile prediction model. The method is established on the principal profile generated in the training phase and each profile point calculates own threshold dynamically, it not only considers the degree that current profile point deviates from the principal profile, but also considers the effects of the change trend of profiles with depth. The approach further increases the performance and effect of the anomaly detection about Argo profile floats’ data.Through experiments using the global Argo profile floats’ data, the result show that the anomaly detection method which is introduced in this paper combining historical profiles and the characteristics effectively, avoids the respectively of detects by static threshold efficiently and improves the accuracy and effect of detection.