Research On The Data Layout Of Strongly Associated Marine Monitoring Data Storage

With the establishment of a full-scale marine three-dimensional monitoring network and the expansion of the digital ocean demonstration effect,marine surveillance data exploded.Marine monitoring data is a big data that features strong data dependency,interdisciplinarity,complex acquisition approaches and diversification of species.However,how to quickly and effectively find the required data from all the monitoring big data is one of the critical needs of ocean researchers for monitoring data management.In order to better organize and manage the large data of ocean monitoring,the layout strategy of data came into being,and gradually became one of the key problems in restricting the effective management and application of data.The characteristics of marine monitoring data,which are massive,strong and real-time updates,have created huge barriers to the rapid storage and querying of data on ocean applications,like polar online investigation,strange tide disaster inversion and ocean aided decision.Moreover,these characteristics also bring the following questions: 1)How to make a static layout strategy of closely linked data to meet the needs of high-performance computing and high-frequency query on the marine applications.2)How to dynamically update the real-time,fast-growing ocean data to realize the goal of reducing the cost of training time and response time of the user's access to the data.3)How to use a reasonable replica layout strategy to ensure the reliability and security of data and improve the response speed of which to some extent.Therefore,this paper proposes static data layout strategy,dynamic data layout strategy and data replica layout strategy.Experimental results show that our strategies improve the utilization of data and the efficiency of ocean application query.This paper takes marine data featuring strong data correlation,geographical area,real-time monitoring and so on as the object of study,and seek for the efficient distribution of data.The main works are listed as follow:(1)The research status and key problems of marine monitoring data management are summarized.The influence of marine monitoring data characteristics on data layout is pointed out based on the analysis of the mechanism structure and function of data layout.Moreover,the definition and symbolic representation of marine monitoring data are put forward,which lays a good foundation for the application of data layout in marine data management.(2)The static data layout strategy based on data dependency is proposed based on the various features of marine monitoring data that is massive,heterogeneous,and strong connected.Firstly,the cloud computing storage model which is suitable for large ocean monitoring data is constructed by analyzing the characteristics of the data.Then,considering the dependency among monitoring tasks,monitoring points and monitoring data,the strategy does the marine data layout including the dependency of monitoring points,the dependency of monitoring data and the overall dependency of monitoring data.Finally,the marine monitoring data are effectively distributed from the above three perspectives.This strategy not only satisfies the storage management requirements of large ocean data,but also stores the data with high correlation degree in the same data center,which has achieved better layout effect.(3)The dynamic data layout strategy based on incremental learning strategy is proposed to solve the problems including dynamic real-time updating and variability of marine monitoring data.According to the formal definition of the value of marine monitoring data,the data storage area is divided into active and non-active areas.At the same time,the incremental learning method is used to learn the incremental part of the data based on the knowledge that has been obtained,rather than re-modeling the overall database after the data increment.Such strategy can effectively compress the size of the sample set and discarding useless samples.The experimental results show that the data layout is effective.(4)The data replica layout strategy based on multiple attribute optimization is proposed to improve the system storage space utilization and the protection of data.Firstly,the new strategy adds the definition of the data copy heat to the dynamic layout strategy former proposed.Then,the updating method of the data copy is established according to the value of the data heat.Meanwhile,the multi-attribute optimization method is used to analyze the key attributes affecting the storage nodes according to influencing factors of the storage nodes.Finally,the data replicas can be found with the nodes which are equipped with the best attributes.The experimental results show that this strategy effectively manages and distributes data replicas.