The Research And Implementation For Mass Storage System Of Numerical Products

With the application of numerical weather prediction research and extension and depth, increasing product data have been forecast to reach tens of P's massive data-level. Every day through various numerical prediction models for numerical prediction of a large number of product data, and with the forecast increase resolution, the amount of data generated forecast products are also growing exponentially, and with the continuous increase in forecast models continue to produce new data type. How to effectively organize, store, manage and distribute vast amounts of forecast products of these data so that data can be forecast products more quickly and more accurately to provide services for the research and application, become urgent problems.Number for the effective management of vast amounts of data and numerical weather prediction products to increase support for complex data types and changes, this work includes: 1, the current large-scale scientific data storage areas related to the storage network, object storage, metadata, object-oriented technology research and analysis, results show that the object storage system can be integrated NAS and SAN advantages, but also has direct access to high-speed SAN and NAS data sharing and other advantages, provide high reliability, cross-platform data sharing and secure storage system structure, while at the application level, object-oriented database is the best solution to manage large volumes of data. 2, analysis of the numerical prediction of the characteristics of product data, product data is the numerical prediction of the collection field, the field can be abstracted into a separate object. 3, presented at the lustre file system, combined with the metadata service, the design of object-oriented data management module, the module replace the original form of a document management system to manage the module. At the application level, the original numerical forecast products from a simple file management system management system into object-oriented database, and this database is designed for massive numerical prediction product design. In object-oriented database in the field of object-oriented abstraction into a multi-dimensional data - data cubes, in the process used to achieve multi-dimensional matrix, due to the flexibility of the data cube, increased support for complex data types can be upgraded and scalability, while building object-oriented data cube tree, in the meta-data services with the help of the data cube using the tree increases the efficiency of data retrieval.Next, in the lustre file system to run the entire system under experimental test results with other file systems running on the results, the results show that the object is to manage the massive storage of lustre numerical forecast products best file system selection data, in this paper, lustre and then compare the test parameter settings, run this paper to find the numerical prediction of mass storage systems, the optimal parameters of the product set, combined with lustre coupling characteristics of the optimized system design, made to small files together into big file transfer program enhance the small file I / O performance. Finally, according lustre characteristics, proposed the use of Lustre cluster file system for multiple integration of specific programs to improve the overall effective utilization of storage resources.