Resource Management In Blue Whale Distributed File System

The recent trends in high-performance computing show that the supercomputer systems are developing quickly, and more than fifty percent of the computer systems listed in the latest the Top500 supercomputers are now labeled as clusters.In the meanwhile, storage subsystem is playing a more and more important role in the whole computer systems. The peripheral storage systems are developing quickly, but they are still the slowest part in the whole system. So they are now the bottlenecks which heavily influenced the supercomputer systems. As more and more information materials are digitalized, more and more storage spaces are needed to store them. Currently, network storage system is the most efficient and direct way to solve these problems.The "Blue Whale" massive network storage system project supported by the National High Technology Research and Development Program (863 program) aims to build a high performance storage subsystem for cluster environment with great scalability. Based on the high speed computer interconnect network, "Blue Whale" stores data on many storage nodes, utilizes the potentials of all the storage nodes, transfers data concurrently between them. The "Blue Whale" distributed File System (BWFS) is the core component of the "Blue Whale" massive network storage system, and provides remote file system service to the applications.With the design, implementation and optimization of the Blue Whale distributed File System (BWFS), this thesis deeply explores the principle model of resource management. The research includes the model itself, the techniques used to design and implement the model and many experiments to prove the model. The detailed results of the research are as the followings:1. The Distributed Layered Resource Management model of BWFS. The DLRM model partitions the whole system into many distributed modules, layered them to simplify the design and implementation. The model adopts the out-of-band data transfer path, and enables load balancing and on-line migration. It is the basis of the good performance and scalability of BWFS.2. Efficient space management. All the resources in BWFS are virtualized into Global Logical Address, and then partitioned into Resource Groups. The space managements are performed concurrently against the RG The techniques of dynamic data/inode allocation strategy and dynamic bitmap with summary information bring efficient space management to BWFS.3. Dynamic Binding technology. All the active metadata in BWFS may be distributed dynamically among the metadata server cluster. So it enables load...