Design And Implementation Of Remote Storage Management Platform Among Multiple Datacenters

With the arrival of cloud computing and big data era, the storage of electronic data growing exponentially. Meanwhile, in recent years, frequent occurrence of natural disasters and man-made accidents results in damage of datacenters’ storage equipment, which makes enterprises, universities, institutions and other units make high demands of the security of data stored in datacenters.There are two widespread probleams in today’s datacenters:first, most of them have urgent need of remote backups, but building a complete remote backups-recovery system requires a lot of money and management costs, which is unlikely to be achieved for units who have limited finance such as universitys. Second, most of them have low storage space utilizations, some of which are even still using backword architectures. In these architectures, resources cannot share among different subsystems and this leads to’information islands’, different storage nodes within the datacenter having unbalanced load and lots of fragments.Based on the above, a vision is proposed in this thesis that multiple datacenters contribute their free storage space to backup data each other through the mechanism of cooperation and consultation. We expect to realize datacenter’s remote backup without the need for additional infrastructure, with the lowest capital investment and the least energy consumption. Through studying distributed storage technology, this thesis design and implement a remote storage management platform prototype system among multiple datacenters which relys on Ceph as the underlying storage environment. This thesis consists of the following parts.(1) We design and implement a remote storage management platform prototype system among multiple datacenters(RSMP-MD). This system abstracts Ceph’s running environment, and then implement monitoring and management of storage resources from system administrators and normal users two angles. On the one hand, this system can help administrators abstract, manage uniformly and monitor real-timely storage resources which distribute on different nodes, and facilitates administrators to locate and solve the system’s failure as well. On the other hand, this system can help normal users achieve sharing their storage space to the storage cluster, apply for and use storage space of the cluster. At the same time, normal users can also manage assigned storage resources’ life cycle.(2) By studying the Ceph’s underlying object location mechanism and principle, we propose a user’s data remote location model based on CRUSH algorithm. The model realize the function that user’s data will locate to remote storage nodes by customizing different rules for different users. We designed and implemented related algorithms, and then verify the algorithms can not only realize the needed function but also ensure the cluster load balancing when new storage nodes are add into the cluster.(3) We test the runction of the RSMP-MD prototype system according to the system’s requirements. The testing results show that the system meet the functional requirements of the system basically and has good scalability.