Research And Implementation Of Multi-Cloud Models Based On The Separation Of Metadata And File

With the development of Internet and intelligent devices,the sharp increase in the amount of data promotes the cloud storage technology to flourish.The data confidentiality is a very important part of cloud storage technology,and the mainstream solution to maintain confidentiality is data encryption.However,the performance consumption of encrypting and decrypting data is inevitable.In particular,this kind performance is unacceptable under the circumstances of huge amount of data or the client without high efficiency.The thesis proposes a multi-cloud model,which is based on the technology of the separation of metadata and file data.The model is an improvement and supplement to the existing storage scheme and it is different from the encryption method.The model can guarantee the confidentiality of data and ensure the efficiency of file access.According to the model proposed above,this thesis designs and implements a system,which satisfies the data storage requirements.The thesis divides the system into several function modules,and introduces the analysis,design and implementation of each module.The function modules of the system mainly include user management,metadata management,file fragmentation and restore,file transfer,data distribution on several clouds and authorization management.The main contents of this thesis are as follows:1.After researching the file carving and file restore technology,the thesis designs and implements a strategy about file fragmentation.The strategy includes file header processing and uneven fragmentation.2.According to the requirements of the fragmentation function,the thesis designs the structure of the system metadata,which shows the hierarchical structure of files in the system and the corresponding relationship between the original file and fragments.SQLite is used for managing the metadata on the client.3.The thesis implements the cloud storage function with multiple nodes.The cloud storage function is based on the Qiniu cloud storage,Kingsoft cloud storage,Baidu cloud storage and self-built cloud storage.The callback function is designed to deal with the message from cloud storage systems.4.The thesis implements a schedule algorithm which can choose a proper cloud storage node as a file storage site.In order to achieve the load balance considering the price and speed factors,the algorithm improved the existing load balancing algorithm.5.As a complete storage system,the system also implements the user management module,data collection module,etc.These modules are mainly used for the access control to the system,the rights management of the file transfer and operations and so on.6.The thesis tests the feasibility of the fragmentation algorithm by the file carving technology,and analyzes the rationality of the multi-cloud distribution algorithm by experiments.At last,the thesis tests the functions and the performance of the system.