Research And Design Of The Cloud Storage System That Based On Fastdfs

Along with the rapid development of information technology, a large amount of data needs to be stored, in order to solve this problem, companies often need to purchase massive storage devices, which not only requires high costs, but also has difficulties in managing and maintaining the massive data. This solution has all sorts of disadvantages. Cloud storage is a technology with the development of cloud computing, it focuses attention on the storage and management of massive data, with the characteristics of elastic expansion, convenient for massive data management and low cost, is considered to be a good solution for data storage in the future. It uses the cluster and distributed file system technology, connects different types of devices together in the network, provides data storage and access as a whole for the outside users can access the resources which stored in the cloud through the internet at any time and any place.FastDFS is an open-source distributed file system, which researched and developed by Chinese. It works well on UNIX-like systems. FastDFS is lightweight, high concurrent access, load balancing, scalability, synchronize files automatically, good performance in store small file, in comparison with other distributed file system. Based on the existing cloud platform of the laboratory, this thesis explores a cloud storage mode which based on FastDFSFirstly, this thesis studies the system architecture, operation and synchronization mechanisms of FastDFS. Secondly, analyses the functional requirements and performance requirements of the system. Design the overall architecture of cloud storage system which based on FastDFS by referring to the classic model of cloud storage architecture. From top to bottom is user access layer, application interfaces layer, management layer and data storage layer. In the user access layer, adopting B/S mode designs user interface by using open-source framework Bootstrap. Load balancing is studied while users access the system. Design REST API interface in application interface layer. In management layer, realize the functional requirements which proposed in the requirement analysis. Implement the operation of file upload, download, delete as well as the operation of the folder for general users. Manage users and monitor cluster for administrator. Meanwhile, study the original cluster monitoring principle of FastDFS, which Add support for monitor CPU, memory usage, Disk read and write as well as network I/O. This thesis also researches on the scheduling algorithm of FastDFS, analysis of its defects and weaknesses exist, then, presents a dynamic load balancing algorithms. Design the interface server by using of epoll and thread pool mode between the management layer and the Fast storage cluster. A large number of concurrent requests could response in short time. Meet the performance requirements of the system. Redefine the API interfaces of FastDFS and encapsulate the new API in the task processing of thread pool. Finally, test the function and performance of the system. Prove that the cloud storage system which based on FastDFS meets the prospective design requirements.