Research And Application Of Large Concurrency Problem Based On FastDFS

With the development of the times,the amount of data information has shown explosive growth,the simple distributed file system has been unable to meet the requirements of massive data processing.At present,even though many scholars have done research on the problem of massive small file processing,there are still a lot of deficiencies in the existing research on dealing with large concurrency problems.Therefore,this paper focuses on how to further improve the FastDFS file processing capabilities,in order to better solve the problem of massive small files stored in the face of large concurrency problems.The main work of this paper is as follows:(1)A brief overview of the research background and significance of the topic,and focus on the analysis of the research status at home and abroad of the distributed file system,No SQL,server cluster.They do a good job for the next study of the theoretical knowledge of this paper.(2)First of all,through the in-depth study and analysis of FastDFS,Redis database,Nginx cluster,summing up their respective advantages and disadvantages.Secondly,in order to solve the problem of large concurrency problems better,this paper puts forward the application of FastDFS,Redis database and Nginx server cluster in the actual project,combining their advantages and making up for their deficiencies.Finally,based on the experimental data,it is verified that the time that the combined structure of FastDFS+Redis+Nginx cluster takes to upload or download files will be less than the time that the original FastDFS+Redis architecture takes.(3)By introducing the Piece mechanism in the FastDFS file storage structure,a large amount of idle space is reused in the process of merging and storing small files.It not only improves the utilization of file storage space,but also optimizes the performance of theFastDFS furtherly.At the same time,through the analysis and comparison of simulation results and experimental data,it is verified that,when dealing with large concurrent requests,comparing to the original FastDFS,the new FastDFS which joined the Piece mechanism not only ensures higher stability of the system and higher document processing capability,but also can improve utilization of file storage space.