Research On Small File Access Optimization In Non-CDN Environment

Nowadays,with the rapid increment in the amount of data,data storage scalability for enterprises are requested continuously.In the new requirements of storage,the traditional data storage methods have been unable to meet the new generation of technical requirements.The distributed storage system with strong scalability,low price characteristics is gradually becoming the choice of many enterprises.However,the common distributed storage products on the market today,either for large object storage designs,or for Internet-based products,are not applicable in an enterprise-wide environment without CDN.On the basis of weedfs,this paper takes distributed storage technology as the core and explores the application of small file distribution and storage in non-CDN environment based on the status quo of small file storage needs.The following research is carried out:First,review the development of the storage system,the current characteristics of the various distributed storage technology.Second,we added a caching system for weedfs,and improved the caching algorithm to adapt to various scenarios for the shortcomings of traditional caching algorithms.Third,combined with the traditional cache algorithm and machine learning technology,to further improve the performance of a real cache system.In this paper,weedfs has been improved via cache system,while the machine learning and traditional caching technology are combined.A new choice is provided for enterprises processing massive small file.the study In this paper would provide a little reference to other industries in choosing the storage system and improving cache algorithm.