Research On Performance Improvementmethod Of File System Metadata Service Based On Shared Log

With the rapid development of internet,big data and other applications and technologies,the number of data continues to increase.In the face of massive data,the traditional data storage system has limitations to some extent.Distributed file system has attracted wide attention because of its high fault tolerance,high capacity and scalability,which are more suitable for storage requirements in the era of large data.Distributed file systems are usually composed of metadata servers and storage servers.Existing distributed file systems use centralized and distributed metadata services or no metadata server to manage metadata.Centralized design is simple and consistent,but when the access load is too large,metadata becomes the performance bottleneck of the system.There is also a single point of failure.Distributed system can solve performance bottleneck and single point failure,but in order to ensure data consistency,it needs lock synchronization operation among nodes,which brings additional performance overhead.In addition,if there is no timely synchronization or synchronization is destroyed,it will lead to data consistency problems.Metadata-free servers eliminate performance bottlenecks,single point failures,and data consistency.However,the metadata of file system can't be managed globally.In the implementation of traversing directories,the efficiency of reading and writing small files is very low,and the client needs to undertake the function of locating files,which also increases the load of the client.In this paper,we find that shared logs have good data consistency,and there is no single I/O bottleneck and single point of failure.Therefore,in order to solve the problem of distributed file system metadata service at this stage,this paper proposes a metadata service model based on shared log.This model solves the performance bottleneck and data consistency problems by using the characteristics of shared log and manages the global metadata information in a more reasonable way.By comparing the experiment with GlusterFS metadata information management,it is found that the model has good metadata operation performance under the same data consistency as GlusterFS.