Research Of Fast Remote File Synchronization Technique Under Limited Network

Fast remote file synchronization techniques are concerned with efficient file synchronization problem over a slow communication link in the case where two parties have the similar files. This problem arises in many application scenarios, e.g., when updating versions of software in a distributed computing environment, synchronizing personal files between different computers and devices or data synchronization in mobile computing environment, etc. More generally, the problem is becoming increasingly common in many network based applications where files and content are widely replicated, frequently modified.Traditional fast remote file synchronization techniques include sliding-block based synchronization, fixed-size chunking based synchronization, variable-size chunking based synchronization and multi-round synchronization. These synchronization algorithms have some obvious shortcomings including low network bandwidth utilization and high implementation complexity. Based on these algorithms, this paper proposes an improved two-round fast remote file synchronization algorithm Tpsync. Through large number of synchronization experiments under different workloads, the results show that the overhead of network bandwidth reduces 12%and overall synchronization time drops 18.9% compared to a very widely open source tool in Linux distributions called Rsync. Another contribution of this paper is that it gives a detailed experimental comparison and analysis to the current synchronization algorithms. Aspects of the assessment include the network bandwidth utilization, CPU overhead and memory overhead. The results of the assessment can give an instruction to the users when they choose file synchronization algorithm. Under different application scenarios, selecting the most appropriate synchronization algorithm can obtain the best performance.This paper firstly introduces the fast remote file synchronization technology research and development status, and elaborates the basic theoretical knowledge related to this research. Then it gives out an analysis and summary to the current synchronization algorithms, and describes these algorithms' advantages and shortcomings. After that, this paper introduces the improved two-round file synchronization algorithm Tpsync, including research background, the basic principles of the algorithm and algorithm description. Then it gives the implementation details of Tpsync prototype and makes a large number of performance comparison experiments between Tpsync and Rsync. After that, it assesses and analyzes the performance differences among current several types of file synchronization algorithms. Finally, this paper gives out summary and prospect of this research.