Improvement And Implementation Of Data Synchronization Based On Cloud Storage

With the increasingly producing speed of data and the rapid development of cloud computing,there is also a highly growing demand on cloud computing platforms.Data update is more and more frequent,people's demand for data synchronization is also increasing.In the cloud storage environment,network bandwidth is still scarce resources,efficient data synchronization algorithm research is necessary,it can reducing network bandwidth consumption.A good synchronization algorithm is that it can overcome the bottleneck of network transmissin as much as possible in the synchronization process.It can achieve the goals that synchronizes the remote data faster and transfers less amount of data to reduce network bandwidth consumption.So,it can achieve the more efficient data synchronization.Remote file synchronization technology can make the data in the remote host to keep the same,this technology has been widely used for remote file backup and update.When the file is updated,you can synchronize the file to the backup center to ensure the validity and security of the data.However,most of the files are updated when the amount of data is very small,if re-backup,the need for a large amount of network traffic,so there is a file differential synchronization algorithm,the file will be updated to synchronize the difference between the backup center.It can effectively reduce the data transmission and save the network traffic.At present,the algorithms base on the different file synchronization are Rsync and RDC.Rsync,which is currently the most widely used remote file synchronization algorithm,the rsync algorithm plays a very important role in the remote file synchronization,it is because of open source,difference transmission,network load is low.So that,it can apply in many fields.According to file synchronization requirements of the cloud storage platform,and on the purpose of reducing network traffic and improving the efficiency of synchronization data,the emphasis of this thesis is to study rsync algorithm and its application in remote file synchronization.The main works of this thesis are as follows:1.The thesis discusses the classical Rsync algorithm which is using in the remote file synchronization,the thesis introduces the basic principle of Rsync algorithm,it analyzes the performance of the algorithm and its application in the remote file synchronization;it also introduces the multi-thread technology and its application in the data synchronization mechanism.2.After completing the research and implementations of data synchronization mechanism,thet improvements are carried out in two aspects.On the one hand,using a parallel way to divide the big data into many sections,and using the multi-thread technology to calculate every piece of the fingerprints,achieve the effect of calculating the fingerprints through multi thread technology,thereby reducing the generation time of fingerprint and improve the efficiency of synchronization system.On the other hand,for the problem of that the delta table in the Rsync algorithm is too big,the thesis uses the way that we compress the delta table before transferring it.In this way,it can reduce the data which is transferred online.So that,it can complete the more efficient and fast data synchronization.3.Make tests on the optimized data synchronization mechanism,and make a com parative analysis of performance between the data synchronization mechanism and the optimized one,analysis of test results,make a summary on the data synchronization mechanism depends on the test results.The experiment result shows that the optimizat ions have improved the efficiency of the data synchronization mechanism.