The Design And Implementation Of Persistence Scheme Based On In-memory Database

In this era of rapid development,Internet technology has been further developed.Information technology plays a significant role in promoting the social economy,and the cornerstone of information technology is database technology,which is the carrier of information storage.Traditional disk database has always played an important role in database technology,and have also achieved good results in traditional OLTP(On-Line Transaction Processing).However,people become increasingly strict with the response speed of application.In many high real-time scenarios,the traditional disk database is disappointing.With the development of memory technology,the capacity of memory is getting larger and larger,and the price is getting lower and lower.Because of its superior access speed,the memory database has been widely used in multifarious scenarios.But currently,there are still many problems to be solved in the memory database,including data security,fast system recovery,crazy memory consumption(occupation)and so forth.These problems stem from the defects of its persistence strategy.Common persistence strategies include log and snapshot,which are only simple backups of data,and the snapshot method still has major problems in data security.In view of the shortcomings of the persistence strategy of the above in-memory database,this thesis puts forward an optimization scheme: ameliorate the persistence strategy on the basis of being fully compatible with the native system.In this study,the persistence part of the native system is separated out and processed by an independent storage engine,and the memory occupancy rate is reduced through the cold data elimination mechanism.In this thesis,the main work will be completed as follows:1)Starting from the background of the current era and investigating deeply the practical significance of the memory data persistence strategies optimization as well as the development status of this field.On the basis of full investigation,the overall architecture design and functional implementation of the system are completed,and its performance reaches a better level.2)In order to solve the problem that the data in the in-memory database is not completely secure,it is proposed to use the independent storage engine as the persistence layer to complete the data persistence work separately.The efficient skip list structure of the storage engine is used to improve the efficiency of data writing and merging.3)In order to solve the problem that the in-memory database occupies a large amount of memory under the long-term operation,a cold data elimination mechanism is proposed to remove infrequently used data from memory,improve memory utilization,and load data into memory when it is reread.4)After the completion of the system design,build a number of workloads to test the system in multiple dimensions,including functional testing and performance testing,compare system with the native system,and analyze the test results based on the results.The experimental results show that,with the optimization of the in-memory database persistence strategy,the database performance meets the expected requirements,and some problems in persistence are solved.