Research And Optimization Of Storage Engine Based On LevelDB

With the maturity of Web 2.0 applications,the Internet has developed rapidly in recent years,bringing challenges to traditional relational databases.These applications are known for large-scale data and require database to support with higher read/write performance for more efficient management.However,the traditional database cannot match the requirements of the second-order response of reading and writing performance under large-scale data storage,and the management cost is higher,the expansion performance is poor.Based on the above reasons,the NoSQL databases have emerged on the road of exploring new technologies.Currently the LevelDB modeled by LSM-Tree is widely used in the key/value database.Many commercial products on the market implement the LevelDB storage engine interface,and there are many secondary developments and applications based on the LevelDB source code,which is enough to see that LevelDB has a pivotal role in the NoSQL.Based on the overall architecture of NoSQL,this thesis deeply studies its internal implementation mechanism and optimizes the shortcomings of write amplification and read amplification in this system.The main work mainly includes the following points:(1)The thesis researches the current development status of NoSQL,and analyzes the advantages and disadvantages of three mainstream storage engines,and further studies LevelDB based on the theoretical basis.(2)This paper proposes an easy-to-understand optimization scheme--key/value separation storage,which solves three problems brought by the optimization scheme: the garbage collection of value,the consistency of key/value and the range query of key.The LSM-Tree module that originally stored the key/value is greatly simplified,which greatly reduces the LSM-Tree size.The garbage collection mechanism is redesigned as the structure is adjusted and the new Listener monitoring mechanism has been added,which greatly improves the read and write performance of the system.At the end of this paper,the benchmark program of LevelDB is used to test the function and performance of the system.The results show that the optimization effect is obvious under the premise of large value.