Research On Large Capacity Write Buffer Cache Techniques For Key-Value Store

Due to their excellent versatility,high reliability and high performance,key-value stores have become the mainstream system architecture of data storage center.Existing key-value storage systems are mostly built based on Log-Structured Merge-Trees.However,the LSM-based systems suffer from write amplification and unpredictable performance due to their expensive compaction.It is observed that the existing key-value storage system based on Log-Structured Merge-Trees can not effectively use large capacity write cache.Therefore,it is of great significance to study how to manage large capacity write buffer cache more efficiently.In order to reduce the write amplification and write stalls of the whole system,the key-value store with optimized write buffer cache,named FlushKV,is proposed.Firstly,the vertical partitioned LSM-Tree is applied to manage the write memory component,which eliminates redundant data before it is written to disk,thus substantially not only improving the scalability of the write memory component,but also reducing the total amount of data written to the external storage device.Secondly,considering the data characteristics such as heat and the key range of the data to be refreshed,a new self-heuristic refresh strategy,named Tiny＿Flush,is designed to reorganize L₀layer in a horizontal grouping structure,based on which the a new combination strategy of L₀layer and L₁layer is designed,thus substantially reducing the frequency and the amount of compaction data of L₀-L₁compaction.Thirdly,the priority scheduling strategy of between flush and compaction of different levels is designed,to allocate I/O bandwidth reasonably and reduce write stalls caused by merge operation.Finally,the write ahead log is relocated and managed in NVM based on key-value pairs,which reclaims the write ahead log data and reduces the overhead on the critical path of write operations.Evaluation results based on prototype show that,FlushKV still maintains excellent performance in larger write buffer cache and effectively alleviates the huge performance fluctuation caused by expensive compaction.Under different write memory sizes,in a typical configuration where the key-value sizes are 16 bytes and 4KB,the Flush’s random write throughput improvement over Level DB ranges from 2.05 times to 3.57 times,and the write amplification of FlushKV is reduced by 28.51%～56.69%.Under different value size,in a configuration where the write buffer caches are set at 256MB,FlushKV can improve random write throughput 1.58 times to 2.94 times compared to Level DB,and reduces the write amplification 43.55%～59.54%over Level DB.