Research And Implementation Of Light-weight Compaction Key-value Storage System

The Shingled Magnetic Recording(SMR)technology can effectively improve the storage density of disks and has been widely used in massive data storage.The NoSQL database represented by Key-Value(K-V)system provides a high performance,high expansion and high flexibility service for big data processing.Log-Structure merge tree(LSM-tree)has been one of the mainstream indexes in K-V systems supporting write-intensive applications.However,the performance of LSM-tree on SMR disk is severely hampered by compaction procedures in LSM-tree and random writes on SMR drives,which incurs significant write amplification.Therefore,researching on the K-V system based on SMR disk has great realistic meaning.This thesis proposes a Light-Weight Compaction(LWC)algorithm based on LSM-tree.Metadata merging and data append writing in LWC can reduce compaction overhead and improve compaction efficiency on the basis of ensuring metadata lookup performance.Metadata aggregation after LWC can reduce read overhead for the next compaction.Based on the physical characteristics of SMR disk,we design and implement LWC-store with a specific designed disk space management strategy.The LWC-store mitigates on disk random writes,reduces I/O amplification and improves write performance.Experiment comparisons between LWC-store and LSM-tree store show that Light-Weight Compaction can effectively improve writing performance.Random write throughput of LWC-store is 9.8 times of LSM-tree store on SMR disk and 4.67 times of LSM-tree store on HDD.LWC-store can also be applied to HDD and SSD,and still have a high performance advantage to LSM-tree store on the same storage medium.