The Design And Implementation Of In-memory Key-value Storage System Based On MIC Architecture

With the popularity of the Internet and smart devices,an attendant problem is that some internet applications need face massive data.When face massive data,traditional relational databases can not continue to provide high throughput and low latency services,and in-memory database(IMKV)which based on NoSQL begin to replace the traditional relational database,to improve the dilemma faced by current relational databases.In addition,thanks to the development of hardware technology,the memory capacity of DRAM increases,the price falls.Because of the time-consuming disk 10,the previous disk storage began to phase out,and gradually turned to memory storage,which makes data access more quickly.Further more,the coprocessor architecture MIC(many integerated core)which introduced by Intel in 2012 provide high memory bandwidth for memory IO,as well as providing a large number of computing units for processing data.The IMKV began to face new opportunities and challenges,this paper is based on the above background,design and implement a massive data oriented IMKV which with the help of high concurrency features of the MIC platform to guarantee high concurrency and low latency services for massive data storage.Design an in-memory key-value storage system,and exploit the MIC architecture to speed up the index operation of the system.The system is designed from three aspects emphatically:(1)data structure,we design the index structure which based on Cuckoo Hash,and make optimization in term of the MIC hardware features;(2)scheduling strategy,according to different delay requirements for GET/SET/DELETE operation,we design periodic delay scheduling strategy;(3)memory management,in order to solve the problem of data transmission and storage between CPU and MIC,we design slab memory management system.Realize an in-memory key value storage system based on the MIC structure for the first time,and carry on a series of test analysis to the system:(1)load test,test the influence of different GET/SET ratio and load of different sig-loc length on system overhead,and analyze the reason,(2)the synchronization mechanism overhead test,test the influence of using different Linux synchronization mechanisms to ensure the security of the insert and delete operations on system overhead,and analyzes the reasons,(3)contrast test,Mega-KV and Memcached are selected to be compared with the key-value storage system based on MIC architecture,analyze their advantages and disadvantages and its reasons.