A Folded-Structure Code With Low Extension Bandwidth In Distributed Storage

Distributed storage system based on erasure code has become the key technology of storing large data due to its high reliability,extensibility and huge storage potential.The distributed storage system needs to adopt a certain extension method to improve the storage capacity and optimize the bandwidth cost during extension of the system for meeting the needs of data increase and user requirement.Ideally,the coding parameters of the storage system should be dynamically adjusted according to the application requirements.For that the codes before and after extension are not related,the system based on the traditional code needs to obtain all the information data during extension and recode the data according to the new coding structure to complete extension,which usually causes a lot of resource consumption and has a detrimental effect on operation of the system.By analyzing the extension process of distributed storage system based on erasure code,this thesis proposes a folded-structure code,whose subcodes are related to each others,and the folded-structure code can reduce the amount of information data transmitted in the network by at least half during extending the storage system and support multiple extensions of the system.In addition,based on folded-structure code,a low-repair-bandwidth FSC with piggybacking is designed,which can reduce both of extension bandwidth and repair bandwidth.The low-repair-bandwidth FSC with piggybacking has better comprehensive performance and can provide higher reliability and extensibility for the distributed storage system.Firstly,this thesis briefly summarizes the problems occuring in the existing storage system,and introduces some existing research on the repair and extension of distributed storage system.Secondly,this thesis summarizes the basic knowledge and extension of distributed storage system based on MDS code.A method of constructing short codes with MDS property according to the generating matrix of MDS long codes is given.At the same time,the basic knowledge of piggybacking framework is briefly introduced.Thirdly,focusing on the high extension bandwidth of traditional MDS code,this thesis gives an new code design,named as folded-structure code(FSC),which effectively reduces the extension bandwidth during system extension and supports multiple extensions of the system.When extend the system from short code to long code,FSC can enable the system to use all the migrated symbols to update the old parity symbols without reading all the systematic symbols,which reducing data reading of systematic symbols by half.Additional,this thesis analysis the contractibility of folded-structure code.Through discussion,the foldedstructure code still has a good performance when converting from long code to short code,and its bandwidth consumption is lower than that of MDS code under the same parameters.Fourthly,based on the folded-structure code,a new low-repair-bandwidth FSC with piggybacking(L-FSC)is designed,which retains the property of low extension bandwidth of folded-structure code and achieves the purpose for reducing repair bandwidth when a single systematic node fails in the system.The analysis shows that with the increase of the number of parity nodes in original system,the rate of repair bandwidth of L-FSC can approach0.5.The design has a certain reference significance for the secondary design of the foldedstructure code.Finally,FSC and L-FSC are compared with several existing codes.