Research On Multi-Strip Repair Of Load Balanced Erasure Code In Heterogeneous Distributed Storage System

In the past few years,data has grown explosively,and some studies have shown that 90%of the world’s data was created within the past two years.The amount of data is constantly increasing,and the speed of data creation is also rapidly improving.At the same time,in order to cope with the occurrence of disk or disk block failures,users have higher requirements for high availability of data.Distributed storage systems are widely used due to their high availability and low storage overhead,and erasure codes are a commonly used redundancy technique in distributed storage systems.In heterogeneous network environments,there may be differences in the bandwidth size of nodes.In the process of erasure code repair,there is a problem of nodes with larger bandwidths transmitting less data,while nodes with smaller bandwidths transmitting more data,resulting in imbalanced load on nodes and wasting their bandwidth resources.At the same time,existing data transmission methods have the problem of imbalanced load on transmission nodes,which increases the possibility of network congestion and greatly reduces the efficiency of erasure code repair.In order to solve the above problems,this article focuses on the multi stripe repair of erasure codes in distributed storage systems.Two methods for optimizing multi stripe repair in heterogeneous scenarios are designed,and relevant technologies and theoretical foundations,design ideas,and experimental results analysis are elaborated in detail.This article proposes a multi band repair data block selection strategy based on balance,which addresses the problem of imbalanced selection of data blocks for erasure code multi band repair in heterogeneous environments.By defining node balance and calculating stripe balance based on node balance,uniform grouping is performed.After grouping,appropriate repair data blocks are selected for each stripe within the group using a data block selection algorithm.At the same time,this article proposes a CNLB-RP balancing pipeline to address the issue of node load imbalance during data transmission repaired by erasure codes.Based on existing pipeline methods,the pipeline transmission structure is optimized by slicing,grouping,repairing,and misplacing methods,which are refined to the read and write load of nodes,achieving read and write load balancing of nodes in pipeline data transmission.After theoretical analysis and experimental testing,the multi band repair data block selection strategy based on balance degree and the CNLB-RP balance pipeline proposed in this article improve the repair efficiency of erasure codes,achieve node load balancing during data transmission,greatly reduce the possibility of network congestion,fully utilize node bandwidth resources,reduce repair time,and improve node load balancing performance.