Performance Optimization Of Distributed Storage Systems Based On Software-Defined Networks

Today,distributed storage systems are an essential solution for storing and managing massive amounts of data in the era of Big Data.And enterprises and research fields have been working on new storage architectures and distributed scheduling strategies to optimize system performance and improve space utilization.Researchers use new storage architectures to monitor the system’s computing,network,and storage resources globally to ensure load balancing to enhance the performance and space utilization of distributed storage systems.Then they use distributed node selection strategies to optimize the system’s performance.At the same time,they ensure that the hot and cold levels of data match the performance of heterogeneous storage devices to avoid cold data residing in the high-performance storage tier.This Strategy will avoid storage pollution caused by cold data living in the high-performance storage tier,thus improving the space utilization of the system.Therefore,it is vital to carry out research on new storage architectures and distributed optimal scheduling strategies.This dissertation mainly focuses on the Ceph distributed storage system and investigates the following issues: 1)In replica mode,the CRUSH algorithm selects storage nodes based only on node storage capacity without considering the impact of network and node load on system performance;2)I/O interference among heterogeneous storage devices leads to system performance degradation,and resource competition among underlying storage devices causes interference between I/O request scheduling;3)The statically distributed data in the system cannot meet the dynamic access performance requirements of clients,that is,how to ensure the match between the temperature level of data and the performance of heterogeneous storage devices.The main innovations of the dissertation are as follows:(1)In terms of improving the CRUSH algorithm: this dissertation introduces a new distributed storage system architecture based on Software Defined Network(SDN),simplifying the tedious configuration of monitoring network status and measurement costs.Based on SDN technology,it monitors the real-time network status and the load of nodes.It establishes a multi-attribute decision mathematical model to obtain the performance weight of the CRUSH algorithm.Finally,the performance of the Ceph system is optimized by improving load balancing.(2)In coordinating I/O interference between devices: this dissertation designs an object-level coordination strategy based on SDN,which uses the Packet-In-Packet-Out(PIPO)communication method and a timed coordination process to provide efficient and cost-controlled coordination services.(3)In hierarchical storage of hot and cold data: this dissertation proposes an object-level migrating strategy based SDN,which builds a model to determine objects’ "hotness" or "coldness" based on various characteristics such as their access time and frequency.By matching the real-time temperature of objects with the performance of the storage layer,this strategy intelligently migrates objects to different storage layers for hierarchical storage.As a result,this strategy can automatically perceive which objects is cold/hot and optimize the allocation of storage space resources accordingly.