Research On Performance Optimization Method Of Distributed Data Sharing System

In recent years,with the development of the Internet of Things and related fields,such as the massive access of heterogeneous devices,the volume of data generated by devices has grown rapidly,and data sharing is facing enormous pressure.As one of the main data distribution methods,the publish-subscribe model supports data producers(consumers)to publish(subscribe)data to(from)the global data domain in an anonymous decoupled manner.The data can be stored on any node,and the uneven distribution of data requires network transmission,which consumes some communication resources.At the same time,the dynamic and flexible data sharing mode increases the complexity of access control policy deployment,posing challenges to data privacy protection.Therefore,studying the performance optimization methods of distributed data sharing systems has significant practical significance for improving the usability,resource utilization efficiency,and security of the entire system.Tasks under the publish-subscribe model often consist of multiple dependent subtasks,and the tasks are executed periodically,resulting in a large amount of data distribution.To address the problem of large network resource consumption in traditional data sharing methods,an algorithm for task offloading based on dependency grouping and path merging is proposed by analyzing the impact of task production and consumption data on data exchange.The algorithm adjusts the distribution of the data domain to reduce unnecessary data transmission.To address the complex problem of deploying data sharing access control policies,an algorithm based on the similar minimum set of access control policy mining is proposed.The authorization mode in the log is combed to reduce the complexity of policy deployment.The main research contents are summarized as follows:(1)An algorithm for task offloading based on dependency grouping and path merging is proposed.Firstly,a non-linear integer programming model for data transmission optimization is established,and an optimization algorithm based on data dependency grouping strategy(GBTO)is designed to minimize data transmission as the goal to unload tasks to computing nodes.Secondly,based on the further exploration of task dependency characteristics,an optimization algorithm based on path merging(GFBTO)is designed to enable tasks in the same task group to fully share the computing resources of the local node.Compared with GBTO,GFBTO enhances task parallelism by increasing the number of tasks unloaded to the same computing node,reducing the operating system scheduling and switching overhead.According to data usage scenarios,GBTO is suitable for CPU-intensive tasks,while GFBTO is more suitable for I/O-intensive tasks.Experimental results show that the data transmission optimization effects of GBTO and GFBTO can reach 80% and 90% of the theoretical optimal solution,respectively.(2)An algorithm based on the similar minimum set of access control policy mining is proposed.Firstly,to address the problem that the expressive power of traditional attribute-based access control policy formal language is insufficient,dynamic environmental attributes and multi-valued attribute conditions are added to the original static attribute and single-valued attribute conditions.Secondly,a theoretical model of attribute-based access control policy mining problem is constructed,and it is proved that the problem is an NP-Hard problem and can only be approximately solved.Finally,an algorithm based on the similar minimum set of access control policy mining(SLSBPM)is designed and implemented to analyze the authorization mode from the access control authorization log,and mine the minimum policy set that is as similar as possible to the original access rules through access behavior,reducing policy redundancy and achieving automatic deployment.Experimental results show that the algorithm has certain improvements in policy semantic similarity,syntactic similarity,and generated rule set size.It can mine more similar authorization modes with as little log data as possible and fewer redundant rules,and can quickly approach the theoretical optimal value of policy set similarity and size.(3)Designed and implemented the distributed data sharing system Mort,which mainly includes modules such as task construction,unloading algorithm,task management,and data distribution.Addressing the dependencies,periodicity,and highfrequency data distribution characteristics of tasks in the publish-subscribe system,the system integrates the GBTO and GFBTO algorithms into the unloading algorithm module,and provides unloading solutions for the constructed task group instances through the global state information maintained by the task management module.Additionally,the SLSBPM algorithm is integrated into the task management module to mine effective access control policies from the log stream and support users in directly modifying data access permissions.After joint analysis,the policies are applied in realtime to task data access requests.Mort not only reduces data transmission in the network but also effectively controls necessary task data access.Experimental results show that the average data sharing delay for network data and local data is only 6ms and 0.4ms,respectively,and the system’s data packet loss ratio is small,with a maximum of only 0.9‰.