Research On Multi-agent Deep Reinforcement Learning Based Resource Provisioning Approach In Edge Computing

By deploying services and performing computing tasks at the edge of the network,edge computing can meet the delay-sensitive service requirements of resource-poor terminal devices.Reasonable resource provisioning can improve the utilization of edge computing resources and the success rate of task execution in edge computing system.The research on resource provisioning in edge computing is divided into two categories:task scheduling and service placement.Task scheduling method based on task execution time estimation is a representative task scheduling method.By finely modeling the task execution process,it estimates the execution time of the task on different edge servers,and schedules the task to the server that is expected to take the shortest time to execute.The service placement method based on load prediction is a representative service placement method.It predicts the future load through historical load information,and places services according to the prediction results.However,the execution of some tasks depends on the execution results of other tasks,and the execution time of cross-server tasks is difficult to estimate,and there is a problem of resource preemption between multiple servers,resulting in inaccurate task completion time estimation.In real-world scenarios,user requests are generated randomly,and the request load is difficult to estimate,resulting in inaccurate load prediction;redundant services are placed between servers,resulting in waste of resources.Aiming at the problem of low success rate of task execution caused by data dependency between tasks,difficulty in estimating the execution time of cross server tasks,and multi server resource preemption,a scheduling algorithm based on multi-agent deep reinforcement learning for dependent tasks is proposed.First of all,considering that the edge computing system is a multi-agent system,it is assumed that the dependent task scheduling of each edge server is controlled by one agent,and each agent can only observe local state information.The dependent task scheduling process unrelated to the past state is modeled as Decentralized Partially Observable Markov Decision Process.Secondly,for the dependent tasks modeled by directed acyclic graph,through the improved graph attention network,the task can aggregate other task features that are dependent on it,and improve the representation of task features.Again,the task feature information and edge server state information are used as the input of the agent policy network,and the task scheduling decision is generated through the Proximal Policy Optimization algorithm.Finally,the cooperation between task scheduling strategies of each agent is established by Counterfactual Multi-Agent Policy Gradients architecture.In this thesis,simulation experiments are carried out on the dependent task scheduling process with different number of service requests,task delay tolerance and the number of edge servers.The experimental results show that the task execution success rate of the proposed algorithm is significantly better than that of similar representative algorithms.Compared with Drag JDEC algorithm,the success rate of task execution is improved by 6.7%on average under different service requests.Aiming at the problem of low success rate of task execution caused by user request load is difficult to predict,and redundant service placement exists between servers,a service placement and resource allocation algorithm based on multi-agent deep reinforcement learning is proposed.First,considering that the service placement and resource allocation actions are discrete-continuous hybrid actions,the process of service placement and resource allocation unrelated to the past state is modeled as Parameterised Action Markov Decision Process.Secondly,Multi-Pass Deep Q-Networks algorithm is used to solve the Parameterised Action Markov Decision Process.And dual network structure and experience playback technology are adopted to enhance the stability of network training.Finally,QMIX architecture is used to establish strategy collaboration among multi-agent to realize the strategy collaboration of service placement and resource allocation among edge servers.In this thesis,simulation experiments are carried out on the service placement and resource allocation process with different number of service types,the number of edge server computing resources and the number of edge servers.The experimental results show that the task execution success rate of the proposed algorithm is significantly better than that of similar representative algorithms.Compared with the PDQN algorithm,the success rate of task execution is improved by 5.2%on average under different number of edge servers.The edge computing resource provisioning method based on multi-agent reinforcement learning proposed in this thesis optimizes task scheduling and service placement,realizes more efficient edge computing resource provisioning,improves the utilization of edge computing resources,and improves the success rate of task execution of edge computing system.