Research On Deep Reinforcement Learning-based Computation Offloading And Resource Allocation In Vehicular Edge Computing

With the advent of the fifth generation mobile communication technology(5G),emerging on-board applications such as automatic driving and safety early warning system are gradually emerging.The limited computation resources of the vehicle itself are difficult to meet the computing and delay requirements of these applications.In vehicular edge computing(VEC),computation offloading technology allows vehicles to offload computional tasks to other edge compuation nodes,so as to solve the problem of insufficient computing power of vehicles.However,the computing resources of edge nodes are also limited.How to reasonably formulate the offloading strategy and make full use of the computing resources of edge nodes has become the key to reduce the task execution time.The traditional optimization algorithm is very complex and lacks good learning ability,so it is difficult to adapt to the dynamic network environment.Therefore,this paper introduces deep reinforcement learning to solve the problem of computation offloading and resource allocation in VEC environment.The main research contents are as follows:(1)An adaptive task offloading strategy based on deep reinforcement learning is proposed.Aiming at the scenario of inseparable tasks,the "end edge cloud" collaborative offloading problem in VEC environment is studied.In order to minimize the processing time of tasks,the offloading problem is described as an optimization problem of minimizing the processing time of tasks under multiple constraints.Because this problem is a mixed integer linear programming problem,it is difficult to solve it by using traditional optimization algorithms such as game theory.In view of the above considerations,this paper proposes a computation offloading scheme COQL based on Q-Learning and an adaptive offloading scheme CODQL based on deep Q-learning(DQN).Simulation results show that CODQL scheme has better performance than COQL scheme.In addition,compared with the baseline method,the proposed scheme can effectively reduce the total task processing time.(2)A task dependent computation offloading and resource allocation scheme based on deep reinforcement learning is proposed.Aiming at the computation offloading problem with task dependency in vehicle application,this paper first uses directed acyclic graph(DAG)to model the built application.Secondly,the priority of each subtask in DAG is determined,and the priority sequence of task scheduling is obtained.Then,the computation offloading problem is transformed into Markov decision process(MDP).In addition,considering that the action of task offloading decision is discrete and the action of computing resource allocation is continuous in the problem of computation offloading,the action space of MDP contains both discrete actions and continuous actions.Aiming at the problem that dqn can not effectively deal with continuous actions,a computation offloading and resource allocation scheme based on the combination of dqn and depth deterministic strategy gradient algorithm(DDPG)is proposed.Simulation results show that the proposed scheme can effectively improve the application completion rate and reduce the average application completion time.