Research On IoV Spectrum Efficiency Optimization Based On Deep Reinforcement Learning

The Internet of Vehicles is an important application of the fifth generation mobile communication.In this network,there are two kinds of communication links,one is a vehicle-to-infrastructure(V2I)communication link,and the other is a vehicle-tovehicle(V2V)communication link.In order to improve the efficiency of spectrum utilization,many studies have focused on interference management through resource allocation algorithm.However,the basis for the implementation of these methods is based on the fact that the base station needs to obtain all the channel state information(CSI)of all the vehicles.In reality,it is difficult for the base station to obtain accurate CSI due to the high-speed movement of vehicles.In order to solve this problem,this paper uses the deep reinforcement learning algorithm(DRL).First,this paper studies the resource allocation problem,considering a single V2 V link as an agent.We take the real-time CSI that can be obtained from this V2 V link and the obtained interference from other vehicles as the state,using the channel selection and transmission power as the agent's action,and using the system's spectrum effect as a reward.Then,we construct reinforcement learning problem and use deep Q network(DQN)to solve this problem.Next,in view of the fact that multiple V2 V links in the system are agents,a multi-agent reinforcement learning model is constructed,and the agents continuously update their own strategy in order to maximize the same reward.The simulation proves that the single agent algorithm is better than the random selection algorithm,which improves the spectrum efficiency of the system.And because the multi-agent algorithm is based on a cooperative model,the simulation result is better than the single-agent algorithm,which further improves the spectrum efficiency of the system.Secondly,this paper studies the resource allocation problem,considering both V2 V links and V2 I links as agents.In order to solve the problem of inconsistent action selection between V2 I link and V2 V link,this paper first allocates channels to V2 V link,and then uses multi-agent deep deterministic policy gradient(MADDPG)to solve the power allocation problem of V2 I links and V2 V links.It can be seen from the simulation results that the resource allocation algorithm based on MADDPG can handle the continuous variable well,which improves the spectrum efficiency of the system.Finally,in order to deal with discrete variables and continuous variables simultaneously,this paper studies the resource allocation algorithm at the base station.The overall system optimization problem is decomposed into two sub-problems.For the problem of power allocation,a linear search algorithm is used to solve it.For the channel allocation problem,this paper uses DQN to solve the channel allocation problem.By comparing with the depth-first search algorithm,it is verified that DQN reduces the complexity of the algorithm while ensuring the performance of resource allocation.In order to further solve the improvement of the robust of the algorithm,this paper proposes an intelligent branch and bound algorithm,using DQN to guide the pruning strategy of the branch and bound algorithm,while ensuring the traversal effect,greatly reducing the complexity of the algorithm and the algorithm is robust.