Resource Optimization Of Vehicular Networks Based On Reinforcement Learning

With the development of mobile communication technology and vehicular networks,more and more in-vehicle services have emerged to improve comfort and convenience of passengers and drivers.However,due to the limited resources of computing,storage and battery capacity of vehicles,it is difficult to effectively support the increasingly requirements for these services.Therefore,appropriate resource optimization schemes are needed to improve user satisfaction by making full use of communication,computing and caching resources in vehicular networks.At present,there are many challenges in the optimization of resources in vehicular networks,such as uneven distribution of vehicles,high mobility of vehicles leading to the rapid channel change,and difference of different services.In addition,the traditional resource management scheme is usually described as optimization problem.However,joint communications,computing and cache resources optimization problems tend to be complex problems,which is usually a NP-hard problem,and it's difficult to find the global optimal solution and the complexity of algorithm is high.As an emerging technology applied to the resource optimization of vehicular network,reinforcement learning has the characteristics of fast decision-making speed and the ability to solve complex problems through learning,so it is a promising method to deal with a series of challenges faced by vehicular networks.Therefore,this thesis studies resource optimization schemes of vehicular networks based on reinforcement learning to improve the performance and benefit in vehicular networks.Firstly,based on the challenges in resource optimization of vehicular networks and recent research status,the reinforcement learning technology in vehicular networks is introduced and the different applications of reinforcement learning technology in different scenarios in vehicular networks are investigated and summarizes.In addition,that different reinforcement learning algorithms apply to different scenarios is summarized.Secondly,in order to solve the problem of resource shortage and resource oversupply caused by the uneven distribution of vehicles in vehicular networks,an access allocation problem of communication,computing and caching resources in vehicular networks is studied,which makes use of the vehicle helpers and the cooperation between RSUs to allow flexible scheduling of resources to improve the network performance.Based on SDN technology,a vehicle-network cooperation deep reinforcement learning framework is proposed to solve the problem of resource access and allocation for vehicle users when the channel changes too fast in vehicular networks,which can use the historical feedback information of vehicles to make reasonable decision without real-time SNR information to make full use of resources in vehicular networks to improve the performance of different vehicle services.Simulation results show that compared with other schemes the proposed scheme can obtain higher long-term rewards and can make different decisions according to different needs to make reasonable and full use of resources in the network.Then,in order to solve the problem of the diversity and difference of services and limited resources in vehicular networks.a service-oriented joint communication,computing and cache resource management problem in vehicular networks is studied,which can make reasonable resource allocation decisions for different services to improve the revenue of network operators.To deal with the high complexity caused by high mobility of vehicles,dynamics network resources and large number of vehicles,a double-scale deep reinforcement learning framework is proposed to reduce the complexity of resource management,which decomposes original problem into two subproblems to reduce the action space dimensions caused by variety of vehicular services and the large number of vehicles,and combines the advantages of on-policy learning and off-policy learning to make network operators can make flexible resource allocation decisions.The proposed scheme can improve user satisfaction of different services and reduce cost of network operators,which lead to high revenue of network operators.The simulation results show that compared with other schemes the proposed scheme has higher revenue of network operators,higher stability when the number of vehicles changes,and can make dynamic resource allocation decisions according to different service demands.Finally,the thesis is summarized and prospected.