Research On Resource Allocation Method Based On Deep Reinforcement Learning In Federated Learning Of Internet Of Vehicles

With the development of Internet of Vehicles(Io V)and Artificial Intelligence(AI)technologies,an increasing number of AI applications are being deployed in vehicular operating systems.Vehicular Edge Compute(VEC)opens up possibilities for intelligent services;however,it also poses risks of exposing user privacy data.Federated Learning(FL),as a promising privacy protection paradigm,synthesizes a global model using only the parameters of locally trained models,thereby avoiding sensitive data leakage.However,introducing FL into Io V faces many challenges.Firstly,as participants in FL,the selection of vehicles affects the training cost and model performance of FL,making the rational selection of participating vehicles crucial to the performance of FL.In addition,FL involves frequent model exchanges,consuming significant communication and computing resources.How to allocate resources properly to reduce communication costs becomes a critical issue.Moreover,gradient quantization is widely used for model compression in FL to reduce transmission overhead.How to allocate quantization levels reasonably based on the high mobility of vehicles and dynamic changes in channels is a key problem for jointly optimizing latency and quantization error.Deep reinforcement learning provides a favorable framework for addressing resource allocation problems.This paper focuses on the resource allocation problem in vehicular FL and researches resource allocation schemes based on deep reinforcement learning.The main research work of this paper is as follows.(1)Vehicles,as participants of FL,have high mobility,and inappropriate participants may cause unnecessary computation and communication overhead or may harm the representation of the global pattern.This part of the work considers the mobility of vehicles and model similarity,and designs a participating vehicle selection algorithm.By defining the residence time of vehicles in the coverage of edge servers and the model similarity,the necessity of participation of vehicles is measured.By setting the threshold of participation necessity,the vehicles participating in local training are selected before the global model is issued in each round of FL,so as to reduce unnecessary computing and communication cost and ensure the representative of the model.Simulation results verify the superiority of the proposed method in FL average cost and test accuracy compared with the baseline method.(2)In a FL system consisting of multiple vehicles and base stations,vehicles share a Non-Orthogonal Multiple Access(NOMA)channel and use Vehicle to Infrastructure(V2I)communication to transmit models to the base stations.The rapid channel variations in the vehicle environment make it infeasible to collect accurate instantaneous channel state information at the base stations for centralized resource management.This work proposes a resource allocation scheme based on Graph Neural Network(GNN)and Multi-Agent Deep Reinforcement Learning(MADRL).Firstly,considering vehicle mobility and channel state uncertainty,communication and computation models for FL of Io V are established.Then,the vehicle network is represented as a directed graph,and low-dimensional features are extracted for each node based on graph information.Subsequently,using the extracted features,the states,actions,and shared reward functions for each intelligent agent are defined,and MADRL is employed for channel and power allocation.Finally,through simulation experiments,the feasibility and effectiveness of the proposed scheme in optimizing transmission costs in vehicular FL are validated.(3)For FL of Io V with gradient quantization compression model,the assignment of quantization levels affects the performance of FL.A small number of quantization levels can significantly reduce the communication overhead,but it will hurt the accuracy of the model.A too large quantization level results in a larger transmission delay.This work proposes a gradient quantization scheme based on deep reinforcement learning.Firstly,considering the gradient quantization of FL,the mobility of vehicles and the uncertainty of channel state,a system model including FL training time and quantization error was established.In addition,the distributed state,action and reward function were defined according to the system model.Then,the optimal strategy of maximizing the long-term discount reward was obtained through the Double Deep Q-Network(DDQN),and a distributed gradient quantization scheme was formulated to make each vehicle perform quantization level allocation based on local observation.Finally,simulation experiments verify the feasibility and effectiveness of the proposed scheme compared with several benchmark schemes in terms of optimizing training time and quantization error.