Communication Optimization And Trust In Distributed Federated Learning

Distributed federated learning is the product of the deep integration of artificial intelligence and edge computing,and it is also the key technology to combine intelligent manufacturing with AI.The industrial Internet of Things can realize the application of artificial intelligence through distributed federated learning.Generalized Internet can realize multi-institution cooperative training global model through distributed federated learning.In the Industrial Internet of Things,distributed federated learning is faced with the challenges of limited computing and communication resources and interaction between model training and communication mechanisms.In the generalized Internet,distributed federated learning is confronted with training motivation and model security problems caused by distrust among institutions.Starting from two application scenarios of industrial Internet of Things and generalized Internet,we will study communication optimization of distributed federated learning,consensus mechanism,and trust between institutions respectively.Federated learning(FL)enables multiple devices to collaboratively train a shared machine learning(ML)model while keeping all the local data private,which is a crucial enabler to implement Artificial Intelligence(AI)at the edge of the Industrial Internet of Things(IIo T)scenario.Distributed FL based on device-to-device(D2D)communications can solve the single point of failure and scaling issue of centralized FL,but subject to the communication resource limitation of D2 D links.Thus it is crucial to reduce the data transmission volume of FL models between devices.In this paper,we propose a quantization-based distributed federated learning(Q-DFL)mechanism in a D2 D network and prove its convergency.Q-DFL contains two phases: in phase I,a local model is trained with the stochastic gradient descent(SGD)algorithm on each IIo T device,and then exchanges the quantified model parameters between neighboring nodes;in phase II,a quantitative consensus mechanism is designed to ensure the local models converge to the same global model.We also propose an adaptive stopping mechanism and a synchronization protocol to fulfill the phase transition from phase I to phase II.Simulation results reveal that with Q-DFL,a 1-bit quantizer can be employed without affecting the model convergence at the price of slight accuracy reduction,which achieves significant transmission bandwidth saving.FL is subject to the motivation and security issues that are both caused by trust issues when the network architecture evolves from centralization to truly decentralization.Thus,it is important to eliminate the mistrust of cooperation between participants in the distributed network.In this paper,we propose a two-tier FL-Blockchain network system in a distributed network to motivate most participants to upload their local models by using cryptocurrency and prevent model parameters from being tampered with.The incentive mechanism is employed to distribute cryptocurrencies while considering the difficulty of training models and the accuracy of models.Meanwhile,a lightweight consensus algorithm for FL is proposed to ensure the shared model is as accurate as possible.The trust issue goes away because of blockchain technology,but the performance issue comes along with it.To improve the throughput performance of transactions,we implement a reputation mechanism at the expense of security.Except for throughput performance,another metric is the accuracy of the shared model.To balance the trade-off between throughput and accuracy,we propose an accuracy dependent throughput management(ADTM)mechanism to guarantee the throughput performance remains in a desirable range while improving the shared model accuracy.Compared to relevant baselines,extensive simulations show that the two-tier FL-Blockchain network system improves the accuracy and keeps the throughput as high as it can reasonably be.