Researches On Grouping Model Aggregations In Distributed Edge Learning

Edge Intelligence trains and deploys deep learning models at the edge of networks close to users and data sources to improve the performance of AI applications,reduce communication overhead,and improve user privacy.Collaborative learning using distributed clusters with high parallelism in edge environments has become very popular.Most of the existing research on distributed edge learning adopts the framework of centralized model aggregation.However,centralized frameworks face the following challenges: non-IID data distribution on edge devices,high communication overhead,backbone network saturation,and single point of failure.Therefore,it is urgent to reduce the communication overhead and improve the performance of the global model under the non-IID setting of edge data.At present,by grouping edge devices,two layers of centralized models are aggregated into three layers,namely,hierarchical model aggregation framework.Hierarchical model aggregation can take advantage of edge aggregation with low communication cost without degrading performance and avoid frequent cloud aggregation with high communication cost.However,most existing device grouping strategies only consider the communication delay between edge devices.In the edge environment,due to the device data non-IID,the difference between group data and global data distribution after grouping will also impair the convergence performance of the global model.Therefore,the tradeoff between groups data distribution and groups communication delay will occur in the process of edge devices grouping.In addition,most of the existing grouping methods are based on K-means,which tends to make the grouping results fall into the local optimum and result in poor convergence performance of the final global model.At the same time,the grouping method based on K-means needs to specify the number of groups artificially,which is extremely inflexible.Therefore,this paper considers the grouping problem of model aggregations for distributed edge learning.The main research contents of this paper are as follows:(1)Based on the model aggregation framework in distributed edge learning,this paper considers the heterogeneous data distribution and network bandwidth resources of edge devices,and firstly explores the device grouping strategy through experiments.In addition,the grouping problem of edge devices is normalized to a two-objective optimization problem based on the grouping strategy,and a heuristic model aggregation grouping optimization algorithm is proposed.Compared with the baseline algorithm in different experimental Settings,the experimental results show that the global models learned from the proposed grouping structure have better convergence performance.(2)Using theoretical values to set grouping goals and guide grouping will lead to inaccurate grouping results and inflexible search process.In this paper,the grouping objective is established based on the real value that can accurately evaluate the advantages and disadvantages of the grouping structure.This value needs to be obtained after the real training and aggregation of the global model on the grouping structure.The new grouping objective is to minimize the communication time when the global model trained on the grouping structure reaches the target accuracy.This paper proposes a learning-based model aggregation grouping algorithm,which can optimize this goal with less computing resource consumption.Experimental results show that this algorithm can achieve better results than other baseline algorithms under different experimental settings.