Research On High-Performance Federated Learning For Edge Networks

As a machine learning technology,federated learning can provide distributed data storage and model training,breaking down data silos while achieving data protection and parallel computing.However,the current mainstream federated learning framework mainly focuses on the improvement of model performance under supervised learning,and lacks consideration for semi-supervised and unsupervised learning.At the same time,the decentralization of training data also faces problems such as unstable model training and low model convergence speed caused by data heterogeneity as well as increased on-device training load and communication load caused by model training and transmission.In order to overcome the above challenges,build a more efficient federated learning framework and improve its practicality in the fields like smart healthcare,this thesis investigates high-performance federated learning schemes and their applications,and the main contributions include the following three aspects:1)Federated contrastive learning for skin lesion intelligent diagnosis: The lack of labeled data,visual similarity between skin lesions and restriction on private data sharing remain the major challenges in skin lesion diagnosis.To tackle these challenges,this thesis proposes a federated contrastive learning framework to break down data silos and enhance the generalizability of diagnostic model to unseen data.Subsequently,by combining data features across participated nodes,the proposed framework can improve performance of contrastive training.To extract discriminative features during on-device training,a contrastive learning based intelligent skin lesion diagnosis scheme in edge computing networks is proposed.Specifically,a contrastive learning based dual encoder network is designed to overcome training sample scarcity by fully leveraging unlabeled samples for performance promotion.Meanwhile,we devise a maximum mean discrepancy based supervised contrastive loss function,it can efficiently explore complex intra-class and inter-class variances of samples.Finally,the diagnosis simulations demonstrate that compared with existing methods,our proposed scheme can achieve superior accuracy in both on-device training and distributed training scenarios2)Uncertainty minimization for personalized federated semi-supervised learning: The main obstacle to achieving robust performance and smooth convergence for federated training is the statistical heterogeneity of distributed data.To tackle this problem,a personalized semi-supervised learning paradigm is proposed,which allows partial-labeled or unlabeled clients to seek labeling assistance from data-related clients(helper agents),thus to enhance their perception of local data;2)based on this paradigm,this thesis designs an uncertainty-based data-relation metric to ensure that selected helpers can provide trustworthy pseudo labels instead of misleading the local training;3)to mitigate the network overload introduced by helper searching,this thesis further develops a helper selection protocol to achieve efficient communication with acceptable performance sacrifice.Experiments show that the proposed method can obtain superior performance and more stable convergence than other related works with partially labeled data,especially in highly heterogeneous setting.3)Computational layered federated learning framework for smart and efficient service systems:Existing federated learning schemes hinge on participating nodes to perform intensive on-device training and network communication,which is a significant burden for energy-constrained mobile devices.This work proposes a computational layered federated learning(CLFL)framework to enable resource-constrained devices to perform computation-efficient on-device training and lightweight message transmitting.First,the network structure and key aspects of the entities in the framework are introduced.Then,this thesis gives the implementation principles of CLFL,and presents two instance schemes that allow devices to participate in joint training without the need for direct gradient computation or continuous data transmission.In order to more intuitively reflect the performance and efficiency of the proposed methods,simulation experiments show the comparison of the proposed method with traditional federated learning,and the experimental results show that the proposed method can largely reduce the complexity of communication and computation.Finally,for the future exploration,this thesis presents four related research challenges of CLFL and offer possible solutions.