Personalized Federated Learning Method For Heterogeneous Data

In real production life,local data of edge devices in fields with strong confidentiality,such as smart medical care,credit and financial risk control,and autonomous driving,cannot be collected to participate in centralized machine learning training due to the limitations in privacy,storage,or communication.Therefore,federated learning,a distributed learning paradigm based on a heterogeneous edge computing environment,is proposed to address this issue.Clients participating in federated learning do not transmit local datasets,which can avoid data leakage effectively,and through multi-party joint modeling,federated learning brings together more available informations and improves data utilization.However,the statistical challenges induced by data distribution among clients is difficult to be dealt with by federated learning,especially in the case of strong heterogeneity between clients.In this issue,personalized federated learning is proposed to provide differentiated suggestions for clients,which aims to optimize the performance of the local model.While most studies in the personalized method only focused on global model optimization,the main weakness is the failure to address multi-model in personalized federated learning.Moreover,historical training information is not adopted to update the corresponding model on each client.Hence,personalized federated learning algorithm with fine-tuning clusters and adaptively updated weights is proposed in this thesis.A framework for personalized federated learning algorithm based on adaptive multi-teacher with clients self-knowledge distillation is constructed,and then a personalized federated learning algorithm for multi-source medical image classification is given.Firstly,a novel fine-tuning clusters is defined based on the previous studies of multi-model federated learning handling techniques,with the purpose of measuring the quality of clusters and adjusting clients.Based on the fine-tuning clusters,an adaptively updated weights with experience loss on clients is presented.Experimental results on twelve datasets with state-of-art personalized federated learning algorithms show that the proposed algorithm can alleviate the statistical challenge caused by heterogeneous data effectively.Secondly,the personalization granularity is extended to finer clients level.After introducing self-knowledge distillation into federated learning,in order to respond to the performance degradation of the local model quickly and adapt to the improvement slowly,an adaptive multi-teacher strategy is constructed based on the fact that there is a deviation between the local model updates of two communication rounds.Experimental results on benchmark datasets show that the proposed algorithm can find a balance between global model and personalization to deal with heterogeneity on clients.Thirdly,a personalized federated learning algorithm for multi-source medical image classification is proposed to verify the effectiveness of the work in this thesis in real-life applications.A framework of gradual and progressive personalization levels,from the clusters to the clients,is designed to deal with statistical challenge caused by clients on real-life datasets.Experimental results on three medical image datasets show that the proposed algorithm can achieve better performance and faster convergence speed.The above researches not only enrich the theory of federated learning algorithms when facing heterogeneous data on clients,but also provide new ideas for solving statistical challenges on personalized federated learning.The thesis includes 26 figures,16 tables and 123 references.