Research On Key Problems Of Federated Learning With Heterogeneous Data

Federated learning(FL)is a promising distributed machine learning paradigm that enables multiple participants to collaboratively train a global model while protecting user privacy and data security.However,in real-world scenarios,the problem of data heterogeneity increases the difficulty of model training in FL,which reduces the training efficiency and accuracy of the global model.In addition,in scenarios with a high degree of data heterogeneity,a single global model cannot well meet the local needs of each participant.This thesis conducts research on the above issues,and the main research contents and contributions are as follows:(1)A Federated Learning algorithm with Exploratory Aggregation named Fed Explore is proposed.Fed Explore measures the amount of information acquired by the global model during local training by comparing the representation similarity between the global model and the local model.During server aggregation,models with more information would be given higher weights.Compared with the traditional average aggregation algorithm,Fed Explore can dynamically finetune the optimization direction of the global model according to the amount of information obtained from the local training,fully exploit the potential of each participant’s local model,and improve the aggregation effect of the global model.Experiments prove that Fed Explore can not only improve the accuracy of the model in data heterogeneous scenarios,but also reduce the round of communication rounds during training.(2)A local dynamic training mechanism is proposed.This method dynamically adjusts the number of local training epochs for each participant by measuring the benefits of the global model after local training on different local data.This thesis conducts experiments by combining the local dynamic training mechanism with Fed Avg,Fed Prox,MOON and Fed Explore proposed in this thesis.The experimental results show that the local dynamic training mechanism can reduce the computing burden of the local device and improve the training efficiency of the model in heterogeneous data scenarios while ensuring the accuracy of the model.(3)A Multi-center Personalized Federated Learning with Model Decoupling named Fed MDC is proposed.Fed MDC divides the overall model into a main module and a personalized module.The personalization of the local model for each participant is realized by maintaining the personalized module locally,aggregating the main module by the server,and finally combine the personalized module and the main module on each local device.In addition,in order to reduce the impact of data heterogeneity on model aggregation,this thesis proposes a grouping strategy based on local representation.On the premise of protecting data privacy,participants with similar data distribution are divided into a group to improve the aggregation effect of the model.Experiments prove that Fed MDC can achieve better model personalization effects in various heterogeneous scenarios,and improve the accuracy of personalized models in local tasks.