Study On Personalized Federated Learning Based On Convolutional Neural Networks

With the emphasis on data privacy protection and the improvement of national data protection laws,centralized machine learning algorithms based on large amounts of data face many challenges,which also makes federated learning algorithms a new research hotspot.This thesis studies the personalized federated learning algorithm,and the specific research content is:(1)For data heterogeneity that exists on the client,such as non-independent and homogeneous data,a personalized federated learning algorithm(PFLGS)based on gradient similarity is proposed.The algorithm implements federated personalized learning on both the client and server sides.In the client-side local training stage,the algorithm obtains a better initialization model by combining the meta-learning algorithm Reptile.In the server model aggregation stage,a similarity matrix between clients is calculated using the gradient updated locally by the client,and a different global model is generated for each client through the similarity matrix.In the test of CIFAR10,CIFAR100,MNIST,EMNIST,compared with Fed Avg,Fed Update,Per Fed Avg,p Fed Me and other algorithms,the accuracy of PFLGS algorithm in the case of IID and Non-IID distribution will be 0.02%-8.45%and 0.3%-12% higher respectively.In addition,for the impact of historical local updates on the model,a personalized federated learning algorithm based on gradient similarity momentum(PFLGS-M)is proposed,which takes the historical local update amount into the aggregation stage of the server model.In the above four dataset tests,compared with the PFLGS algorithm,the accuracy of the PFLGS-M algorithm under the condition that the data is IID and non-IID distribution will be 0.11%-0.22% and 0.34%-0.79% higher respectively.(2)For the federated learning model in this thesis adopts the characteristics of CNN network,and proposes a personalized federated learning framework based on hierarchy.The framework divides the network model into local personalization layer and federated layer,the federated layer uses the data of multiple clients for collaborative training,and the personalization layer uses local data for training,which can well realize federated learning personalization and improve model performance.Based on the hierarchical personalization framework and the idea of server dynamically generating the weights during model aggregation,a hierarchical personalization algorithm based on gradient similarity(HPFLGS)is proposed,which dynamically generates the weights during model aggregation by dividing the federated layer into multiple personalization layers and using the similarity of the local update amount of the client to dynamically generate the weights during model aggregation.In the above four dataset tests,compared with Fed Avg,Fed Update,Per Fed Avg,and p Fed Me algorithms,the accuracy of HPFLGS algorithm in the case of IID and Non-IID distribution will be 0.15%-8.81% and 0.6%-13.4%,respectively.In addition,a hierarchical personalized federated learning algorithm based on gradient similarity momentum(HPFLGS-M)is proposed,which incorporates the similarity of historical local update quantities to improve model performance.In the above four dataset tests,compared with the HPFLGS algorithm,the accuracy of the HPFLGS-M algorithm in the case of IID and non-IID distribution will be 0.11%-0.38% and 0.11%-0.95%,respectively.