Performance Optimization Of Semi-Supervised Federated Learning Systems Based On Information Entropy

In the Internet of Things era,the spread of artificial intelligence has made data a strategic resource.With the popularity of intelligent applications and increased awareness of privacy protection,operators have raised an urgent need for data exploitation with privacy protection.In order to use data to train models while safeguarding the security of distributed data,researchers have proposed federated learning techniques to achieve the goal of training shared models without the restriction of transmitting raw data.However,a core problem faced by federated learning is that no more data labels exist in real scenarios.On the one hand,manual labeling solutions are time-consuming and labor-intensive,and there are problems such as privacy violations and even legal violations.On the other hand,the existing semi-supervised federated learning schemes are too expensive regarding computational resources.They are unsuitable for scenarios where unlabeled data dominate the data set.In order to solve the above problems,this paper first proposes the loss function based on information entropy,an adaptive loss model based on inverse triangulation,and a confidence evaluation model based on regularized entropy from the perspectives of optimizing the utilization of unlabeled data and constructing a model for semisupervised loss.Finally,this paper proposes a semi-supervised federated learning performance optimization scheme based on information entropy.In particular,this paper proposes a general semi-supervised federated learning strategy that can be applied to many practical problems and is not limited to a particular scenario or a specific DNN model,with good transferability.Specifically,the innovations and outstanding contributions of this paper are as follows.(1)This paper presents the first information entropy-driven optimization technique to better utilize the unlabeled data in distributed federated learning scenarios.Firstly,this paper proposes an information entropy-based loss function for unlabeled data through the study of information entropy theory,followed by a property analysis to illustrate its feasibility and effectiveness as a loss function.The paper then proposes a confidence evaluation model based on information entropy to optimize the training process.Theoretical and experimental analyses show that the proposed information entropy-based technique can be applied to scenarios with fewer data labels.(2)To dynamically optimize the joint loss composition of semi-supervised federated learning,this paper first presents the adaptive loss model for semi-supervised federated learning.Specifically,this paper proposes using the inverse triangulation technique for adaptive scaling and the derivation of the set of hyper-parameters and finally proposes an adaptive algorithm.Based on this algorithm,this paper also proposes an adaptive joint loss model for semi-supervised federated learning.Experiments show that the model works much better than the traditional scheme.(3)This paper proposes an information entropy-based performance optimization strategy for semi-supervised federated learning to improve model performance in semi-supervised federated learning while reducing computational overheads.Specifically,this paper proposes a confidence evaluation model based on regularised entropy,which is mathematically proven sensitive to data and can be used for federated learning training optimization and convergence evaluation.This paper then integrates the information entropy loss function and the adaptive scaling factor adjustment scheme to propose a set of semi-supervised federated learning performance optimization schemes.Extensive experiments on the Fashionmnist and Cifar10 datasets show that our proposed scheme can improve the model accuracy by up to 6.42% compared to the current state-of-the-art semi-supervised federated learning schemes.The theoretical analysis also shows that our scheme can reduce the computational overhead of unlabeled processing data by nearly half for the endpoints involved in federated learning compared to the mainstream semi-supervised federated learning schemes.