Research On Optimization And Application Of Heterogeneous Data Fusion In Federated Learning

With the extensive development of artificial intelligence technology represented by deep learning in recent years,more and more tasks use deep neural network model and achieve good results.The model based on deep neural network needs to use a large amount of high-quality data for training,but a large amount of high-quality training data needs to be collected from the real node that generates data.Due to the demand of privacy protection in recent years,the data of each data node cannot be uniformly sent to one server,so federated learning comes into being.Federated learning makes use of the data of each node to train a model uniformly while protecting the privacy of each data node.Heterogeneous data is characteristic of the datasets of each data node used by federated learning.According to the heterogeneous data form of each node,there are horizontal federated learning,vertical federated learning and federated transfer learning.This paper proposes improved algorithms for horizontal and vertical federated learning respectively to improve the performance of federated learning algorithms.In the scenario of horizontal federated learning,the data of each data node has the same features,but the data distribution among the nodes is non-independent and identically distributed.So data heterogeneity is at the data distribution level.For image classification tasks in horizontal federated learning,the existing algorithms perform poorly when the data distribution of different data nodes is extremely unbalanced,so FedNKD is proposed in this paper.FedNKD uses noise data for knowledge distillation to improve algorithm performance.Because the data distribution of different data nodes varies greatly,the model trained by each node using local data only has part of specialized knowledge.FedNKD uses knowledge distillation to gather specialized knowledge of each data node into a model,so that it has better performance in global data distribution.In the scenario of vertical federated learning,each data node has the same batch of data,but different features of the same data sample are distributed in different data nodes.So data heterogeneity is at the data feature level.At present,there are few relevant researches on time series anomaly detection in vertical federated learning.Therefore,this paper proposes Coeus:an unsupervised time series anomaly detection model based on Transformer under the background of vertical federated learning.Coeus can efficiently fuse the features of the same data sample.And a matching automatic anomaly fraction threshold selection algorithm is proposed to improve the performance of the Coeus algorithm.FedNKD and Coeus have been verified on multiple datasets,demonstrating their excellent performance.They are compared with the comparison algorithm,they improve by several percentage points.