Research On Privacy-preserving In Federated Learning

The Data island has been a critical problem during the extensive practical deployment of artificial intelligence technologies such as deep learning.As a distributed learning method with privacy guarantee,federated learning has gradually attracted people’s attention.However,at present,the research about federal learning is still in its infancy,so that there are many challenges in its practical application.Firstly,the accuracy of neural network-based deep learning models depends not only on the size of the datasets,but also on the quality of the data.The existing federated learning schemes usually assume that participants have performed data cleaning locally.However,local processing under the background knowledge of local datasets can seriously influence the accuracy of data cleaning,so that disrupt the availability of federated model.Secondly,when the participants are enterprise users with sufficient computing resources,the existing security aggregation protocols in federated learning often require participants to use the same key,resulting in too strong security assumptions.Due to ideal environmental assumptions,federated learning is difficult to apply in practice.Finally,when the participants are terminal users with limited computing resources,the overhead of secure aggregation in existing federal learning schemes grows quadratically in the number of participants.Thus,these schemes have strict restrictions on the number of participants in the federal system.In order to comprehensively improve the applicability of model,so that promote the practical deployment of federal learning.This thesis,focused on the privacy requirements in the federal environment,studies the privacy preserving data federated cleaning and privacy preserving parameter aggregation in federal learning,and the main content as follows:For the requirements to optimize the quality of federated data,an efficient distributed privacy preserving data cleaning protocol for federated system is proposed.We utilize a mixed secure multi-party computing framework to implement LDOF based outlier detection.According to the characteristics of the calculation function,the arithmetic circuit is used to perform the arithmetic operation in the data cleaning algorithm,and the sorting operation is performed through the boolean circuit.Based on the flexible circuit combination,our scheme achieves efficient and accurate cleaning of local data on arbitrarily partitioned datasets under the background of global data.In addition,we propose an optimization method for horizontally partitioned datasets.We further optimize the execution efficiency of the scheme,by using local sensitive hash(LSH)function to reduce the amount of data involved in secure computation.Compared with the basic scheme,the implementation efficiency of the optimized scheme is almost doubled.Aiming at the problem that the security assumption of federated learning for enterprise users is too strict,a federated learning scheme with joint encryption/decryption mechanism is proposed.This scheme is based on the threshold paillier homomorphic encryption system generated by the distributed key.Our scheme constructs a key generation protocol by combining secret sharing technology,which achieves that participants can jointly generate a homomorphic cryptosystem without a trusted center,and all participants only possess the share of the private key about the cryptosystem.Compared with the existing homomorphic cryptographic federated learning schemes using the same key,which can not against the collusion between any participant and the server,the scheme in this thesis can still be secure in the case of collusion between partial participants(less than the threshold)and the server.Thus,our scheme is more practical in real-world.Moreover,the data coding technology is introduced to package the plaintext data,so that one encryption operation can process more plaintext data,which greatly improves the efficiency of the overall scheme and further improves the practicability of the scheme.Aiming at the expensive secure aggregation cost of federated learning for terminal devices,a lightweight federated learning scheme for massive Io T terminals is proposed.To protect the privacy of the individual local data,we utilize the additive masking scheme to protect user parameters.The secret sharing technique is adopted to eliminate the masks such that global model can be aggregated correctly with privacy guarantee.Compared with the existing secret sharing based federated learning schemes in which participants need to share the new mask during each round of parameter aggregation,the participants in our scheme only need to share the mask once in the initialization phase of federated learning.In the subsequent learning process,participants utilize the secure masks reusing protocol proposed in this paper to safely and reliably reuse the same set of masks in the case of multiple rounds of parameter aggregation,and do not need to interact with other participants in the whole process.Therefore,our scheme greatly reduces the computation and communication overhead of participants,and its acceptable computation and communication costs make federated learning applicable to large-scale terminal clusters.In addition,targeted solutions are proposed for the collusion between some participants and cloud server in the actual deployment.Finally,we conduct comprehensive test experiments on real Io T terminal devices(smart phones and Raspberry Pi)to evaluate the performance of proposed schemes.Extensive experimental results are provided to validate the superiority of our proposed schemes in terms of the computation and the communication overhead while preserving the accuracy of the model and the privacy of the data.