Research On Privacy-preserving Deep Learning Based On Homomorphic Encryptions

Deep learning has achieved great success in speech recognition,image processing,and other fields.Although deep learning provides new solutions for these applications,the training model of deep learning requires a large amount of data.Therefore,the service providers need to collect a large amount of participant data,and these data may contain sensitive information about companies or users,such as medical records,account information,business operation status,etc.The applications of these sensitive information in deep learning can easily lead to the leakage of sensitive data of enterprises or users.At the same time,with the increasing awareness of personal privacy protection of users and the continuous introduction of national information security laws and regulations,various fields pay more and more attention to the research of multi-participant privacy-preserving deep learning.At present,a lot of data in artificial intelligence exists in the form of data islands.In addition,the restrictions on data security applications and the constraints of laws and regulations aggravate the formation of data islands.Federated deep learning,as the mainstream technology of distributed collaborative computing,is a collaborative training and prediction model based on deep learning algorithms without data exchange by multi-source users to realize secure mining and utilization of multi-source island data value.Although federated deep learning can solve the collaborative computing paradigm of “raw data is not out of the domain,and data is available and invisible” and solve the problem of data islanding,it can still leak the privacy of user data.Aiming at the problems in existing privacy-preserving deep learning models or frameworks,this dissertation designs novel privacy-preserving deep learning schemes based on homomorphic encryption that applies to specific scenarios.The main works are as follows.(1)Aiming at the problems of image deep learning,such as privacy disclosure,uncomputable problem of some complex nonlinear functions in the ciphertext,training error in the training process,a novel privacy-preserving image classification deep learning scheme(PIDL)is proposed.In PIDL,two training classification methods(PIDLSC and PIDLSL)of deep learning are designed,that is,two groups of ciphertext activation functions and cost functions – sigmoid activation function + cross-entropy cost function of ciphertext or softmax activation function + log-likelihood cost function of ciphertext are used to construct two privacy-preserving deep learning methods,which realize deep learning training under ciphertext data.In this scheme,the error in the training process is reduced by improving the training mode of the sigmoid activation function.The scheme improves the ciphertext sigmoid activation function + ciphertext square error cost function in some original privacypreserving deep learning schemes for existing ciphertext activation function and ciphertext cost function,and deep learning training and classification for privacy protection are implemented based on Paillier encryption algorithm,protecting the privacy of training data and training model.Security analysis and performance evaluation demonstrate that,on the premise of ensuring the security and correctness of the scheme,although protocols have more rounds of interaction during servers,compared with existing schemes,the proposed scheme has lower communication costs and higher accuracy.(2)Aiming at the problems that some nonlinear functions in the deep learning model cannot be directly calculated under ciphertext or need to improve the calculation methods and the original multi-key deep learning model has low efficiency,a privacy-preserving deep learning scheme(PDLHR)with homomorphic re-encryption and secure computing toolkit is proposed.Firstly,a homomorphic re-encryption scheme based on BCP cryptosystem is presented.The scheme can convert ciphertexts under different public keys into ciphertexts under the same public key under the premise of guaranteeing homomorphism,which is more simplified than the existing re-encryption schemes based on BCP cryptosystem.To realize efficient ciphertext calculation,a secure computing toolkit is designed,and the ciphertext training processes such as ciphertext activation functions and ciphertext cost function are constructed.Compared with the prior works,PDLHR realizes efficient ciphertext training under multi-key deep learning,and protects the privacy of input data,training model and inference results.Security analysis and performance evaluation show that although the scheme has the problem of interaction rounds,it alleviates the interactions to a certain extent through packet invocation,and the encryption and decryption efficiency of the scheme is better than that of the original schemes.(3)Aiming at the problems that multi-participant ciphertext collaborative computing is difficult,and privacy preserving federated learning training has a slow convergence rate,a practical and efficient privacy-preserving federated deep learning scheme(PEPFL)is proposed.Firstly,a distributed El Gamal cryptographic scheme for federated learning is presented,which can solve the problem of multiple keys in federated learning.Then,a novel privacy-preserving federated learning framework is designed using momentum gradient descent(MGD)and convolutional neural network(CNN)as well as the designed cryptographic system.In this scheme,users first generate their public and private key pairs and send their public keys to the aggregation server.Then the aggregation server produces the joint public key and sends it to all users.These users encrypt their data under the joint public key through the proposed distributed El Gamal cryptography and upload them to trainers.Trainers train these ciphertext data in the local models,and then the aggregation server and the trainers cooperate to update the ciphertext momentums and ciphertext weights.Finally,the security analysis and performance evaluation of the proposed scheme are carried out.Results show that compared with existing schemes,our scheme has lower communication and computational costs while guaranteeing security,and ensuring higher efficiency of encryption and decryption.(4)Aiming at the problems of high communication costs,frequent failures,and lowquality of training data in federated learning in the Internet of Things(Io T)environment,a privacy-preserving federated deep learning scheme based on Elliptic Curve Cryptosystem with dynamic and fairness data is proposed.Firstly,a multi-key EC-El Gamal cryptographic system(MEEC)adapted to federated learning is proposed,although the encryption scheme needs to be encrypted under the same algebraic structure,it solves the problem of cooperative calculation of users with multiple keys in federated learning,reduces communication and computational costs,and improves the encryption efficiency.Secondly,multi-participant user dynamic quitting and joining algorithms are designed to prevent the influence of communication failure or user dynamic joining and exiting on model training or prediction.Finally,the security analysis and performance evaluation of the proposed scheme are carried out.Results show that,while ensuring security,the encryption and decryption efficiency of our scheme is lower than that of learning with the error(LWE)encryption algorithm,but better than that of other comparison schemes.At the same time,the total efficiency is superior to the comparison of several encryption schemes,and it also verifies that the selection of the training parameter threshold has a direct influence on the training accuracy.