Research On Key Techniques Of Differentially Private Transfer Learning

Traditional machine learning usually has two basic assumptions:1)there are suffcient labeled data;2)training and testing data are independent and identically distributed.Transfer learning relaxes these two assumptions,which exploits knowledge present in labeled training data from a source domain to enhance a model’s performance in a target domain,which has little or no labeled target training data.However,in many real-world applications,the source and target domains may come from different institutes.Knowledge transfer or sharing between the two domains might pose a potential privacy risk when the data of the two domains contain sensitive information.Differential privacy proposed in recent years provides a feasible way to address the privacy preservation problem of transfer learning.Unlike the traditional privacy protection models,differential privacy offers a more robust privacy guarantee without making any assumptions about the adversary’s prior knowledge.An in-depth analysis of existing research on differentially private transfer learning reveals that the research is still in its infancy.This is mainly reflected in the fact that the assumptions made by the related research work are too idealistic and difficult to be satisfied in practical application scenarios.Therefore,this thesis conducts in-depth research on the differentially private transfer learning problem for three typical types of transfer learning,i.e.,domain adaptation,multi-task learning,and transfer reinforcement learning.This thesis makes the following contributions:(1)To domain adaptation,a differentially private correlation alignment approach for domain adaptation called PRIMA is presented,where privacy protection is achieved through adding perturbation to the exchanged information,i.e.,the covariance matrix of the target data,and the gradients during model training.In PRIMA,since perturbing the covariance matrix will inject a large amount of noise in high dimensions,a random subspace ensemble based covariance perturbation method is proposed,which splits the feature spaces of source and target data into several low dimensional non-overlapping subspaces.In particular,an optimization problem is formalized to determine the dimensions of the subspaces.Moreover,since perturbing the covariance matrix may destroy its positive semi-definiteness,a shrinking based method is developed for the recovery of positive semi-definiteness of the covariance matrix,which ensures that the utility of the recovered covariance matrix is better than the perturbed one.Furthermore,when perturbing the gradients during model training on each subspace,the private models with poor performance negatively impact the overall performance of the ensemble.To maximize the ensemble performance,an ordering-based ensemble pruning method is put forward to select a subset of all the private models to form the final pruned ensemble.Extensive experiments conducted on benchmark datasets confirm the effectiveness and efficiency of the presented approach.(2)To multi-task learning,a differentially private multi-task relationship learning approach called DRUPE is presented,where privacy protection is achieved through perturbing the gradients at each task node.In particular,the amount of noise added to the gradient increases as the data dimension increases,which makes the method of directly perturbing the gradient perform poorly in high dimensions.To solve the above problem,a joint feature selection and gradient sparsification method is proposed.This method first removes the specific features,and only features that are useful for all tasks are retained.Then,during stochastic gradient descent,the unimportant gradients are made zero,and noise is added(only to nonsparse gradients)to achieve differential privacy.Moreover,the noise added at each gradient update cumulatively increases the variance of the gradient,which leads to a slower convergence rate.For this reason,a variance reduction based gradient calibration method is developed,which first estimates the gradient error from the previous perturbed gradients and then calibrates the current perturbed gradient by subtracting the gradient error term.Furthermore,to alleviate the negative effect caused by the inaccurate task relationships that are inferred from the private task models,a task relationship calibration method is put forward,which uses the successive projection algorithm to calibrate the inaccurate pairwise relationships between the tasks.Experimental results on both synthetic and real-world datasets confirm the performance of the presented approach.(3)To transfer reinforcement learning,a differentially private intertask mapping approach for transfer reinforcement learning called DPTRL is presented,where privacy protection is achieved through adding perturbation to the exchanged information,i.e.,the target data and the source policy.In DPTRL,by introducing sparsity into random projection,a transfer function construction method is proposed under differential privacy,in which the sparsity degree and the optimal projection dimension of the projection matrix are determined by preserving the distances between the source and target state-action pairs as much as possible.Moreover,to reduce the negative effect of the target rewards that are perturbed during the private transfer function construction,a noisy reward correction method is developed to reduce the variance of bellman estimation and improve the performance of the final learned policy.Furthermore,during learning the source policy,an adaptive variance scaling method is put forward,which automatically adapts per-episode noise addition based on a derived optimal learning rate series to achieve higher performance.Extensive experiments are conducted to corroborate the efficiency of the presented approach.