Research On Auditing Label Differentially Private Deep Learning

With the rapid development of the big data industry,the need to ensure data security and Exploit data value has become increasingly urgent.Differentially private deep learning algorithm has therefore been widely studied in academia and applied in industry.By adjusting the privacy budget,differential privacy controls the randomness introduced,and realizes the trade-off between algorithm utility and data privacy.However,since differential privacy provides the worst-case privacy guarantee to individual privacy,it actually provides more privacy protection than the theoretical value,which causes troubles in the value of the privacy budget.Hence,privacy auditing is often used to measure the real privacy protection provided by the algorithm,and provide guidance for the selection of the parameter.Label differentially private deep learning,which is suitable for label-only privacy scenarios and has better utility while achieving privacy protection,has received extensive attention in recent years.In practical applications,there are also problems such as difficulty in choosing the privacy parameter,and unclear privacy protection.At present,Great researches have been achieved in privacy auditing methods.They take poisoning attacks and membership inference attacks as auditing means to approach the worst-case assumption of label differential privacy.However,most of these methods are designed for DP-SGD(Differential Privacy Stochastic Gradient Descent).When they are directly applied to label differentially private deep learning,two problems will be encountered: one is that they need to consume a lot of time to get the final auditing result;the other is that they have poor migration effect or unable to adapt to label differentially private deep learning.Therefore,this thesis aims to study efficient and accurate label differentially private deep learning auditing methods.The main work of this thesis is as follows:(1)Aiming at the high time-consuming problem caused by repeated simulation statistics in privacy auditing,this thesis takes the worst-case assumption of label differential privacy as an entry point,and proposes three core algorithm components of the privacy auditing architecture.Combined with the formula relationship between the sample pair judgment accuracy and the lower bounding of privacy budget in the theoretical proof,this thesis proposes a heuristic privacy auditing architecture of one-time model training and repeated experimental judgment,which greatly saves time costs.(2)Aiming at the problem of few and ineffective auditing methods for label differentially private deep learning,this thesis analyzes the worst-case assumption of label differential privacy,and on the basis of the given heuristic privacy auditing architecture,the core algorithm components are specifically designed to make them as close as possible.Firstly,this thesis proposes an auditing method based on label poisoning,which constructs adjacent data sets close to the worst case.Secondly,an auditing method based on shadow model is proposed,which constructs the attacker close to the worst case.Finally,combining the two methods,an auditing method combining label poisoning and shadow model is proposed.Experiments show that the heuristic auditing framework proposed in this thesis has better time utility while adapting to most audit methods.And the three auditing methods proposed in this thesis can improve the accuracy of auditing results.Due to the dual considerations of time utility and metric utility,the auditing method combining label poisoning and shadow model is the optimal solution among the three methods.Finally,the open-source implementation of the auditing methods in this thesis can be found on Git Hub: https://github.com/Cloudness Zhang/Auditing＿Label Private＿Deep Learning.