Research On Membership Inference Attack Defense Methods

The privacy of the training data of machine learning models faces the threat of membership inference attacks,where an attacker can infer whether the accessed data originates from the training data of the model through normal access to the model.There exist many defensive studies against membership inference attacks,which are mainly implemented in two ways: one is by reducing the overfitting degree of the model,and the other is by hiding the membership information.However,existing membership inference defense methods still suffer from large defense costs and the problem that privacy and utility cannot be well traded off.In this thesis,we conduct an in-depth study of membership inference attacks and defense methods to design efficient privacy-preserving schemes for model training data from the perspective of model training optimization and model structure optimization,respectively,with the goal of trade-off between model privacy and utility with limited defense costs.The main contributions of this thesis are summarized in the following two points:(1)This paper proposes a secure and effective training optimization approach,named Squeeze-Loss,which is designed by thoroughly studying privacy defense schemes against membership inference attacks.This approach aims to decrease the gap between training loss and test loss of the model.From the perspective of model training optimization,this approach first introduces a gradient ascent mechanism to reduce the overfitting of the machine learning model and decrease the loss gap between training data and test data.Additionally,a validation data set is introduced to detect whether the model is experiencing an overfitting trend.Once overfitting is detected,the gradient ascent mechanism is triggered to reduce the gap between the model’s losses on training and test data.Theoretical analysis and experimental verification show that this approach achieves privacy protection while maximizing the utility of the model.(2)This paper proposes a novel privacy-utility trade-off defense framework(KDDPSGD).To address the issue of model performance loss caused by differential privacy in defending against membership inference attacks,an effective solution is designed to mitigate this problem.The solution starts from the perspective of model structure optimization,combining knowledge distillation technology to achieve compression of model structure and parameters while ensuring the performance of the smaller model as much as possible.Next,differential privacy technology is applied to the compressed model.Under a quantified privacy budget,this solution can mitigate the performance loss caused by differential privacy to the model and achieve good privacy protection.To demonstrate the effectiveness of this framework on model privacy and utility performance,the privacy and utility of the defense framework are evaluated under various membership inference attacks.