Research On Key Technologies Of Privacy Security For Machine Learning Models

The accelerated advancement of artificial intelligence(AI)technology has led to its ubiquitous implementation across various technical domains,thereby establishing its status as an essential core technology.Nonetheless,AI technology frequently relies on the contributions of multiple users to supply vast quantities of data as the technical basis.Consequently,the development of secure and trustworthy AI models,safeguarding the privacy of the involved data,ensuring the robustness and reliability of the model functions,and guaranteeing the compliance and legality of the model have emerged as critical factors constraining the evolution of AI.Addressing the present security challenges and privacy risks encountered by these models,this study investigates the application of privacy computing techniques to protect key technological aspects within AI models,with a focus on model privacy,security,robustness,and legal compliance.In accordance with the comprehensive life cycle of AI models,this thesis primarily accomplishes the following tasks:1.Before model training:privacy preservation of training datasets.Machine learning(ML)models are vulnerable to many inference attacks including membership inference,model inversion,and attribute inference.Unfortunately,there lack of a general solution for model privacy preservation since previous works only focus on defending against specific inference attacks.In this work,we identify the crucial implicit cause for the success of inference attacks is that too much redundant and task-irrelevant private information is exposed to model training.By leveraging this important observation,we propose a new defense named least information learning(LIL)to protect model privacy,which to the best of our knowledge is the first general defense that is not restricted to any specific inference attack.Specifically,to achieve LIL,we propose a novel dataset selection method by leveraging the information theory-based representative dataset selection at the dataset level and model explanation-guided sample feature selection at the feature level.Finally,we empirically evaluate the proposed LIL on four popularly studied datasets and five model architectures.Comprehensive experimental results demonstrate that LIL-trained models can effectively mitigate the aforementioned inference attacks with negligible accuracy loss.We also further explore the impact of LIL on model robustness and find LIL improves the ML’s security against two commonly used model function attacks,i.e.,adversarial example and poisoning attack.More importantly,LIL significantly reduces model training time thanks to the reduced computational amount and faster convergence,benefiting from the proposed dataset selection.2.During model training:detection and defense of poisoning attack.Model poisoning attacks greatly jeopardize the application of federated learning(FL).The effectiveness of existing defenses is susceptible to the latest model poisoning attacks,leading to a decrease in prediction accuracy.Besides,these defenses are intractable to distinguish benign outliers from malicious gradients,which further compromises the model generalization.In this work,we propose a novel defense including detection and aggregation,named RECESS,to serve as a“vaccine”for FL against model poisoning attacks.Different from the passive analysis in previous defenses,RECESS proactively queries each participating client with a delicately constructed aggregation gradient,accompanied by the detection of malicious clients according to their responses with higher accuracy.Further,RECESS adopts a newly proposed trust scoring based mechanism to robustly aggregate gradients.Rather than previous methods of scoring in each iteration,RECESS takes into account the correlation of clients’performance over multiple iterations to estimate the trust score,bringing in a significant increase in detection fault tolerance.Finally,we extensively evaluate RECESS on typical model architectures and four datasets under various settings including white/black-box,crosssilo/device FL,etc.Experimental results show the superiority of RECESS in terms of reducing accuracy loss caused by the latest model poisoning attacks over five classic and two state-of-the-art defenses.3.During model application:model parameter security.Public intelligent services enabled by machine learning algorithms are vulnerable to model extraction attacks that can steal confidential information of the learning models through public queries.Though there are some protection options such as differential privacy(DP)and monitoring,which are considered promising techniques to mitigate this attack,we still find that the vulnerability persists.In this article,we propose an adaptive query-flooding parameter duplication(QPD)attack.The adversary can infer the model information with black-box access and no prior knowledge of any model parameters or training data via QPD.We also develop a defense strategy using DP called monitoring-based DP(MDP)against this new attack.In MDP,we first propose a novel real-time model extraction status assessment scheme called Monitor to evaluate the situation of the model.Then,we design a method to guide the differential privacy budget allocation called APBA adaptively.Finally,all DP-based defenses with MDP could dynamically adjust the amount of noise added in the model response according to the result from Monitor and effectively defends the QPD attack.Furthermore,we thoroughly evaluate and compare the QPD attack and MDP defense performance on real-world models with DP and monitoring protection.4.During model application:model function security.Network intrusion detection systems(IDS)are often considered effective measures to thwart cyber attacks.Currently,state-of-the-art(SOAT)IDSs are mainly based on machine learning(ML)including deep learning(DL)models,which suffer from their own security issues,especially evasion attacks by using adversarial examples.However,previous related studies mostly assume that the adversary knows the information of the target model more or less,and/or focuses on extracted features rather than the traffic sample itself,which causes the severe drawback on attack feasibility in real-world scenarios.In this thesis,we investigate a more realistic black-box scenario where the adversary can only morph the traffic sample,and obtain the results,i.e.,accepted or rejected,without other knowledge.We devise a practical black-box attack strategy that successfully leverages the model extraction and transfer attack to evade the detection of the target IDS.Then we implement an automatic framework called EvadeD to evaluate our proposed attack strategy by executing our evasion attack against the typical and SOAT IDSs.Experimental results show the effectiveness of our attack strategy in terms of successful evasion.5.After model deployment:machine unlearning.Recently users’ right-to-beforgotten is stipulated by many laws and regulations.However,only removing the data from the dataset is not enough,as machine learning models would memorize the training data once the data is involved in model training,increasing the risk of exposing users’ privacy.To solve this problem,currently,the straightforward method,naive retraining,is to discard these data and retrain the model from scratch,which is reliable but brings much computational and time overhead.In this thesis,we propose an exact unlearning architecture called ARCANE.Based on ensemble learning,we transform the naive retraining into multiple one-class classification tasks to reduce retraining cost while ensuring model performance,especially in the case of a large number of unlearning requests not considered by previous works.Then we further introduce data preprocessing methods to reduce the retraining overhead and speed up the unlearning,which includes representative data selection for redundancy removal,training state saving to reuse previous calculation results,and sorting to cope with unlearning requests of different distributions.We extensively evaluate ARCANE on three typical datasets with three common model architectures.Experiment results show the effectiveness and superiority of ARCANE over both the naive retraining and the state-of-the-art method in terms of model performance and unlearning speed.