Research On Security And Privacy-Preservation Techniques In Machine Learning

Machine learning is one of the techniques of Artificial Intelligence.In recent years,machine learning models,especially neural network based deep learning models,are widely applied in the fields of medical care,image processing,and cyberspace security.Nevertheless,there are security and privacy threats in all phases of machine learning systems.Specifically,the main threats in the data collection phase are poisoning attacks,where an adversary adds some malicious data to the dataset to degrade the usability of the trained model;The main threats in the model training phase are the backdoor attacks,where an adversary maliciously manipulates the model training process to make the model show desired backdoor behavior of the adversary;The main threats in the phase of model prediction are adversarial example attacks,where an adversary generates an adversarial sample by adding small perturbations to a normal sample to induce a misclassification of the machine learning model;The main threats in the phase of model release are model stealing attacks,where an adversary reconstructs a substitute model that is as similar as possible to the released model by performing API queries.This severely infringes the intellectual property rights of the model owner.However,the research on these security and privacy threats is still in its infancy.This dissertation investigates security and privacy protection techniques in machine learning from four types of attacks against machine learning: poisoning attacks,backdoor attacks,adversarial example attacks and model stealing attacks.1.Research on security threats and techniques of poisoning attacks in the data collection phase.To address the problem that the existing poisoning attack schemes tend to fall into the local optimum and lead to low efficiency,this dissertation develops two efficient poisoning attack schemes.Specifically,to accelerate the convergence speed,this dissertation proposes a poisoning attack scheme using the momentum algorithm.In addition,to avoid falling into local optimum during the optimization process,this dissertation proposes a poisoning attack scheme using the Adam algorithm.Finally,this dissertation also proposes a defense method that uses Mahalanobis distance to detect poisoned samples.2.Research on security threats and techniques of backdoor attacks in the model training phase.To address the problem that existing work lacks research on backdoor attack for incremental learning,this dissertation designs a backdoor attack for incremental learning and explores the security of the incremental learning process.Specifically,this dissertation proposes a backdoor attack scheme that the backdoor stays dormant in the pre-trained model,and is only activated in the downstream model(also called student model)after an incremental learning process.The backdoor causes the downstream model to misclassify samples with backdoor triggers to the target label.The attack is highly effective and stealthy,and is able to evade existing backdoor detection methods.3.Research on security threats and techniques of adversarial example attacks in the model prediction phase.To address the problem that most of the existing adversarial example attacks only focus on the digital domain and become less effective in the physical world,this dissertation proposes a black-box adversarial example attack through data augmentation.Specifically,two attack algorithms are proposed to meet different attack objectives:Efficient-AATR employs a greedy strategy to generate adversarial samples with fewer queries;Effective-AATR employs an adaptive particle swarm optimization algorithm to search for the most effective adversarial samples within a given number of queries.The attack scheme can generate more natural and effective adversarial samples in the physical world.4.Research on security threats and techniques of model stealing attacks in the model release phase.To address the problem that the existing defense schemes against model stealing attacks are less effective,this dissertation proposes a comprehensive defense framework against model stealing attacks.Specifically,the defense framework consists of four closely connected phases: the defender first uses adversarial training to weaken the effectiveness of model stealing attacks;then,malicious query detection is used to distinguish malicious queries and flag malicious users;afterwards,the defender performs an adaptive response strategy against the malicious user’s query;finally,the adaptive response results can be used both as a model watermark to verify the ownership of any suspicious model and to further reduce the effectiveness of the model stealing attack(i.e.,reducing the performance of the model stolen by the attacker).This dissertation validates the effectiveness and robustness of the above schemes through extensive experiments,and demonstrates their superior performance by comparing them with existing works.