Research On Deep Model Watermarking Under Multiple Attacks

We are in the era of big data.Every day,huge amounts of data are generated by business,scientific,and social activities taking place all around us.With data coming from sensors,streaming video,satellite and medical imagery,and from interactions with cloud computing,data-driven approaches to decision making are being applied in areas as diverse as medicine,business,advertising,and entertainment.Among numerous data-driven techniques,deep learning(DL)is the fastest-growing one,which has achieved great success in various fields.Training a good deep learning model(i.e.,deep model)requires high-quality labeled data,expensive computing resources and the wisdom of designers,which makes it have high business value and knowledge attributes.Therefore,the deep model can be regarded as the intellectual property(IP)of the model owner.To enable deep models to better empower various industries,protecting the IP of deep models has become a hot issue in the digital economy era.The deep model IP protection can be regarded as the extension of the problem of IP protection to the new carrier,namely deep models.Digital watermarking is a main technique for IP protection.Inspired by digital watermarking,deep model watermarking(i.e.,model watermarking)came into being,whose main purpose is to embed the watermark information in the deep model.After the model is forged or stolen by the attacker,the model owner can extract the previously embedded watermark from these illegal models,so as to verify the ownership.According to whether the internal information(such as model weight or model structure)of the illegal model can be obtained in the verification stage,model watermarking is further divided into two modes:white-box model watermarking and black-box model watermarking.As a new research field,model watermarking still faces many challenges:white-box model watermarking and black-box model watermarking are faced with the threat of ambiguity attacks and model extraction attacks,respectively;in addition,model watermarking also needs to resist adaptive attacks in order to carry out effective ownership verification in the practical scenario.In view of the above key scientific problems,this dissertation first studies how to resist ambiguity attacks in white-box model watermarking and how to resist model extraction attacks in black-box model watermarking.Then,this dissertation explores black-box model watermarking against adaptive attacks.Finally,this dissertation proposes a model watermarking algorithm which can resist multiple attacks.The main work and innovations are briefly described in four aspects as follows:1.White-box Model Watermarking Against Ambiguity AttacksAmbiguity attacks mean that the attacker attempts to use a forged watermark to pass the ownership verification,which causes forensic ambiguity.However,existing black-box model watermarking methods can not resist the ambiguity attack,and only a few white-box model watermarking methods can resist the ambiguity attack,but they have the limitation of use,that is,such methods need to change the network structure of the original model,which will lead to a performance degradation of the model.To this end,this dissertation first analyzes the reasons for the limitations of existing methods,and then proposes a practical white-box model watermarking algorithm against ambiguity attacks.This method is based on a novel passport-aware normalization,which is generally applicable to most existing model structures.Therefore,it causes no structure change in the target model.Compared with existing methods,the proposed method only needs to add another passport-aware branch for IP protection.This new branch is jointly trained with the target model but discarded in the inference stage.In the verification stage,the private passport-aware branch is added back for ownership verification.Extensive experiments verify the effectiveness of the proposed method in both image and 3D point cloud recognition models.2.Black-box Model Watermarking Against Model Extraction AttacksModel extraction attacks mean that even if only having access to the application programming interface(i.e.,API)of the target model,the attacker can still obtain a large number of high-quality labeled data by continuously querying the model API,and then train his own surrogate model to steal the function of the target model.Existing white-box model watermarking methods are difficult to resist model extraction attacks.Only a few black-box model watermarking methods consider model extraction attacks,but they only focus on the classification model.To remedy it,this dissertation proposes a novel model watermarking framework to protect the more valuable image processing model.Specifically,a special task-agnostic barrier is added after the target model,which embeds a unified and invisible watermark into its outputs.When the attacker trains one surrogate model by using such input-output pairs,the hidden watermark will be learned and extracted out from the outputs of the surrogate model,which can be used by the model owner to claim the ownership.In order to further enhance the ability of watermark extraction,a two-stage training strategy is designed in this dissertation.Experiments demonstrate that this method can resist model extraction attacks under different network structures and loss functions.In addition to protecting the IP of the deep model,this method can also be used for the IP protection of precious data for model training.3.Black-box Model Watermarking Against Adaptive AttacksAdaptive attacks mean that the attacker can make the model ownership verification failed with the knowledge of the corresponding model watermarking method.Taking the common backdoor-based black-box model watermarking method as an example,attackers can utilize anomaly detection or data pre-processing to inspect trigger pattern or remove the backdoor.It is found that most model watermarking methods are not robust under such adaptive attacks.In order to enhance the robustness against such adaptive attacks,a robust and invisible trigger generation method is proposed in this dissertation.Specifically,this method first extracts the structure of images,and embeds the watermark information into these structure areas to generate the trigger pattern.Because the image structure can keep its semantics unchanged during data pre-processing,the generated trigger pattern is inherently robust to data pre-processing.Then,this dissertation uses the deep network to embed this trigger pattern into the cover image in a invisible way to bypass anomaly detection.Extensive experiments demonstrate that the proposed trigger generation method is suitable for different datasets and network structures,and related model watermarking method is flexible for different application scenarios,such as ownership verification for a single user or multiple users.4.Black-box Model Watermarking Against Multiple AttacksBased on the above research,aiming at the image processing model with high commercial value,this paper proposes a black-box model watermarking method against multiple attacks,which is robust against both model extraction attacks and adaptive attacks.Based on the robustness analysis of model watermarking under multiple attacks,this paper finds that the fragility of previous algorithm is due to its assumption of "the whole image consistency",which will be destroyed if attackers use some common adaptive attacks such as data pre-processing.To address it,this paper proposes a novel consistency,namely "structure consistency",based on which a new deep structure-aligned model watermarking algorithm is designed.Specifically,the embedded watermarks are designed to be aligned with physically consistent image structures(such as edges or semantic regions).Experiments demonstrate that this method can resist both adaptive attacks and model extraction attacks.In addition,different physical structures can be used flexibly for different image processing tasks.