Research On DeepFake Refinement And Detection

In recent years,deep learning has brought convenience to people’s daily life,but it has also inevitably hindered the progress of society.Deepfake technology is a typical example.It can arbitrarily modify images or videos to generate fake images that are difficult to distinguish between real and fake.Although this technology has brought a lot of progress to the field of digital image processing,some people have maliciously used this technology in the fields of politics,pornography,etc.,which has brought a lot of harm to the society.Therefore,this thesis focuses on the research of DeepFake forensics technology to help people judge the authenticity of images and ensure the safe application of DeepFake technology.In machine learning,data augmentation is an effective way to improve model performance.From the perspective of data augmentation,this thesis considers constructing difficult samples that are hard to identify by the detection models as new training data to help improve the performance of the detection model.Existing DeepFake images have obvious flaws.Detection techniques can easily identify their authenticity.Therefore,how to repair DeepFake images from multiple angles and construct difficult samples that are more difficult to detect is a challenge.To solve this problem,this thesis proposes three different types of fake image refinement methods from the perspectives of frequency domain,spatial domain and semantics,successfully achieving the goal of constructing difficult samples.In addition,due to the lack of localization methods in the existing forensics technology,this thesis also proposes a DeepFake detection and localization framework.Experiments show that the difficult samples constructed by the refinement methods can improve the performance of the model.In summary,the research contents and contributions of this thesis are as follows:· Spatial frequency domain signal refinement method of fake images based on shallow reconstruction In order to repair the flaws in the fake image spatial frequency domain,this thesis proposes a post-processing image spatial frequency domain signal refinement method.The main idea is to linearly project or sparsely encode DeepFake images based on a dictionary learned from real image datasets,thereby eliminating artifacts in DeepFake images.The method does not need to know any information about the network that generates the DeepFake images.Experiments on three state-of-the-art DeepFake detectors demonstrate the effectiveness of the method.· Spatial frequency domain refinement method of fake images based on deep predictve filtering network In order to repair the defects of the image spatial and frequency domain,as well as improve the efficiency and domain adaptability of the refinement method,this thesis designs a deep predictive filtering method to perform the image refinement from the spatial frequency domain.The main idea is to destroy the artifacts by adding noise to the DeepFake image,and then restore the image through a predictive filtering denoising method.This thesis also propose an intelligent noise-adding technique based on adversarial attacks,thereby reducing the damage to image quality.· Semantic refinement of fake Images based on adversarial learning In addition to the defects in the spatial frequency domain,the fake images may also have semantic error.Adversarial attack is a technique that generates fake images with incorrect semantics,thereby misleading the neural network.In order to repair the semantics of fake images,by studying additive-based and filtering-based adversarial defense denoising methods,we find that filtering-based methods have better defense performance under both image-level and semantic-level losses.In addition,considering that the filtering-based methods are not general to different attack strength,this thesis proposes a perturbation-aware filtering method to repair the semantics of images.Finally,this thesis proposes and validates the hypothesis that adversarial training and adversarial denoising can be combined to achieve better adversarial defense performance.· Bobust detection and localization method for fake images Due to the lack of localization methods in the existing forensics technology.In this thesis,a grayscale fake region localization map is proposed to represent the fake region.Then a general deepfake image detection and localization framework is proposed.In order to improve the generality of the model across face attributes,this thesis proposes an approach to introduce an attention mechanism into the framework using face parsing.In order to improve the cross-method generality of the model,this thesis proposes partial data augmentation and single-sample clustering methods.The detection and localization method is also robust to image degradation.