Deepfakes Detection Based On Invisible Noise Fingerprint

As the main information medium in social networks,video and images carry people’s pursuit in expressing themselves and recording life.In addition to taking photos and shooting video,netizens also synthesize fake videos and images through computer algorithms.Although these fake videos and images are of a certain entertainment,those about portrait synthesis may become a source of disinformation and violate the privacy rights of relevant users.Therefore,the research on face forgery,especially the technologies that use face-swap technology to generate fake faces,has important practical significance for network public opinion,security protection,and judicial evidence collection.Most of the existing studies are based on passive detection of tampering defects in forgery algorithms,which lacks generality.The detection algorithm will fail when the tampering algorithm is adjusted and improved accordingly.The active defense of the forgery algorithm is the latest research direction proposed.The current research work is carried out on the facial attribute editing model.For the face-swap algorithm,which is more threatening in the face forgery algorithm,there is still no effective defense strategies.Aiming at the shortcomings of existing research,to improve the accuracy of fake face detection and the effectiveness of defense measures,this study proposes a deepfakes detection method based on invisible noise fingerprints.The purpose is to solve the technical challenge of how to design an efficient implicit fingerprint noise addition network on the premise of ensuring image quality for the challenge of deepfakes detection.The main work of the thesis is as follows:(1)Proposing a new proactive deepfakes detection framework based on invisible noise fingerprints for the first time.Different from the existing proactive defense methods,the model framework can implicitly embed invisible noise fingerprint features into the face-swap model through training data without affecting the quality of the fake face generated by the face-swap model.Therefore,it is convenient to detect and trace the source of the generated fake face without being discovered by the attacker.(2)Designing a fingerprint noise-adding network based on the encoder-decoder structure.The noise-adding network is trained by the reconstruction loss function and the discriminative loss function,and the noise fingerprint feature is embedded into the high-level semantic features of the image by using the convolutional neural network.Therefore,the noise feature in the face data can be made invisible while ensuring that the noise fingerprint is invisible.It can be implicitly embedded into the face swapping network,thereby improving the detection performance of the deepfakes discriminator for the generated fake faces.(3)Designing a deepfakes detection network model based on the principle of generative confrontation.The real face,the fake face generated by the face-swap network,and the noised face synthesized by the noise-adding network are used as training data,and the high-level semantic features of the face are extracted through the convolutional neural network,and the corresponding discrimination is alternately trained using the principle of generative confrontation.network,noise-adding network and face-changing network,so as to realize the recognition of false portrait data containing noisy fingerprints.(4)Designing experiments to verify the robustness and generalization of the implicit noise fingerprint proposed in this study based on the data generated by the noise-added network.At the same time,compared with the existing noise addition methods,the robustness and superiority of the method in this study are verified by experiments.Finally,a prototype system for false portrait detection based on implicit noise fingerprints is implemented.