Research On Face-swapped Videos Detection Based On MaskGAN

With the rapid development of the Internet,image tampering methods represented by Deepfake technology have developed rapidly.Various tampered images and tampered videos flood social media and news reports,seriously threatening media credibility,judicial justice,and even posing huge challenges to national security.However,on the one hand,the existing face-swapped video detection algorithms generally have the problem of poor generalization,and the accuracy rate drops significantly on datasets with unknown tampering methods.On the other hand,the current common deepfake video detection methods are feature detection,rarely using inconsistent feature information between video frames.In order to solve the problem of poor generalization of existing algorithms,this paper detects the ubiquitous facial fusion features in the face-changing process,and fuses pictures with similar facial landmarks to replace the face distortion in Deepfake,using generative adversarial networks.In the form of automatic selection of the best face-swap region,the face-swap image dataset with only fusion features is generated.Use this dataset to train the detection network to extract the fusion features.Further,for the video,this paper detects the fusion features and the interframe inconsistency features that are common in the face-changing process.First,align the picture with the face of each frame of the video,and then use the generative adversarial network to automatically select the facechanging area,and then filter out the noise by masking the facial information and background information to obtain a face-changing video data set with only fusion features.Use this video data set to train the detection network to extract fusion features and time series features in the video,so as to achieve tampering.detection.The main results of this paper include:(1)In view of the problem of poor generalization of existing algorithms,a face fusion algorithm based on MaskGAN and an image tampering detection algorithm based on DeeplabV3+are proposed to generate a face-changing image dataset containing only fusion features and use it to train a tampering detection model.The algorithm generator uses U-Net and sSE to extract features from face images,and realizes Mask generation and face fusion;the discriminator uses a convolution layer to discriminate the face-changing pictures generated by the generator to form MaskGAN.Then,the obtained face-changing pictures are used as training sets and input into the improved DeeplabV3+for training,so that the network can extract the fusion feature circle generated during the facechanging process from the face-changing pictures,so as to identify the authenticity of the face-changing images.This paper achieves accurate face swapping with only the introduction of fusion features,and generates a face swapping dataset with fusion labels.It solves the problems of over-fitting and poor generalization that exist in the existing algorithms,and after a large number of experimental data,it is proved that the algorithm in this paper can perform well in the case of unknown tampering methods,and the baseline algorithms perform poorly in FaceSwaps,Celeb-DF and Celeb-DF.Achieved 23.02%and 6.9%cross-domain AUC performance improvement,respectively.(2)In view of the problem that the existing fusion feature extraction algorithm cannot generate face-changing video,and thus cannot use interframe features,and the existing spatio-temporal feature detection algorithm only extracts the spatial features within the frame,a Delaunay triangulation-based Segmentation affine transformation-MaskGAN’s facechanging video generation algorithm and EfficientNet-LSTM-based continuous frame tampering detection algorithm generate face-changing videos that only contain fusion features and inconsistent features between frames,and use them to train detection models.The algorithm first uses Delaunay triangulation to divide the face into several small triangles,and then performs piecewise affine transformation on each small triangle to achieve smooth alignment of the two faces.face video.And by masking the facial information and background information,it is ensured that the generated face-swapping video only contains fusion features and interframe inconsistent features.Then,EfficientNet-b0 is used to extract fusion features,and LSTM is used to combine the extracted fusion features with temporal features,to achieve tamper detection without tampering with the data set for training,and verify the effectiveness of the method through experiments.