Deepfake Detection In Multi-Face Scenarios Using Multi-Feature Fusion

Thanks to the rapid development of the computer vision field,computer vision technology is now used in a variety of fields.People can use face recognition technology to authenticate people,and people can use computer vision technology to edit and modify animations and videos of people.However,the improper use of computer vision technology can also pose a threat to social life.The use of face forgery techniques to improperly replace the faces of people in videos can produce fake videos,and these newly created fake videos are highly deceptive.If there is no way to detect such fake videos well,it will pose a serious threat to social stability and personal privacy.Nowadays,most researchers in the field of deepfake detection focus on single face videos,while in real life,a video often contains multiple faces.Existing single-face detection methods are difficult to be directly applied to such multi-face scenarios,and thus they lack adaptability to realistic environments.In the multi-face scenarios,we need to consider multiple face images comprehensively,for which we propose a mean to fuse the multi-face data in the data pre-processing stage in this thesis.Based on the fusion of multi-face data,we consider the detection of artifacts in the multi-face scenarios from both local and global perspectives in this thesis.For this purpose,we propose the detection method that fuse deep and shallow features in this thesis.At the same time,face forgery means can lead to differences in the frequency domain between real face images and forged face images.In this thesis,we propose the method to combine frequency domain features for multi-face video detection,and fuse multi-face image data and frequency domain data separately during data pre-processing.In summary,the main contributions of this thesis are as follows:1.We propose a multi-face fake video detection method based on fusion of deep and shallow features.In order to be able to process multi-face videos,we first propose a method to fuse multi-face image data in the data pre-processing stage in this thesis.And based on this,in order to enable the model to perceive the artifacts from both global and local perspectives,we use VGG16 as the framework of the feature extractor and fuse the multi-level features extracted from it in this thesis.We conduct sufficient experiments to verify the performance and the rationality of the structure of the proposed model,and the experimental results prove that the proposed model can well solve the fake video detection problem in multi-face scenarios.2.We propose a multi-face fake video detection method combining frequency domain features.The forgery means can lead to differences in the frequency domain between real and fake face images.And in order to take full advantage of the differences between real and fake face images in the frequency domain,we consider the use of frequency domain features in this thesis and propose a fusion method for multi-face frequency domain data and image data.In order to better handle the frequency domain data and image data,we design the feature extractor specially in this thesis and fuse the two on this basis.The experimental results demonstrate that the proposed model can well solve the multi-face fake video detection problem under the condition of introducing frequency domain data.