Research On Blind Forensics Of Digital Image Content Tampering Based On Semantic-agnostic Features

Driven by the rapid development of artificial intelligence,new tampering technologies and applications such as intelligent face-swapping based on Deep Fake and image generation based on Chat GPT continue to emerge and harvest a large number of users.Tampered images about major news events,face forgeries spoofing celebrities,and computer-generated images at will may pose serious threats to information security,judicial forensics,and national security while providing unlimited imagination and creative space for public entertainment.For common image content tampering operations,i.e.,splicing,copy-move,and removal,etc.,digital image content tampering blind forensics can use the attribute features of the image itself to achieve image authenticity identification and tampering area location without any a priori knowledge of image processing,which has become a research hotspot in current image forensics.Based on convolutional neural networks,this paper investigates the blind forensics of digital image content tampering by exploiting the semantic agnosticism of content tampering features.The main works are as follows:(1)A method for image tampering localization based on semantic agnostic boundary features is proposed.Current image blind forensics methods based on tamper boundary localization rely on boundary branches simply stacked by convolutional layers,and cannot fully learn generic boundary cues detached from the semantic content of the image,making it difficult to accurately detect tampered boundaries.Based on this,this paper proposes a dual-stream network consisting of a region module and a boundary module to learn spatial features and boundary features,respectively.To identify semantic-agnostic boundary traces,the boundary module activates tampered boundary information from rich spatial features using edge gate components deployed on different levels of the region module.The Collaborative loss supervision scheme containing Sobel filter-based edge agreement loss and crossentropy loss are proposed for fine-grained guidance of boundary cues.Quantitative and qualitative experimental results show that the proposed method can more clearly locate the pixel-level boundaries between the tampered and non-tampered regions.(2)A method for image tampering localization with explicit interaction between tampering foreground and background is proposed.Current image tampering localization methods tend to focus excessively on the tampered foreground without sufficiently considering the semantic agnosticism of the connection between the tampered foreground and background,which leads to fuzzy predictions for localized regions that are closely related to the background semantics.Based on this,this paper proposes a mutual complementary network that can facilitate the flow of information between the tampered foreground and background.To solve the problem that existing methods do not pay attention to the background of the untampered region,two independent encoders with preprocessing layers are designed to extract the tampered foreground features and the background features,respectively.To fully facilitate the information interaction between the tampered foreground and background,the mutual attention mechanism based on self-feature attention and cross-feature attention is designed to complement and strengthen the ability to characterize tamper boundaries and local regions that fit the semantics.The cooperative loss-supervision scheme based on the tampered foreground and background is used to further facilitate the separation of the tampered foreground and background during model training.Quantitative and qualitative experimental results show that the proposed method can significantly improve tampered foreground prediction and background prediction,and enable complete tampered region localization in image scenes with complex content semantics.(3)A method for image tampering localization based on uncertainty constraints is proposed.Existing image blind forensics methods mostly treat each pixel in the image equally during the model learning process,causing them to potentially generate high probability of false detections for pixel regions that undergo post-processing such as compression,blurring,etc.,to mask traces,especially in cross-dataset scenarios.Based on this,this paper proposes an uncertainty-constrained parallel network,which mitigates the interference from the semantic content of the image by discarding the shallow features in feature learning,and utilizes the uncertainty-constrained idea to strengthen the model’s learning of false detection pixels.Attention-guided partial decoders in different input modes are designed to efficiently integrate tampered edges and tampered semantics by discarding shallow coded features that contain the semantic structure of the image and adaptively modeling the spatial and channel dimensions of intermediate and deep coded features.Uncertainty constraint-based total structure loss is proposed to amplify the loss weights on misdetected pixel regions by dynamically updating the uncertainty prediction.Quantitative and qualitative experimental results show that the proposed approach can achieve leading performance in tampering localization accuracy,acrossdataset generalization,and robustness against post-processing attacks.(4)A method for image tampering detection and localization that can remove semantic features incrementally is proposed.The latest image blind forensics methods tend to directly assume that an image is tampered with,employing an aggregation-based attention mechanism to overlearn the limited tampering space without being able to fully reveal the difference between the real image and the tampered image,which makes them ineffective for detecting real images in Internet scenarios.Based on this,this paper proposes a progressive subtraction network that can perform tamper detection and localization tasks simultaneously by adaptively mining the hidden tampering traces under the content semantics.To address the problem that the property of convolutional neural networks that tend to learn the semantic content of images introduces semantic noise for generic tampering features,the semantic-agnostic manipulation attention consisting of the multi-scale feature iterative fusion module,the multi-kernel feature fusion residual module,and the subtraction operation is designed.Among them,the subtraction operation is used to explicitly remove the semantic features related to the image content learned by the multi-scale feature iterative fusion module and the multi-kernel feature fusion residual module from the encoded features.Image-level and pixel-level losses are supervised to facilitate the network’s simultaneous learning of image authenticity differences and statistical differences within tampered images.Quantitative and qualitative experimental results show that the proposed method can promote both accuracy in detecting image authenticity and robustness in localizing tampered regions.In summary,for the problems that still exist in current digital image blind forensics research in terms of image tampering localization and image authenticity identification,from the perspective of capturing semantically agnostic features such as tamper boundary features,complementarity features,and uncertainty features,this paper provides in-depth research on locating tampering boundaries,reducing false predictions,improving the generalization across datasets,and constructing detection and localization models,etc.,which can positively promote the development of multimedia information security and related technologies,and further improve the reliability of image forensics related applications.