Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-grained consistency learning, applied to every image patch and text token, both detects manipulated news media and points to the forged face or forged words, setting new state-of-the-art grounding results on the DGM4 benchmark.

desk verdict Solid fine-grained consistency framework for DGM4 with real grounding gains, though single-run results and a strong baseline confound the exact SOTA margin. read the letter →

arxiv 2506.05890 v1 pith:OQSSLYGU submitted 2025-06-06 cs.CV

Yiheng Li , Yang Yang , Zichang Tan , Huan Liu , Weihua Chen , Xu Zhou , Zhen Lei This is my paper
classification cs.CV
keywords multi-modalmediamanipulationDGM4consistencylearninggroundingdeepfakedetectioncontextualsemanticvision-language
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that fine-grained consistency, computed between every pair of image patches and between every patch or token and the other modality's global meaning, is what lets a detector both decide that a news item is manipulated and point to the forged face or forged words. This matters because existing DGM4 methods detect manipulation reasonably well but localize the manipulated content poorly, and the paper reports its largest gains exactly in grounding: over the prior best method, image localization IoU at the 0.75 threshold rises by 6.78 points and text-grounding F1 by 3.18 points, with binary and multi-label classification also improving. The claim, stated sympathetically, is that consistency is a learnable, supervisable signal that separates forged from genuine content at the patch and token level, and that using it to gate attention yields more reliable fine-grained predictions.

What carries the argument

The load-bearing object is the fine-grained consistency matrix. For within-modality context, Eq. 1 maps a pair of embeddings to a cosine similarity rescaled into $[0,1]$, giving an $n\times n$ matrix $M_{\text{pat}}$ (or $M_{\text{tok}}$) supervised by whether the two patches or tokens share the same source. For across-modality semantics, a global embedding of the other modality is produced by attention (Eq. 2) and each local embedding is compared with it (Eq. 3), producing an $n\times 1$ consistency vector supervised by the patch or token's genuineness. These matrices do double duty: a consistency loss (Eq. 5) supervises the construction itself, and the scores select reliable and suspicious subsets that a forgery-aware attention module mixes with residual connections; text grounding is decided directly by a threshold on the semantic consistency score, replacing a separate grounding head.

What would settle it

Take the trained CSCL model and recompute the semantic consistency matrix of Eq. 3 using a reference embedding built only from tokens or patches that the ground-truth mask marks as genuine, instead of Eq. 2's attention over all tokens; if image or text grounding metrics do not degrade relative to the paper's reported numbers, the claim that the all-token global reference drives the grounding gains is falsified.

Watch

Extended reading notes

Core claim

On the DGM4 task, the paper proposes Contextual-Semantic Consistency Learning (CSCL), a framework with two cascaded decoders per modality. The Contextual Consistency Decoder builds an $n\times n$ consistency matrix comparing every image patch with every other patch, and likewise every text token with every other token, supervised so that pairs from the same source score 1 and cross-source pairs score 0. The Semantic Consistency Decoder aggregates the other modality into one global embedding and scores each patch or token against it, supervised so that genuine content scores 1 and manipulated content scores 0. These matrices then guide forgery-aware reasoning and aggregating modules that attend selectively to the most reliable and the most suspicious content. The paper reports that this configuration reaches 96.34 AUC, 92.48 mAP, 84.07 IoUm, and 76.62 text F1 on DGM4, surpassing all prior methods and particularly improving the localization of manipulated content.

Load-bearing premise

The load-bearing premise is that the global embedding of the other modality, built by attending over all its patches or tokens, is a trustworthy reference for semantic consistency even when parts of that modality are themselves forged; if that reference is contaminated, genuine content can be scored as inconsistent and false positives rise.

Editorial extensions

If this is right

  • If the reported gains hold, fine-grained consistency supervision is a practical substitute for heavy task-specific decoders, since text grounding comes from thresholding a consistency score rather than training a separate token-level classifier.
  • Because the improvements are concentrated in localization, consistency learning may be the component that moves multi-modal forgery detection from judging whether media is fake toward showing where and what was faked.
  • Because the consistency matrices are supervised directly by manipulation masks, the same architecture trains end-to-end using the DGM4 labels without extra grounding annotations.
  • The ablation results indicate that contextual and semantic consistency are complementary: either one alone helps, and using both on the same modality gives the largest gains, with the benefit also appearing across different backbone encoders.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if consistency matrices are the active ingredient, the same construction could transfer to other multimodal forgery settings, such as audio-visual deepfakes or out-of-context images, wherever patch- or token-level ground truth can be derived from splice or replacement masks.
  • Beyond the paper: the threshold filter for text grounding suggests that grounding can be treated as a ranking problem on consistency scores, so a natural extension is calibrating the threshold per manipulation type or per news domain rather than using one global value.
  • Beyond the paper: Section 3.3 notes that the other modality may be partially forged while still using all of its fine-grained embeddings to build the global reference; a concrete test is to recompute semantic consistency using only known-genuine tokens and measure how much of the reported IoUm and F1 gain depends on that choice.
  • Beyond the paper: the pairwise $n\times n$ contextual matrix grows quadratically with the number of patches, so on higher-resolution images or longer documents one could test whether sampling patch pairs or using a low-rank approximation preserves the grounding gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents CSCL, a framework for detecting and grounding multi-modal media manipulation on the DGM4 benchmark. The method augments a METER-style multi-modal encoder with two cascaded decoders: a Contextual Consistency Decoder (CCD) that builds a within-modality pairwise consistency matrix over image patches and text tokens, and a Semantic Consistency Decoder (SCD) that scores each local embedding against a global embedding of the other modality. Both consistency matrices are supervised with BCE losses derived from the manipulation masks, and forgery-aware reasoning/aggregating modules are used to refine embeddings before the final classifiers. The authors report state-of-the-art results on DGM4, with particularly large gains on image and text grounding, and provide code and pretrained weights.

Significance. If the reported results are reliable, CSCL would be a meaningful advance for the DGM4 grounding task, and the release of code and weights is a concrete asset that makes independent verification feasible. The ablation studies in Tables 2-4 support the contribution of each individual module, and the backbone-generalization experiment in Table 3 is a strength. The central limitation is that the headline SOTA claim rests on single-run numbers without statistical analysis and is partly confounded by a stronger baseline implementation, so the magnitude of the claimed improvements is not yet established.

major comments (4)
  1. [§4.3, Tables 1 and 2] The headline SOTA claim is confounded by the baseline. Table 2 shows that the authors' own baseline without any consistency decoder already attains 81.21 IoUm and 73.79 F1, which is higher than Wang et al.'s published 80.83 IoUm and 73.44 F1 in Table 1. The reported gains of +3.24 IoUm and +3.18 F1 over Wang et al. therefore cannot be attributed solely to CSCL; part of the margin may come from the stronger base implementation or training recipe. Please provide a controlled comparison in which Wang et al. is re-run under the same backbone and training setup, or report the CSCL improvement over the same-code baseline together with confidence intervals.
  2. [§4.2 and Figs. 7-8] All headline numbers are single runs with no standard deviations, confidence intervals, or significance tests. The hyperparameters k_image=16, k_text=8, and threshold=0.5 are selected from the ablation curves in Figs. 7-8, but no validation split is described, so these choices appear to be tuned on the test data. Because the DGM4 test set is small and the reported SOTA gaps (e.g., +6.78 IoU75) concern a few points, the improvements could lie within sampling noise. Please report means and standard deviations over at least three seeds, use a fixed validation split for hyperparameter selection, and provide significance tests for the main comparisons.
  3. [§3.3, Eqs. (2)-(3)] The semantic consistency score is computed against a global embedding of the other modality that is aggregated from all fine-grained embeddings, including forged ones. The paper itself acknowledges in §3.3 that 'the content of another modality may be partially forged,' yet Eq. (2) uses all text tokens (and analogously all image patches) to form the global embedding. When the other modality contains manipulated tokens, the global embedding is contaminated, so genuine patches in the target modality may look inconsistent and be misclassified. This is a failure mode of the core mechanism, not a cosmetic issue. Please analyze this scenario (e.g., ablating a robust aggregation that excludes tokens flagged as forged by the contextual decoder) or provide evidence that the supervision in Eq. (5) prevents the contamination from hurting grounding.
  4. [§3.3-3.4, Eq. (5), Threshold Filter] The text-grounding mechanism is a direct supervised classifier in disguise. The semantic consistency matrix S_tok is trained with a BCE loss whose ground truth is the manipulation mask, and text grounding is obtained by thresholding S_tok. Thus the 'consistency score' is not an unsupervised measure of image-text agreement; it is a per-token binary classifier supervised by the same labels used for the task. The statement in §3.3 that 'we no longer need to provide additional prediction head' is therefore misleading. Please either acknowledge this supervision explicitly and discuss what the consistency formulation adds beyond a standard token classifier, or provide evidence that the score behaves as a consistency measure (e.g., on genuine but semantically mismatched pairs).
minor comments (6)
  1. [§3.4, Eq. (5)] The text says 'The loss L_m can be obtained by Eq. (5)' but the equation defines L_c; please align the notation.
  2. [Abstract and §4.1] The dataset description '230 image-text news pairs, including 77426 genuine pairs and 152574 manipulated pairs' is internally inconsistent; clarify whether there are 230 images or 230 image-text pairs, and how the two numbers relate.
  3. [Figs. 7-8] The horizontal axes in Figs. 7 and 8 are unlabeled; please add explicit axis labels and clarify whether the curves are computed on train, validation, or test data.
  4. [Table 2] The rows with two checkmarks are ambiguous because the caption does not specify which of C.I., C.T., S.I., S.T. are active in each row; please label each row or use explicit component names.
  5. [§3.3 and Abstract] There is a typo 'consistency sore' after Eq. (3) (should be 'consistency score'), and 'avaliable' in the abstract should be 'available'.
  6. [Table 1] The identifier 'Ours CVPR'25' is potentially confusing for an arXiv preprint; please use a neutral identifier or provide a formal citation if the paper has been accepted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the consistency scores that drive grounding are learned functions of embeddings, trained on labels but evaluated on held-out data, and the SOTA claims rest on external baselines and controlled ablations.

full rationale

The derivation chain is self-contained, and no load-bearing step reduces to its own inputs. The headline empirical claim — new SOTA on DGM4, especially for grounding (+3.24 IoUm, +2.13 IoU50, +6.78 IoU75, +3.18 F1 over Wang et al.) — is tested against externally published baselines (HAMMER, UFAFormer, Wang et al.) and is additionally supported by controlled ablations (Tables 2-4), so the central claim has independent empirical content, even though single-run numbers without error bars and a re-implemented baseline that already exceeds Wang et al.'s IoUm are comparability and statistical-significance risks that belong in correctness review, not in circularity review. The strongest apparent circularity candidate is the text-grounding rule: Section 3.3 defines S_tok as a cosine similarity between a token embedding and a global image embedding (Eq. 3), Section 3.4 supervises the semantic consistency matrix with BCE against the token-level manipulation mask (Eq. 5), and the Threshold Filter turns S_tok into the grounding decision. This is not circular: S_tok is a learned function of embeddings, not the ground-truth label, the BCE loss is ordinary supervised training of a token-level scoring function, and the reported F1 is computed on held-out DGM4 test data, so the prediction is not forced by construction. The one genuine caveat is that the Threshold Filter's 0.5 threshold is selected using the F1 curve in Fig. 8 on the evaluation data (a mild optimism bias), but the curve is flat near the peak (76.4-76.6 across thresholds 0.3-0.7), so this does not amount to a fitted parameter renamed as a prediction. Self-citations to works with overlapping authorship (UFAFormer [23], HFI-Net [32], F2Trans [33]) are used only as related work and competitors, not as load-bearing premises, uniqueness theorems, or ansatz authorities. No equation in the paper is equal to another by construction, and no prediction quantity coincides with a fitted input, so the honest verdict is no circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities; it constructs supervised consistency matrices from existing labels. The load-bearing inputs are the fine-grained ground-truth masks and the pretrained METER/ViT/RoBERTa features, plus three hand-tuned hyperparameters.

free parameters (3)
  • k_image = 16
    Number of most similar and most dissimilar image patches selected in forgery-aware reasoning; chosen via Fig.7 ablation on DGM4.
  • k_text = 8
    Number of most similar and most dissimilar text tokens selected in forgery-aware reasoning; chosen via Fig.7 ablation.
  • threshold_text_grounding = 0.5
    Threshold on semantic consistency score S_tok for marking text tokens as manipulated; selected in Fig.8 ablation.
assumptions (3)
  • domain assumption DGM4 ground-truth manipulation masks are accurate at both image-patch and text-token level.
    The consistency matrices and their losses in Section 3.4 are defined against these masks; any noise in masks propagates into training and evaluation.
  • domain assumption The global embedding of the other modality, computed by attention over all fine-grained embeddings (Eq. 2), is a usable reference even when parts of that modality are forged.
    Section 3.3 acknowledges partial forgery but still aggregates all embeddings; this assumption underlies the semantic consistency scores.
  • domain assumption Same-source consistency is a valid binary criterion for forgery (both manipulated or both genuine patches are consistent).
    Used to build ground-truth contextual matrices in Section 3.2; no justification that same-source pairs are always mutually consistent across manipulations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation." pith.science (2026). https://pith.science/paper/OQSSLYGU

@misc{pith2026250605890,
  author       = {Pith},
  title        = {Pith review of: Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OQSSLYGU}},
  note         = {Machine review of arXiv:2506.05890}
}
read the original abstract

To tackle the threat of fake news, the task of detecting and grounding multi-modal media manipulation DGM4 has received increasing attention. However, most state-of-the-art methods fail to explore the fine-grained consistency within local content, usually resulting in an inadequate perception of detailed forgery and unreliable results. In this paper, we propose a novel approach named Contextual-Semantic Consistency Learning (CSCL) to enhance the fine-grained perception ability of forgery for DGM4. Two branches for image and text modalities are established, each of which contains two cascaded decoders, i.e., Contextual Consistency Decoder (CCD) and Semantic Consistency Decoder (SCD), to capture within-modality contextual consistency and across-modality semantic consistency, respectively. Both CCD and SCD adhere to the same criteria for capturing fine-grained forgery details. To be specific, each module first constructs consistency features by leveraging additional supervision from the heterogeneous information of each token pair. Then, the forgery-aware reasoning or aggregating is adopted to deeply seek forgery cues based on the consistency features. Extensive experiments on DGM4 datasets prove that CSCL achieves new state-of-the-art performance, especially for the results of grounding manipulated content. Codes and weights are avaliable at https://github.com/liyih/CSCL.

Figures

Figures reproduced from arXiv: 2506.05890 by the authors.

Figure 1
Figure 1. Comparison of fine-grained feature process between our method and existing methods. (a) Previous methods [43, 44] adopt shallow and deep decoders to process embeddings for dif￾ferent sub-tasks. (b) Current SOTA methods [23, 47] conduct a unified multi-modal decoder for embeddings, but they ignore the consistency relationship between genuine and forged content. (c) Our method explores the consistency learning to achi… view at source ↗
Figure 2
Figure 2. The overall architecture of CSCL. CSCL can be divided into contextual consistency decoder and semantic consistency decoder. These decoders construct fine-grained consistency matrices and a use consistency loss for supervision. In each decoder, a forgery-aware reasoning or aggregating module is used to reduce the interference of confused content and deeply explore forgery cues. 2.2. Multi-modal deepfake detection Wit… view at source ↗
Figure 3
Figure 3. Histogram of genuine, forged and confused image patches. Genuine patches are marked by green , forged patches are marked by red (down right), and confused patch is marked by blue (down left). Observe the high-frequency range to determine consistency. text embeddings. There may be some distinctive informa￾tion that differs between the outputs of uni-modal encoders, which is the key clue to distinguishing authenticity… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: F1 scores of four manipulation types in fine-grained manipulation classification. FS, FA, TS, TA denotes face swap, face attribute, text swap, text attribute, respectively. ing 96.34% AUC, 92.48% mAP, 84.07% IoUm and 76.62% F1 on binary classification, multi-label clas…
Figure 5
Figure 5. Figure 5: Visualization of detection and grounding results. Here, red box and text indicate the prediction of manipulated faces and words, while green box and text represent the corresponding ground truth [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Visualization of fine-grained features distribution used in constructing consistency. (a) contextual consistency of an image, (b) semantic consistency of an image, (c) contextual consistency of text, and (d) semantic consistency of text. The forged content is marked by…
Figure 8
Figure 8. Figure 8: Threshold value selection in Threshold Filter. 4.5. Discussion The ability of distinguishing features corresponding to dif￾ferent source data is an important prerequisite for imple￾menting CSCL. As shown in [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Detecting AI-Generated Video: A Vision-Language Dual-View Survey

    cs.CV 2026-07 conditional novelty 6.0 of 10

    AIGC-V detection should be treated as factual fidelity verification and organized by a four-layer vision-language dual-view taxonomy spanning cues, motion, cross-modal consistency, and world-level reasoning.

Reference graph

Works this paper leans on

63 extracted references · 46 canonical work pages · cited by 1 Pith paper

  1. [1]

    Open- domain, content-based, multi-modal fact-checking of out- of-context images via online resources

    Sahar Abdelnabi, Rakibul Hasan, and Mario Fritz. Open- domain, content-based, multi-modal fact-checking of out- of-context images via online resources. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14940–14949, 2022. 3

    work page 2022

  2. [2]

    Exposing the deception: Uncover- ing more forgery clues for deepfake detection

    Zhongjie Ba, Qingyu Liu, Zhenguang Liu, Shuang Wu, Feng Lin, Li Lu, and Kui Ren. Exposing the deception: Uncover- ing more forgery clues for deepfake detection. InProceed- ings of the AAAI Conference on Artificial Intelligence, pages 719–728, 2024. 2

    work page 2024

  3. [3]

    Aligned and non-aligned double jpeg detection using convolutional neural networks.Jour- nal of Visual Communication and Image Representation, 49: 153–163, 2017

    Mauro Barni, Luca Bondi, Nicolo Bonettini, Paolo Bestagini, Andrea Costanzo, Marco Maggini, Benedetta Tondi, and Stefano Tubaro. Aligned and non-aligned double jpeg detection using convolutional neural networks.Jour- nal of Visual Communication and Image Representation, 49: 153–163, 2017. 2

    work page 2017

  4. [4]

    Audio-visual person-of-interest deep- fake detection

    Davide Cozzolino, Alessandro Pianese, Matthias Nießner, and Luisa Verdoliva. Audio-visual person-of-interest deep- fake detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 943–952,

    work page

  5. [5]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 3, 6

    arXiv 2010

  6. [6]

    An empirical study of training end-to-end vision-and-language transformers

    Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, et al. An empirical study of training end-to-end vision-and-language transformers. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18166–18176, 2022. 4, 6, 7

    work page 2022

  7. [7]

    Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks.Commu- nications of the ACM, 63(11):139–144, 2020. 1

    2020

  8. [8]

    Delving into the local: Dynamic in- consistency learning for deepfake video detection

    Zhihao Gu, Yang Chen, Taiping Yao, Shouhong Ding, Jilin Li, and Lizhuang Ma. Delving into the local: Dynamic in- consistency learning for deepfake video detection. InPro- ceedings of the AAAI Conference on Artificial Intelligence, pages 744–752, 2022. 3

    work page 2022

Show all 63 references
  1. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  2. [10]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 9729–9738, 2020. 2

    2020

  3. [11]

    Detection of fake images via the ensemble of deep representations from multi color spaces

    Peisong He, Haoliang Li, and Hongxia Wang. Detection of fake images via the ensemble of deep representations from multi color spaces. In2019 IEEE international conference on image processing (ICIP), pages 2299–2303. IEEE, 2019. 2, 7

    2019

  4. [12]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 1

    2020

  5. [13]

    Fighting fake news: Image splice detection via learned self-consistency

    Minyoung Huh, Andrew Liu, Andrew Owens, and Alexei A Efros. Fighting fake news: Image splice detection via learned self-consistency. InProceedings of the European conference on computer vision (ECCV), pages 101–117,

  6. [14]

    Bihpf: Bilateral high- pass filters for robust deepfake detection

    Yonghyun Jeong, Doyeon Kim, Seungjai Min, Seongho Joe, Youngjune Gwon, and Jongwon Choi. Bihpf: Bilateral high- pass filters for robust deepfake detection. InProceedings of the IEEE/CVF Winter Conference on Applications of Com- puter Vision, pages 48–57, 2022. 1

    2022

  7. [15]

    Multimodal fusion with recurrent neural networks for rumor detection on microblogs

    Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, and Jiebo Luo. Multimodal fusion with recurrent neural networks for rumor detection on microblogs. InProceedings of the 25th ACM international conference on Multimedia, pages 795– 816, 2017. 3

    2017

  8. [16]

    Countering malicious deepfakes: Survey, battleground, and horizon.International journal of computer vision, 130(7):1678–1734, 2022

    Felix Juefei-Xu, Run Wang, Yihao Huang, Qing Guo, Lei Ma, and Yang Liu. Countering malicious deepfakes: Survey, battleground, and horizon.International journal of computer vision, 130(7):1678–1734, 2022. 1

    2022

  9. [17]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. InProceedings of naacL-HLT, page 2. Minneapolis, Minnesota, 2019. 6

    2019

  10. [18]

    Mvae: Multimodal variational autoencoder for fake news detection

    Dhruv Khattar, Jaipal Singh Goud, Manish Gupta, and Va- sudeva Varma. Mvae: Multimodal variational autoencoder for fake news detection. InThe world wide web conference, pages 2915–2921, 2019. 3

    2019

  11. [19]

    Vilt: Vision- and-language transformer without convolution or region su- pervision

    Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision- and-language transformer without convolution or region su- pervision. InInternational conference on machine learning, pages 5583–5594. PMLR, 2021. 6

    2021

  12. [20]

    Align before fuse: Vision and language representation learn- ing with momentum distillation.Advances in neural infor- mation processing systems, 34:9694–9705, 2021

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learn- ing with momentum distillation.Advances in neural infor- mation processing systems, 34:9694–9705, 2021. 8

    2021

  13. [21]

    Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection

    Jiaming Li, Hongtao Xie, Jiahong Li, Zhongyuan Wang, and Yongdong Zhang. Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6458–6467,

  14. [22]

    Towards multimodal dis- information detection by vision-language knowledge inter- action.Information Fusion, 102:102037, 2024

    Qilei Li, Mingliang Gao, Guisheng Zhang, Wenzhe Zhai, Jinyong Chen, and Gwanggil Jeon. Towards multimodal dis- information detection by vision-language knowledge inter- action.Information Fusion, 102:102037, 2024. 1, 3, 6

    2024

  15. [23]

    Unified frequency-assisted trans- former framework for detecting and grounding multi-modal manipulation.International Journal of Computer Vision, pages 1–18, 2024

    Huan Liu, Zichang Tan, Qiang Chen, Yunchao Wei, Yao Zhao, and Jingdong Wang. Unified frequency-assisted trans- former framework for detecting and grounding multi-modal manipulation.International Journal of Computer Vision, pages 1–18, 2024. 1, 2, 3, 6, 7

    2024

  16. [24]

    Fka-owl: Ad- vancing multimodal fake news detection through knowledge- augmented lvlms

    Xuannan Liu, Pei Pei Li, Huaibo Huang, Zekun Li, Xing Cui, Weihong Deng, Zhaofeng He, et al. Fka-owl: Ad- vancing multimodal fake news detection through knowledge- augmented lvlms. InACM Multimedia 2024, 2024. 1

    2024

  17. [25]

    Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

    Yinhan Liu. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019. 3, 6

    1907 arXiv

  18. [26]

    Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5,

    Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5,

    arXiv

  19. [27]

    Newsclip- pings: Automatic generation of out-of-context multimodal media.arXiv preprint arXiv:2104.05893, 2021

    Grace Luo, Trevor Darrell, and Anna Rohrbach. Newsclip- pings: Automatic generation of out-of-context multimodal media.arXiv preprint arXiv:2104.05893, 2021. 3

    2021 arXiv

  20. [28]

    Gener- alizing face forgery detection with high-frequency features

    Yuchen Luo, Yong Zhang, Junchi Yan, and Wei Liu. Gener- alizing face forgery detection with high-frequency features. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 16317–16326, 2021. 6

    2021

  21. [29]

    Forensic similarity for digital images.IEEE Transactions on Information Forensics and Security, 15:1331–1346, 2019

    Owen Mayer and Matthew C Stamm. Forensic similarity for digital images.IEEE Transactions on Information Forensics and Security, 15:1331–1346, 2019. 3

    2019

  22. [30]

    Exposing fake images with forensic similarity graphs.IEEE Journal of Selected Topics in Signal Processing, 14(5):1049–1064, 2020

    Owen Mayer and Matthew C Stamm. Exposing fake images with forensic similarity graphs.IEEE Journal of Selected Topics in Signal Processing, 14(5):1049–1064, 2020. 3

    2020

  23. [31]

    Detecting gan- generated imagery using saturation cues

    Scott McCloskey and Michael Albright. Detecting gan- generated imagery using saturation cues. In2019 IEEE in- ternational conference on image processing (ICIP), pages 4584–4588. IEEE, 2019. 2

    2019

  24. [32]

    Hierarchical frequency-assisted interactive networks for face manipulation detection.IEEE Transac- tions on Information Forensics and Security, 17:3008–3021,

    Changtao Miao, Zichang Tan, Qi Chu, Nenghai Yu, and Guodong Guo. Hierarchical frequency-assisted interactive networks for face manipulation detection.IEEE Transac- tions on Information Forensics and Security, 17:3008–3021,

  25. [33]

    F 2 trans: High-frequency fine-grained transformer for face forgery detection.IEEE Transactions on Information Forensics and Security, 18:1039–1051, 2023

    Changtao Miao, Zichang Tan, Qi Chu, Huan Liu, Honggang Hu, and Nenghai Yu. F 2 trans: High-frequency fine-grained transformer for face forgery detection.IEEE Transactions on Information Forensics and Security, 18:1039–1051, 2023. 1

    2023

  26. [34]

    Self-supervised distilled learning for multi-modal mis- information identification

    Michael Mu, Sreyasee Das Bhattacharjee, and Junsong Yuan. Self-supervised distilled learning for multi-modal mis- information identification. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2819–2828, 2023. 3

    2023

  27. [35]

    Laa-net: Localized artifact attention network for quality-agnostic and generalizable deepfake de- tection

    Dat Nguyen, Nesryne Mejri, Inder Pal Singh, Polina Kuleshova, Marcella Astrid, Anis Kacem, Enjie Ghorbel, and Djamila Aouada. Laa-net: Localized artifact attention network for quality-agnostic and generalizable deepfake de- tection. InProceedings of the IEEE/CVF Conference on ...

    2024

  28. [36]

    Capsule-forensics: Using capsule networks to detect forged images and videos

    Huy H Nguyen, Junichi Yamagishi, and Isao Echizen. Capsule-forensics: Using capsule networks to detect forged images and videos. InICASSP 2019-2019 IEEE interna- tional conference on acoustics, speech and signal processing (ICASSP), pages 2307–2311. IEEE, 2019. 2

    2019

  29. [37]

    Avff: Audio-visual feature fusion for video deepfake detection

    Trevine Oorloff, Surya Koppisetti, Nicol `o Bonettini, Di- vyaraj Solanki, Ben Colman, Yaser Yacoob, Ali Shahriyari, and Gaurav Bharaj. Avff: Audio-visual feature fusion for video deepfake detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog...

    2024

  30. [38]

    Deepfake generation and detection: A benchmark and survey.arXiv preprint arXiv:2403.17881,

    Gan Pei, Jiangning Zhang, Menghan Hu, Guangtao Zhai, Chengjie Wang, Zhenyu Zhang, Jian Yang, Chunhua Shen, and Dacheng Tao. Deepfake generation and detection: A benchmark and survey.arXiv preprint arXiv:2403.17881,

  31. [39]

    Deepfake text detec- tion: Limitations and opportunities

    Jiameng Pu, Zain Sarwar, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed, and Bimal Viswanath. Deepfake text detec- tion: Limitations and opportunities. In2023 IEEE sympo- sium on security and privacy (SP), pages 1613–1630. IEEE,

  32. [40]

    Thinking in frequency: Face forgery detection by min- ing frequency-aware clues

    Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao. Thinking in frequency: Face forgery detection by min- ing frequency-aware clues. InEuropean conference on com- puter vision, pages 86–103. Springer, 2020. 2

    2020

  33. [41]

    Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsu- pervised multitask learners.OpenAI blog, 1(8):9, 2019. 1

    2019

  34. [42]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  35. [43]

    Detecting and grounding multi-modal media manipulation

    Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and grounding multi-modal media manipulation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 6904–6913, 2023. 1, 2, 3, 5, 6, 7

    2023

  36. [44]

    Detecting and grounding multi-modal media manip- ulation and beyond.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    Rui Shao, Tianxing Wu, Jianlong Wu, Liqiang Nie, and Zi- wei Liu. Detecting and grounding multi-modal media manip- ulation and beyond.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1, 2, 3, 5, 6

    2024

  37. [45]

    Learning on gradients: Generalized arti- facts representation for gan-generated images detection

    Chuangchuang Tan, Yao Zhao, Shikui Wei, Guanghua Gu, and Yunchao Wei. Learning on gradients: Generalized arti- facts representation for gan-generated images detection. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 12105–12114, 2023. 2

    2023

  38. [46]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 4, 5

    2017

  39. [47]

    Exploiting modality- specific features for multi-modal manipulation detection and grounding

    Jiazhen Wang, Bin Liu, Changtao Miao, Zhiwei Zhao, Wanyi Zhuang, Qi Chu, and Nenghai Yu. Exploiting modality- specific features for multi-modal manipulation detection and grounding. InICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (IC...

    2024

  40. [48]

    Noise based deepfake de- tection via multi-head relative-interaction

    Tianyi Wang and Kam Pui Chow. Noise based deepfake de- tection via multi-head relative-interaction. InProceedings of the AAAI Conference on Artificial Intelligence, pages 14548– 14556, 2023. 2

    2023

  41. [49]

    Eann: Event adver- sarial neural networks for multi-modal fake news detection

    Yaqing Wang, Fenglong Ma, Zhiwei Jin, Ye Yuan, Guangxu Xun, Kishlay Jha, Lu Su, and Jing Gao. Eann: Event adver- sarial neural networks for multi-modal fake news detection. InProceedings of the 24th acm sigkdd international confer- ence on knowledge discovery & data mining, pa...

  42. [50]

    Add: Frequency attention and multi- view based knowledge distillation to detect low-quality com- pressed deepfake images

    Simon Woo et al. Add: Frequency attention and multi- view based knowledge distillation to detect low-quality com- pressed deepfake images. InProceedings of the AAAI Con- ference on Artificial Intelligence, pages 122–130, 2022. 2

    2022

  43. [51]

    Luke: Deep contextualized entity representations with entity-aware self-attention.arXiv preprint arXiv:2010.01057, 2020

    Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, and Yuji Matsumoto. Luke: Deep contextualized entity representations with entity-aware self-attention.arXiv preprint arXiv:2010.01057, 2020. 6

    2010 arXiv

  44. [52]

    Unified contrastive learning in image-text-label space

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, and Jianfeng Gao. Unified contrastive learning in image-text-label space. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19163–19173, 2022. 2

    2022

  45. [53]

    Avoid-df: Audio-visual joint learning for detecting deepfake

    Wenyuan Yang, Xiaoyu Zhou, Zhikai Chen, Bofei Guo, Zhongjie Ba, Zhihua Xia, Xiaochun Cao, and Kui Ren. Avoid-df: Audio-visual joint learning for detecting deepfake. IEEE Transactions on Information Forensics and Security, 18:2015–2029, 2023. 1

    2015

  46. [54]

    Masked relation learning for deepfake detection

    Ziming Yang, Jian Liang, Yuting Xu, Xiao-Yu Zhang, and Ran He. Masked relation learning for deepfake detection. IEEE Transactions on Information Forensics and Security, 18:1696–1708, 2023. 3

    2023

  47. [55]

    Dynamic differ- ence learning with spatio-temporal correlation for deepfake video detection.IEEE Transactions on Information Foren- sics and Security, 2023

    Qilin Yin, Wei Lu, Bin Li, and Jiwu Huang. Dynamic differ- ence learning with spatio-temporal correlation for deepfake video detection.IEEE Transactions on Information Foren- sics and Security, 2023. 3

    2023

  48. [56]

    Bootstrapping multi-view rep- resentations for fake news detection

    Qichao Ying, Xiaoxiao Hu, Yangming Zhou, Zhenxing Qian, Dan Zeng, and Shiming Ge. Bootstrapping multi-view rep- resentations for fake news detection. InProceedings of the AAAI conference on Artificial Intelligence, pages 5384– 5392, 2023. 3

    2023

  49. [57]

    De- fending against neural fake news.Advances in neural infor- mation processing systems, 32, 2019

    Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. De- fending against neural fake news.Advances in neural infor- mation processing systems, 32, 2019. 1

    2019

  50. [58]

    Multi-attentional deep- fake detection

    Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu. Multi-attentional deep- fake detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2185– 2194, 2021. 6

    2021

  51. [59]

    Learning self-consistency for deepfake detection

    Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning self-consistency for deepfake detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 15023–15033, 2021. 2, 3, 4

    2021

  52. [60]

    A survey of deep facial attribute analysis.International Journal of Computer Vision, 128:2002–2034, 2020

    Xin Zheng, Yanqing Guo, Huaibo Huang, Yi Li, and Ran He. A survey of deep facial attribute analysis.International Journal of Computer Vision, 128:2002–2034, 2020. 1

    2002

  53. [61]

    Two-stream neural networks for tampered face detection

    Peng Zhou, Xintong Han, Vlad I Morariu, and Larry S Davis. Two-stream neural networks for tampered face detection. In2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pages 1831–1839. IEEE,

  54. [62]

    Multi-modal fake news detec- tion on social media via multi-grained information fusion

    Yangming Zhou, Yuzhou Yang, Qichao Ying, Zhenxing Qian, and Xinpeng Zhang. Multi-modal fake news detec- tion on social media via multi-grained information fusion. InProceedings of the 2023 ACM international conference on multimedia retrieval, pages 343–352, 2023. 3

    2023

  55. [63]

    Generalizing to the future: Mitigating entity bias in fake news detection

    Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Dand- ing Wang, and Fuzhen Zhuang. Generalizing to the future: Mitigating entity bias in fake news detection. InProceed- ings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, p...

    2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.