Pith. sign in

REVIEW 3 major objections 6 minor 80 references

MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MADBench is the first audio deepfake benchmark that treats speech and background audio as independent manipulable components, and its results show that fake background audio is easier to detect than synthetic speech while also degrading…

desk verdict A genuinely new component-aware audio-visual benchmark with a thoughtful design, but the real/fake component asymmetry lets detectors shortcut on separation artifacts, so the headline findings are provisional until a control is added. read the letter →

arxiv 2608.09593 v1 pith:ZF2XZCEL submitted 2026-08-10 cs.SD cs.AI

Yanqiu Li , Yang Xiao , Jisheng Bai , Bin Chen , Hong Jia , Ting Dang This is my paper · ORCID
classification cs.SDcs.AI
keywords audiodeepfakedetectionenvironmentalspeechsynthesiscomponent-levelevaluationaudio-visualbenchmarksceneconsistencymultimodallargelanguagemodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that audio deepfakes in real video should be evaluated as two independent acoustic components rather than one undifferentiated stream: the speech track and the environmental soundscape, each with its own generative pipeline and artifact profile. It introduces MADBench, a benchmark built from real videos with a fixed visual stream, in which the speech component, the background component, or both are replaced by generated audio, and every sample is labeled accordingly. Using this testbed, the paper shows that general-purpose audio-visual encoders detect manipulated background audio more reliably than synthetic speech, that existing pretrained deepfake detectors operate near chance on both components, and that a fake background track degrades detection of fake speech. The purpose of the benchmark is to make these cross-component effects visible, rather than collapsing all audio manipulation into a single real/fake label.

What carries the argument

The load-bearing construction is the modality-component hierarchy: each video's audio stream is decomposed into a speech stem $\hat{s}$ and an environmental residual $\hat{e} = x - \hat{s}$ using speech separation, with mixture consistency requiring the components to sum back to the original waveform. Fake speech is produced by same-identity and cross-identity TTS and by voice conversion; fake environmental audio is produced by text-to-audio, video-to-audio, and audio-to-audio generators, under scene-matched and scene-mismatched conditions. Assembly re-mixes real speech with real environmental audio through the same RMS-normalization pipeline used for fake samples, so the four labels R/S/E/Q differ only in which acoustic component was manipulated. This controlled construction is what lets the benchmark attribute detection differences to the speech or environmental component rather than to visual or assembly differences.

What would settle it

Train or run any detector on the task of distinguishing original unseparated real clips from the benchmark's reassembled real clips (real speech stem plus real environmental residual re-mixed through the same pipeline). If it exceeds chance, separation artifacts are present and the construction is asymmetric; applying the same check with fake components passed through an identical separation step would test whether the reported environmental-audio advantage survives when both sides see the same processing.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that component-level annotation changes what audio deepfake detection can measure. From each source video it constructs four sample types—real, speech-fake, environment-fake, and joint-fake—with the visual stream held fixed, and reports that pretrained audio-visual deepfake detectors transfer near chance (mean binary AUC 0.518–0.534 even after fitting new heads), while frozen general-purpose audio-visual encoders reach above 0.91 AUC for any-fake detection and support four-way R/S/E/Q attribution. It further reports that environmental audio manipulation is consistently easier to detect than speech manipulation across encoders and protocols, including scene-matched ones, and that fake environmental audio interferes with speech-specific detection (interference gap up to 0.138) while the converse does not hold. The paper interprets these results as evidence that environmental audio is a distinct forensic domain whose manipulation is obscured when benchmarks use a single audio label.

Load-bearing premise

The benchmark's real audio components come from a separation model, and only the fake components are generated cleanly; if that separation leaves detectable artifacts in the real stems, the measured differences between real and fake components could reflect the separation pipeline rather than the forgery itself.

Editorial extensions

If this is right

  • Pretrained audio-visual deepfake detectors should not be assumed transferable to authentic-video, component-level forgeries; with native scores or fitted heads they remain near chance (mean AUC below 0.54).
  • Frozen general-purpose audio-visual encoders are strong baselines for this setting, reaching above 0.91 AUC for any-fake detection and macro-F1 up to 0.745 for four-way R/S/E/Q classification.
  • Manipulated environmental audio is a distinct and more detectable forgery source than synthetic speech, so background-audio synthesis deserves its own evaluation axis and countermeasure research.
  • Fake background audio can obscure speech-specific forgery cues while the converse does not, so single-label benchmarks overestimate detector robustness in realistic jointly manipulated clips.
  • Audio alone, not the visual stream, carries the strongest direct forensic signal for manipulation labels; video's main role is scene-consistency verification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims: if the asymmetry holds in the wild, background-audio manipulation is the more dangerous attack surface—it is easier to spot, yet it can hide a simultaneous voice forgery, so practical detectors should flag suspicious ambience even when speech sounds clean.
  • Because the real samples are re-assembled from separated stems while fake components are not re-separated, a detector that exploits separation artifacts could inflate the reported gaps; this could be tested by separating the fake components too or by training on the original unseparated mixes.
  • The near-chance zero-shot omni-model results suggest that component-level attribution requires explicit forensic supervision or component-conditioned prompting, both of which MADBench's labels now make trainable.
  • The scene-matched/mismatched taxonomy is a tunable difficulty axis: future generators could be prompted with the fine-grained scene labels to produce forgeries that close the gap between synthetic and real ambience, stress-testing the benchmark's hardest settings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces MADBench, a benchmark for audio deepfake detection in authentic-video settings where speech and environmental audio are treated as separate forgery components. The dataset is built from AVSpeech clips, with speech separated using MossFormer2 and the residual treated as environmental audio; fake speech is generated via TTS and voice conversion, and fake environmental audio via TTA, VTA, and ATA generation, producing real, speech-fake, environment-fake, and joint-fake variants under scene-matched and scene-mismatched protocols. The authors evaluate pretrained A-V detectors, frozen A-V encoders, audio-only baselines, and zero-shot omni models, reporting that pretrained detectors and omni models perform near chance, frozen A-V encoders transfer well, environmental manipulation is easier to detect than speech, and fake environmental audio asymmetrically degrades speech detection.

Significance. If the construction asymmetry were removed, MADBench would fill a genuine gap: no existing benchmark independently controls speech and environmental audio over authentic video, and the component-level protocol plus leakage controls (speaker-level splits, fixed video, balanced generation) are thoughtful. The finding that frozen A-V encoders provide transferable component-level cues while task-specific detectors do not is potentially useful, and the paper's detailed QC and leakage controls are strengths. However, the central empirical claims are currently undermined because real components are MossFormer2 separation outputs while fake components are clean generations, so reported differences may reflect separation artifacts rather than manipulation. The paper is therefore not yet a rigorous foundation for component-aware audio deepfake detection, but the issue appears addressable with a substantial revision.

major comments (3)
  1. [Section 3.2 and 3.5] The construction treats real and fake components asymmetrically. In Section 3.2, the real speech component is MossFormer2's output s_hat and the real environmental component is the residual e_hat = x - s_hat. In Sections 3.3 and 3.4, fake speech and fake environmental components are generated cleanly and are never passed through the same separation model. Consequently, every real component is a separation output while every fake component is not, so a detector can separate real from fake by detecting the presence or absence of separation artifacts (e.g., musical noise, spectral holes, leakage) rather than manipulation. The QC checks described in Section 3.2 (ASR coverage, VAD/ASR absence of speech, minimum energy) rule out gross speech leakage but do not rule out artifact-level shortcuts. Because this asymmetry affects every real-versus-fake comparison in Tables 2 through 5, the validity of MADBench as a benchmark and the specific findings (e.g., the environmental advantage ΔE−S and the interference gaps G_S in Table 3) rest on an untested assumption.
  2. [Section 3.5] The paper states that “all sample types share an identical construction process, so that labels reflect component differences rather than assembly differences,” but this is inaccurate. The mixing procedure is identical, but the component-generation process is not: real components come from MossFormer2 separation, while fake components are clean generations. The claim should be revised, and the authors should either (a) pass fake components through the same separation pipeline so that both classes receive identical processing, or (b) provide a control experiment showing that detectors cannot distinguish real from fake on the basis of separation artifacts alone (e.g., train a detector to separate the residual e_hat from clean generated environmental audio, or s_hat from clean TTS, and show that this detector does not transfer to MADBench). Without such a control, the paper's central claims are not supported.
  3. [Table 3 and Section 5.2] The reported environmental-audio advantage and interference gap G_S are confounded by the construction asymmetry. For example, G_S compares AUC(S vs. R) with AUC(Q vs. E). In S vs. R, both groups contain real (separated) environmental audio; in Q vs. E, both groups contain fake (clean) environmental audio. The detector's ability to exploit the presence or absence of separation artifacts differs between these two set pairs, so the gap does not isolate interference from manipulation. A positive G_S could simply reflect the model using the environmental component's processing history as a shortcut. The authors should re-analyze these quantities on a sympathetically processed subset before making the interference claim.
minor comments (6)
  1. [Table 3 caption] The table header does not clearly separate the columns for ΔE−S and G_S/G_E; the S and E rows contain different numbers of numeric entries, making it hard to determine which value belongs to which metric. Please reformat the header and clarify the caption.
  2. [Section 3.3] The description that fake speech is “placed into the original speech regions, with non-speech regions left silent” creates another asymmetry: the real speech stem from MossFormer2 may contain residual noise outside speech regions, while the fake speech component is silent there. Although mixing with an environmental component may partially mask this, it should be acknowledged and controlled.
  3. [Section 3.4] For scene-mismatched VTA and ATA samples, the paper states that the generated track is replaced with one from an incompatible scene rather than being generated for that mismatch. Please clarify whether the generation-branch balance reported in Table 1 reflects the final samples or the initially generated components, since the replacement changes the effective branch distribution.
  4. [General] The paper does not state where the dataset and benchmark code will be released. Please include a repository or availability statement for reproducibility.
  5. [Table 5] The text notes that the evaluated AudioMosaic checkpoint is fine-tuned on the EnvSDD TTA split, which is important context. Please add this information to the table caption so that readers do not mistake it for a frozen pretrained model.
  6. [Section 6] The conclusion states that MADBench establishes a rigorous foundation, but given the construction asymmetry discussed above, this claim is too strong for the current version. A limitation paragraph should be added, or the claim should be softened.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: MADBench is a constructed empirical benchmark whose findings are measurements on a new dataset, not quantities forced by definition or by a self-citation chain.

full rationale

The paper does not attempt a derivation that reduces to its own inputs. Section 3.2 defines the environmental component as the mixture-consistency residual e-hat = x - s-hat from MossFormer2, which is a dataset-construction choice rather than a predicted quantity. Section 3.5 states that the real sample is reassembled from separated components through the same mixing procedure as fake samples, so differences between real and fake samples are at least not introduced by the final mixing step. The asymmetry that real components are separation outputs while fake components are cleanly generated is a real construct-validity threat that could let detectors exploit separation artifacts as shortcuts, but it does not make any reported AUC or classification result equivalent to an equation by construction, nor does it rename a fit as a prediction. Fitting logistic-regression heads on the training split and calibrating on validation is standard evaluation practice, not circularity. The self-citations to EnvSDD and related challenge papers appear only as related work and are not load-bearing for the central first-benchmark claim. No circular step meeting the quoted-reduction standard was found.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The benchmark rests on empirical construction choices rather than a mathematical derivation. The main load-bearing assumptions are the fidelity of speech separation, the validity of the scene taxonomy, and the representativeness of the chosen generators.

assumptions (4)
  • domain assumption MossFormer2 separation yields speech and environmental components that are faithful to the original mixture and free of systematic artifacts.
    Section 3.2 defines the real components via separation; if separation introduces artifacts, real components differ from fake components for reasons unrelated to forgery.
  • domain assumption The visual-scene taxonomy, built with Qwen2.5-VL summaries and manual verification, correctly captures which environmental sounds are consistent with each visible scene.
    Section 3.4 uses the taxonomy to define scene-matched and scene-mismatched conditions; incorrect labels would invalidate scene-consistency conclusions.
  • domain assumption The generated fake speech and fake environmental audio are representative of real-world audio deepfakes.
    Sections 3.3 and 3.4 select a set of six speech generators and three environmental generation paradigms; generalization beyond these generators is assumed.
  • domain assumption QC criteria (ASR coverage, VAD, CLAP matching) are sufficient to ensure the generated components preserve the intended content and are free of speech/music leakage.
    Sections 3.3 and 3.4 rely on these automatic checks; thresholds are not fully quantified in the text.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection." pith.science (2026). https://pith.science/paper/ZF2XZCEL

@misc{pith2026260809593,
  author       = {Pith},
  title        = {Pith review of: MADBench: A Benchmark for Modality-Aware Audio Deepfake Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZF2XZCEL}},
  note         = {Machine review of arXiv:2608.09593}
}
read the original abstract

Recent advances in speech synthesis and audio generation have made high-fidelity acoustic forgery low-cost and difficult to attribute, enabling a realistic attack scenario in which speech and background audio are independently manipulated over otherwise authentic video. Yet existing research either focuses on visual manipulation, addresses speech detection in isolation, or conflates speech and non-speech audio as a single undifferentiated audio stream, overlooking the distinct forensic challenges posed by background audio. This conflation is consequential: the two acoustic components arise from fundamentally different generative mechanisms, exhibit distinct artifact profiles, and pose different challenges to detection systems. We introduce MADBench, the first benchmark that treats speech and environmental audio as distinct acoustic components, enabling component-aware evaluation of audio deepfake detection across independently manipulated forgery sources. We benchmark representative state-of-the-art detectors and multimodal large language models under a unified protocol. Our experiments reveal that environmental audio manipulation is more detectable than synthetic speech across general-purpose encoders, while existing pretrained detectors fail on both acoustic components, and manipulated environmental audio asymmetrically degrades speech deepfake detection, findings entirely invisible under the single-label paradigm of prior benchmarks. MADBench establishes a rigorous foundation for future research into robust, component-aware audio deepfake detection.

Figures

Figures reproduced from arXiv: 2608.09593 by the authors.

Figure 1
Figure 1. Overview of the MADBench dataset construction and benchmark evaluation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

80 extracted references · 55 canonical work pages

  1. [1]

    Proceedings of the IEEE/CVF international conference on computer vision , pages=

    Faceforensics++: Learning to detect manipulated facial images , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages=

  2. [2]

    arXiv preprint arXiv:2006.07397 , year=

    The deepfake detection challenge (dfdc) dataset , author=. arXiv preprint arXiv:2006.07397 , year=

    arXiv 2006

  3. [3]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Celeb-df: A large-scale challenging dataset for deepfake forensics , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    work page

  4. [4]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Jiang, Liming and Li, Ren and Wu, Wayne and Qian, Chen and Loy, Chen Change , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    work page

  5. [5]

    arXiv preprint arXiv:2108.05080 , year=

    FakeAVCeleb: A novel audio-video multimodal deepfake dataset , author=. arXiv preprint arXiv:2108.05080 , year=

    arXiv

  6. [6]

    2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA) , pages=

    Do you really mean that? content driven audio-visual deepfake dataset and multimodal method for temporal forgery localization , author=. 2022 International Conference on Digital Image Computing: Techniques and Applications (DICTA) , pages=. 2022 , organization=

    work page 2022

  7. [7]

    2024 , isbn =

    Cai, Zhixi and Ghosh, Shreya and Adatia, Aman Pankaj and Hayat, Munawar and Dhall, Abhinav and Gedeon, Tom and Stefanov, Kalin , title =. 2024 , isbn =. doi:10.1145/3664647.3680795 , booktitle =

    arXiv 2024

  8. [8]

    Computer Speech & Language , volume=

    ASVspoof 2019: A large-scale public database of synthesized, converted and replayed speech , author=. Computer Speech & Language , volume=. 2020 , publisher=

    2019

Show all 80 references
  1. [9]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=

    Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=. 2023 , publisher=

    2021

  2. [10]

    arXiv preprint arXiv:2408.08739 , year=

    ASVspoof 5: Crowdsourced speech data, deepfakes, and adversarial attacks at scale , author=. arXiv preprint arXiv:2408.08739 , year=

    arXiv

  3. [11]

    ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=

    Aasist: Audio anti-spoofing using integrated spectro-temporal graph attention networks , author=. ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=. 2022 , organization=

    2022

  4. [12]

    arXiv preprint arXiv:2111.02813 , year=

    Wavefake: A data set to facilitate audio deepfake detection , author=. arXiv preprint arXiv:2111.02813 , year=

    arXiv

  5. [13]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=

    The partialspoof database and countermeasures for the detection of short fake speech segments embedded in an utterance , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=. 2023 , publisher=

    2023

  6. [14]

    arXiv preprint arXiv:2209.15352 , year=

    Audiogen: Textually guided audio generation , author=. arXiv preprint arXiv:2209.15352 , year=

    arXiv

  7. [15]

    , title =

    Liu, Haohe and Chen, Zehua and Yuan, Yi and Mei, Xinhao and Liu, Xubo and Mandic, Danilo and Wang, Wenwu and Plumbley, Mark D. , title =. Proceedings of the 40th International Conference on Machine Learning , articleno =. 2023 , publisher =

    2023

  8. [16]

    IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=

    Audioldm 2: Learning holistic audio generation with self-supervised pretraining , author=. IEEE/ACM Transactions on Audio, Speech, and Language Processing , volume=. 2024 , publisher=

    2024

  9. [17]

    Proceedings of Interspeech , pages=

    EnvSDD: Benchmarking Environmental Sound Deepfake Detection , author=. Proceedings of Interspeech , pages=. 2025 , doi=

    2025

  10. [18]

    ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , year=

    Environmental Sound Deepfake Detection Challenge: An Overview , author=. ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , year=

    2026

  11. [19]

    INTERSPEECH 2026 , year=

    The First Environmental Sound Deepfake Detection Challenge: Benchmarking Robustness, Evaluation, and Insights , author=. INTERSPEECH 2026 , year=

    2026

  12. [20]

    2026 IEEE International Conference on Multimedia and Expo (ICME) , year=

    Overview of ESDD2: Environment-Aware Speech and Sound Deepfake Detection Challenge , author=. 2026 IEEE International Conference on Multimedia and Expo (ICME) , year=

    2026

  13. [21]

    ICASSP Grand Challenge Evaluation Plan , year=

    ESDD 2026: Environmental Sound Deepfake Detection Challenge Evaluation Plan , author=. ICASSP Grand Challenge Evaluation Plan , year=

    2026

  14. [22]

    ICME Grand Challenge Evaluation Plan , year=

    ESDD2: Environment-Aware Speech and Sound Deepfake Detection Challenge Evaluation Plan , author=. ICME Grand Challenge Evaluation Plan , year=

  15. [23]

    arXiv preprint arXiv:2605.20266 , year=

    A survey of large audio language models: Generalization, trustworthiness, and outlook , author=. arXiv preprint arXiv:2605.20266 , year=

    arXiv

  16. [24]

    Proceedings of Interspeech , pages=

    VCapAV: A Video-Caption Based Audio-Visual Deepfake Detection Dataset , author=. Proceedings of Interspeech , pages=. 2025 , doi=

    2025

  17. [25]

    2016 24th European signal processing conference (EUSIPCO) , pages=

    TUT database for acoustic scene classification and sound event detection , author=. 2016 24th European signal processing conference (EUSIPCO) , pages=. 2016 , organization=

    2016

  18. [26]

    ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Clotho: An audio captioning dataset , author=. ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2020 , organization=

    2020

  19. [27]

    2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=

    Audio set: An ontology and human-labeled dataset for audio events , author=. 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP) , pages=. 2017 , organization=

    2017

  20. [28]

    2019 , publisher=

    TAU Urban Acoustic Scenes 2019 Openset, Development Dataset , author=. 2019 , publisher=. doi:10.5281/zenodo.2591503 , url=

    2019 doi

  21. [29]

    International conference on machine learning , pages=

    Learning transferable visual models from natural language supervision , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  22. [30]

    ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Clap learning audio concepts from natural language supervision , author=. ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2023 , organization=

    2023

  23. [31]

    2023 , volume=

    Chen, Sanyuan and Wu, Yu and Wang, Chengyi and Liu, Shujie and Tompkins, Daniel and Chen, Zhuo and Che, Wanxiang and Yu, Xiangzhan and Wei, Furu , booktitle=. 2023 , volume=

    2023

  24. [32]

    IEEE Journal of Selected Topics in Signal Processing , volume=

    Wavlm: Large-scale self-supervised pre-training for full stack speech processing , author=. IEEE Journal of Selected Topics in Signal Processing , volume=. 2022 , publisher=

    2022

  25. [33]

    2023 , eprint=

    ImageBind: One Embedding Space To Bind Them All , author=. 2023 , eprint=

    2023

  26. [34]

    2018 IEEE international workshop on information forensics and security (WIFS) , pages=

    Mesonet: a compact facial video forgery detection network , author=. 2018 IEEE international workshop on information forensics and security (WIFS) , pages=. 2018 , organization=

    2018

  27. [35]

    Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

    Lips don't lie: A generalisable and robust approach to face forgery detection , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages=

  28. [36]

    Proceedings of the 28th ACM international conference on multimedia , pages=

    Not made for each other-audio-visual dissonance-based deepfake detection and localization , author=. Proceedings of the 28th ACM international conference on multimedia , pages=

  29. [37]

    Proceedings of the 2023 ACM International Conference on Multimedia Retrieval , pages=

    AVForensics: Audio-driven deepfake video detection with masking strategy in self-supervision , author=. Proceedings of the 2023 ACM International Conference on Multimedia Retrieval , pages=

    2023

  30. [38]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  31. [39]

    ACM Transactions on Multimedia Computing, Communications and Applications , volume=

    Joint audio-visual attention with contrastive learning for more general deepfake detection , author=. ACM Transactions on Multimedia Computing, Communications and Applications , volume=. 2024 , publisher=

    2024

  32. [40]

    ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    End-to-end anti-spoofing with rawnet2 , author=. ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2021 , organization=

    2021

  33. [41]

    2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , pages=

    Lip sync matters: A novel multimodal forgery detector , author=. 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , pages=. 2022 , organization=

    2022

  34. [42]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

    Lost in translation: Lip-sync deepfake detection from audio-video mismatch , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages=

  35. [43]

    Advances in Neural Information Processing Systems , volume=

    Lips are lying: Spotting the temporal inconsistency between audio and visual in lip-syncing deepfakes , author=. Advances in Neural Information Processing Systems , volume=

  36. [44]

    IEEE Transactions on Human-Machine Systems , year=

    AV-Lip-Sync+: Leveraging AV-HuBERT to Exploit Multimodal Inconsistency for Deepfake Detection of Frontal Face Videos , author=. IEEE Transactions on Human-Machine Systems , year=

  37. [45]

    ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Audio-visual deepfake detection with local temporal inconsistencies , author=. ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2025 , organization=

    2025

  38. [46]

    AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake , year=

    Yang, Wenyuan and Zhou, Xiaoyu and Chen, Zhikai and Guo, Bofei and Ba, Zhongjie and Xia, Zhihua and Cao, Xiaochun and Ren, Kui , journal=. AVoiD-DF: Audio-Visual Joint Learning for Detecting Deepfake , year=

  39. [47]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =

    Oorloff, Trevine and Koppisetti, Surya and Bonettini, Nicol\`o and Solanki, Divyaraj and Colman, Ben and Yacoob, Yaser and Shahriyari, Ali and Bharaj, Gaurav , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , month =. 2024 , pages =

    2024

  40. [48]

    AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection , year=

    Hashmi, Ammarah and Shahzad, Sahibzada Adil and Lin, Chia Wen and Tsao, Yu and Wang, Hsin-Min , journal=. AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection , year=

  41. [49]

    Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization , year=

    Katamneni, Vinaya Sree and Rattani, Ajita , booktitle=. Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization , year=

  42. [50]

    2025 , eprint=

    Qwen2.5-Omni Technical Report , author=. 2025 , eprint=

    2025

  43. [51]

    2026 , eprint=

    MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction , author=. 2026 , eprint=

    2026

  44. [52]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Face2Face: Real-Time Face Capture and Reenactment of RGB Videos , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2016 , doi=

    2016

  45. [53]

    ACM Transactions on Graphics , volume=

    Deferred Neural Rendering: Image Synthesis Using Neural Textures , author=. ACM Transactions on Graphics , volume=. 2019 , doi=

    2019

  46. [54]

    and Rubinstein, Michael , title =

    Ephrat, Ariel and Mosseri, Inbar and Lang, Oran and Dekel, Tali and Wilson, Kevin and Hassidim, Avinatan and Freeman, William T. and Rubinstein, Michael , title =. 2018 , issue_date =. doi:10.1145/3197517.3201357 , journal =

    2018

  47. [55]

    ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Zhao, Shengkui and Ma, Yukun and Ni, Chongjia and Zhang, Chong and Wang, Hao and Nguyen, Trung Hieu and Zhou, Kun and Yip, Jiaqi and Ng, Dianwen and Ma, Bin , title=. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=....

    2024

  48. [56]

    Chen, Yushen and Niu, Zhikang and Ma, Ziyang and Deng, Keqi and Wang, Chunhui and Zhao, Jian and Yu, Kai and Chen, Xie , booktitle=. F5-. 2025 , address=. doi:10.18653/v1/2025.acl-long.313 , url=

    2025 doi

  49. [57]

    and Zhou, Y

    Zhou, S. and Zhou, Y. and He, Y. and Zhou, X. and Wang, J. and Deng, W. and Shu, J. , title =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2026 , doi =

    2026

  50. [58]

    2024 , eprint=

    Zero-shot Voice Conversion with Diffusion Transformers , author=. 2024 , eprint=

    2024

  51. [59]

    The Thirteenth International Conference on Learning Representations , year=

    Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement , author=. The Thirteenth International Conference on Learning Representations , year=

  52. [60]

    2020 , doi=

    Desplanques, Brecht and Thienpondt, Jenthe and Demuynck, Kris , booktitle=. 2020 , doi=

    2020

  53. [61]

    The Fourteenth International Conference on Learning Representations , year=

    AudioX: A Unified Framework for Anything-to-Audio Generation , author=. The Fourteenth International Conference on Learning Representations , year=

  54. [62]

    2025 , eprint=

    MMAudio: Taming Multimodal Joint Training for High-Quality Video-to-Audio Synthesis , author=. 2025 , eprint=

    2025

  55. [63]

    2024 , eprint=

    FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds , author=. 2024 , eprint=

    2024

  56. [64]

    2024 , eprint=

    Stable Audio Open , author=. 2024 , eprint=

    2024

  57. [65]

    2025 , eprint=

    SAO-Instruct: Free-form Audio Editing using Natural Language Instructions , author=. 2025 , eprint=

    2025

  58. [66]

    Computer Vision and Image Understanding , volume=

    Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization , author=. Computer Vision and Image Understanding , volume=. 2023 , doi=

    2023

  59. [67]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  60. [68]

    2021 , eprint=

    AST: Audio Spectrogram Transformer , author=. 2021 , eprint=

    2021

  61. [69]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    Self-Supervised Video Forensics by Audio-Visual Anomaly Detection , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2023 , doi=

    2023

  62. [70]

    arXiv preprint arXiv:2305.13774 , year=

    Add 2023: the second audio deepfake detection challenge , author=. arXiv preprint arXiv:2305.13774 , year=

    2023 arXiv

  63. [71]

    ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Add 2022: the first audio deep synthesis detection challenge , author=. ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2022 , organization=

    2022

  64. [72]

    and Wilson, Kevin and Thorpe, Jeremy and Chinen, Michael and Patton, Brian and Saurous, Rif A

    Wisdom, Scott and Hershey, John R. and Wilson, Kevin and Thorpe, Jeremy and Chinen, Michael and Patton, Brian and Saurous, Rif A. , booktitle=. Differentiable Consistency Constraints for Improved Deep Speech Enhancement , year=

  65. [73]

    ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=

    Compspoof: A dataset and joint learning framework for component-level audio anti-spoofing countermeasures , author=. ICASSP 2026-2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages=. 2026 , organization=

    2026

  66. [74]

    arXiv preprint arXiv:2512.19687 , year=

    Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning , author=. arXiv preprint arXiv:2512.19687 , year=

  67. [75]

    Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

    Cav-mae sync: Improving contrastive audio-visual mask autoencoders via fine-grained alignment , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages=

  68. [76]

    arXiv preprint arXiv:2501.15368 , year=

    Baichuan-omni-1.5 technical report , author=. arXiv preprint arXiv:2501.15368 , year=

  69. [77]

    arXiv preprint arXiv:2607.02770 , year=

    Gemma 4 technical report , author=. arXiv preprint arXiv:2607.02770 , year=

    arXiv

  70. [78]

    2024 , eprint=

    XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detection , author=. 2024 , eprint=

    2024

  71. [79]

    2026 , eprint=

    Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTOR , author=. 2026 , eprint=

    2026

  72. [80]

    2026 , eprint=

    AudioMosaic: Contrastive Masked Audio Representation Learning , author=. 2026 , eprint=

    2026

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.