Audio-Visual Deepfake Detection With Local Temporal Inconsistencies

Djamila Aouada; Enjie Ghorbel; Marcella Astrid

REVIEW 3 major objections 4 minor 1 cited by

Audio-Visual Deepfake Detection With Local Temporal Inconsistencies

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that audio-visual deepfakes can be caught by measuring fine-grained temporal mismatches between the audio and visual streams at every time step, rather than relying on a single global consistency score.

desk verdict Promising temporal fine-grained AV deepfake detector, but the reported SOTA margins are selected on the test set and the cross-dataset claim rests on 140 videos. read the letter →

arxiv 2501.08137 v4 pith:3YKSPEVY submitted 2025-01-14 cs.CV cs.CRcs.MMcs.SDeess.AS

Marcella Astrid , Enjie Ghorbel , Djamila Aouada This is my paper · ORCID

classification cs.CVcs.CRcs.MMcs.SDeess.AS

keywords audio-visualdeepfakedetectiontemporalinconsistencyfine-grainedclassificationpseudo-fakeaugmentationattentionmechanismDFDCFakeAVCelebcross-datasetgeneralization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that audio-visual deepfakes leave small timing errors between the sound and image tracks, and that a detector can catch them by comparing the two streams at every moment instead of using one global score. To do this, the authors introduce a temporal distance map, meaning the per-time-step L2 distance between visual and audio features, plus an attention mechanism that down-weights irrelevant moments such as background sound. They also train on pseudo-fakes built by locally reordering, repeating, flipping, or translating short chunks of a clip, so the model learns fine-grained temporal inconsistencies. With the best configuration, the reported AUC is 98.0% on the DFDC test set and 87.0% when transferred to FakeAVCeleb, both above the compared state-of-the-art methods.

What carries the argument

The central mechanism is the temporal distance map $m_t = \lVert f^v_t - f^a_t \rVert$, computed from a shallow 3D convolutional visual encoder and a 1D convolutional audio encoder aligned to the same $T'$ time steps. A cross-attention map weighs each time step by the correlation between projected audio and visual features, and the classifier consumes the attended map $\hat{m} = m \odot a$. Training also relies on pseudo-fakes generated by local temporal manipulations over a random contiguous sub-sequence: replacement with another clip, frame repetition, flipping, and left/right translation, so the model sees subtle temporal inconsistencies instead of only global audio-visual mismatches.

What would settle it

Train the reported model on the DFDC subset and apply it to a set of real deepfakes whose only manipulation is a full audio-track replacement or a constant audio delay; if the AUC falls to the level of the no-augmentation baseline, the local temporal-edit proxy is the reason the cross-dataset result does not transfer.

Watch

Extended reading notes

Core claim

The paper's central claim is that temporal fine-grained inconsistencies are a usable forensic signal for audio-visual deepfakes. The proposed representation is a distance vector $m$ of length $T'$, where each entry $m_t$ is the L2 distance between the visual feature and the audio feature at time step $t$, multiplied element-wise by a softmax attention map before classification. The authors report that this local distance map outperforms both a single global distance ($T'=1$) and a longer map ($T'=15$), and that their pseudo-fake augmentation, especially replacing a local sub-sequence with a chunk from another clip, gives the largest cross-dataset gain.

Load-bearing premise

The load-bearing premise is that pseudo-fakes created by locally editing a real clip, namely repeating, flipping, translating, or swapping a few frames, faithfully mimic the timing errors found in real deepfakes, so a detector trained only on those edits will transfer to real forgeries.

Editorial extensions

If this is right

A detector can score audio-visual alignment at every time step and flag fakes without identity-specific training or spatial artifact cues.
Using $T'=7$ time steps outperforms both a single global distance ($T'=1$) and a longer map ($T'=15$), so the useful signal is local but not overly fine.
Attention over the temporal distance map is what lifts cross-dataset AUC on FakeAVCeleb from 72.14% to 80.34%.
Replacing a short sub-sequence with a chunk from a different clip is the most effective pseudo-fake operation, suggesting that content substitution is the artifact class that generalizes best.
With the chosen configuration, the reported AUCs of 98.0% (DFDC) and 87.0% (FakeAVCeleb) exceed all compared state-of-the-art methods in the paper.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The authors do not test partial or localized forgeries; a natural extension is to use a threshold on the attended distance map to identify which temporal segments in a video are fake.
Because temporal local distance outperforms the spatial local distance of the prior method they extend, combining both temporal and spatial fine-grained maps could be more robust than either alone.
The synthetic edits cover only simple timing distortions; testing the same training recipe on fakes with continuous audio delay or pitch shifts would reveal whether the proxy captures a broader class of real artifacts.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Hi,

You asked about the Astrid et al. audio-visual deepfake detection paper. Bottom line: there is a real nugget here — a temporal distance map that compares audio and visual features at each time step, plus an attention mechanism over time, and a set of simple local temporal pseudo-fake augmentations (repeat, flip, translate). As far as I can tell, the temporal fine-grained treatment is new, and the augmentations are a nice extension of their own spatial fine-grained predecessor [9]. The cross-dataset result on FakeAVCeleb is the strongest selling point: 87% AUC vs 84.5% for FGI is a meaningful gap, assuming it holds up.

And there's the rub. The paper selects its best configuration (T'=7, attention, clip replacement) using test-set AUC from both the in-dataset and cross-dataset tables. No held-out validation split is described. That means the reported 98.0% and 87.0% are the best of several configurations looked at on the same test sets, not independent evaluations. The in-dataset gap over FGI is only 0.3 points (98.0 vs 97.7), which is almost certainly within noise — no error bars or significance tests anywhere. The cross-dataset gap is larger (2.5 points) but sits on only 140 videos, with no variance estimate. The stress-test note is correct: the claimed superiority is not independently established from these tables.

The pseudo-fake proxy assumption is also worth flagging. The paper trains on locally edited sequences (repeating, flipping, translating chunks) as if they represent real deepfake temporal artifacts. The replacement augmentation is plausible; the others are more speculative. The cross-dataset improvement suggests some generalization, but it's not a direct validation on real forgeries.

That said, the method is clearly described, the ablations are sensible, and the idea is honest. The protocol flaw is fixable: use a validation split for hyperparameter selection, train several seeds, report mean/variance. The paper would be more credible after that. As is, it's a solid workshop/conf submission with a promising direction, not a proven SOTA claim.

I'd send it to review — a serious referee will catch the selection issue and the small test set, and the authors can address it. Worth a reading group discussion on test-set selection in deepfake detection, maybe.

Best.

Referee Report

3 major / 4 minor

Summary. The paper proposes an audio-visual deepfake detector that combines a temporal distance map between audio and visual features, an attention mechanism over time steps, and pseudo-fake data augmentation based on local temporal manipulations (clip replacement, repetition, flipping, and translation). The method is trained on a DFDC subset and evaluated in-dataset on DFDC and cross-dataset on FakeAVCeleb, reporting video-level AUC of 98.0% and 87.0%, respectively, with the configuration T'=7 plus attention and clip-replacement augmentation.

Significance. If the reported results are valid, the paper makes a useful contribution by shifting fine-grained inconsistency modeling from the spatial to the temporal domain and by exploring several local temporal augmentation strategies. The architecture is simple, the augmentation idea is clearly described, and the cross-dataset evaluation direction is appropriate. However, the central SOTA claim is currently undermined by a test-set selection protocol: the augmentation type and T' value are chosen from the same test sets on which the final comparison is reported. The cross-dataset evidence is also based on only 140 videos without uncertainty quantification. These issues are fixable, but they must be addressed before the claimed superiority over prior methods can be accepted.

major comments (3)

[Section 3.2.1 and Section 3.2.3, Tables 1 and 2] The configuration used for the SOTA comparison, namely clip-replacement augmentation and T'=7, is selected by comparing rows of Table 1 and Table 2 on the DFDC test set and the FakeAVCeleb cross-dataset test set. No held-out validation split is described; the implementation details mention only checkpoint selection by lowest training loss. Consequently, the reported AUC values of 98.0% and 87.0% in Tables 3 and 4 are selected-test estimates rather than independent evaluations, and the claimed margins over FGI (97.7% and 84.5%) do not follow. Please introduce a proper validation split for model selection, or use nested cross-validation, and report the performance of the selected configuration on a truly held-out test set.
[Section 3.1, FakeAVCeleb cross-dataset evaluation] The cross-dataset evaluation uses only 70 real and 70 fake videos, and no error bars, confidence intervals, or significance tests are reported. A 2.5 AUC point difference over FGI (87.0% vs 84.5%) on such a small set may be within sampling variability. Please report per-video bootstrap confidence intervals or results over multiple training seeds, and state whether the difference is statistically significant.
[Section 2.1.2 and Section 3.2.1] The pseudo-fake examples are generated by local temporal edits (replacing, repeating, flipping, translating) and the classifier is trained only on these synthetic inconsistencies. The paper does not provide evidence that these edits are a faithful proxy for the temporal artifacts produced by real deepfake pipelines. Please analyze where the model attends in real fakes, or evaluate on real forged videos with known temporal manipulation, to support the claim that the method detects actual temporal inconsistencies rather than synthetic edit patterns.

minor comments (4)

[Abstract and full text] The dataset name is inconsistently typeset as 'FakeA VCeleb' in the abstract and 'FakeAVCeleb' elsewhere; please unify.
[Section 2.1.2, Eq. (3)] The indexing in the repeating formula C = A_{i+⌊a/p⌋*p} is unclear; it should be made explicit that each element in the chunk is drawn from the position i + p*⌊a/p⌋, or be rewritten with clearer notation.
[Fig. 2 caption and Section 2.1.2] There is a typo 'we uses f = 2' in the flipping example; also, the caption lists six subfigures (a)-(f) but the text describes five manipulation types, which may confuse readers.
[General] No code or trained models are mentioned as being released; providing code would strengthen reproducibility of the augmentation and attention implementation.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: pseudo-fake edits and temporal distance map are operational definitions, evaluated on external benchmarks; minor self-citation to [9] is not load-bearing.

full rationale

The paper's derivation is not circular. Pseudo-fakes are generated by explicit local temporal operations (replacement Eq. 2, repetition Eq. 3, flipping Eq. 4, translation Eqs. 5-6) from real samples; the model is then trained to separate these edited samples from original ones, and evaluated on external DFDC and FakeAVCeleb test videos. The distance map in Eq. 8 is an operational definition of the model's input representation, not a restatement of the target AUC. The only material self-citation is [9], which supplies the clip-replacement augmentation and the cross-attention idea; this is not load-bearing proof of the present contribution, and [9] is itself used as the strongest baseline (FGI). The claim that temporal local distance outperforms spatial distance is supported by the reported comparison rather than assumed in the construction. The validation-protocol concern raised by the skeptic--selecting T'=7 and clip-replacement by inspecting the same DFDC/FakeAVCeleb test tables, then reporting those selected numbers as the final 'Ours' results--is real but is an overfitting/selection-bias issue, not an equation-level circularity or a fitted parameter renamed as a prediction. The weakest assumption is the proxy validity of the pseudo-fake edits for real deepfake artifacts, but that is an external-validity concern, not a circularity. I therefore keep the circularity score low.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the representativeness of pseudo-fakes and on the meaningfulness of per-step feature distances. The main free parameter that materially changes conclusions is T', selected on test performance.

free parameters (4)

rmin = ≈0
Lower bound for pseudo-fake modification length (Eq. 1); chosen by hand, not fit.
rmax = 1
Upper bound for modification length; chosen by hand.
lmin = 2
Minimum modification length set in Section 2.1.1.
T' = 7
Temporal feature size selected based on test AUC in Table 2; a post-hoc choice that affects reported results.

assumptions (3)

domain assumption Per-time-step L2 distance between independently extracted audio and visual features is a meaningful measure of temporal inconsistency.
Used in Eq. (8) to define the distance map; no separate validation that L2 in feature space captures semantic timing mismatches.
ad hoc to paper Pseudo-fakes generated by local temporal manipulations are representative of real deepfake artifacts.
The entire augmentation strategy depends on this; the paper provides no evidence linking these synthetic edits to actual forgery pipelines.
domain assumption The attention mechanism, based on softmax of feature inner products, suppresses irrelevant audio such as background noise.
Used in Eqs. (9)-(10); the improvement with attention is shown only in aggregate, not analyzed per failure case.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Audio-Visual Deepfake Detection With Local Temporal Inconsistencies." pith.science (2026). https://pith.science/paper/3YKSPEVY

@misc{pith2026250108137,
  author       = {Pith},
  title        = {Pith review of: Audio-Visual Deepfake Detection With Local Temporal Inconsistencies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3YKSPEVY}},
  note         = {Machine review of arXiv:2501.08137}
}

read the original abstract

This paper proposes an audio-visual deepfake detection approach that aims to capture fine-grained temporal inconsistencies between audio and visual modalities. To achieve this, both architectural and data synthesis strategies are introduced. From an architectural perspective, a temporal distance map, coupled with an attention mechanism, is designed to capture these inconsistencies while minimizing the impact of irrelevant temporal subsequences. Moreover, we explore novel pseudo-fake generation techniques to synthesize local inconsistencies. Our approach is evaluated against state-of-the-art methods using the DFDC and FakeAVCeleb datasets, demonstrating its effectiveness in detecting audio-visual deepfakes.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey
cs.CV 2024-11 conditional novelty 4.0 of 10

The paper presents a comprehensive survey of passive deepfake detection methods, with a taxonomy across modalities and an analysis of deployment-oriented properties beyond detection accuracy.

Reference graph

Works this paper leans on

27 extracted references · 24 canonical work pages · cited by 1 Pith paper

[9]

Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’,

Heather Chen and Kathleen Magramo, “Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’,” CNN

work page
[1]

As their quality im- proves, distinguishing real from fake becomes increasingly difficult, highlighting the need for effective detection systems

INTRODUCTION Audio-visual deepfakes have beneficial applications but pose significant risks when misused [1, 2]. As their quality im- proves, distinguishing real from fake becomes increasingly difficult, highlighting the need for effective detection systems. One approach for detecting audio-visual deepfakes is to identify inconsistencies between audio and...

work page arXiv
[2]

METHODOLOGY As shown in Fig. 1, we address the fine-grained deepfake detection problem at the temporal level from two perspec- tives, namely: (1) data augmentation with pseudo-fake gener- ation (Section 2.1); and (2) architectural design for capturing fine-grained audio-visual distances in the temporal dimension (Section 2.2). arXiv:2501.08137v4 [cs.CV] 1...

work page Pith review arXiv 2025
[3]

This method, depicted in Fig

Replacing with another clip C = Bi:i+l−1, (2) where Bi:i+l−1 is a subsequence from a randomly selected data sample. This method, depicted in Fig. 2(b) is used in [9]. In this work, we also consider other methods

work page
[4]

In the example of Fig

Repeating C = Ai+⌊ a p ⌋∗p where a = {0, 1, ..., l− 1}, (3) where p is the number of repetitions, randomly chosen from 2 to l for each generated pseudo-fake. In the example of Fig. 2(c), we use p = 2

work page
[5]

In the example of Fig

Flipping C = Ai+2f⌊ a f ⌋+f −1−a where a = {0, 1, ..., l− 1}, (4) where f is the flipping frequency, randomly chosen from 2 to l for each pseudo-fake. In the example of Fig. 2(d), we uses f = 2

work page
[6]

Translating left or right C = Ai+min(l−1,a+v) where a = {0, 1, ..., l− 1}, or (5) C = Ai+max(0,a−v) where a = {0, 1, ..., l− 1}, (6) 𝐀1 𝐀2 𝐀3 𝐀4 𝐀5 𝐀6 𝐀7 𝐀8 𝐀1 𝐀2 𝐀3 𝐀3 𝐀5 𝐀5 𝐀7 𝐀8 𝐀1 𝐀2 𝐀4 𝐀3 𝐀6 𝐀5 𝐀7 𝐀8 𝐀1 𝐀2 𝐀5 𝐀6 𝐀6 𝐀6 𝐀7 𝐀8 𝐀1 𝐀2 𝐁3 𝐁4 𝐁5 𝐁6 𝐀7 𝐀8 𝐀1 𝐀2 𝐀3 𝐀3 𝐀3 𝐀4 𝐀7 𝐀8 (a) (b) (c) (d) (e) (f) Fig. 2. Given (a) an original data sequence, we generate...

work page
[7]

Experimental setup Dataset

EXPERIMENTS 3.1. Experimental setup Dataset. Following prior works [3, 9, 17], we train on a sub- set of the DFDC dataset [15] with 15,300 training and 2,700 test videos, balancing real and fake videos in the training set while maintaining the original test set distribution ( in- dataset protocol). To evaluate the generalization capability of the model, w...

work page

Show all 27 references

[8]

Our approach includes both data augmentation and architectural design strategies

CONCLUSION We propose detecting audio-visual deepfakes by identifying temporal inconsistencies. Our approach includes both data augmentation and architectural design strategies. For the aug- mentation, we experiment various manipulation techniques to create pseudo-fakes. As fo...
[10]

Deepfake video targeting zelen- sky’s wife linked to russian disinformation campaign, cnn analysis shows,

Gianluca Mezzofiore, “Deepfake video targeting zelen- sky’s wife linked to russian disinformation campaign, cnn analysis shows,” CNN
[11]

Not made for each other- audio-visual dissonance-based deepfake detection and localization,

Komal Chugh, Parul Gupta, Abhinav Dhall, and Ra- manathan Subramanian, “Not made for each other- audio-visual dissonance-based deepfake detection and localization,” in Proceedings of the 28th ACM inter- national conference on multimedia, 2020, pp. 439–447

2020
[12]

Deepfake video detection using audio-visual con- sistency,

Yewei Gu, Xianfeng Zhao, Chen Gong, and Xiaowei Yi, “Deepfake video detection using audio-visual con- sistency,” in Digital Forensics and Watermarking: 19th International Workshop, IWDW 2020, Melbourne, VIC, Australia, November 25–27, 2020, Revised Selected Pa- pers 19. Spring...

2020
[13]

Self- supervised video forensics by audio-visual anomaly de- tection,

Chao Feng, Ziyang Chen, and Andrew Owens, “Self- supervised video forensics by audio-visual anomaly de- tection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10491–10503

2023
[14]

Statistics-aware audio-visual deepfake detector,

Marcella Astrid, Enjie Ghorbel, and Djamila Aouada, “Statistics-aware audio-visual deepfake detector,”arXiv preprint arXiv:2407.11650, 2024

2024 arXiv
[15]

Laa-net: Localized artifact attention network for quality-agnostic and generalizable deepfake detection,

Dat Nguyen, Nesryne Mejri, Inder Pal Singh, Polina Kuleshova, Marcella Astrid, Anis Kacem, Enjie Ghor- bel, and Djamila Aouada, “Laa-net: Localized artifact attention network for quality-agnostic and generalizable deepfake detection,” in Proceedings of the IEEE/CVF Conference ...

2024
[16]

Multi- attentional deepfake detection,

Hanqing Zhao, Wenbo Zhou, Dongdong Chen, Tianyi Wei, Weiming Zhang, and Nenghai Yu, “Multi- attentional deepfake detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 2185–2194

2021
[17]

Detecting audio-visual deepfakes with fine-grained in- consistencies,

Marcella Astrid, Enjie Ghorbel, and Djamila Aouada, “Detecting audio-visual deepfakes with fine-grained in- consistencies,” arXiv preprint arXiv:2408.06753, 2024

2024 arXiv
[18]

Avoid-df: Audio-visual joint learning for detecting deepfake,

Wenyuan Yang, Xiaoyu Zhou, Zhikai Chen, Bofei Guo, Zhongjie Ba, Zhihua Xia, Xiaochun Cao, and Kui Ren, “Avoid-df: Audio-visual joint learning for detecting deepfake,” IEEE Transactions on Information Forensics and Security, vol. 18, pp. 2015–2029, 2023

2015
[19]

Avt2-dwf: Improving deepfake de- tection with audio-visual fusion and dynamic weighting strategies,

Rui Wang, Dengpan Ye, Long Tang, Yunming Zhang, and Jiacheng Deng, “Avt2-dwf: Improving deepfake de- tection with audio-visual fusion and dynamic weighting strategies,” arXiv preprint arXiv:2403.14974, 2024

2024 arXiv
[20]

Avfakenet: A unified end-to-end dense swin trans- former deep learning model for audio–visual deepfakes detection,

Hafsa Ilyas, Ali Javed, and Khalid Mahmood Malik, “Avfakenet: A unified end-to-end dense swin trans- former deep learning model for audio–visual deepfakes detection,” Applied Soft Computing , vol. 136, pp. 110124, 2023

2023
[21]

V oice-face homogene- ity tells deepfake,

Harry Cheng, Yangyang Guo, Tianyi Wang, Qi Li, Xi- aojun Chang, and Liqiang Nie, “V oice-face homogene- ity tells deepfake,” ACM Transactions on Multimedia Computing, Communications and Applications, 2023

2023
[22]

Do you really mean that? content driven audio- visual deepfake dataset and multimodal method for tem- poral forgery localization,

Zhixi Cai, Kalin Stefanov, Abhinav Dhall, and Munawar Hayat, “Do you really mean that? content driven audio- visual deepfake dataset and multimodal method for tem- poral forgery localization,” in 2022 International Con- ference on Digital Image Computing: Techniques and Applic...

2022
[23]

The deepfake detection challenge (dfdc) dataset,

Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Fer- rer, “The deepfake detection challenge (dfdc) dataset,” arXiv preprint arXiv:2006.07397, 2020

2006 arXiv
[24]

Fakeavceleb: A novel audio-video multimodal deepfake dataset,

Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S Woo, “Fakeavceleb: A novel audio-video multimodal deepfake dataset,” in Proc. Conf. Neural Inf. Process. Syst. Datasets Benchmarks Track, 2021

2021
[25]

Emotions don’t lie: An audio-visual deepfake detection method using affec- tive cues,

Trisha Mittal, Uttaran Bhattacharya, Rohan Chandra, Aniket Bera, and Dinesh Manocha, “Emotions don’t lie: An audio-visual deepfake detection method using affec- tive cues,” inProceedings of the 28th ACM international conference on multimedia, 2020, pp. 2823–2832

2020
[26]

Evaluation of an audio-video multimodal deepfake dataset using unimodal and multimodal detec- tors,

Hasam Khalid, Minha Kim, Shahroz Tariq, and Si- mon S Woo, “Evaluation of an audio-video multimodal deepfake dataset using unimodal and multimodal detec- tors,” in Proceedings of the 1st workshop on synthetic multimedia-audiovisual deepfake generation and detec- tion, 2021

2021
[27]

Adam: A method for stochastic optimization,

Diederik P Kingma and Jimmy Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014

2014 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [9]

Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’,

Heather Chen and Kathleen Magramo, “Finance worker pays out $25 million after video call with deepfake ‘chief financial officer’,” CNN

work page

[2] [1]

As their quality im- proves, distinguishing real from fake becomes increasingly difficult, highlighting the need for effective detection systems

INTRODUCTION Audio-visual deepfakes have beneficial applications but pose significant risks when misused [1, 2]. As their quality im- proves, distinguishing real from fake becomes increasingly difficult, highlighting the need for effective detection systems. One approach for detecting audio-visual deepfakes is to identify inconsistencies between audio and...

work page arXiv

[3] [2]

METHODOLOGY As shown in Fig. 1, we address the fine-grained deepfake detection problem at the temporal level from two perspec- tives, namely: (1) data augmentation with pseudo-fake gener- ation (Section 2.1); and (2) architectural design for capturing fine-grained audio-visual distances in the temporal dimension (Section 2.2). arXiv:2501.08137v4 [cs.CV] 1...

work page Pith review arXiv 2025

[4] [3]

This method, depicted in Fig

Replacing with another clip C = Bi:i+l−1, (2) where Bi:i+l−1 is a subsequence from a randomly selected data sample. This method, depicted in Fig. 2(b) is used in [9]. In this work, we also consider other methods

work page

[5] [4]

In the example of Fig

Repeating C = Ai+⌊ a p ⌋∗p where a = {0, 1, ..., l− 1}, (3) where p is the number of repetitions, randomly chosen from 2 to l for each generated pseudo-fake. In the example of Fig. 2(c), we use p = 2

work page

[6] [5]

In the example of Fig

Flipping C = Ai+2f⌊ a f ⌋+f −1−a where a = {0, 1, ..., l− 1}, (4) where f is the flipping frequency, randomly chosen from 2 to l for each pseudo-fake. In the example of Fig. 2(d), we uses f = 2

work page

[7] [6]

Translating left or right C = Ai+min(l−1,a+v) where a = {0, 1, ..., l− 1}, or (5) C = Ai+max(0,a−v) where a = {0, 1, ..., l− 1}, (6) 𝐀1 𝐀2 𝐀3 𝐀4 𝐀5 𝐀6 𝐀7 𝐀8 𝐀1 𝐀2 𝐀3 𝐀3 𝐀5 𝐀5 𝐀7 𝐀8 𝐀1 𝐀2 𝐀4 𝐀3 𝐀6 𝐀5 𝐀7 𝐀8 𝐀1 𝐀2 𝐀5 𝐀6 𝐀6 𝐀6 𝐀7 𝐀8 𝐀1 𝐀2 𝐁3 𝐁4 𝐁5 𝐁6 𝐀7 𝐀8 𝐀1 𝐀2 𝐀3 𝐀3 𝐀3 𝐀4 𝐀7 𝐀8 (a) (b) (c) (d) (e) (f) Fig. 2. Given (a) an original data sequence, we generate...

work page

[8] [7]

Experimental setup Dataset

EXPERIMENTS 3.1. Experimental setup Dataset. Following prior works [3, 9, 17], we train on a sub- set of the DFDC dataset [15] with 15,300 training and 2,700 test videos, balancing real and fake videos in the training set while maintaining the original test set distribution ( in- dataset protocol). To evaluate the generalization capability of the model, w...

work page