{"id":"1be845c4-b60d-4ad1-8660-013486a8490d","arxiv_id":"2506.07358","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A 0.48M-parameter single-stream network with iterative audio-visual fusion outperforms larger two-stream baselines on DF-TIMIT, FakeAVCeleb, and DFDC deepfake detection benchmarks.","lead":"This paper introduces SS-AVD, a compact 0.48M-parameter network that detects videos manipulated in vision, audio, or both. It fuses audio and visual information at every layer instead of at the end, and reports stronger accuracy than larger two-stream detectors on three deepfake benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Identity overlap in random video-level splits likely inflates the reported superiority and 'unseen type' generalization.","rationale":"The reader's weakest_assumption exactly identifies the most load-bearing issue: the random video-level splits do not guarantee identity-disjoint partitions, so the model can exploit identity-specific cues. This concern directly threatens both the claimed 'superiority' on the three benchmarks and the 'unseen types of deepfakes' generalization, because the cross-method evaluation also shares source identities. No other issue is as central: the missing error bars affect statistical confidence but not the core validity of a single result; the hyperparameter sensitivity analysis on the DFDC test set is troubling but only affects one dataset and could be remedied by validation-based selection; the missing AVoiD-DF comparison affects completeness, not correctness of the reported gains. Identity leakage, by contrast, could invalidate the entire comparison, since all reported numbers may be inflated. Because the reader already accounts for this in a CONDITIONAL verdict, the stress-test confirms that conditional status rather than moving to reject; the method may still be sound once evaluated under identity-disjoint splits. The recommended concrete test settles the concern by quantifying the effect of identity overlap.","tokens_in":14662,"tokens_out":7861,"duration_ms":98193,"concrete_test":"Re-run the experiments in Tables 1 and 3 using identity-disjoint splits: group all videos by subject/speaker ID (DF-TIMIT subject, VoxCeleb2 speaker ID for FakeAVCeleb and real videos, DFDC actor ID) and split at the identity level so no identity appears in both training and test. Compare the resulting ACC/AUC with the reported numbers. If whole-video AUC drops by more than a few points, or if SS-AVD no longer outperforms the baselines, the reported superiority is at least partially attributable to identity leakage. Also report the number of overlapping identities in the current random splits as a diagnostic.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of superiority and generalization rests on the evaluation protocol described in the 'Splits' paragraph: 'We randomly select parts of the videos for evaluation... We split the training, validation, and test subsets in each evaluation dataset at the rate of 0.75, 0.1, and 0.15, respectively.' These splits are random at the video level and are never checked for identity overlap. In DF-TIMIT, all videos come from only 32 subjects, so the same subject is almost guaranteed to appear in both training and test. In FakeAVCeleb and DFDC, the datasets are built from repeated appearances of the same actors/speakers, and the cross-method evaluation in the 'Cross-method Evaluation' section uses FakeAVCeleb generated by two face-swap methods from the same underlying real videos. Under such splits, a model can achieve high accuracy by recognizing identity-specific or recording-specific traces (a particular face texture, voice timbre, background, or camera) rather than by learning general forgery artifacts. This would inflate all reported numbers in Tables 1 and 3 and would specifically invalidate the claim of 'superiority in ... unseen types of deepfakes', because the 'unseen' method still shares identities and source footage with the training set. The paper provides no subject-level split protocol and no analysis of identity overlap, leaving the central claim unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SS-AVD, a single-stream audio-visual deepfake detector. It uses stacked collaborative audio-visual learning (CAVL) blocks, each containing a visual preprocessing module and a self-attention-based audio-visual module, to fuse audio and visual features throughout the network rather than only at the classification stage. A multi-modal classification module with multi-modal style-shuffle augmentation (MMSSA) and latent-shuffle augmentation (LSA) is introduced to improve modality-specific and whole-video predictions. The model has only 0.48M parameters. Experiments on DF-TIMIT, FakeAVCeleb, and DFDC report accuracy/AUC values for visual, audio, and whole-video detection, and a cross-method evaluation on FakeAVCeleb is used to support a claim of generalization to unseen deepfake types. The paper concludes that SS-AVD outperforms prior audio-visual joint detectors while being substantially lighter.","tokens_in":14854,"tokens_out":5012,"duration_ms":53902,"significance":"If the reported results hold under a rigorous evaluation protocol, the paper makes a useful contribution: it demonstrates that a parameter-efficient detector (0.48M parameters, at least 11x smaller than the next smallest baseline) can perform iterative multimodal fusion and achieve competitive or superior performance, which is relevant for deployment on resource-constrained devices. The architecture is clearly motivated, and the ablation study gives some evidence for the contribution of each proposed component. However, the central empirical claims are currently supported only by an evaluation protocol with random video-level splits, test-set hyperparameter tuning, and single-run results with no variance estimates. Until these issues are addressed, the quantitative superiority and the 'unseen deepfake' generalization claims are not established.","major_comments":[{"comment":"The evaluation protocol uses random video-level splits (0.75/0.1/0.15) and never checks for identity overlap. Since DF-TIMIT has only 32 subjects, the same subject is almost guaranteed to appear in both training and test; FakeAVCeleb and DFDC are also built from repeated actors/speakers, and the cross-method evaluation uses forgery methods applied to the same underlying real videos. Under these splits, the model can succeed by recognizing identity- or recording-specific traces rather than general forgery artifacts, which would inflate the results in Tables 1 and 3 and would invalidate the 'unseen types of deepfakes' claim. Please re-run with subject-disjoint (identity-exclusive) splits and report the degree of identity overlap in the current splits.","section":"Splits paragraph"},{"comment":"The hyperparameters gamma2 and gamma3 are tuned on the DFDC test set, and the best configuration from this tuning ({0.1, 1.0}, ACC 86.48) is then used for the DFDC results reported in Table 1. This is test-set leakage: the reported DFDC gains over baselines partially reflect selection on the test set rather than an unbiased evaluation. The hyperparameters should be selected on the validation split and the test set used only once, or the DFDC numbers should be explicitly labeled as tuning results.","section":"Table 5"},{"comment":"The authors state that Emotions and VFD results are taken directly from their original papers, while the other baselines are retrained on the authors' random splits. Because the original papers use their own train/test protocols, the entries in Table 1 are not directly comparable, and the claimed superiority over these methods (e.g., whole-video AUC improvements on FakeAVCeleb) may reflect protocol mismatch rather than model quality. All baselines should be evaluated under the same identity-disjoint splits, or entries from other protocols should be clearly marked and excluded from superiority claims.","section":"Comparison Methods"},{"comment":"All results appear to be from a single training run with no error bars or multiple seeds. Many of the reported advantages are small (e.g., Table 3 FaceSwap whole-video AUC 85.39 vs 83.87 for 2+1 Stream, and Table 4 differences around 0.5-1%), so without variance estimates the 'outperforms' claim is not statistically grounded. Please report mean and standard deviation over at least three seeds.","section":"Experiments (Tables 1, 3, 4, 5)"}],"minor_comments":[{"comment":"In the sentence defining input shapes, 'donates' should be 'denotes'; additionally, Eq. (1) contains an unmatched parenthesis in 'P= ReLU(Projc(F''v),'.","section":"Design of SS-AVD"},{"comment":"The phrase 'y_1/2 indicts a pseudo-label' should be 'indicates', and the definition of the pseudo-label could be made more explicit.","section":"Adversarial Loss"},{"comment":"Generating audio labels for DFDC by comparing hash values of audio tracks is a heuristic that may introduce label noise; please discuss its accuracy or provide a validation on a subset with known audio manipulations.","section":"Datasets (DFDC)"},{"comment":"The checkmark/cross notation in the ablation table is hard to read; using explicit 'with/without' columns or a legend would improve clarity.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of the journal. The main risk is the evaluation protocol; if the authors can provide identity-disjoint splits and unbiased hyperparameter selection, the contribution could be publishable. I do not see a fundamental flaw in the architecture itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a real architectural contribution—a 0.48M-parameter single-stream network that fuses audio and visual features at every stage via CAVL blocks, plus a classification module with style-shuffle and latent-shuffle augmentation. The idea is clean and the ablation study supports the design choices. But don't trust the headline numbers yet. The evaluation protocol has a load-bearing flaw: splits are random at the video level with no identity-disjoint guarantee. In DF-TIMIT, all videos come from 32 subjects; in FakeAVCeleb and DFDC, the same actors appear across the dataset. So the model can easily learn identity-specific or recording-specific traces, which would inflate every ACC/AUC in Tables 1 and 3. The cross-method evaluation is especially vulnerable, since the FaceSwap and Fsgan fakes in FakeAVCeleb are generated from the same underlying real videos. The claim of 'unseen types of deepfakes' is therefore not supported.\n\nThere are other soft spots. The hyperparameters gamma2 and gamma3 were selected by looking at DFDC test accuracy (Table 5), which is tuning on the test set. No error bars or multiple seeds. The DFDC audio labels are derived from a hash-comparison proxy, not ground truth. And AVoiD-DF, which is cited, is missing from the comparison table—that's a notable gap. No code release either.\n\nWhat the paper does well: the architecture description is clear, the ablation study is sensible, and the parameter count is genuinely striking. The idea of continuous cross-modal fusion instead of late fusion is worth taking seriously. If the results can be reproduced under identity-disjoint splits and with proper seeds, this could be a useful baseline for edge-device deepfake detection.\n\nMy take: the reader's conditional verdict is right. The paper deserves peer review, but it needs major revision on evaluation. The authors should re-run with identity-disjoint splits, report multiple seeds, move hyperparameter selection to validation, and add AVoiD-DF and other recent baselines. I'd send it out, but I'd tell the authors the generalization claims are not currently supported.","headline":"The architecture is genuinely interesting and the parameter count is striking, but the random video-level splits with identity overlap make the reported superiority and 'unseen deepfake' claims unsupported.","tokens_in":15485,"tokens_out":3527,"would_cite":false,"duration_ms":39678,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single-stream network that fuses audio and visual features at every layer claims to beat larger two-stream deepfake detectors while using only 0.48M parameters.","keywords":["audio-visual deepfake detection","single-stream multi-modal learning","lightweight network","self-attention fusion","spatial attention","style-shuffle augmentation","cross-method generalization","deepfake forensics"],"falsifier":"Run the same SS-AVD training on FakeAVCeleb or DFDC with an identity-disjoint split where no speaker or face appears in both train and test, then compare AUC scores to the paper's random-split numbers; if the gap to the baselines collapses or the absolute AUC drops sharply, the claimed generalization does not survive identity separation.","tokens_in":14383,"feed_emoji":"🕵️","tokens_out":7652,"duration_ms":70133,"temperature":0.7,"pith_summary":"This paper tries to establish that joint audio-visual deepfake detection does not require two separate neural networks that learn audio and visual features in isolation and fuse them only at the end. It proposes a single-stream network, SS-AVD, that fuses audio and visual features at every layer through iterative multi-modal learning blocks, and claims this design is both more accurate and far smaller than existing joint detectors. With only 0.48M parameters, the authors report state-of-the-art results on the DF-TIMIT, FakeAVCeleb, and DFDC benchmarks, including higher accuracy on audio-only, visual-only, and whole-video classification, and better cross-method generalization to unseen forgery types. The practical payoff is that a capable detector could run on resource-limited devices where larger two-stream models cannot.","feed_headline":"0.48M-parameter network beats larger deepfake detectors","feed_subtitle":"Fusing sound and vision at every layer, the single-stream model stays small and accurate.","key_machinery":"The Collaborative Audio-Visual Learning (CAVL) block is the central object: it combines a Visual Preprocessing Module (VPM), which applies frame-based spatial attention to visual features, with a Self-Attention-based Audio-Visual Module (SAAVM), which chunks visual frames into tokens and pools audio into matching tokens, then feeds both through a shared self-attention layer to learn spatial-temporal correlations between the modalities. Stacking CAVL blocks across four pyramid stages produces iterative audio-visual fusion throughout the network. A multi-modal classification module then adds two training strategies: Multi-Modal Style-Shuffle Augmentation (MMSSA), which shuffles feature style statistics between samples so classifiers rely on content rather than style, and Latent-Shuffle Augmentation (LSA), which pairs visual and audio features from different samples to make the whole-video classifier resistant to audio-visual mismatches. The loss combines classification, adversarial, and contrast terms.","core_discovery":"The central claim is that continuous cross-modal fusion during feature learning, rather than late fusion of independently learned audio and visual features, yields better deepfake detection with a fraction of the parameters. The authors report that SS-AVD, a single-stream network with 0.48M parameters, outperforms or matches state-of-the-art joint audio-visual detectors on DF-TIMIT, FakeAVCeleb, and DFDC, and also outperforms them in a cross-method evaluation where training and test deepfakes are produced by different face-swap methods. The design rests on the premise that fusing modalities throughout the network exploits their natural correlations and avoids redundant layers, which is what makes the model small and effective at the same time.","pith_inferences":["The single-stream iterative-fusion principle is a general recipe: the same CAVL-style architecture could be applied to other audio-visual reasoning tasks, such as lip-sync verification or audio-visual speech recognition, where cross-modal correlation is the signal.","A natural stress test the authors do not run is a strictly identity-disjoint split; measuring performance under that condition would isolate whether the reported generalization comes from general fake traces or from speaker and face familiarity.","Because the paper's hyperparameter search was deliberately small, the reported 0.48M-parameter model may be a lower bound on achievable accuracy; tuning the loss weights and the style-shuffle mixing factor could push scores higher.","The style-shuffle and latent-shuffle ideas could be plugged into existing two-stream detectors as a drop-in training scheme, potentially improving their generalization without architectural changes."],"forward_implications":["A detector with 0.48M parameters is practical for mobile and edge deployment, where existing joint detectors with more than 5M parameters are too heavy.","Iterative multi-modal fusion can flag partial forgeries, such as fake visual with real audio or real visual with fake audio, using correlations established early in the network rather than only at the final classification stage.","The style-shuffle and latent-shuffle augmentations should make the classifier less dependent on compression traces or device fingerprints, improving robustness to unseen deepfake generation methods.","The reported results suggest the approach transfers across datasets with different manipulation types, since it leads on DF-TIMIT, FakeAVCeleb, and DFDC simultaneously."],"supporting_citations":[{"why":"Baseline 2+1 Stream, a late-fusion two-stream joint detector whose reported results the paper must beat on visual, audio, and whole-video tasks.","marker":"(Zhou and Lim 2021)"},{"why":"Baseline BA-TFD for temporal forgery localization and a source for the contrast loss formulation.","marker":"(Cai et al. 2022)"},{"why":"Baseline VFD, which measures face-voice homogeneity for deepfake detection and is compared on whole-video AUC.","marker":"(Cheng et al. 2022)"},{"why":"Baseline MultiModalTrace, a two-ResNet-stem approach with MLP fusion that the paper compares on all three datasets.","marker":"(Raza and Malik 2023)"},{"why":"Baseline MRDF, which uses two feature encoders and a transformer for audio-visual fusion.","marker":"(Zou et al. 2024)"},{"why":"Provides the DF-TIMIT dataset used to evaluate visual-only and whole-video detection.","marker":"(Korshunov and Marcel 2018)"},{"why":"Provides the FakeAVCeleb dataset, the main benchmark for audio-visual deepfake detection and cross-method evaluation in this paper.","marker":"(Khalid et al. 2021)"},{"why":"Provides the DFDC dataset used for large-scale evaluation; audio labels are derived from its audio tracks.","marker":"(Dolhansky et al. 2020)"},{"why":"Inspires the style-shuffle augmentation by reducing style bias, which the MMSSA strategy is built on.","marker":"(Nam et al. 2021)"}],"fun_headline_variants":["Single-stream fusion beats heavy deepfake detectors at 0.48M params","Tiny 0.48M-parameter model fuses sound and vision to detect deepfakes","Continuous cross-modal fusion yields 0.48M-param deepfake detector","Single-stream network with 0.48M params detects deepfakes better","Fuse audio and visual at every layer: lightweight deepfake detector"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The random video-level train and test splits assume no person appears in both training and testing, so the model's apparent generalization to unseen deepfakes could partly come from recognizing familiar identities rather than general forgery cues.","fun_headline_variants_meta":{"raw":{"variants":["Single-stream fusion beats heavy deepfake detectors at 0.48M params","Tiny 0.48M-parameter model fuses sound and vision to detect deepfakes","Continuous cross-modal fusion yields 0.48M-param deepfake detector","Single-stream network with 0.48M params detects deepfakes better","Fuse audio and visual at every layer: lightweight deepfake detector"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000511,"raw_usage":{"total_tokens":2506,"prompt_tokens":989,"completion_tokens":1517,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":1413}},"tokens_in":605,"tokens_out":1517,"duration_ms":11409,"temperature":1.0,"reasoning_tokens":1413,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:36:09.589258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same SS-AVD training on FakeAVCeleb or DFDC with an identity-disjoint split where no speaker or face appears in both train and test, then compare AUC scores to the paper's random-split numbers; if the gap to the baselines collapses or the absolute AUC drops sharply, the claimed generalization does not survive identity separation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline 2+1 Stream, a late-fusion two-stream joint detector whose reported results the paper must beat on visual, audio, and whole-video tasks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Baseline BA-TFD for temporal forgery localization and a source for the contrast loss formulation."},{"cited_title":"Voice-Face Homogeneity Tells Deepfake","cited_arxiv_id":"2203.02195","evidence_quote":"Baseline VFD, which measures face-voice homogeneity for deepfake detection and is compared on whole-video AUC."},{"cited_title":"A.; and Malik, K","cited_arxiv_id":null,"evidence_quote":"Baseline MultiModalTrace, a two-ResNet-stem approach with MLP fusion that the paper compares on all three datasets."},{"cited_title":"Cross-Modality and Within-Modality Regularization for Audio-Visual DeepFake Detection","cited_arxiv_id":"2401.05746","evidence_quote":"Baseline MRDF, which uses two feature encoders and a transformer for audio-visual fusion."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Inspires the style-shuffle augmentation by reducing style bias, which the MMSSA strategy is built on."}],"review_version":1}