{"id":"29230dda-93c0-4390-aa3b-edeebcb886cd","arxiv_id":"2506.17592","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SELFI adaptively fuses face identity embeddings with visual features via a learned relevance gate, improving cross-dataset deepfake detection AUC by about 3% on average.","lead":"This paper studies when a face's identity helps or hurts deepfake detection and proposes SELFI, a detector that dynamically mixes identity and visual features. On four public benchmarks it raises cross-dataset AUC by about 3% on average, with the largest gain on the DFDC dataset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adaptive-fusion claim is not isolated: no static-fusion baseline is reported, and the IAFM gain over concatenation is small (0.4% average) and inconsistent across backbones, so the per-sample relevance gate may not be the source of the reported gains.","rationale":"I considered the reader's flagged assumption about IResNet100 pretraining overlap with test identities. That concern is plausible but less decisive: the identity-only classifier must still learn a mapping from embeddings to real/fake labels on FF++, so pretraining overlap with test identities does not by itself explain cross-dataset AUC gains, and the paper's Table 5 auxiliary-feature ablation partially addresses whether the gain is due to identity semantics versus other feature sources. The concern I find more load-bearing is that the distinctive mechanism of SELFI, the per-sample relevance gate, is never compared against static fusion in any table. The only non-adaptive identity baseline is concatenation (FAIA-only), and the incremental gain of IAFM over it is small, within likely seed noise, and inconsistent across backbones. The cross-manipulation table also lacks a static-fusion SELFI row. The paper's controlled ablations and backbone-agnostic experiments are genuine strengths, and the authors' limitations paragraph is honest about identity-embedding quality; however, the central claim that adaptive selection is preferable to either extreme requires isolating the gate itself. Since the missing comparison is a straightforward addition rather than a fundamental flaw, the reader's CONDITIONAL verdict stands, and I flag this as an additional condition rather than a verdict change.","tokens_in":14396,"tokens_out":6852,"duration_ms":76293,"concrete_test":"Re-run the Table 3 and Table 2 protocols with three additional fusion baselines using the same CLIP backbone and training setup: (a) fixed convex combination with rho=0.5; (b) a single scalar rho learned globally, not conditioned on the input; (c) the full IAFM but with the gate input f_fi permuted across samples or removed. Report frame-level AUC on CDFv2, DFD, DFDCP, and the FF++ cross-manipulation grid, with at least 3 seeds and mean plus/minus standard deviation. If (a) or (b) matches SELFI within one standard deviation, or if (c) does not degrade performance, then the per-sample relevance mechanism is not responsible for the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that per-sample relevance gating (Eqs. 4-5) is preferable to both unconditional identity use and identity suppression. The controlled evidence for this claim is Table 3: FAIA-only (concatenation, no gate) averages 0.845, and adding IAFM gives 0.849, a 0.4% gain, while on DFDCP performance drops from 0.844 to 0.830. Appendix Table 8 shows the same inconsistency: on EfficientNet-B4, IAFM lowers the average from 0.759 to 0.745; on ResNet34 it raises it from 0.749 to 0.761. No seeds or error bars are reported, so these differences are within plausible noise. More importantly, no ablation compares IAFM against a static fusion rule, such as a fixed scalar rho=0.5, a global learned scalar, or simple averaging. Because Eq. 5 is a convex combination that discards part of the identity stream when rho<1, any gain over concatenation could reflect reduced input dimensionality or regularization rather than per-sample selection. The cross-manipulation results in Table 2 compare SELFI against CLIP and prior methods, but not against SELFI with static fusion, so they do not isolate the adaptive mechanism. The identity-vs-other-source ablation in Table 5 addresses ensemble effects, but not adaptivity. Thus the distinctive claim of 'selective' fusion is not yet supported by the presented evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the role of face identity in deepfake detection and proposes SELFI, a framework that extracts identity embeddings from a frozen face recognition model (IResNet100), projects them into a forgery-relevant space via a trainable adapter (FAIA), and adaptively fuses them with visual features through a per-sample relevance gate (IAFM). The authors first report that identity embeddings alone can classify real vs. fake faces, and that their transferability varies across manipulation types. They then evaluate SELFI trained on FaceForensics++ on four cross-dataset benchmarks (CDFv2, DFD, DFDC, DFDCP), reporting average frame-level AUC improvements over prior methods, and provide ablations over modules, backbones, and auxiliary feature sources. The central claim is that selective, relevance-guided fusion of identity is preferable to both suppressing identity and using it unconditionally.","tokens_in":14624,"tokens_out":4325,"duration_ms":43416,"significance":"If established, the paper would help reconcile conflicting views on identity in deepfake detection by showing that the utility of identity cues is manipulation-dependent and that a learned per-sample gate can exploit this. The work is empirically grounded: cross-dataset test sets are held out from training, the identity extractor is frozen, and the auxiliary-source ablation in Table 5 partially controls for ensemble effects. The controlled comparisons across backbones in Table 4 (e.g., +5.6% average AUC for CLIP) are a strength, and the proposed modules are simple and potentially reusable. However, the distinguishing element of the method—adaptive per-sample fusion—is not yet isolated from static fusion, and the possibility of identity leakage through the pretrained face recognizer is not addressed. The paper is therefore of interest to the community, but the evidence for the 'selective' mechanism requires additional experiments.","major_comments":[{"comment":"The claim that per-sample relevance gating (IAFM) is preferable to static identity usage is not supported by the presented evidence. On CLIP (Table 3), adding IAFM to FAIA-only changes average AUC from 0.845 to 0.849, and on DFDCP the score drops from 0.844 to 0.830; on EfficientNet-B4 (Table 8) the average drops from 0.759 to 0.745, while on ResNet34 it rises from 0.749 to 0.761. These differences are small and inconsistent, and no static-fusion baseline is reported—for example, a fixed scalar rho=0.5, a global learned scalar, or simple averaging of the two feature vectors. Because Eq. (5) is a convex combination that discards part of the identity stream when rho<1, any gain over concatenation could be due to reduced input dimensionality or implicit regularization rather than per-sample selectivity. The paper should compare IAFM against static fusion rules and report multiple seeds and error bars to establish that the adaptive mechanism itself is beneficial.","section":"§5.3, Tables 3 and 8; Eq. (5)"},{"comment":"The frozen IResNet100 model used for identity embeddings is pretrained on large-scale face recognition datasets (e.g., MS-Celeb-1M), which may contain identities that appear in the evaluation benchmarks CDFv2 and DFDC. If the recognizer has memorized those identities, the reported cross-dataset gains could be inflated by identity memorization rather than by a transferable forgery cue. The paper neither measures the overlap between the recognizer's training identities and the test identities nor ablates the effect, for example by evaluating on identities absent from the recognizer's training set. This is load-bearing for the headline cross-dataset results and should be addressed.","section":"§1 and §3.1, Eq. (1)"},{"comment":"The module and backbone ablations in Tables 3 and 4 omit the DFDC benchmark, while Table 5 and the headline results include it. DFDC is the benchmark on which the largest claimed gain (6%) is based, so its absence from the ablations makes it difficult to verify that the improvement on DFDC is attributable to the proposed modules. Please report the missing DFDC numbers for the ablation rows, or explicitly state why DFDC was excluded from these experiments.","section":"§5.3, Tables 3–5"}],"minor_comments":[{"comment":"The notation for the auxiliary loss is inconsistent: Eq. (7) and the text use L_fag, while Table 8 and some ablation descriptions use L fs; the table header also writes 'EfficientNetb4' instead of 'EfficientNet-B4'. Please unify the notation throughout.","section":"§4.3 and Table 8"},{"comment":"There is a typo in the row label 'Random Intialization'—it should be 'Random Initialization'.","section":"Table 5"},{"comment":"The abstract states that SELFI outperforms prior state-of-the-art by an average of 3.1% AUC, while the text in §5.2 says it achieves an average improvement of 3.5% over LSDA. Please clarify which comparison is being reported and the exact sets of datasets, since these numbers differ.","section":"§5.2"},{"comment":"The text cites specific numbers such as '84.6% → 74.5%' in describing Fig. 2(b), but the corresponding Table 6 reports 0.846 and 0.745 in different row/column positions; the figure and table should be cross-referenced more clearly so the reader can map the patterns to the quantitative results.","section":"§3.2 and Table 6"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable empirical study with a useful negative/positive finding about identity features, but the distinctive 'selective fusion' claim needs a static-fusion control and seed variance analysis. The identity-leakage concern is also significant in this benchmark-based field and should be addressed before the paper can be recommended for acceptance. The missing DFDC numbers in the key ablations are an editorial gap that should be fixed. Given the scope of the requested experiments, major_revision seems appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: the part its title advertises—selective, relevance-gated fusion—is not supported by the experiments as written. The controlled ablations show identity features help, but they never compare the adaptive gate against a static fusion rule (fixed rho, global learned scalar, simple averaging). The only evidence for the gate is Table 3's 0.4% average gain over concatenation, which goes negative on DFDCP, and the appendix shows it negative on EfficientNet-B4. Without seeds or error bars, that difference is noise. So the 'selective' claim needs extra work; the 'identity helps' claim is on firmer ground.\n\nWhat's actually new: the empirical categorization of identity cue behavior across manipulation types—transferable for DF/FS, method-specific for F2F, ineffective for NT—is useful and not something I'd seen laid out so directly. The identity-only detection result with a frozen recognizer is also a clean demonstration. And the auxiliary-source ablation (random vs ImageNet vs deepfake-trained vs face identity) is a good control showing the gain is not just ensemble size.\n\nSoft spots, in order of severity. First, the missing static-fusion baseline means the core mechanism of IAFM is not isolated; a convex combination that downweights the identity stream could help via regularization or reduced dimensionality. That is the main issue. Second, no error bars or multiple seeds anywhere; given the small delta in question, that matters. Third, the headline SOTA comparison in Table 1 mixes backbones (CLIP vs LSDA and others), so the 3.1% average improvement is not attributable to SELFI alone; the controlled Table 4 comparisons against the same backbone are more honest, and they are decent. Fourth, the frozen IResNet100 was trained on massive face data that likely overlaps with celebrities in CDFv2 and DFDC; the paper does not measure this, so some of the cross-dataset gain could be identity memorization. That is a concrete leak risk, not a fatal blow, but it needs a check. Minor: Table 2's protocol is not described, and code isn't out yet.\n\nIf I were refereeing, I'd ask for a static-fusion control and repeated runs; the identity analysis alone is worth reporting, and the overall direction seems sound. This is not a desk reject; it is a revise-and-resubmit with a specific experimental gap.","headline":"SELFI's adaptive gating claim is not isolated by the experiments; the identity-feature analysis is solid, but the paper needs a static-fusion baseline and error bars before the headline mechanism can be believed.","tokens_in":15260,"tokens_out":2281,"would_cite":false,"duration_ms":23334,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adaptively gating face identity features per image—rather than always suppressing or always using them—improves deepfake detection generalization across datasets and manipulation types.","keywords":["deepfake detection","face identity","adaptive fusion","generalization","cross-dataset evaluation","forgery-aware identity adapter","identity-aware fusion module"],"falsifier":"Retrain SELFI with a face recognizer that provably never saw the benchmark identities—for example, trained only on synthetic faces or on a disjoint identity set—and compare cross-dataset AUC. If the reported 3.1% average gain and 6% DFDC gain shrink sharply or disappear, the gains depend on identity memorization rather than on a generalizable forgery cue.","tokens_in":14079,"feed_emoji":"🕵️","tokens_out":11107,"duration_ms":92939,"temperature":0.7,"pith_summary":"The paper sets out to settle a dispute in deepfake detection: should a detector suppress face identity cues as a source of bias, or lean on them as forensic evidence? Its answer, supported by experiments, is neither: identity is genuinely informative, but its value depends on the manipulation method. SELFI therefore extracts identity embeddings from a frozen face recognizer, projects them into a forgery-relevant space, and uses a per-image relevance score to blend them with visual features. On four cross-dataset benchmarks the framework reports an average gain of 3.1% frame-level AUC over prior methods, with a 6% margin on the DFDC benchmark. A sympathetic reading is that the paper's core contribution is evidence that the identity-versus-visual trade-off should be learned per sample, not decided in advance.","feed_headline":"Selective identity fusion lifts deepfake AUC by 6%","feed_subtitle":"SELFI weighs identity cues per image, adding 3.1% average AUC across four benchmarks.","key_machinery":"The load-bearing mechanism is the soft fusion rule in Eq. (5): $f_{\\text{fused}} = \\rho \\cdot f_{\\text{fi}} + (1-\\rho) \\cdot f_{\\text{vis}}$, where $\\rho$ is a scalar in $[0,1]$ produced by a two-layer relevance predictor that takes the concatenated identity and visual features. The identity feature comes from FAIA: a frozen face-recognition embedding (IResNet100, 512 dimensions) projected by a learnable matrix into the visual backbone's feature space, with a forgery-aware cross-entropy loss applied to that projection so the identity space is aligned with the fake-detection task. The gate $\\rho$ is what lets the model amplify identity when it helps and suppress it when it would hurt.","core_discovery":"The paper's central claim is that selective identity use beats both extreme policies: discarding identity features entirely or trusting them unconditionally. The empirical discovery underneath it is that identity embeddings alone can separate real from fake faces, but they generalize unevenly across manipulation types—transferable for DeepFakes and FaceSwap, method-specific for Face2Face, and largely ineffective for NeuralTextures. SELFI operationalizes selectivity by predicting a relevance scalar $\\rho$ for each input and computing the fused feature as $\\rho$ times the projected identity feature plus $(1-\\rho)$ times the visual feature, trained jointly with a forgery-aware guidance loss on the identity branch. If the claim holds, identity should be treated as a conditional forensic cue whose weight is learned from the input, not fixed by design choice.","pith_inferences":["A natural check on the claim is whether the frozen face recognizer's training data overlaps with the benchmark identities; if overlap exists, part of the reported cross-dataset gain could be identity memorization rather than a transferable forgery cue. The paper does not measure this overlap.","A testable extension is to replace the single scalar gate with a per-channel or per-region gating vector, which may preserve identity cues where they are reliable and discard them where they are not.","The same per-sample relevance mechanism could be applied to other strong semantic features such as pose, expression, or lighting, turning SELFI from an identity-specific fix into a general template for bias-aware fusion.","Since the paper itself notes that identity embeddings degrade for partially visible or extreme-angle faces, conditioning the relevance gate on face quality or landmark confidence is a concrete next step."],"forward_implications":["Cross-dataset generalization improves by an average of 3.1% frame-level AUC across Celeb-DF v2, DFD, DFDC, and DFDCP, and by 6% on DFDC.","Identity-aware features outperform other auxiliary sources—random, ImageNet, and deepfake-trained features—by 13.9 to 23.8% AUC, indicating the gain comes from identity semantics, not feature ensembling.","The fusion module transfers across backbones, adding 5.6% average AUC to CLIP, 2.6% to ResNet34, and 1.6% to EfficientNet-B4.","In cross-manipulation tests, training on one manipulation type and testing on others, SELFI beats CLIP and the implicit-identity baseline by up to 4.0% average AUC."],"supporting_citations":[{"why":"Supplies the frozen IResNet100 face-recognition model whose 512-d identity embeddings carry the identity signal.","marker":"[9]"},{"why":"Backbone (CLIP) that provides the visual features and the 768-d space into which identity features are projected.","marker":"[33]"},{"why":"FaceForensics++ is the training set and the source of the four manipulation types used in the cross-manipulation analysis.","marker":"[34]"},{"why":"Celeb-DF v2 is one of the four cross-dataset test benchmarks.","marker":"[26]"},{"why":"DeepfakeDetection (DFD) is one of the four cross-dataset test benchmarks.","marker":"[2]"},{"why":"Provides the DFDC and DFDCP test benchmarks, including the DFDC set where SELFI reports a 6% improvement.","marker":"[19]"},{"why":"A recent state-of-the-art cross-dataset detector whose reported scores SELFI claims to surpass by 3.5% average AUC.","marker":"[45]"},{"why":"Provides the benchmark protocol and baseline scores used in the main comparison tables.","marker":"[47]"},{"why":"Represents the view that identity is a harmful implicit bias, the position SELFI argues against.","marker":"[10]"},{"why":"Represents the opposing view that identity is a strong forensic cue, the position SELFI also qualifies.","marker":"[18]"}],"fun_headline_variants":["Identity cues help deepfake detection only when used selectively","SELFI adapts identity use per face, boosting deepfake AUC 6% on DFDC","Don't suppress or trust identity blindly—selective fusion wins","Adaptive identity weighting lifts deepfake detection by 6% AUC"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim presumes that the frozen face recognizer's identity embeddings are a transferable and leakage-free representation of face identity for the test benchmarks, with no overlap between the recognizer's training identities and the benchmark faces.","fun_headline_variants_meta":{"raw":{"variants":["Identity cues help deepfake detection only when used selectively","SELFI adapts identity use per face, boosting deepfake AUC 6% on DFDC","Don't suppress or trust identity blindly—selective fusion wins","Adaptive identity weighting lifts deepfake detection by 6% AUC"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000281,"raw_usage":{"total_tokens":1684,"prompt_tokens":984,"completion_tokens":700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":622}},"tokens_in":600,"tokens_out":700,"duration_ms":7216,"temperature":1.0,"reasoning_tokens":622,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:06:36.149153+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain SELFI with a face recognizer that provably never saw the benchmark identities—for example, trained only on synthetic faces or on a disjoint identity set—and compare cross-dataset AUC. If the reported 3.1% average gain and 6% DFDC gain shrink sharply or disappear, the gains depend on identity memorization rather than on a generalizable forgery cue.","supporting_citations":[{"cited_title":"Contributing data to deepfake detection research, 2019","cited_arxiv_id":null,"evidence_quote":"DeepfakeDetection (DFD) is one of the four cross-dataset test benchmarks."},{"cited_title":"Deepfake detection challenge","cited_arxiv_id":null,"evidence_quote":"Provides the DFDC and DFDCP test benchmarks, including the DFDC set where SELFI reports a 6% improvement."},{"cited_title":"Implicit identity leakage: The stumbling block to improving deepfake detection generalization","cited_arxiv_id":null,"evidence_quote":"Represents the view that identity is a harmful implicit bias, the position SELFI argues against."},{"cited_title":"Implicit identity driven deepfake face swapping detection","cited_arxiv_id":null,"evidence_quote":"Represents the opposing view that identity is a strong forensic cue, the position SELFI also qualifies."}],"review_version":2}