{"id":"deb0d4a4-8609-4536-b811-8ce64f092b82","arxiv_id":"2608.09427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Fake images and videos systematically produce lower-magnitude features in pretrained models, so a simple norm-based score detects deepfakes without training a classifier.","lead":"Fake media tends to activate pretrained AI models less strongly than real media, and this simple magnitude difference can flag deepfakes without any training. The finding yields a competitive zero-shot detector and shows the signal strengthens as foundation models grow.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Section 3 causal-attribution experiment does not isolate semantic shift from low-level fingerprints, leaving the semantic-shift mechanism and its generalization claims underdetermined.","rationale":"The reader's weakest assumption correctly identifies the causal-attribution gap as the most load-bearing concern. The detection phenomenon itself—lower magnitude and lower L1/L2 ratio for fakes across many benchmarks—is clearly and plausibly documented, so I do not see a basis for rejection. However, the paper's headline interpretation that the effect is 'primarily associated with semantic shifts' and the implied promise of robustness to unseen generators depend on isolating semantic shift from low-level generative fingerprints. The Section 3 control does not achieve this isolation: the SD1.5 autoencoder reconstruction is a weak proxy for full generative fingerprints, and the chosen OOD-real datasets are confounded by low-level differences. I also considered the post hoc layer selection as an alternative concern; it is real and should be addressed with a validation split, but it affects the magnitude of the reported numbers rather than the existence of the phenomenon or the causal story. Therefore the reader's CONDITIONAL verdict remains appropriate, and my stress-test does not change it. The concrete test above would settle whether the low-level confound actually explains the norm gap, which is the condition on which the paper's more ambitious claims depend.","tokens_in":19949,"tokens_out":8517,"duration_ms":99412,"concrete_test":"On GenImage SD1.5, take a matched set of real images and apply three conditions: (i) SD1.5 VAE reconstruction exactly as in Section 3; (ii) low-level transforms (JPEG re-encoding at several quality levels, Gaussian blur, resize) matched so the transformed real set has the same sharpness/quality histograms as the fake set; (iii) the original real and true SD1.5 fake sets. Compute DINOv3-7B CLS-token L1-norm distributions for all conditions. If the transformed-real distribution in condition (ii) overlaps the fake distribution, low-level artifacts alone can explain the norm gap and the semantic-shift conclusion is unsupported. If condition (ii) stays near the real distribution while fakes remain low, the semantic mechanism is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central interpretation of the paper—that lower feature magnitude marks semantic shift and therefore generalizes to unseen generators—rests on Figure 2, but that experiment is confounded. The reconstruction control uses only the SD1.5 VAE encode-decode path, which injects decoder-specific artifacts but omits the full generative pipeline: diffusion sampling noise, prompt conditioning, and generator-specific frequency spectra that characterize actual SD1.5 fakes. A small norm shift for VAE reconstructions therefore does not bound the low-level contribution present in real fakes. Similarly, the OOD-real controls (EuroSAT and MAVOS-DD Arabic) differ from the encoder's pretraining distribution in sensor, resolution, compression, and language, not only in semantics, so their low norms are consistent with a low-level mechanism. The paper's own qualitative examples reinforce this worry: blurry or cluttered real frames receive low norms, and fake images with low norms are described as having a 'synthetic style.' This leaves a plausible alternative reading in which NormFake detects compression, blur, or other low-level generative artifacts rather than semantic unfamiliarity. The empirical detection result on the tested benchmarks is well documented and may survive, but the load-bearing explanatory and generalization claim is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a consistent empirical phenomenon: frozen self-supervised foundation models produce lower-magnitude feature representations for fake media than for real media, across image and video domains and several backbones. The authors operationalize this as two parameter-free scores, magnitude(h)=||h||_1 and sparsity(h)=||h||_1/||h||_2, and show that these scores alone achieve competitive deepfake detection ROC-AUC on GenImage and on six audio-visual video benchmarks, without training a classifier. They also report analyses aimed at attributing the norm gap to semantic shift rather than low-level generative fingerprints, and a scaling study showing that larger DINOv3 backbones yield stronger detection. The central detection claim is straightforward, falsifiable, and largely supported by the reported tables; the causal attribution and the universality of the phenomenon are less well supported.","tokens_in":20160,"tokens_out":5147,"duration_ms":54587,"significance":"If the core phenomenon holds, this is a valuable and surprising result: a zero-shot, parameter-free statistic of frozen SSL features can compete with trained deepfake detectors, and the connection to the familiarity hypothesis in OOD detection is conceptually useful. The method has no fitted parameters and no calibration stage, which rules out a circular-fitting concern for the detector itself; the paper also evaluates on a broad set of benchmarks and generators and includes informative comparisons with prior real-only and fake-aware methods. The main significance risk is that the paper's explanatory claim—that the effect is primarily semantic—rests on a confounded control experiment, and that the reported test-set layer selection and absence of error bars make the precise quantitative claims less secure. With additional controls and more careful statistical reporting, the contribution would be a strong baseline for the field.","major_comments":[{"comment":"The experiment intended to isolate semantic shift from low-level generative fingerprints is confounded. The SD1.5 autoencoder reconstruction injects only the VAE encoder-decoder path, omitting the diffusion sampling noise, prompt conditioning, and generator-specific frequency characteristics that are present in actual SD1.5 fakes; a small norm shift for VAE reconstructions therefore does not bound the low-level contribution in real fakes. Similarly, the out-of-distribution real controls (EuroSAT and MAVOS-DD Arabic) differ from the encoder's pretraining distribution in sensor, resolution, compression, and language, not only in semantics. Since the qualitative examples in Section 5 show that blurry or cluttered real frames receive low norms, the data are consistent with a low-level mechanism as well as a semantic one. The conclusion that reduced feature magnitude is 'primarily associated with semantic shifts' is therefore underdetermined. I would ask for additional controls, for example full-pipeline generation with fixed semantics, low-level-only perturbations such as compression or blur, and out-of-distribution real data matched in acquisition statistics, before the attribution claim is accepted.","section":"Section 3, 'Why does magnitude differ?' and Figure 2"},{"comment":"The quantitative comparison is weakened by test-set layer selection and the absence of error bars. The caption of Table 2 states that for NormFake 'we report the best-performing layer for each variant,' and the text explains that the sparsity variant uses the penultimate block; this layer choice is made after inspecting test performance, so the reported 95.1% mean may be optimistic. In addition, all ROC-AUC values in Tables 1-3 and Figures 6-7 are single point estimates with no standard errors or confidence intervals, so statements such as 'trailing by just one percentage point' (Table 1) or 'outperforming' prior methods (Tables 1 and 2) cannot be evaluated for statistical significance. I recommend reporting confidence intervals and a validation-based or pre-specified layer-selection rule, or explicitly labeling the reported numbers as oracle-layer results.","section":"Section 4, Tables 1 and 2, and supplementary Figure 8"},{"comment":"The claim that NormFake provides 'consistently strong performance across all evaluated backbones' is not supported by the numbers in Table 3. On GenImage, NormFake (magnitude) achieves 57.3% for BEiT-L, 61.9% for OpenCLIP-G/14, and 73.5% for SigLIP2-Giant, while PE-Core-G14 and DINOv3-7B reach 87.7% and 86.9%. The abstract's statement that the phenomenon holds 'across multiple pretrained models' should therefore be qualified: the effect is strong for some backbones but weak or near-chance for others, and the conditions under which the norm gap appears remain to be characterized. This matters because the paper's framing as an 'implicit' property of foundation models depends on the breadth of the empirical generalization.","section":"Section 5, Table 3"}],"minor_comments":[{"comment":"The sentence 'rather the the low-level fingerprints' contains a typo and should read 'rather than the low-level fingerprints.'","section":"Section 3, paragraph after Figure 2"},{"comment":"The sentence 'This proves that NormFake, despite its simplicity, acts as a strong baseline' is too strong given the single point estimates in Table 1; 'demonstrates' or 'suggests' would be more appropriate.","section":"Section 4, 'Evaluation: Video deepfake detection'"},{"comment":"The caption contains '6,7B' and the horizontal axis labels include spaces such as 'ViT -S'; these formatting issues should be corrected.","section":"Figure 6 caption and axis labels"},{"comment":"There are several typos, including 'Evaluted' in the Table 3 caption, 'seperability' in Section 5, and 'supplmentary' in Section 4; a careful proofreading pass is needed.","section":"Supplementary material, Section 9 and main text"},{"comment":"The limitations paragraph on pretraining-data contamination is welcome, but it does not mention the potential sensitivity of NormFake to low-level image quality factors such as compression or blur; adding this to the limitations would align the paper with its own qualitative observations.","section":"Section 6, Limitations"}],"recommendation":"major_revision","confidential_remarks":"The empirical detection result is likely to survive revision, and the parameter-free nature of NormFake is a genuine strength. The main risk is over-interpretation of the attribution experiment: I would require additional controls for isolating semantic shift from low-level fingerprints, a validation-based or pre-specified layer-selection protocol, and confidence intervals for the main ROC-AUC comparisons before recommending acceptance. I do not see a circularity problem in the detector itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper's core empirical claim—frozen self-supervised features scored by L1 norm or L1/L2 ratio give a solid, training-free deepfake detector across a wide range of image and video benchmarks—survives reading. The causal interpretation attached to it, that the effect is driven primarily by semantic shift, does not; the supporting experiment is confounded. This is still a useful paper and should go to review, but the authors need to fix the reporting and soften the mechanism claim.\n\nWhat is actually new: prior training-free methods used calibrated distance scores, whitened-space likelihoods, or reconstruction error. Showing that the raw feature magnitude (and even more so the L1/L2 sparsity ratio) separates real from fake across multiple backbones, eight generators, and six video benchmarks is not something I've seen laid out this systematically. The scale plot—bigger DINOv3 models give better zero-shot detection—is a genuinely useful finding. And the comparison tables are informative: NormFake is competitive with methods trained on fake data, which is surprising. The authors also cite the familiarity-hypothesis literature and acknowledge the connection to OOD feature-norm effects, so they are not overselling novelty.\n\nThe soft spot is the causal story. Section 3 claims semantic shift matters more than low-level fingerprints, but the two controls do not isolate these factors. The SD1.5 reconstruction passes only through the autoencoder encode-decode path; no sampling noise, no prompt conditioning, so it does not actually inject the artifacts that real SD1.5 generations have. EuroSAT and MAVOS-DD Arabic are used as 'semantic' OOD, but they also differ in sensor, resolution, compression, and language. The paper's own qualitative figures show blurry or cluttered real frames receiving low norms, which is consistent with a low-level or difficulty-based mechanism. So the phrase 'primarily semantic shift' is stronger than the evidence.\n\nSmaller issues: the layer choice for the sparsity variant is selected after inspecting test performance (the penultimate block on GenImage), and there is no validation split for that selection. All tables are single ROC-AUC point estimates with no confidence intervals, and no code is provided. These are fixable, but they matter for trusting the exact numbers. None of this breaks the central detection claim; it just means the paper should be read as a strong empirical baseline with an open mechanism.\n\nBottom line: if you build or benchmark deepfake detectors, NormFake is a baseline you will want to cite. Send it to review, but the authors should release code, pick layers on a validation split, report uncertainty, and either run a cleaner mechanism experiment or drop the 'semantic shift' phrasing.","headline":"A strong training-free deepfake baseline from feature norms, with a causal interpretation the evidence doesn't yet support.","tokens_in":20684,"tokens_out":3873,"would_cite":true,"duration_ms":38126,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fake images and videos systematically yield lower-magnitude, sparser representations in pretrained self-supervised foundation models, and a score based on those statistics detects deepfakes competitively without any learned classifier.","keywords":["deepfake detection","self-supervised learning","feature norm","sparsity","anomaly detection","out-of-distribution detection","foundation models","generated image detection"],"falsifier":"A decisive experiment would hold semantics fixed while varying only the generative pipeline: take real images from a domain the encoder knows well, create a 'fake' set with a generator fine-tuned on that same domain (so content is semantically in-distribution), and check whether the $\\ell^1$-norm gap appears; a gap here would mean low-level fingerprints, not semantic shift, carry the signal, overturning the paper's causal explanation.","tokens_in":19699,"feed_emoji":"🎭","tokens_out":11997,"duration_ms":99863,"temperature":0.7,"pith_summary":"The paper claims that pretrained self-supervised foundation models are already implicit deepfake detectors: fake images and videos produce feature vectors with systematically lower magnitude and higher sparsity than real ones, across models, datasets, and both image and video domains. On this basis the authors propose NormFake, which scores a sample by either the $\\ell^1$ norm of its representation or the $\\ell^1/\\ell^2$ sparsity ratio, and show that these two statistics rival far more complex trained detectors, reaching 95.1% mean ROC-AUC on the GenImage image benchmark and 82.4% on six audio-visual video benchmarks. They attribute the effect mainly to semantic shift: generated content lies outside the real-world distribution the encoder was pretrained on, so the model encodes it with weaker, more concentrated activations, while low-level generative fingerprints play a comparatively small role. If the claim is right, representation learning itself yields a scalable, classifier-free deepfake-detection signal that strengthens as foundation models grow.","feed_headline":"A single feature statistic catches deepfakes without a classifier","feed_subtitle":"Fake content makes foundation models output smaller, sparser feature vectors—enough to beat specialized detectors.","key_machinery":"The load-bearing object is the representation-statistic score, defined directly on the frozen features of a pretrained self-supervised encoder: magnitude($h$) = $\\|h\\|_1$ and sparsity($h$) = $\\|h\\|_1/\\|h\\|_2$, where $h$ is the feature vector of an image (the CLS token for DINOv3) or of a video frame (the visual AV-HuBERT embedding). The sparsity ratio is the Hoyer sparsity measure: invariant to global rescaling and a numerically stable lower bound on true $\\ell^0$ sparsity; both statistics are lower for fake than for real samples, so the fakeness score is their negative. These statistics carry the entire argument because no classifier, learning step, or calibration is applied, and the implicit anomaly-detection capability of the foundation model is read off directly. The causal analysis additionally rests on two controlled perturbations, out-of-domain real inputs and autoencoder-reconstructed real inputs run through the same encoders, used to attribute the norm gap to semantic shift rather than low-level fingerprints.","core_discovery":"The central discovery is a property of frozen self-supervised representations rather than a new detector architecture: across diverse backbones (including DINOv3-7B for images and AV-HuBERT for video), fake samples consistently yield features with lower $\\ell^1$ norm than real samples, and the ratio $\\|h\\|_1/\\|h\\|_2$ shows that fake representations are also sparser, meaning their activation energy concentrates in fewer dimensions. The paper operationalizes this as anomaly detection: because foundation models are pretrained on real media, real inputs are in-distribution and produce large, diffuse activations, while fake inputs are out-of-distribution and produce small, concentrated activations, so simply negating the magnitude or sparsity score separates the classes without any learned classifier, and the separation widens with model scale. The authors trace the cause to semantic shift using two controls: out-of-domain real data (EuroSAT satellite imagery and MAVOS-DD Arabic videos) that also show reduced norms but not as low as fakes, and SD1.5 autoencoder reconstructions of real images that inject generative fingerprints while preserving semantics and stay close to the real distribution. They conclude that semantic deviation, not pixel-level fingerprints, is the dominant driver of the norm gap.","pith_inferences":["Editorial inference: if the norm gap is really tracking pretraining-distribution shift, then NormFake's reliability is hostage to pretraining data; a future self-supervised encoder trained on corpora already containing large amounts of generated media should show a compressed norm gap, exactly the contamination failure the paper lists as a limitation.","Editorial inference: the same magnitude/sparsity statistics may extend to audio-only deepfake detection, but the paper's own audio experiments on FakeAVCeleb (sparsity near or below chance) suggest the phenomenon is far weaker outside the visual domain, so any such extension would need fresh evidence rather than an assumption of transfer.","Editorial inference: NormFake could serve as a prior or regularizer for trainable detectors, for example by penalizing a learned classifier whenever its features' $\\ell^1/\\ell^2$ ratio leaves the range typical of real samples, which the paper names as future work but does not test.","Editorial inference: the strong correlation between NormFake and audio-visual synchronization methods (SpeechForensics, FACTOR) hints that a single underlying failure, semantic drift of generated media, produces both visual feature shrinkage and audio-visual desynchronization, so detectors across modalities may be measuring facets of the same defect rather than independent cues."],"forward_implications":["A frozen foundation model becomes a zero-shot deepfake detector: NormFake (sparsity) reaches 95.1% mean ROC-AUC across eight generators in GenImage without ever seeing a fake sample during training.","The signal transfers across domains and generators: on six audio-visual video benchmarks NormFake (sparsity) averages 82.4% ROC-AUC using only visual AV-HuBERT features, outperforming several fake-aware and real-only methods and landing about one point behind the best multimodal baseline.","The discriminative strength scales with backbone size: average GenImage performance rises steadily from the 21M-parameter ViT-S to the 6.7B-parameter ViT-7B, with gains saturating near ViT-H+, while two prior real-only baselines degrade when moved to the larger DINOv3-7B backbone.","Because the effect is dominated by semantic shift, NormFake works best on generators whose outputs deviate semantically (continuous latent diffusion models such as SD1.4/1.5, Wukong, Midjourney) and is comparatively weaker on generators whose fakes are semantically close to real images (BigGAN, VQDM), where pixel-level artifact detectors excel.","Per-layer analysis shows the deepest representations separate real from fake best, and for AV-HuBERT the final LayerNorm substantially restores separability that the raw last transformer block loses, indicating the signal is carried by semantic features rather than early low-level cues."],"supporting_citations":[{"why":"Supplies DINOv3, the self-supervised image encoder whose CLS-token features (including the 7B model) exhibit the lower-magnitude/lower-sparsity pattern on GenImage.","marker":"Siméoni et al. 2025"},{"why":"Supplies AV-HuBERT, the audio-visual self-supervised model whose visual-only features carry the norm gap on the video benchmarks.","marker":"Shi et al. 2022"},{"why":"Supplies GenImage, the million-scale image benchmark with eight generators and class-aligned real/fake pairs used for all image evaluations.","marker":"Zhu et al. 2023"},{"why":"Supplies FakeAVCeleb, the audio-visual deepfake dataset used to establish the phenomenon and evaluate video performance.","marker":"Khalid et al. 2021"},{"why":"Defines the sparsity ratio ||h||1/||h||2 used as the NormFake (sparsity) score; the paper cites it as the source of the sparsity measure.","marker":"Hoyer 2004"},{"why":"Establishes that the ℓ1/ℓ2 ratio is a numerically stable lower bound on true ℓ0 sparsity, justifying its use as a discriminative statistic.","marker":"Lopes 2013"},{"why":"Provides the familiarity hypothesis that the paper invokes to explain why out-of-distribution (fake) inputs produce lower-magnitude features.","marker":"Dietterich and Guyer 2022"},{"why":"Supplies the SD1.5 autoencoder used to reconstruct real GenImage samples, injecting generative fingerprints while preserving semantics in the causal analysis.","marker":"Rombach et al. 2022a"}],"fun_headline_variants":["Fake media shrinks AI feature vectors - a built-in deepfake detector","No training needed: Foundation models flag deepfakes via feature size","Semantic gap, not pixel noise, is why foundation models detect fakes","Scale up any foundation model, get a better zero-shot deepfake spotter","Deepfake detection emerges from representation magnitude, no classifier"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that fake media get smaller feature norms because they fall outside the encoder's pretraining distribution in a semantic sense, not because of low-level generative fingerprints or coincidental dataset differences; if the norm gap is actually driven by low-level artifacts, NormFake's generalization to unseen generators and domains is not assured.","fun_headline_variants_meta":{"raw":{"variants":["Fake media shrinks AI feature vectors - a built-in deepfake detector","No training needed: Foundation models flag deepfakes via feature size","Semantic gap, not pixel noise, is why foundation models detect fakes","Scale up any foundation model, get a better zero-shot deepfake spotter","Deepfake detection emerges from representation magnitude, no classifier"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00073,"raw_usage":{"total_tokens":3269,"prompt_tokens":946,"completion_tokens":2323,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":562,"completion_tokens_details":{"reasoning_tokens":2229}},"tokens_in":562,"tokens_out":2323,"duration_ms":18390,"temperature":1.0,"reasoning_tokens":2229,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:33:53.755784+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive experiment would hold semantics fixed while varying only the generative pipeline: take real images from a domain the encoder knows well, create a 'fake' set with a generator fine-tuned on that same domain (so content is semantically in-distribution), and check whether the $\\ell^1$-norm gap appears; a gap here would mean low-level fingerprints, not semantic shift, carry the signal, overturning the paper's causal explanation.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies AV-HuBERT, the audio-visual self-supervised model whose visual-only features carry the norm gap on the video benchmarks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies GenImage, the million-scale image benchmark with eight generators and class-aligned real/fake pairs used for all image evaluations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies FakeAVCeleb, the audio-visual deepfake dataset used to establish the phenomenon and evaluate video performance."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the sparsity ratio ||h||1/||h||2 used as the NormFake (sparsity) score; the paper cites it as the source of the sparsity measure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that the ℓ1/ℓ2 ratio is a numerically stable lower bound on true ℓ0 sparsity, justifying its use as a discriminative statistic."},{"cited_title":"G.; and Guyer, A","cited_arxiv_id":null,"evidence_quote":"Provides the familiarity hypothesis that the paper invokes to explain why out-of-distribution (fake) inputs produce lower-magnitude features."}],"review_version":1}