{"id":"42afd2e7-fc58-408a-b97f-dea6baa8b582","arxiv_id":"2412.05943","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Adversarial transferability in denoisers is attributed to Gaussian noise concentrating on a typical set, and a low-density Gaussian sampling defense (TS) is shown to improve robustness while preserving denoising accuracy.","lead":"Deep-learning image denoising models are easily fooled by tiny adversarial perturbations, and the same perturbation transfers across very different models. This paper explains the transferability through the geometry of Gaussian noise, the typical set, and proposes a cheap noise-sampling defense that improves robustness without hurting denoising quality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorems only prove Gaussian concentration; the causal step from typical-set deviation to denoising failure is asserted in Section 3.4.5, not derived.","rationale":"The paper's strongest claim is that adversarial samples fail because they leave the typical set of the training Gaussian noise. That claim is load-bearing for both the transferability explanation and the TS defense: if low typical-set density is not what makes models fail, the theory reduces to standard concentration-of-measure facts about Gaussian vectors, and TS becomes an unremarkable noise-augmentation scheme. I read the theorems carefully: they are broadly correct information-theoretic statements, but they stop at log-density bounds. The missing link is a statement of the form: for the trained denoiser f, the reconstruction error is large whenever -1/n log f(n+ξ) deviates from h(N(0,σ²)). No such statement appears, and it is not a corollary of the preceding lemmas. The paper itself concedes in Section 5 that 'some insights were not rigorously proven,' yet the abstract retains the word 'prove.' The empirical observations are consistent with a simpler magnitude-based story, and the Table 4 mixed-noise results show that ordinary higher-variance Gaussian augmentation nearly matches the TS defense, which further weakens the proposed mechanism. My concern aligns with the reader's weakest_assumption. The reader's REJECT verdict is appropriate: the central theoretical contribution is an assertion rather than a derivation, and the defense experiments lack matched baselines, variance reporting, and code. I would keep the verdict at REJECT unless the causal link is demonstrated by the proposed log-density-controlled experiment and the TS comparisons are strengthened.","tokens_in":27867,"tokens_out":7640,"duration_ms":84598,"concrete_test":"Controlled log-density ablation on a fixed clean image u and n ~ N(0, σ²I). For a range of perturbation norms λ around the PGD budget, construct two families: ξ_par(λ) = λ n/||n|| and ξ_orth(λ) = λ e, with e ⟂ n, ||e|| = 1. For each input u+n+ξ, compute Δlog = -[log f(n+ξ) - log f(n)] and the PSNR drop ΔPSNR relative to denoising u+n. Plot ΔPSNR as a function of Δlog for both families. The paper's mechanism predicts a single monotone curve independent of direction; if the parallel family shows large ΔPSNR at Δlog values where the orthogonal family shows no effect, then typical-set deviation is not the causal variable—the direction-dependent gradient structure of the attack is.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is causal: adversarial samples fail because they deviate from the typical set of the training Gaussian noise. Section 3.4.5 states: 'deep image denoising models only learn from the typical set during training, they never encounter perturbations that deviate samples from this set. Thus, the models have difficulties to generalize on the adversarial samples.' The preceding theorems do not establish this. Theorem 3.6 bounds the L2 norm of Gaussian noise; Theorems 3.8 and 3.13 show that a perturbed noise n+ξ lies in a larger typical set A_B2 or A_B∞. None of these statements involve the denoiser f, its training objective, or its output error. From 'n+ξ is outside A_ε' it does not follow that f fails; no Lipschitz, smoothness, or distributional condition links log-density to reconstruction error. The phrase 'only learn from the typical set' is a metaphor, not a proven property of trained networks. Observation 2, that only perturbations parallel to the Gaussian noise are harmful, is exactly what needs explanation: for large n, a parallel perturbation of norm η changes log-density by about η√n/σ, while an orthogonal perturbation of the same norm changes it by about η²/(2σ²). Thus the experiments are also consistent with a simpler mechanism: models are sensitive to increases in effective noise magnitude. TS does not rescue the theory: Table 4 shows that mixing ordinary Gaussian noise at levels 25.5 and 26 produces nearly the same robustness gains as TS-Pres. and TS-Def., so TS behaves like generic higher-variance augmentation. Unless a controlled experiment ties denoising error directly to log-density, the abstract's 'prove ... causing the models to fail' is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies adversarial transferability in deep image denoising models. It proposes that all denoising models learn the same Gaussian-noise typical set; adversarial perturbations push the noisy input out of this typical set, causing model failure and explaining why adversarial samples transfer across architectures. The authors prove concentration bounds: Theorem 3.6 locates Gaussian noise on a sphere; Lemmas 3.7 and 3.12 bound the log-density change under L2/L∞ perturbations; Theorems 3.8 and 3.13 show the perturbed sample remains in a larger typical set; Propositions 3.14–3.16 bound typical-set volumes. They then propose TS sampling, which draws Gaussian noise with lower log-density during training, and report experiments on DnCNN, DnCNN-B, CTNet, and DeamNet showing reduced performance loss under Denoising-PGD attacks while preserving Gaussian denoising quality, plus comparisons with mixed-noise training. Central claims: (i) high transferability is caused by the common Gaussian-noise distribution rather than structure or dataset; (ii) adversarial samples deviate from the typical set; (iii) such deviation is the cause of model failure; (iv) TS sampling defends by sampling a larger typical set.","tokens_in":28118,"tokens_out":3965,"duration_ms":32730,"significance":"If the causal story were established, the paper would give a first-principles explanation for a striking phenomenon (near-universal adversarial transferability in denoising) and a parameter-light defense that improves rather than sacrifices denoising quality, with the typical-set mechanism connecting to the existing 'concentration of measure' robustness literature. The paper also ships a self-contained AEP derivation for the Gaussian case and supports it with a hypothesis-testing experimental sequence (excluding architecture and training-data similarity, showing local patch attacks, and demonstrating direction dependence). As it stands, however, the load-bearing claim that typical-set deviation causes model failure is not derived and the defense experiments do not isolate that mechanism, so the significance is conditional on a gap that the manuscript itself acknowledges only implicitly.","major_comments":[{"comment":"The central causal claim is asserted, not proven: the text states that because models 'only learn from the typical set during training, they never encounter perturbations that deviate samples from this set. Thus, the models have difficulties to generalize on the adversarial samples.' None of the preceding theorems (3.6, 3.8, 3.13) involve the denoiser f, its loss, or its output error; they only bound the log-density of the input distribution. No Lipschitz, smoothness, or training-consistency condition links log-density under the training Gaussian to reconstruction error. As written, an input with low Gaussian density could in principle be denoised perfectly (e.g., a constant offset is perfectly corrected by any shift-equivariant denoiser), so the inference from concentration of measure to failure needs a theorem or a controlled experiment.","section":"§3.4.5, item 3"},{"comment":"The observation that only perturbations parallel to the Gaussian noise are harmful is consistent with a simpler mechanism than typical-set deviation: for n large, a parallel perturbation of norm η changes the log-density by about η√n/σ, while the observed severe failures occur at exactly that alignment. The paper's Theorems 3.8 and 3.13 give only set-containment results; Remark 1 claims the 'maximum difference' in density for parallel perturbations, which is a statement about the input distribution, not about the model. The experiment in §3.1 thus does not discriminate between 'deviation from the typical set' and 'increase of effective noise magnitude' as the cause of failure.","section":"§3.1, Observation 2 and §3.4.3, Remark 1"},{"comment":"The defense experiments do not isolate the proposed mechanism. Mixing Gaussian noise at levels 25.5 and 26 yields robustness gains on Set12-Adv-L2 of 29.9174 and 30.0874 dB, respectively, essentially matching TS-Pres.'s 30.0871 dB, while Table 4 shows only single-run averages with no standard deviation, number of runs, or significance test. Since plain higher-noise training achieves the same robustness numbers, the claim that TS works 'because it samples the larger typical set in the direction of the Gaussian noise' is not supported; a minimal control would be training on Gaussian noise with the same effective L2 magnitude without the TS screening step.","section":"§4.3, Table 4"},{"comment":"The bounds in Lemma 3.7 are not tight in the claimed sense. The proof uses the inequality ∥x+ξ∥²₂ ≤ ∥x∥²₂+η²+2η∥x∥₂, whose equality requires ξ to be a nonnegative multiple of x; the matching upper bound uses the reverse inequality with no lower-bound justification beyond symmetry. The resulting log-density bounds in the proof of Theorem 3.8 contain a sign inconsistency: the interval Ω is written with the lower endpoint η²−2η√(nσ²(1+2ϵ)) over 2nσ² minus ϵ, but Lemma 3.7's lower bound is negative with the same numerator; the final B₂ expression is justified only by taking absolute values. These issues do not invalidate the qualitative claim that log-density decreases, but the stated theorem as written is not a rigorous derivation.","section":"Theorem 3.8 and Lemma 3.7"}],"minor_comments":[{"comment":"Equation (2.4) is garbled: the first and third lines both contain L(f₁(x), y) < L(x, y), which is dimensionally inconsistent (L operates on two images but is compared to a scalar), and the formatting of the case structure is broken. Please rewrite the definition of transferability cleanly.","section":"§2.7, Eq. (2.4)"},{"comment":"The proof of Theorem 3.6 uses the calculus step '2σ²(log A + n(h(X) − ϵ)) < ∥x∥²₂' where log A = −(n/2)log(2πσ²); substituting h(X)=log(√(2πe)σ) gives boundary terms that cancel. The intermediate algebra is not shown and the current displayed inequality is not self-evident; a three-line derivation would help.","section":"§3.4.2, Theorem 3.6"},{"comment":"The prose interpretation of the volume bounds contains a typo: '2(η²/2σ² n⁻¹)2(η/σ n^{-1/2})' is written as though it decreases 'exponentially as n increases,' which is incorrect for the first factor, and the notation mixing n⁻¹ with the exponent n is confusing. Also, the claimed strict ordering Vol(A_ϵ) ≤ Vol(A_B2) ≤ Vol(A_B∞) is asserted from upper bounds rather than proven.","section":"§3.4.4, Propositions 3.15 and 3.16"},{"comment":"All reported numbers are point estimates from a single training run with no error bars, seeds, or statistical tests; given the small average differences between TS-Pres. and the normal model (e.g., 30.390 vs. 30.387 dB on Set12-Gaussian), it is not possible to verify the claim of 'marginally improved denoising performance.'","section":"§4.2, Tables 2–4"},{"comment":"There are several citation inconsistencies: the FGSM citation in §3.3.3 is given as '[25]' (Szegedy et al.) when the text refers to Goodfellow et al., and the reference list contains '[31]' K. Zhang et al. but the text cites '[33]' for SSIM; please verify all in-text citation numbers against the bibliography.","section":"§1 and §3.3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a sound and self-contained Gaussian typical-set calculation, but the central claim (out-of-typical-set implies model failure) is a missing derivational link, not a cosmetic issue. The comparison in Table 4 with mixed-noise training is the most damaging piece of evidence because it offers a competing explanation of the defense's success; the authors should be asked to run the minimal control (equivalent-magnitude higher-variance Gaussian training, with multiple seeds) or to present a theoretical argument showing that the log-density screening is what drives the robustness gain. A fair revision could either add a theorem with explicit assumptions on the denoiser (e.g., Lipschitz continuity in the input and a training loss that penalizes deviation from the clean image) or reframe the paper's contribution as an empirical/concentration-bounds study with a weaker causal claim. I recommend major revision rather than rejection because the concentration theorems and the hypothesis-testing sequence are valuable and the missing causal link is, in principle, addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth reading for the empirical observation: adversarial perturbations transfer across structurally unrelated denoisers, the adversarial regions form a single connected set aligned with the Gaussian noise direction, and the same patterns appear across images. That part is solid and builds cleanly on the authors' earlier work [17]. The typical-set framing is a nice way to think about why high-dimensional Gaussian noise concentrates on a sphere shell, and the TS sampling defense is a simple, cheap data augmentation that does not hurt Gaussian denoising.\n\nBut the central claims outrun the evidence. The abstract says the authors \"prove that adversarial samples deviate slightly from the typical set ... causing the models to fail.\" The theorems only prove Gaussian concentration bounds: Lemma 3.7 bounds the log-density change, Theorem 3.8 puts the perturbed noise in a larger typical set, and the volume results are textbook AEP. None of these involve the denoiser, its training objective, or its error. Section 3.4.5 simply asserts that because the model \"only learns from the typical set during training,\" it fails on samples outside it. That is a metaphor, not a theorem. No Lipschitz or distributional condition links log-density to reconstruction error. The causal step is unproven, and the own experiments in Table 4 make the problem concrete: mixing ordinary Gaussian noise at levels 25.5 and 26 gives almost the same robustness gains as TS-Pres. and TS-Def. So TS behaves like generic higher-variance augmentation, which is consistent with a simpler story: models are sensitive to increases in effective noise magnitude, not specifically to typical-set deviation.\n\nThe empirical work also has standard weaknesses: single runs, no code, missing hyperparameters (K, mixing ratio), and comparisons only against DnCNN-style training, not against matched-compute adversarial training or random noise augmentation. The theoretical sections are mostly textbook material, though the application to denoising is new.\n\nAll that said, the paper is not a waste of time. The transferability phenomenon is real and striking, the typical-set explanation is a clear hypothesis worth testing, and the TS defense is a harmless practical trick. The fit between theory and experiments is not there yet, but it is a legitimate research question. I would send it to peer review with the expectation of heavy revision: replace \"prove\" with \"hypothesize,\" add a controlled experiment that varies log-density independently of noise magnitude, report variance across seeds, and compare TS against higher-variance Gaussian augmentation. With those changes, the paper could make a decent contribution.","headline":"Interesting empirical phenomenon and a plausible-sounding story, but the proof of causality is missing and the defense gains look like generic higher-variance augmentation.","tokens_in":28736,"tokens_out":1097,"would_cite":false,"duration_ms":12915,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["94A08","68U10","68Q32"],"pacs":[],"model":"deepseek-v4-flash","headline":"Adversarial noise fools every denoising model because it leaves the Gaussian typical set; sampling from that outer set restores robustness.","keywords":["image denoising","adversarial transferability","typical set","asymptotic equipartition property","adversarial defense","out-of-distribution sampling","Gaussian noise","high-dimensional geometry"],"falsifier":"Construct an input whose log-density is the same as an adversarial sample but whose perturbation direction is not aligned with the Gaussian noise direction, and test whether the denoiser fails equally hard; if the failure disappears, the claim that typical-set deviation alone, rather than direction, is the causal mechanism would be undermined.","tokens_in":27610,"feed_emoji":"🛡️","tokens_out":5243,"duration_ms":49785,"temperature":0.7,"pith_summary":"This paper argues that the striking transferability of adversarial attacks across deep image denoising models—CNN, Transformer, unfolding, and plug-and-play alike—is a consequence of the geometry of Gaussian noise in high dimensions. Because denoising models are trained on i.i.d. Gaussian noise, they effectively learn the typical set that this noise occupies, which is a thin spherical shell. The paper proves that even very small adversarial perturbations move the input outside this typical set, with the largest deviation occurring when the perturbation points along the Gaussian noise direction; all models fail for the same reason, which is why a single perturbation defeats every model. This diagnosis yields a defense: a training strategy, Out-of-Distribution Typical Set Sampling (TS), that augments Gaussian noise with samples of lower log-probability density, drawn from the larger typical set that adversarial samples actually live in. The authors show that TS training improves robustness against L2- and L∞-bounded adversarial attacks while preserving or slightly improving ordinary denoising performance.","feed_headline":"Adversarial noise fools every denoiser for one statistical reason","feed_subtitle":"Gaussian typical-set geometry explains why one perturbation breaks all models; sampling outside it restores robustness.","key_machinery":"The typical set of i.i.d. Gaussian noise, defined through the asymptotic equipartition property and the weak law of large numbers, is the object that carries the argument. In n dimensions the typical set is a thin spherical shell with radius √(nσ²) and volume bounded by approximately (√(2πe)σ)^n; the paper derives explicit bounds (Theorem 3.6, Theorem 3.8, Theorem 3.13, Propositions 3.14–3.16) showing that an adversarial perturbation of size η keeps the perturbed vector inside a larger typical set while moving it out of the training set. This geometry tells the authors where to sample: their TS algorithm repeatedly draws Gaussian noise and keeps samples with progressively lower log-pdf, so training covers the larger typical set that adversarial samples actually occupy.","core_discovery":"The central discovery, on the paper's own terms, is that the manifold of adversarial perturbations for a deep denoising model is essentially the complement of the Gaussian typical set from which training noise is drawn, so the same small perturbation that moves one sample off that manifold transfers to every model trained on Gaussian noise. Using the asymptotic equipartition property, the paper proves that a Gaussian vector in n dimensions lies, with probability approaching one, in a typical set of volume roughly (√(2πe)σ)^n and concentrated on the sphere ‖x‖₂² ≈ nσ²; an adversarial perturbation of radius η in L2 or L∞ leaves this set but remains in a slightly larger typical set whose size is bounded in Propositions 3.15 and 3.16. The paper therefore claims that the high adversarial transferability observed in image denoising is not a property of the attack, but of the shared Gaussian-noise statistics that all denoisers learn.","pith_inferences":["A natural extension of the paper's logic is that input preprocessing that projects a perturbed image back onto the typical set, for instance by rescaling the norm of the noise component, could blunt attacks without any retraining.","If the mechanism is correct, the same typical-set analysis should apply to any regression task trained on a known noise distribution, not just Gaussian denoising, and to other noise families with similar concentration-of-measure behavior.","The theory predicts that high transferability will disappear if training noise is sampled uniformly from a broader shell rather than from the strict typical set; this is testable by retraining an existing model on TS-augmented data and measuring transfer across different architectures."],"forward_implications":["If the typical-set explanation is correct, any denoiser trained with i.i.d. Gaussian noise will exhibit adversarial transferability regardless of architecture, dataset, or training procedure.","The TS training strategy can be applied to any denoising model without changing its structure, improving robustness with no loss, or a slight gain, in Gaussian denoising quality.","Because adversarial samples are confined to a slightly larger typical set, robustness can be increased by sampling from that set directly instead of generating adversarial samples, avoiding the computational cost of adversarial training.","The sphere-shell picture predicts that a model's failure under perturbation can be predicted by measuring how far the input lies from this shell, which could guide detection as well as defense."],"supporting_citations":[{"why":"Provides the Denoising-PGD attack and the initial observation that adversarial samples transfer across diverse denoising models.","marker":"[17]"},{"why":"Supplies the weak law of large numbers, the asymptotic equipartition property, and the volume properties of typical sets used in the proofs.","marker":"[26]"},{"why":"Defines the PGD attack and adversarial training formulation that the paper's experiments and defense discussion build on.","marker":"[14]"},{"why":"Supplies the DnCNN architecture and residual-learning setup used as the primary test bed for the transferability and TS experiments.","marker":"[8]"},{"why":"Provides the SwinIR Transformer model used to test transferability across CNN and Transformer structures, rejecting the structural-similarity hypothesis.","marker":"[10]"},{"why":"Supplies the plug-and-play denoiser DPIR, one of the model families used to establish transferability beyond pure deep networks.","marker":"[30]"},{"why":"Supplies the unfolding-model DeamNet, extending the transferability observation to model-data hybrid architectures.","marker":"[20]"},{"why":"Provides the differential entropy formula for a Gaussian variable used in Lemma 3.5 to compute the radius and volume of the typical set.","marker":"[24]"}],"fun_headline_variants":["Why one adversarial noise breaks all denoisers: typical set geometry","Gaussian typical set explains cross-model adversarial transfer in denoisers","Adversarial transfer in denoisers: A typical set theorem and defense","Out-of-distribution sampling thwarts adversarial attacks on denoisers","The typical set flaw: Why denoisers are universally fooled and how to fix it"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument rests on the premise that a denoiser trained only on Gaussian-noise inputs from the typical set will fail on any input that leaves this set, i.e., low log-density under the training Gaussian distribution is by itself sufficient to cause large denoising error.","fun_headline_variants_meta":{"raw":{"variants":["Why one adversarial noise breaks all denoisers: typical set geometry","Gaussian typical set explains cross-model adversarial transfer in denoisers","Adversarial transfer in denoisers: A typical set theorem and defense","Out-of-distribution sampling thwarts adversarial attacks on denoisers","The typical set flaw: Why denoisers are universally fooled and how to fix it"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1557,"prompt_tokens":941,"completion_tokens":616,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":518}},"tokens_in":557,"tokens_out":616,"duration_ms":5555,"temperature":1.0,"reasoning_tokens":518,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:11:15.789257+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct an input whose log-density is the same as an adversarial sample but whose perturbation direction is not aligned with the Gaussian noise direction, and test whether the denoiser fails equally hard; if the failure disappears, the claim that typical-set deviation alone, rather than direction, is the causal mechanism would be undermined.","supporting_citations":[{"cited_title":"Evaluating Similitude and Robustness of Deep Image Denoising Models via Adversarial Attack","cited_arxiv_id":"2306.16050","evidence_quote":"Provides the Denoising-PGD attack and the initial observation that adversarial samples transfer across diverse denoising models."},{"cited_title":"Thomas and A","cited_arxiv_id":null,"evidence_quote":"Supplies the weak law of large numbers, the asymptotic equipartition property, and the volume properties of typical sets used in the proofs."},{"cited_title":"Mkadry, A","cited_arxiv_id":null,"evidence_quote":"Defines the PGD attack and adversarial training formulation that the paper's experiments and defense discussion build on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the DnCNN architecture and residual-learning setup used as the primary test bed for the transferability and TS experiments."},{"cited_title":"Liang, J","cited_arxiv_id":null,"evidence_quote":"Provides the SwinIR Transformer model used to test transferability across CNN and Transformer structures, rejecting the structural-similarity hypothesis."},{"cited_title":"Zhang, Y","cited_arxiv_id":null,"evidence_quote":"Supplies the plug-and-play denoiser DPIR, one of the model families used to establish transferability beyond pure deep networks."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the unfolding-model DeamNet, extending the transferability observation to model-data hybrid architectures."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the differential entropy formula for a Gaussian variable used in Lemma 3.5 to compute the radius and volume of the typical set."}],"review_version":1}