{"id":"45c5cae9-0739-4bdf-a3bf-9ec5eff75867","arxiv_id":"2505.20955","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Removing high-frequency components from reconstruction-error scores improves membership inference attacks on diffusion models, demonstrated on DDIM and Stable Diffusion.","lead":"Membership inference attacks on diffusion models work by measuring how well the model reconstructs an image, but they are thrown off by the model's unstable handling of high-frequency image details. This paper shows that filtering out high-frequency content before computing the reconstruction error consistently improves attack accuracy across models and datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Proposition 4.2's proof is invalid: Eq. C.11 does not follow algebraically from Eq. C.10, and the asserted link between σ_H/σ_M and membership advantage is not established.","rationale":"The reader's weakest_assumption identifies the same vulnerable point: the theoretical result in Proposition 4.2 and the supporting Eq. 8. I agree, and I locate the concrete failure even earlier than the reader did: the algebraic step from Eq. C.10 to Eq. C.11 is invalid when k≠1, and the move from a standard-deviation ratio to a membership advantage is not justified by Yeom et al. as cited. The empirical part of the paper is substantially stronger: the filter improves three independent baselines across DDIM and fine-tuned Stable Diffusion settings, the ablation shows low sensitivity to s and rt over a broad range, and the defense experiments give a coherent picture. Those results would survive even if the proof were withdrawn or corrected. The main tables lack error bars and multiple-seed reporting, which is a secondary rigor concern, but it does not change the assessment. The appropriate verdict remains CONDITIONAL: the practical claims can stand after the theoretical demonstration is repaired or downgraded and after adding statistical rigor. Since the reader already reached CONDITIONAL, no verdict adjustment is needed.","tokens_in":24171,"tokens_out":5353,"duration_ms":54955,"concrete_test":"Independently re-derive Eq. C.11 from Eq. C.10 by substituting k = h_M/h_H. If the correct rearrangement is l_M^2 + h_M^2 > (l_M+Δ)^2 + h_H^2 − 2Δ·sqrt((l_M+Δ)^2+h_M^2) + Δ^2, then Eq. C.11 is not equivalent to Eq. C.10 unless h_M = h_H. Then run a Monte Carlo check of the proposition's conclusion: sample independent normal low- and high-frequency score components with variances (l_M^2, h_M^2) and (l_H^2, h_H^2), choose Δ = l_H − l_M, and compute the optimal-threshold Yeom advantage before and after hard high-frequency removal. If the advantage does not increase in cases satisfying the proposition's condition, the theoretical claim fails; if it does increase, the proof still needs correction to be valid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's only formal support for the claimed mechanism is Proposition 4.2, but the proof in Appendix C does not go through. The transition from Eq. C.10 to Eq. C.11 is algebraically wrong. Substituting k = h_M/h_H into Eq. C.10 gives h_M^2 - h_H^2 > 2l_MΔ + 2Δ^2 - 2Δ·sqrt((l_M+Δ)^2 + h_M^2), which rearranges to l_M^2 + h_M^2 > (l_M+Δ)^2 + h_H^2 - 2Δ·sqrt((l_M+Δ)^2 + h_M^2) + Δ^2. Eq. C.11 instead has l_M^2 + h_H^2 on the left, effectively replacing h_M^2 with h_H^2; this does not follow when h_M ≠ h_H. The subsequent steps (C.12-C.17) therefore lack a valid premise. Separately, even if the standard-deviation ratio inequality were proven, the conclusion about membership advantage does not follow: under the paper's own normality assumption, the optimal-threshold Yeom advantage depends on the means μ_M and μ_H as well as on σ_M and σ_H, and the monotone relationship σ_H/σ_M ∝ Adv in Eq. 8 is asserted rather than derived. The experiments consistently show empirical gains, and the attenuating filter (s=0.2) is not the hard removal analyzed in the proposition, so the abstract's claim that the deficiency is 'theoretically demonstrated' is unsupported as written, although the empirical contribution can stand independently.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies membership inference attacks (MIAs) against diffusion models. The authors unify error-based attacks (Naive, SecMI, PIA) into a single reconstruction-distance paradigm, observe that diffusion models reconstruct low-frequency content more reliably than high-frequency content, and argue that the high-frequency component of the reconstruction score injects variance that drowns the membership signal. They propose FreMIA, a plug-and-play module that applies a Fourier-domain mask to attenuate high-frequency components before computing the membership score. They report consistent improvements in ASR, AUC, and TPR@1%FPR over three baselines on DDIM (STL10-U, CIFAR-100, Tiny-IN) and fine-tuned Stable Diffusion (Pokémon, MS-COCO, Flickr), with ablations, defense evaluations, and additional experiments in the appendix. They also state Proposition 4.2, intended to prove that high-frequency filtering increases the membership advantage.","tokens_in":24387,"tokens_out":9221,"duration_ms":84583,"significance":"The empirical finding is potentially valuable: a simple, low-overhead preprocessing step improves attack performance across multiple datasets and model types, with especially large gains in TPR@1%FPR. The paper is careful in its experimental design: same-distribution member/hold-out splits, standard metrics, baseline default parameters, ablation over s and r_t, weaker-overfitting and defense settings, and a public code link. If the frequency-deficiency mechanism is correct, this is a practical contribution to privacy auditing of diffusion models. However, the theoretical demonstration is not currently sound: the proof of Proposition 4.2 contains an algebraic error, and the asserted link between the standard-deviation ratio and membership advantage is not established. The paper's significance at this stage rests on the empirical results rather than on the theory.","major_comments":[{"comment":"The transition from Eq. (C.10) to Eq. (C.11) is algebraically incorrect. Substituting k = h_M/h_H into Eq. (C.10) gives h_M^2 - h_H^2 > 2 l_M Δ + 2Δ^2 - 2Δ√((l_M+Δ)^2 + h_M^2). Eq. (C.11) instead asserts l_M^2 + h_H^2 > (l_M+Δ)^2 + h_H^2 - 2Δ√((l_M+Δ)^2 + h_M^2) + Δ^2, which after canceling l_M^2 + h_H^2 reduces to √((l_M+Δ)^2 + h_M^2) > l_M + Δ, a trivially true statement for h_M > 0. This is not equivalent to Eq. (C.10). Since the subsequent steps (C.12)-(C.17) all rely on Eq. (C.11), the proof of Proposition 4.2 does not go through. The authors should provide a correct derivation or remove the theoretical claim.","section":"Appendix C, Eqs. (C.10)-(C.12)"},{"comment":"The statement that σ_H/σ_M ∝ Adv_M(A) is presented as a consequence of Yeom et al. (2018), but it is not derived there and is false in general under the paper's own normality assumption. For Gaussian score distributions with means μ_H, μ_M and variances σ_H^2, σ_M^2, the optimal-threshold advantage is a function of both the mean shift and the variances; the variance ratio alone does not determine it. Consequently, the argument that a reduction in σ_H/σ_M implies a reduction in membership advantage is not justified, and Proposition 4.2, even if its algebra were corrected, would not establish an attack improvement. Please replace Eq. (8) with a direct computation of the advantage (or AUC) for the relevant score distribution, or explicitly restrict the claim to the empirical observation.","section":"Section 4.3, Eq. (8)"},{"comment":"Proposition 4.2 concerns hard removal of the high-frequency components (the filter with s=0), but the proposed module and all experiments use multiplicative attenuation with s=0.2, and the recommended range is s ∈ [0.0, 0.3]. The proposition therefore does not apply to the actual method. If the proof is repaired, it should be extended to the attenuating mask, or the theoretical claim should be explicitly limited to the s=0 case and the experimental improvement argued separately.","section":"Section 4.3 / Section 5.1, Eqs. (9)-(10) and s=0.2"},{"comment":"The validation of the constraint k^2 > f is performed on the same datasets and methods used to measure the attack improvement (Table 7 vs. Tables 2-3). Since all reported values of f are below 1, the condition is automatically satisfied whenever k ≥ 1, and the few k < 1 cases still pass. This is a post-hoc precondition check, not a predictive test of the theory; the text in Section 4.3 should not present it as independent support for Proposition 4.2.","section":"Appendix D.2 / Table 7"}],"minor_comments":[{"comment":"The text says the best performance is achieved at 's=5 and rt=0.2', but Table 4 is indexed by s as columns and rt as rows; the intended setting is s=0.2 and rt=5. The same typo appears in Appendix D.8.","section":"Section 5.4 / Appendix D.8"},{"comment":"The sentence 'When rt = 1, the high-frequency threshold is set too low, leading to most high- and low-frequency components of the image being filtered' is inaccurate for the mask in Eq. (10): with rt=1 and s=0.2, only components with radius > 1 are attenuated, and low-frequency components are not filtered. Please rephrase.","section":"Section 5.4"},{"comment":"Normality tests are reported only for MS-COCO and Flickr; the text nonetheless claims that 'the scores of baselines conform to a normal distribution'. Please qualify the claim to the tested datasets, or report normality tests for the remaining datasets.","section":"Appendix D.2, Table 8"},{"comment":"The choice of the high/low frequency boundary radius (5 for MS-COCO and Flickr, 2 for CIFAR-100 and Tiny-IN) is not justified. A sentence explaining how this radius was selected would improve reproducibility.","section":"Section 4.2 / Figure 1"},{"comment":"The DDIM posterior distribution in Eq. (1) is typeset incorrectly, with missing parentheses and ambiguous square-root scope. Please fix the equation.","section":"Eq. (1)"},{"comment":"The 'Avg+' rows are not defined in the captions; please state explicitly that the average is taken over the three baseline attacks.","section":"Tables 2 and 3"}],"recommendation":"major_revision","confidential_remarks":"The empirical study is strong and the proposed filter is simple, well-evaluated, and likely useful in practice. The main risk is that the theoretical section overclaims: the proof of Proposition 4.2 is invalid, and the asserted link to membership advantage is not established. I would not recommend rejection because the empirical contribution can stand on its own, but the authors need to either fix the proof or reframe the contribution as empirical before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: the empirical result is real, the theory isn't. The paper shows that a Fourier mask that attenuates high-frequency components improves three established grey-box MIAs on DDIM and fine-tuned Stable Diffusion, with gains that are consistent and sometimes large, especially at TPR@1%FPR. That is a useful, near-zero-cost plug-in for privacy auditing, and the frequency-domain diagnosis of why these attacks struggle is a genuinely new angle. Credit where due: they unify Naive/SecMI/PIA into a single reconstruction-error paradigm, visualize the high-frequency-variance effect, and back it with ablations, defenses, no-text settings, and a public code repo. The empirical package is extensive and internally consistent.\n\nThe soft spot is the theoretical demonstration, and it's not minor. Proposition 4.2's proof in Appendix C has an algebraic error: substituting k = h_M/h_H into Eq. C.10 gives a left-hand side involving h_M^2, but Eq. C.11 writes l_M^2 + h_H^2, effectively swapping h_M for h_H. That doesn't follow when h_M ≠ h_H. Even if that step were fixed, Eq. 8 asserts a monotone link between σ_H/σ_M and membership advantage without deriving it; under normality, Yeom's advantage also depends on means, and the claimed proportionality is not established. So the abstract's claim that high-frequency deficiency 'theoretically demonstrates' the effect is unsupported. The experiments use an attenuating filter (s=0.2), not the hard removal the proposition analyzes, so the theory wouldn't cover them even if it were correct.\n\nTwo more moderate concerns. The hyperparameters (s, rt) are selected on the evaluation data, and the k^2>f constraint is validated on the same datasets used to show improvement — that's checking a precondition, not independent confirmation. The lack of error bars and multiple seeds in the main tables also makes the improvements harder to judge, though the consistency across many settings helps. The normality tests are okay but a bit thin.\n\nIn the end, the paper is worth reading and worth refereeing, but the theory should be separated from the empirical claim. Either fix the proof or reframe it as a motivating heuristic; the empirical finding stands on its own. I'd send it to peer review with a clear request for revision: correct or remove Proposition 4.2, add error bars, and temper the abstract.","headline":"Short version: the Fourier-filter trick is a solid empirical contribution, but Proposition 4.2's proof is algebraically wrong, so the 'theoretical demonstration' claim should not be relied upon.","tokens_in":24994,"tokens_out":4368,"would_cite":true,"duration_ms":37305,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Fourier filter that suppresses high-frequency image detail makes membership inference attacks on diffusion models substantially stronger, by removing score variance the model itself injects when reconstructing edges and textures.","keywords":["Membership Inference Attacks","Diffusion Models","Frequency Domain","Fourier Transform","High-Frequency Deficiency","Privacy","Reconstruction Error","Text-to-Image Models"],"falsifier":"Measure $\\sigma_H/\\sigma_M$ and attack accuracy before and after applying the filter on a dataset where high-frequency content is deliberately uncorrelated with membership—for example, by equalizing high-frequency energy across member and hold-out sets; if the ratio does not improve or accuracy does not rise, the proposed mechanism is not responsible for the gains. A second check is to compare hard removal ($s=0$) with the paper's attenuation ($s=0.2$): the variance-removal argument predicts hard removal should be at least as strong, so a clear advantage for $s=0.2$ would indicate the theory's premise is incomplete.","tokens_in":23875,"feed_emoji":"🔍","tokens_out":5030,"duration_ms":47938,"temperature":0.7,"pith_summary":"This paper argues that existing membership inference attacks on diffusion models underperform because of a \"high-frequency deficiency\": diffusion models faithfully reconstruct low-frequency structure but handle high-frequency details with high variance. That variance contaminates the reconstruction-error scores that attacks rely on, so images rich in high-frequency content tend to be misclassified as hold-out data and flat images as members. The paper unifies the Naive, SecMI, and PIA attacks into a single reconstruction-distance paradigm, shows theoretically that high-frequency variance weakens the membership advantage, and proposes a plug-and-play Fourier-domain filter that attenuates high-frequency components before the score is computed. Across DDIM and fine-tuned Stable Diffusion models, the filter raises attack success, AUC, and true-positive rate at low false-positive rate, with negligible extra time cost. The practical message is that any error-based membership inference attack on a diffusion model is leaving accuracy on the table by scoring images in the spatial domain.","feed_headline":"A Fourier filter makes diffusion privacy leaks easier to detect","feed_subtitle":"Diffusion models reconstruct edges unreliably; filtering that noise reveals which images were in the training set.","key_machinery":"The load-bearing mechanism is the high-frequency Fourier filter combined with a variance-ratio theory of membership advantage. The paper transforms predicted and target images with the discrete Fourier transform, multiplies the spectrum by a mask that suppresses components above a radius $r_t$ (keeping a fraction $s$ of their amplitude), transforms back, and then computes the reconstruction distance. The theoretical engine is Proposition 4.2, which assumes that member and hold-out score variances decompose into independent low- and high-frequency parts. Removing high-frequency variance then changes the ratio $\\sigma_H/\\sigma_M$; when the high-frequency variance on members is larger than on hold-out data ($k>1$), the ratio rises, and via the Yeom et al. (2018) relation $\\sigma_H/\\sigma_M \\propto \\mathrm{Adv}_M(A)$, the attack's membership advantage grows. The mask is what does the work: it strips away the noisy high-frequency component so that the residual score tracks how the model fits the global structure of the image, which is where membership is expressed.","core_discovery":"The paper's central claim is that the membership signal in diffusion models concentrates in the low-frequency part of the reconstruction error, while the high-frequency part is dominated by model variance rather than by membership. It formalizes a general paradigm, $A(x_i,\\theta)=\\mathbf{1}[\\|x_{i,t}-x_{i,t}^{\\mathrm{target}}\\|_q \\le \\tau]$, and shows that the main existing attacks are instances of it. Within that paradigm it observes empirically that membership scores rise with the high-frequency content of the image, that failed member predictions have higher high-frequency content than failed hold-out predictions, and that pixel-level reconstruction error concentrates at high-frequency locations. The theoretical engine is Proposition 4.2: under normality and independence of low- and high-frequency score components, removing the high-frequency component raises $\\sigma'_H/\\sigma'_M$ above $\\sigma_H/\\sigma_M$, increasing membership advantage in the sense of Yeom et al. (2018). The remedy is a Fourier mask, $\\beta_{i,t}(r)=s$ for $r>r_t$ and $1$ otherwise (implemented with $s=0.2$, $r_t=5$), applied to both the predicted and target images before taking the distance. Experiments show consistent gains for Naive, SecMI, and PIA on DDIM (STL10-U, CIFAR-100, Tiny-IN) and fine-tuned Stable Diffusion (Pokémon, MS-COCO, Flickr), including under weaker overfitting, missing captions, and two defenses.","pith_inferences":["Beyond the attacks tested here, any reconstruction-based score on diffusion models—such as pixel-wise error maps used in memorization detection or data auditing—could carry the same high-frequency noise, so the filter may transfer to those tasks.","The implementation attenuates high-frequency components with $s=0.2$ rather than removing them as the theory assumes; an image- or timestep-adaptive mask that more closely matches the theory's hard removal could yield further gains.","The low-to-high frequency generation pattern also appears in text-to-image and video diffusion models, suggesting the deficiency, and the filter, apply beyond the grey-box reconstruction attacks demonstrated in this paper.","If the variance-ratio mechanism is correct, a stronger defense should explicitly reduce high-frequency variance on member images rather than only augmenting data; the adaptive defense proposed here mitigates but does not eliminate the attack, leaving room for further defensive work."],"forward_implications":["Any error-based membership inference attack that fits the reconstruction-distance paradigm can be strengthened by inserting the high-frequency filter before computing the score, with no additional training or model access.","The magnitude of the gain scales with dataset complexity: on Tiny-IN the filter added over seven points to average attack success rate and over eleven points to TPR@1% FPR across the three baselines.","The filter remains effective under weaker overfitting, when captions are absent or machine-generated, and when defenses such as data augmentation or the SSei memory-mitigation method are applied, though gains shrink with the baseline's overall strength.","The gain in the $\\sigma_H/\\sigma_M$ ratio after filtering directly corroborates the theoretical account: observed ratios increase from values near 1 to substantially larger values, matching the predicted improvement in membership advantage.","On pre-trained Stable Diffusion models the filter provides little benefit because the baselines themselves operate near random-guessing in that setting, so the deficiency is not the binding constraint there."],"supporting_citations":[{"why":"Supplies the membership-advantage definition and the $\\sigma_H/\\sigma_M$ ratio that Proposition 4.2 uses to prove filtering amplifies advantage.","marker":"Yeom et al., 2018"},{"why":"Defines the Naive loss-based attack that the paper unifies into the general paradigm and uses as a baseline.","marker":"Matsumoto et al., 2023"},{"why":"Provides SecMI, the DDIM-inversion attack used as a baseline and as an instance of the reconstruction-distance paradigm.","marker":"Duan et al., 2023"},{"why":"Provides PIA, the proximal-initialization attack used as a baseline and as an instance of the paradigm.","marker":"Kong et al., 2023"},{"why":"Establishes that diffusion models recover low-frequency components first and high-frequency components later with greater uncertainty, motivating the high-frequency deficiency.","marker":"Yang et al., 2023"},{"why":"Offers a frequency-space perspective showing high-frequency components are masked early in training, supporting the claim that high-frequency reconstruction is unstable.","marker":"Falck et al., 2025"},{"why":"Supplies the SSei memory-mitigation defense used to test the filter under an active countermeasure.","marker":"Wen et al., 2024"},{"why":"Provides the CLID likelihood-based attack used to demonstrate the filter's generality beyond the three main baselines.","marker":"Zhai et al., 2024"}],"fun_headline_variants":["Frequency-domain filter sharpens diffusion membership attacks","Low-pass filter exposes diffusion model training members","Fourier masking reveals diffusion membership signals","High-frequency noise hides diffusion training data clues","Filtering high-frequency error improves diffusion MIAs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The theory assumes that membership advantage is proportional to the ratio of the standard deviations of member and hold-out scores under normal distributions, that high- and low-frequency score components contribute independent variances, and that the filter removes all high-frequency variance, while the experiments use partial attenuation ($s=0.2$) rather than full removal.","fun_headline_variants_meta":{"raw":{"variants":["Frequency-domain filter sharpens diffusion membership attacks","Low-pass filter exposes diffusion model training members","Fourier masking reveals diffusion membership signals","High-frequency noise hides diffusion training data clues","Filtering high-frequency error improves diffusion MIAs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000274,"raw_usage":{"total_tokens":1707,"prompt_tokens":1082,"completion_tokens":625,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":698,"completion_tokens_details":{"reasoning_tokens":558}},"tokens_in":698,"tokens_out":625,"duration_ms":6637,"temperature":1.0,"reasoning_tokens":558,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:43:52.358626+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure $\\sigma_H/\\sigma_M$ and attack accuracy before and after applying the filter on a dataset where high-frequency content is deliberately uncorrelated with membership—for example, by equalizing high-frequency energy across member and hold-out sets; if the ratio does not improve or accuracy does not rise, the proposed mechanism is not responsible for the gains. A second check is to compare hard removal ($s=0$) with the paper's attenuation ($s=0.2$): the variance-removal argument predicts hard removal should be at least as strong, so a clear advantage for $s=0.2$ would indicate the theory's premise is incomplete.","supporting_citations":[],"review_version":1}