{"id":"d9d97b3f-2554-4bf7-a65e-5e501c95afb6","arxiv_id":"2506.15711","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A shadow-model-based defense adds sample-specific noise to medical images in federated learning, weakening gradient inversion attacks while keeping model accuracy near baseline.","lead":"Federated learning lets hospitals train shared AI models without uploading patient images, but attackers can sometimes reconstruct the training images from the model updates. This paper proposes a defense that adds targeted noise to the images before training, using a shadow model to guess which image regions an attacker would recover, and tests it on chest X-rays and eye scans.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eqs. (10)-(11) make noise strongest where the shadow model reconstructs worst, contradicting the prose; the defense's vulnerable-region localization is unverified, so the reported privacy-utility tradeoff may be an artifact.","rationale":"The central claim is that ShadowDef achieves state-of-the-art privacy with minimal utility loss because it localizes noise to regions where a real gradient-inversion attacker would succeed. For that claim to hold, the noise map must be a faithful, correctly-oriented indicator of attack vulnerability. The paper provides no direct evidence for that localization: L_MSE in Eq. (6) is a pixel-level fitting loss, so M measures the shadow model's own fitting error, not the attacker's success. More importantly, the equations and prose disagree on the sign of the mapping from M to noise. This is the most load-bearing issue because it affects the interpretation of every headline number in Tables 1-4: if the implementation follows Eq. (10), the defense may simply be adding high-frequency noise, and the 'shadow model interpretability' contribution is not operational; if it follows the prose, the implementation contradicts the methods section, so the reported experiments cannot be reproduced from the paper alone without inspecting the code. This concern is concrete and testable, unlike general worries about adaptive attacks, and it is a necessary prerequisite for any adaptive-attack evaluation to be meaningful. The reader's weakest_assumption already identified this, and the appropriate action is to condition acceptance on correcting the inconsistency and validating the localization premise.","tokens_in":29233,"tokens_out":8061,"duration_ms":91220,"concrete_test":"1) Inspect the released code to determine the sign of the noise map: is N2 computed from Eq. (10) (monotone increasing in reconstruction error M) or as the prose describes (decreasing)? 2) Reproduce the noise map on 100 ChestXRay images and compute the spatial rank correlation (e.g., Spearman) between the noise map and the per-pixel absolute error of the actual model-based GIA reconstruction on the same images without defense. If the correlation is negative (noise strongest where attack error is smallest), the 'vulnerable region' rationale is unsupported. 3) Run a defense-aware variant of model-based GIA that adds a denoising/TV regularizer to the attack objective; if reconstructed-image PSNR against the original improves by more than a few dB, the SOTA privacy claim is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 (Noise generation) defines the relative noise map from the pixel-wise reconstruction error M = ||x_i - x_i^rec||^2. Eq. (10) sets N1 = softmax(M/T), and Eqs. (12)-(13) apply a monotone histogram equalization before a second softmax, so the resulting noise N2 is monotonically increasing in M: the largest noise is applied where the shadow model's reconstruction error is largest. The text immediately below Eq. (13), however, states that 'regions with smaller reconstruction errors receive stronger noises, appearing darker in the noise map N2'. These cannot both be true. The released code (github.com/tekap404/ShadowDef) would resolve which behavior is implemented, but the paper as written does not. This matters because the central mechanism is that the error map identifies where a real GIA attacker would succeed, so noise should be targeted at those vulnerable regions. If the implementation follows the equations, noise is instead maximal where the proxy attacker (the shadow model) fails—typically high-frequency edges and texture—and the method's reported privacy-utility tradeoff could be explained simply by adding high-frequency noise that is less task-relevant, with no evidence that the shadow model localizes actual privacy leakage. Conversely, if the prose is correct, the equations and figure labeling need correction, and the premise that M correlates with real attack success is still untested: no experiment reports a spatial correlation between M and the per-pixel error of an actual GIA, and all evaluations use non-adaptive attackers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ShadowDef, a federated-learning defense against gradient inversion attacks (GIAs). A StyleGAN3-based shadow model is pre-trained on public data and fine-tuned per client during FL training; its per-pixel reconstruction error against the true local image is converted into a noise map that is added to the local images before the real gradient update. Grad-CAM++ is used to reduce noise in task-relevant foreground regions, and the noise scale is scheduled to increase over training. The authors report large degradations in reconstruction quality metrics (e.g., ChestXRay whole-image PSNR 15.09 to 11.36, SSIM 0.486 to 0.286) with less than 1% F1 loss, comparing favorably with DP, gradient sparsification, clipping, Soteria, OUTPOST, and Censor on two medical datasets. Ablations, computational cost, segmentation, ViT, and additional GIA variants are covered in the main text and supplementary material.","tokens_in":29377,"tokens_out":4719,"duration_ms":63444,"significance":"If the central mechanism were established, the paper would contribute a genuinely different defense paradigm: instead of perturbing gradients indiscriminately, it uses an interpretable shadow model to localize privacy-sensitive image regions and injects targeted image-space noise, achieving a favorable privacy-utility tradeoff. The paper has real strengths: the code is released, the experimental scope is broad (two medical datasets, model-based and optimization-based GIA, CI-Net, MKOR, segmentation, ViT, and face images), and the reported LPIPS/SSIM gains are substantial. However, the key claim that the shadow-model error map identifies where a real GIA attacker would succeed is currently untested, and the noise-generation equations contradict the accompanying prose about which regions receive stronger noise. These are load-bearing gaps for the paper's central claim, so the result is promising but not yet established.","major_comments":[{"comment":"The noise-generation equations and the prose are inconsistent. Eq. (10) defines N1 = softmax(M/T), where M is the pixel-wise MSE between the original image and the shadow reconstruction, and softmax is monotonically increasing in M. Eq. (12) applies histogram equalization G and another softmax; since histogram equalization is a nondecreasing transformation of N1, N2 is also monotonically increasing in M. Thus the largest noise is applied where the shadow model reconstructs worst. However, the text immediately below Eq. (13) states that \"regions with smaller reconstruction errors receive stronger noises, appearing darker in the noise map N2.\" These claims cannot both be true. This matters directly for the paper's mechanism: the narrative is that the error map identifies where a real attacker would succeed, so noise should target those vulnerable regions. If the implementation follows the equations, the defense instead adds maximal noise where the proxy attacker fails, typically high-frequency edges and texture, and the reported privacy gains could be explained by generic high-frequency perturbation rather than by correct localization of privacy leakage. The paper must resolve this contradiction and validate the localization premise, for example by comparing the spatial correlation between M and the per-pixel reconstruction error of the actual evaluated GIA.","section":"Sec. 4.3, Eqs. (10)-(13)"},{"comment":"The shadow model is fitted to the very images being defended. In Sec. 4.2, latent codes z_j are optimized by minimizing ||x - x_s|| on the true images, and in Eq. (6) the shadow fine-tuning loss includes L_MSE, the mean squared error between reconstructed and actual images. Consequently, the noise map M is computed from a model that has effectively seen the true image content, and the defense evaluation then measures how badly the attacker reconstructs that same true image. The reported PSNR/SSIM/LPIPS gains may therefore partly measure GAN fitting error rather than a demonstrated reduction of attack vulnerability. This is not merely a hypothetical concern: a shadow model that memorizes the defended image will produce an error map that is correlated with that image's high-frequency details, and adding noise there will trivially poison pixel-space similarity metrics. To support the paper's claim, the authors should test the defense against a defense-aware attacker, or at minimum evaluate whether the advantage persists when the shadow model is fine-tuned without L_MSE on the evaluated samples, or on a held-out client's images that were not used for noise-map fitting.","section":"Sec. 4.2-4.3, Eq. (6)"},{"comment":"The evaluation uses only non-adaptive attackers. Both the model-based GIA (GIAS) and the optimization-based GIA are run as black-box reconstruction procedures against the protected gradients, with no attempt to model, invert, or remove the defense's noise. Since the defense is conceptually a form of input perturbation, the natural and standard stress test for a GIA defense is an adaptive attacker who knows the defense algorithm and can, for example, train a denoiser on the public data or add a noise-robustness term to the inversion objective. The paper's own Section 6 claim of \"state-of-the-art privacy protection\" is therefore not yet supported against a determined adversary. The authors should run at least one adaptive attack experiment, even a simple one that pre-processes gradients or reconstructed images with a known-noise model, to show that the reported large PSNR/SSIM drops are not an artifact of attacking a perturbed objective with an unmodified optimizer.","section":"Sec. 5.2.1 and Sec. 6"}],"minor_comments":[{"comment":"The caption of Table 2 says \"Comparison of our method with SOTA defense methods against model-based GIA,\" but the table and surrounding text report results for optimization-based GIA; the caption should be corrected.","section":"Table 2 caption"},{"comment":"The temperature T of the softmax in Eq. (10) is never given a value in the hyperparameter list, although it controls the sharpness of the relative noise map; please specify it and the number of histogram-equalization bins if different from the grayscale range.","section":"Sec. 5.1"},{"comment":"The terminal round r_shadow is defined in the text but its default value is not listed with the other hyperparameters; the value 20 is mentioned later in Sec. 5.1, but this should be stated explicitly at the definition site.","section":"Sec. 4.3"},{"comment":"The sentence describing the effect of removing histogram equalization reports numerical degradations \"by 0.011, 0.77, 0.08, and 0.038\" that do not match the differences between the w/o Equ row and the Ours row in Table 3; please check these numbers and state which metric each value refers to.","section":"Sec. 5.2.2, Table 3"},{"comment":"The phrase \"consistently over 1.5% times in LPIPS and SSIM\" is unclear and should be rewritten; also, the claim \"minimal task performance degradation\" is not supported by the VGGFace2 results in the supplementary material, where F1 drops from 0.916 to 0.791, so the scope of the claim should be qualified to medical-image tasks.","section":"Abstract and Sec. 6"},{"comment":"There are numerous typographical and grammatical errors (e.g., \"proventive\", \"ealier\", \"neccessary\", \"generalizale\") that should be corrected during copyediting.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's central idea is interesting and the experimental breadth is a clear strength, but the internal inconsistency between Eqs. (10)-(13) and the prose, together with the unvalidated localization premise and the absence of adaptive attackers, makes the current SOTA claim premature. I would like the authors to clarify the actual noise-map behavior with code or a corrected figure, and to add the spatial-correlation or adaptive-attack experiment before this can be accepted. I did not find evidence of deliberate misrepresentation; the issues are best read as gaps in presentation and evaluation design."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The new thing here is that the defense operates in image space per sample, using a shadow model's reconstruction-error map to decide where to add noise, plus Grad-CAM++ to protect task-relevant regions and a time-increasing noise schedule. That combination is not in the cited prior literature, and the paper backs it with a broad empirical comparison: two medical datasets, model-based and optimization-based GIA, plus CI-Net and MKOR in the supplement, and the code is public. The strongest evidence is the matched comparison against image-level DP in Table 4, where targeted noise beats uniform noise at lower F1 cost. That alone makes the paper worth reading.\n\nThe soft spots are real and central. First, there is a direct contradiction between the equations and the prose in Section 4.3. Eqs. (10)–(13) define the noise map through a softmax after a monotone histogram equalization, so the map is monotonically increasing in the reconstruction error M. The text immediately below says regions with smaller reconstruction errors receive stronger noise. They can't both be right. The released code would settle it, but the paper doesn't, and the stated mechanism depends on which is true.\n\nSecond, the shadow model is fitted with access to the true image—L_MSE in Eq. (6) and the latent-code fitting in Section 4.2. A real GIA attacker doesn't have that. So the error map M partly encodes GAN fitting error, not validated attack vulnerability. There is no experiment showing that M spatially correlates with where an actual GIA succeeds, and all evaluated attackers are non-adaptive. A defense-aware adversary could behave differently.\n\nThird, the target-region defense metrics reuse the same Grad-CAM++ mask that suppresses noise, so those numbers are partly circular. Fourth, the method has many free hyperparameters—noise scale, softmax temperature, CAM thresholds, momentum, five loss weights—with almost no sensitivity analysis beyond the fine-tuning rounds table.\n\nWho's this for? People working on image-space defenses in FL, especially for medical images. The idea is clever and the empirical base is real, but the central mechanism is not yet demonstrated. It deserves a serious referee, not a desk reject, but the referees should push for a fix of the contradiction, a defense-aware attacker evaluation, and a direct spatial-correlation test between M and actual GIA leakage before the headline claim is accepted.","headline":"A novel image-space defense with a broad empirical sweep, undercut by an internal math/prose contradiction and an unverified vulnerability-localization mechanism; deserves peer review but needs major revision.","tokens_in":30128,"tokens_out":4648,"would_cite":false,"duration_ms":56591,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated-learning clients can block gradient inversion attacks with targeted image noise guided by a shadow model, cutting reconstruction quality sharply without hurting task accuracy.","keywords":["federated learning","gradient inversion attack","privacy defense","shadow model","medical images","noise injection","Grad-CAM++"],"falsifier":"Compare the per-pixel reconstruction-error map produced by an actual undefended GIA on the same images and model against the noise map this method generates: if the noise does not concentrate where the attacker reconstructs best, or if a defense-aware attacker who also fine-tunes a shadow model on the noisy gradients recovers images with near-FedAvg PSNR, the targeted-noise premise fails.","tokens_in":28814,"feed_emoji":"🛡️","tokens_out":7617,"duration_ms":82502,"temperature":0.7,"pith_summary":"Federated-learning clients normally send gradients that a curious server can invert into recognizable patient images. This paper argues that the right defense is to attack the vulnerability where it lives: find the image regions that a gradient-inversion attacker could actually reconstruct, and add noise exactly there instead of perturbing gradients blindly. To do that, each client runs a shadow model—a GAN pretrained on public data and fine-tuned on its own images to mimic an almost-optimal attacker—and uses the pixel-wise reconstruction error as a sensitivity map. Noise is then concentrated on sensitive areas while a Grad-CAM++ foreground mask softens the perturbation on task-critical anatomy, with noise amplitude growing as training progresses because attacks become stronger. On two medical datasets the method cuts whole-image reconstruction quality substantially while keeping F1 nearly unchanged, and it beats gradient-perturbation defenses across the board.","feed_headline":"Shadow attacker guides noise that stops gradient inversion","feed_subtitle":"Targeted pixel noise cuts reconstructed-image quality while F1 barely moves on medical datasets.","key_machinery":"The load-bearing object is the shadow model: a StyleGAN3 generator pretrained on public data, whose per-sample latent codes are first fitted to the client's images and whose generator weights are then fine-tuned on a per-sample gradient-inversion loss so that its reconstruction imitates a strong attacker. From the pixel-wise MSE map between the true image and the shadow reconstruction, the method builds a noise map through an inverse softmax, histogram equalization, momentum averaging across rounds, and a sign-scaled subtraction of the Grad-CAM++ foreground mask, with overall amplitude scaled up by training epoch. The map tells the client where an attacker could reconstruct the image, and applying that noise to the images before real local training breaks the gradient-to-image mapping that GIAs rely on.","core_discovery":"The central claim is that targeted image-space noise guided by a shadow model provides a strictly better privacy-utility tradeoff than gradient-domain defenses. Operationally, against model-based GIA on ChestXRay the paper reports whole-image PSNR falling from 15.09 (undefended FedAvg) to 11.36 and SSIM from 0.486 to 0.286 while F1 drops only from 0.978 to 0.967; on EyePACS PSNR falls from 12.71 to 9.93 and F1 from 0.870 to 0.861. The same framework holds against optimization-based GIA, CI-Net, and a segmentation task, and it protects the foreground regions that Grad-CAM++ identifies as task-critical even though noise there is deliberately weakened. The conclusion the authors draw is that leakage risk cannot be read off gradient magnitudes alone; the spatial pattern of what an attacker can reconstruct is what must be protected.","pith_inferences":["A direct next step the paper does not run is an adaptive attacker who knows the noise-adding procedure and fits its own shadow model to the noisy gradients; if such an attacker recovers images almost as well as from undefended gradients, the targeted-noise advantage would shrink.","Because the method relies on batch-normalization statistics for attack strength, it would naturally be re-derived for architectures without BN; the paper itself notes that both GIA families fail on Vision Transformers, leaving transformer-specific inversion as an open problem the defense could be adapted to later.","The VGGFace2 result suggests that on natural images the Grad-CAM++ foreground is too broad or the noise too damaging; borrowing per-identity saliency maps or face-region priors could extend the method beyond medical images.","One testable extension is to use the same shadow-model sensitivity map to allocate a fixed privacy budget across pixels, connecting the method to differential privacy with spatially varying noise."],"forward_implications":["Against model-based GIA, whole-image PSNR drops by 3.73 dB and SSIM by 0.20 on ChestXRay, and by 2.78 dB and 0.166 on EyePACS, while F1 stays within one percentage point of undefended training.","The defense transfers to optimization-based GIA, to CI-Net, and to an MRI segmentation task, indicating the sensitivity-map idea is not tied to one attack family or image modality.","Noise amplitude that increases with training epoch matches the empirical trend that GIA grows stronger as batch-normalization statistics become accurate, so the defense stays ahead of the attack through the whole federated-learning process.","Because the noise is added to images rather than gradients, the method doubles as an interpretable audit tool: the maps show which anatomical regions are most leak-prone, aligning with data-minimization and transparency requirements in privacy regulation.","The shadow model is updated for only the first twenty global rounds, keeping the added time modest (about 720 seconds versus 566 for FedAvg on EyePACS), so the privacy gain does not require heavy per-round computation."],"supporting_citations":[{"why":"Supplies the strong-assumption GIA formulation with BN statistics, the ChestXRay/EyePACS setup, and the RDLV metric that the paper's evaluation builds on.","marker":"Hatamizadeh et al. (2023)"},{"why":"Defines the model-based GIA (GIAS) with a GAN prior that the paper uses as its main attack and as the basis for the shadow model.","marker":"Jeon et al. (2021)"},{"why":"Provides the StyleGAN3 generator and pretraining strategy used as the shadow model backbone.","marker":"Karras et al. (2021)"},{"why":"Grad-CAM++ is used to build the foreground activation map that suppresses noise on task-critical regions.","marker":"Chattopadhay et al. (2018)"},{"why":"Defines federated averaging, the undefended FedAvg baseline, and the FL objective the defense modifies.","marker":"McMahan et al. (2017)"},{"why":"Differential privacy is the main gradient-perturbation baseline that the method is compared against and that motivates the image-level DP ablations.","marker":"Abadi et al. (2016)"},{"why":"Deep Leakage from Gradients is the canonical optimization-based GIA and the basis of the gradient-matching loss.","marker":"Zhu et al. (2019)"},{"why":"Soteria, a representation-perturbation defense, serves as a comparison baseline for task performance and privacy metrics.","marker":"Sun et al. (2021)"},{"why":"OUTPOST is the adaptive gradient-sparsification defense compared against as a SOTA baseline.","marker":"Wang et al. (2024a)"},{"why":"Censor, which uses orthogonal subspace Bayesian sampling, is a recent SOTA defense baseline in the comparison tables.","marker":"Zhang et al. (2025)"}],"fun_headline_variants":["Shadow model targets noise to foil gradient inversion","Pixel-level noise defense beats gradient obfuscation","Guided noise protects medical data from gradient attacks","Shadow-guided noise preserves accuracy while blocking inversion","Targeted noise defense thwarts image reconstruction"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense works only if the shadow model's reconstruction-error map marks the exact pixel regions where a real gradient-inversion attacker would succeed, so that adding noise there harms the attack more than it harms the task.","fun_headline_variants_meta":{"raw":{"variants":["Shadow model targets noise to foil gradient inversion","Pixel-level noise defense beats gradient obfuscation","Guided noise protects medical data from gradient attacks","Shadow-guided noise preserves accuracy while blocking inversion","Targeted noise defense thwarts image reconstruction"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00017,"raw_usage":{"total_tokens":1313,"prompt_tokens":1034,"completion_tokens":279,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":210}},"tokens_in":650,"tokens_out":279,"duration_ms":3816,"temperature":1.0,"reasoning_tokens":210,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:14:18.899828+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the per-pixel reconstruction-error map produced by an actual undefended GIA on the same images and model against the noise map this method generates: if the noise does not concentrate where the attacker reconstructs best, or if a defense-aware attacker who also fine-tunes a shadow model on the noisy gradients recovers images with near-FedAvg PSNR, the targeted-noise premise fails.","supporting_citations":[{"cited_title":", author Yin, H","cited_arxiv_id":null,"evidence_quote":"Supplies the strong-assumption GIA formulation with BN statistics, the ChestXRay/EyePACS setup, and the RDLV metric that the paper's evaluation builds on."},{"cited_title":", author Lee, K","cited_arxiv_id":null,"evidence_quote":"Defines the model-based GIA (GIAS) with a GAN prior that the paper uses as its main attack and as the basis for the shadow model."},{"cited_title":", author Aittala, M","cited_arxiv_id":null,"evidence_quote":"Provides the StyleGAN3 generator and pretraining strategy used as the shadow model backbone."},{"cited_title":", author Moore, E","cited_arxiv_id":null,"evidence_quote":"Defines federated averaging, the undefended FedAvg baseline, and the FL objective the defense modifies."},{"cited_title":", author Chu, A","cited_arxiv_id":null,"evidence_quote":"Differential privacy is the main gradient-perturbation baseline that the method is compared against and that motivates the image-level DP ablations."},{"cited_title":", author Liu, Z","cited_arxiv_id":null,"evidence_quote":"Deep Leakage from Gradients is the canonical optimization-based GIA and the basis of the gradient-matching loss."},{"cited_title":", author Li, A","cited_arxiv_id":null,"evidence_quote":"Soteria, a representation-perturbation defense, serves as a comparison baseline for task performance and privacy metrics."},{"cited_title":"CENSOR: Defense Against Gradient Inversion via Orthogonal Subspace Bayesian Sampling","cited_arxiv_id":"2501.15718","evidence_quote":"Censor, which uses orthogonal subspace Bayesian sampling, is a recent SOTA defense baseline in the comparison tables."}],"review_version":1}