{"id":"cf28afb2-e038-4c85-9f20-f55f1b9cc560","arxiv_id":"2507.09602","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DRAGD and DRAGDP reconstruct erased federated-learning images by sequentially matching post-unlearning and pre-unlearning gradients, with DRAGDP adding a public-image prior.","lead":"This paper introduces DRAGD, an attack that reconstructs data deleted from a federated learning model by matching gradients collected before and after the unlearning step, and DRAGDP, which starts from public images to improve face reconstructions. It claims a defense called FedANI, but that defense is never described, and the experimental support is weak.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DRAGD is only validated on single-batch raw gradients, not on the multi-step model updates actually exchanged in FedAvg; the claimed real-world vulnerability is therefore unsupported.","rationale":"The reader's weakest assumption was that the server can access exact per-client local gradients before and after unlearning, which is a realistic concern about threat-model assumptions. My identified concern is related but distinct: even when the server has access to per-client updates, those updates in FedAvg are multi-step model deltas, not the raw batch gradients that Algorithm 1 is designed to invert. This is a more fundamental mismatch because it affects every realistic FL deployment, not only those with secure aggregation or clipping. The reader's rationale did mention the unrealistic evaluation (e.g., near-perfect Part reconstruction) but did not explicitly flag the single-batch-gradient versus multi-step-update discrepancy; hence 'partial' agreement. The concern reinforces the REJECT verdict rather than changing it, because the paper's central effectiveness claim is not supported under realistic communication semantics.","tokens_in":13020,"tokens_out":7203,"duration_ms":83148,"concrete_test":"Implement a standard FedAvg round for a single target client on MNIST/LeNet: start from a global model, train locally for E=5 epochs with mini-batches on D_f∪D_r to obtain the pre-unlearning update Δθ* = θ*_{t+1} - θ*_t, then retrain from the same θ*_t for E=5 epochs on D_r to obtain Δθ_u. Replace the raw gradients ∇θ* and ∇θ_u in Algorithm 1 with these model deltas and run DRAGD. If reconstruction of the forgotten data fails (e.g., SSIM < 0.3) while the raw-gradient version succeeds, the method does not transfer to actual federated gradient exchanges.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DRAGD exploits gradient discrepancies during federated unlearning to reconstruct forgotten data. For this claim to hold in a real FL system, the attack must operate on the information the server actually receives from a target client. In the experiments (Section VI.B), the 'pre-unlearning gradient' is computed as a single forward/backward pass over the entire selected subset (e.g., 16 images) and the 'post-unlearning gradient' over the remaining subset (e.g., 4 images). But the text states that FedAvg is used (Section VI.A), and in FedAvg each client performs multiple local SGD epochs and communicates a model delta, not a raw batch gradient. A multi-step model update is not the gradient of a single loss function on the client's data; it is an accumulated parameter change that standard DLG-style inversion, which Equation (2) and Equation (3) implement, cannot directly invert. The paper never evaluates DRAGD against such model deltas, so the empirical support only covers an idealized setting where the server observes exact per-client batch gradients. Even setting aside secure aggregation (which the paper cites as a defense in Section II.C), the exchanged object in vanilla FedAvg is a model update, not the 'gradient' that Algorithm 1 inverts. This mismatch means the headline claim that DRAGD exposes a 'critical privacy vulnerability' in federated unlearning systems is not established by the provided evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DRAGD (Data Reconstruction Attack based on Gradient Difference) and an enhanced variant DRAGDP, which aim to reconstruct client data erased during federated unlearning. The attack uses a pair of gradients: the pre-unlearning gradient computed on the full local dataset and the post-unlearning gradient computed on the remaining data, and optimizes synthetic inputs to match these gradients, with the remaining-data reconstruction fixed as an anchor. DRAGDP replaces the random initialization of the forgotten data with public prior images. The manuscript also claims a defense called FedANI, which is described only as injecting noise into sensitive gradient components. Experiments are reported on MNIST, CIFAR-10, and LFW with LeNet, ResNet18, and ConvNet64 models, comparing against DLG, IG, and CPL baselines, and reporting MSE, PSNR, and SSIM.","tokens_in":13256,"tokens_out":4158,"duration_ms":44804,"significance":"The paper addresses an important and timely problem: privacy risks introduced by federated unlearning, specifically whether gradient discrepancies before and after deletion enable data reconstruction. If the proposed attack were shown to work under the actual information exchanged in federated learning (e.g., FedAvg model deltas) and were properly compared with existing defenses, it would be a meaningful contribution to the security and privacy literature. However, the current manuscript does not provide such evidence. The experimental protocol does not match the stated federated learning setting, the claimed defense FedANI is never defined or evaluated, and the quantitative results lack error bars or repeated trials. The near-perfect reconstruction of the anchor 'Part' set further weakens the external validity of the reported attack performance. As presented, the claims are not substantiated by the experimental or theoretical content of the paper.","major_comments":[{"comment":"The experiments do not evaluate the attack on the information actually exchanged in FedAvg. Section VI.A states that FedAvg is used and that gradients are computed 'using the FedAvg algorithm,' but the attack in Algorithm 1 and Eq. (2)-(3) inverts raw per-client batch gradients (e.g., a single forward/backward pass over 16 or 4 images). In FedAvg, clients perform multiple local SGD steps and communicate a model delta, which is not the gradient of a single loss over the local data. The paper never tests DRAGD/DRAGDP against such multi-step model updates, so the central claim that DRAGD exposes a critical vulnerability in federated unlearning systems is not supported by the provided experiments.","section":"Section VI.A, VI.B, Algorithm 1, Eq. (2)-(3)"},{"comment":"FedANI is presented in the abstract and conclusion as a practical defense and Section VI.A announces that its performance will be assessed, but the paper contains no definition, algorithm, or experimental result for FedANI. The only mention is a parameter setting in Section VI.A ('In FedANI, the gradient noise coefficient is set to 1...'). This missing evaluation is a load-bearing omission for the claimed contribution, which explicitly includes FedANI as a defense.","section":"Abstract, Section VI.A, Section VII"},{"comment":"All quantitative results are reported as single numbers with no error bars, confidence intervals, or number of independent trials. The text acknowledges 'occasional reconstruction failures' in DRAGD (Section VI.B), but the frequency or impact of these failures is never quantified. Given the stochasticity of gradient-inversion optimization, the claims that DRAGD and DRAGDP 'significantly outperform' existing methods are not statistically supported.","section":"Tables I-IV"},{"comment":"The 'Part' reconstruction on MNIST achieves MSE 5.18e-5 and SSIM 1.000, essentially a perfect reconstruction of the remaining data. DRAGD and DRAGDP fix this Part as an anchor during the second-stage attack (Algorithm 1, Eq. (3)). The paper does not analyze how sensitive the attack is to imperfect Part reconstruction, which would be the realistic case under noise, clipping, or multi-step model updates. Without such sensitivity analysis, the reported attack performance may be an artifact of an unrealistically accurate anchor.","section":"Table I and Section V, Eq. (3)"},{"comment":"The threat model assumes the adversarial server can access exact per-client raw gradients before and after unlearning and can isolate the target client's updates. In a realistic federated system with multi-client aggregation or secure aggregation—which the paper itself lists as a defense in Section II.C—the server would not observe these raw per-client gradients. The paper does not discuss how DRAGD would operate under such constraints, so the claimed real-world applicability of the attack is not established.","section":"Section III.B"}],"minor_comments":[{"comment":"The manuscript contains numerous typos and garbled text, e.g., 'VER recent years' at the start of Section I, 'gl obal' in the Abstract, and inconsistent notation in Eq. (2), Eq. (3), and Algorithm 1 (e.g., the subscripts on the gradient symbols are incorrect or ambiguous). A thorough copyedit is needed.","section":"Throughout"},{"comment":"Figure numbering is inconsistent with the text: the MNIST results are introduced as 'Fig.3' but the figure is labelled Fig.2, and the LFW results are introduced as 'Fig.4' but the figure is labelled Fig.3. The CIFAR-10 figures are similarly mis-referenced.","section":"Section VI.B"},{"comment":"The baseline descriptions are inaccurate. For example, 'IG' is described as focusing on data points with the greatest influence on the global model, but the cited Inverting Gradients method (Geiping et al. [25]) is a gradient-matching attack, not an influence-based method. This misdescription undermines the credibility of the comparison.","section":"Section VI.A.3"},{"comment":"The reference list contains errors: reference [37] duplicates reference [32] (both are SecureML), and several entries have incomplete venue or formatting information.","section":"References"}],"recommendation":"reject","confidential_remarks":"The manuscript is not ready for publication in its current form. The core experimental protocol does not match the described federated learning setting, the claimed defense FedANI is entirely absent, and the results are presented without statistical support. These are not presentation issues; they are load-bearing gaps that make the central claims unsubstantiated. The paper appears to be a preliminary draft rather than a complete research article."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is sensible: reconstruct the remaining data from the post-unlearning gradient, fix it, then recover the forgotten data from the pre-unlearning gradient. That two-stage composition is a natural extension of DLG/IG, and applying it to federated unlearning is a fair novelty even if the mathematics is not new. The public-prior variant DRAGDP shows a real, measured improvement on faces, and the ablation that fixes the anchor data is thoughtful and informative. Credit where earned: the authors know what they are doing empirically in the narrow setup they chose.\n\nThat said, the paper has two major soft spots. First, FedANI is promised in the abstract and conclusion but never defined, implemented, or evaluated anywhere. That is not a minor omission; the paper advertises both an attack and a defense, and the defense is absent. Second, the experimental setup is unrealistic in a way that undercuts the headline claim. The 'pre-unlearning gradient' is computed as a single forward/backward pass over a small batch, and the 'post-unlearning gradient' over the remainder. In actual FedAvg, the server receives model updates after multiple local steps, which are not the gradients of a single loss function and cannot be inverted by the DLG-style objective in Equations (2) and (3). The stress-test note is on target here. The paper never evaluates against such model deltas, so the claimed real-world vulnerability in federated unlearning is not established.\n\nThe near-perfect 'Part' reconstruction (MSE 5.18e-5) also biases the subsequent forgotten-data reconstruction, and the lack of error bars or repeated trials makes the numbers hard to interpret. The baselines (DLG, IG, CPL) are appropriate but not strong, so the claimed 'significant outperformance' is credible only within this controlled setting.\n\nThe reader's verdict is fair. The general concern—that gradient exchange during unlearning may leak deleted data—is valid and worth investigating, but this paper does not demonstrate it under realistic assumptions. I would not cite this work in its current form, and I would not send it to peer review as is; the missing defense and the FedAvg mismatch are load-bearing issues that would require a major rewrite. A better version, with FedANI specified and experiments rerun on actual multi-step model deltas, would be worth a fresh look.","headline":"A cleanly presented two-stage gradient-inversion attack, but the paper is structurally incomplete and the experiments don't match the realistic FedAvg setting.","tokens_in":13797,"tokens_out":2437,"would_cite":false,"duration_ms":30522,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federated unlearning leaks the very data it is supposed to erase: two gradients, taken before and after deletion, are enough to rebuild the forgotten images.","keywords":["federated learning","federated unlearning","gradient inversion","data reconstruction attack","privacy leakage","gradient difference","honest-but-curious server","public data prior"],"falsifier":"Run the same DRAGD pipeline against a server that only observes the aggregated update of several clients (say, ten, with one of them unlearning) instead of the target client's isolated gradient; if high-fidelity images are still recovered from the aggregate, the attack generalizes beyond the paper's setup, and if not, the reported reconstruction quality depends on the isolated-gradient assumption.","tokens_in":12796,"feed_emoji":"🕵️","tokens_out":5091,"duration_ms":53713,"temperature":0.7,"pith_summary":"The paper tries to establish that federated unlearning, a mechanism meant to erase a client's data from a shared model, creates a new privacy leak: the server can compare the client's gradient before deletion with the gradient after retraining, and the difference between them encodes the erased samples. It proposes DRAGD, a two-stage attack that first reconstructs the kept data from the post-unlearning gradient and then, with those reconstructions fixed, recovers the forgotten data from the pre-unlearning gradient. The enhanced version DRAGDP initializes the search with public images, which sharply improves fidelity on structured data such as faces. If the claim is right, the unlearning operation itself is the attack surface, not just the ordinary gradient sharing during training.","feed_headline":"Attack rebuilds data that federated unlearning tried to erase","feed_subtitle":"A pre-deletion and post-deletion gradient pair carries enough signal to recover erased images, experiments show.","key_machinery":"The load-bearing object is the gradient discrepancy, the difference between the pre-unlearning gradient $\\nabla \\theta^*$ and the post-unlearning gradient $\\nabla \\theta^u$. DRAGD feeds synthetic noise through the retrained model, minimizes the squared Euclidean distance between the resulting gradient and the real post-unlearning gradient to recover the remaining data, and then freezes those recovered images while minimizing the gradient mismatch of the forgotten batch against the pre-unlearning model. DRAGDP uses the same two-stage optimization but replaces the random initialization of the forgotten batch with public prior images.","core_discovery":"The central claim is that the gradient pair available during federated unlearning—the update computed on the full dataset before removal and the update computed on the remaining data after retraining—carries enough information to reconstruct the supposedly erased samples. The paper shows this by reconstructing the remaining data first, then treating that reconstruction as a fixed anchor while optimizing a synthetic version of the forgotten data against the pre-unlearning gradient. In experiments on MNIST, CIFAR-10, and LFW, DRAGD recovers images with lower mean squared error and higher SSIM than the DLG and IG gradient-inversion baselines, and DRAGDP improves further when public data provide a good starting point. The message is that gradient differences act as a privacy fingerprint of the removed data.","pith_inferences":["A natural extension, not tested in the paper, is to replace the Euclidean gradient-matching objective with a cosine-similarity loss and a generative prior; the paper's own baseline [25] suggests this raises fidelity for larger batches.","Because DRAGDP succeeds when a public prior overlaps with the target domain, the same attack may work on medical or biometric data whenever a public corpus with similar structure exists.","The paper proposes FedANI as a defense but reports no defense experiments in the main evaluation; treating FedANI as validated would require a separate study.","A defender could concentrate on breaking the first stage: if reconstruction of the remaining data is degraded, the frozen anchor for the second stage disappears, and the whole attack loses its grip."],"forward_implications":["A server that merely follows the federated protocol can reconstruct a client's erased images without any access to client hardware or raw data.","The attack transfers across model families: it works on LeNet, ConvNet64, and ResNet18, with more complex models exposing at least as much gradient information.","Public-data priors turn hard reconstructions, such as facial images, into feasible ones, so the risk is highest exactly where the data is most sensitive.","Secure aggregation, gradient clipping, or multi-client mixing should block the attack, because the two-stage procedure relies on an isolated per-client gradient; the paper's own threat model assumes that isolation."],"supporting_citations":[{"why":"Supplies the gradient-matching optimization that DRAGD adapts to the unlearning setting.","marker":"[23]"},{"why":"Provides the high-fidelity gradient inversion baseline (IG) that DRAGD is compared against.","marker":"[25]"},{"why":"Supplies the CPL baseline that shows prior knowledge entering as a partial image.","marker":"[42]"},{"why":"Defines client-level federated unlearning, the operation whose gradient pair the attack exploits.","marker":"[9]"},{"why":"Frames the open privacy question about federated unlearning that the paper addresses.","marker":"[10]"},{"why":"Supports the premise that unlearning itself creates comparative gradient information that attackers can use.","marker":"[17]"}],"fun_headline_variants":["Gradient differences leak erased data in federated unlearning","Unlearning's gradient gap lets attackers rebuild deleted data","Federated unlearning leaves a gradient trail to erased images","Attack recovers erased data from federated unlearning's gradient gap"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack assumes the server receives the exact, unaggregated gradient of the target client both before and after unlearning; with secure aggregation or multi-client updates, those raw per-client gradients are not visible.","fun_headline_variants_meta":{"raw":{"variants":["Gradient differences leak erased data in federated unlearning","Unlearning's gradient gap lets attackers rebuild deleted data","Federated unlearning leaves a gradient trail to erased images","Attack recovers erased data from federated unlearning's gradient gap"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000465,"raw_usage":{"total_tokens":2270,"prompt_tokens":843,"completion_tokens":1427,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":1358}},"tokens_in":459,"tokens_out":1427,"duration_ms":9422,"temperature":1.0,"reasoning_tokens":1358,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T17:51:40.835656+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same DRAGD pipeline against a server that only observes the aggregated update of several clients (say, ten, with one of them unlearning) instead of the target client's isolated gradient; if high-fidelity images are still recovered from the aggregate, the attack generalizes beyond the paper's setup, and if not, the reported reconstruction quality depends on the isolated-gradient assumption.","supporting_citations":[{"cited_title":"Verifi: Towards verifiable federated unlearning,","cited_arxiv_id":null,"evidence_quote":"Supplies the gradient-matching optimization that DRAGD adapts to the unlearning setting."},{"cited_title":"Inverting gradients-how easy is it to break privacy in federated learning?,","cited_arxiv_id":null,"evidence_quote":"Provides the high-fidelity gradient inversion baseline (IG) that DRAGD is compared against."},{"cited_title":"QUOTIENT: Two-party secure neural network training and prediction,","cited_arxiv_id":null,"evidence_quote":"Supplies the CPL baseline that shows prior knowledge entering as a partial image."},{"cited_title":"Wu et al","cited_arxiv_id":null,"evidence_quote":"Defines client-level federated unlearning, the operation whose gradient pair the attack exploits."},{"cited_title":"Secure and efficient federated learning with provable performance guarantees via stochastic quantization,","cited_arxiv_id":null,"evidence_quote":"Frames the open privacy question about federated unlearning that the paper addresses."},{"cited_title":"Federated unlearning and its privacy threats,","cited_arxiv_id":null,"evidence_quote":"Supports the premise that unlearning itself creates comparative gradient information that attackers can use."}],"review_version":1}