{"id":"c7542eed-c042-445f-bd47-fc7df3bf9bf2","arxiv_id":"2508.06789","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ULIA recovers labels of forgotten data from federated unlearning model updates, achieving near-perfect attack success on class-level and client-level unlearning.","lead":"A new attack called ULIA infers the labels of data that clients asked to forget in federated unlearning, by comparing model weights before and after the forgetting process. In tests on MNIST and CIFAR-10 it recovered forgotten labels almost always, which matters because unlearning is supposed to protect privacy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ULIA's core gradient-reconstruction step (Eqs. 8-9) is unvalidated: the reported ASR could come from simpler parameter-change cues or threshold tuning rather than from the reconstructed forgotten-data gradient.","rationale":"The reader's weakest_assumption correctly flags the idealized full-batch assumptions behind the reconstruction chain. I agree that this is the most fragile part of the paper's argument, but I would sharpen it: the absence of any ablation or ground-truth gradient validation means we do not know whether the reported ASR is actually produced by the reconstructed G_f or by simpler signals already present in δ_local or δ_global. If the ablation shows that δ_local alone performs as well as ULIA, then the paper's proposed gradient-label mapping adds nothing and the central contribution is not established. If the cosine-similarity check fails, then Equation (9) is not a valid estimator of the forgotten gradient, and the paper's interpretive claim is false even if the empirical attack works. The reader's verdict of CONDITIONAL is appropriate; my concern reinforces the need for these checks rather than changing the verdict. I did not find grounds for outright rejection: the empirical tables, while missing error bars and baselines, are consistent with the existence of label-revealing traces in unlearning, and the high ASRs are not contradicted by visible evidence. The recommended concrete tests would either validate the mechanism or force a reinterpretation of the results.","tokens_in":7287,"tokens_out":8890,"duration_ms":117299,"concrete_test":"Run ULIA on the Table 1/2 settings with three ablations: (a) replace G_f in Eq. (10) with δ_local, (b) replace it with δ_global, and (c) replace it with a random gradient vector of the same norm; also compute ASR for random label selection. If (a) or (b) matches ULIA's ASR, the reconstruction in Eq. (9) is not the source of the attack. Additionally, in a controlled setting with known forgotten data, compute the cosine similarity between G_f from Eq. (9) and the true average gradient of the forgotten samples on the same model; if the cosine similarity is not significantly above zero under the paper's mini-batch batch-size-64 setting, Eq. (9) is invalid and the reported ASR must be explained by another mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanism of ULIA is Equation (8)-(9): the difference between global and local parameter changes, δ_global - δ_local, is claimed to be proportional to the forgotten data's gradient contribution, so dividing by an estimated learning rate reconstructs G_f. This is asserted, not derived, and it presupposes that (i) the global update is exactly the weighted sum of local updates with known aggregation weights, (ii) each local update is a single full-batch SGD step with a known learning rate, and (iii) all non-target clients' contributions either cancel or are negligible in the difference. The paper's own evaluation uses SGD with batch size 64 (Section 5), so mini-batch noise and multiple local steps violate (ii); under non-IID data, (iii) is implausible because δ_global aggregates heterogeneous client updates and δ_global - δ_local is dominated by other clients, not by the forgotten samples. No ablation is reported—e.g., using δ_local alone, δ_global alone, or a random baseline—so the ASR gains in Tables 1-3 may be driven by simpler cues (e.g., local update magnitude or class-level shifts) rather than by the reconstructed G_f. Additionally, the significance threshold λ is set to a fixed value on the test settings without independent calibration, so the reported ASR may be optimistically tuned. The central claim that ULIA infers labels through gradient reconstruction therefore rests on an unvalidated equation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces ULIA, a label inference attack against federated unlearning (FU). The attack assumes that the difference between pre- and post-unlearning global and target-client model parameters can be used to reconstruct the gradient contribution of the forgotten data. It estimates a learning rate from norm ratios, derives a gradient difference, and maps output-layer gradient magnitudes to label predictions using a significance threshold λ. The authors evaluate ULIA on MNIST and CIFAR-10 under three FU methods (FedEraser, Rapid Retrain, SGA-EWC) and three unlearning levels (sample, class, client), reporting near-100% Attack Success Rate (ASR) for class- and client-level unlearning and degraded but still high ASR for sample-level unlearning, including under non-IID data.","tokens_in":7666,"tokens_out":5441,"duration_ms":63897,"significance":"If validated, the result would constitute a meaningful new privacy threat: federated unlearning is intended to erase data influence, and ULIA claims to recover labels from the parameter differences induced by unlearning. The underlying signal—cross-entropy gradients carry label-specific information in the output layer—is standard and plausible, and the paper is among the first to frame this specifically for FU. However, the paper's contribution is not yet established: the central reconstruction equations are asserted without a rigorous derivation, no baselines or ablations are reported, and the significance threshold is fixed without calibration or sensitivity analysis. The high reported ASRs are plausible but could arise from simpler cues. The paper would be a useful contribution after these validation gaps are addressed.","major_comments":[{"comment":"Equation (8) asserts that δ_global - δ_local equals a term proportional to the forgotten data's gradient contribution. This reduction from Eq. (7) is valid only when all non-target clients have zero local parameter change. The paper's own definitions state that class-level unlearning removes class f from every client, and client-level unlearning removes one client and retrains with the remaining clients. In those cases, δ_global - δ_local,K contains a sum over other clients' local changes, which is not generally zero and is not proportional to client K's forgotten gradient. Thus Eq. (9)'s reconstruction chain is not justified for two of the three claimed unlearning levels. Please derive the exact expression separately for each FU level or restrict the claim to sample-level unlearning.","section":"Equation (8)-(9), Section 3"},{"comment":"No baseline or ablation is reported. All tables report only ULIA's ASR. To support the claim that high ASR comes from the reconstructed gradient G_f, the paper needs comparisons against (i) random label guessing adjusted for class prior, (ii) attacks using δ_local alone, (iii) attacks using δ_global alone, and (iv) simple non-gradient heuristics such as the per-output-unit magnitude of parameter change. Without these controls, the 100% ASR for class- and client-level unlearning could be explained by trivial parameter-shift cues rather than by the gradient-label mapping mechanism central to the paper.","section":"Tables 1-3, Section 5"},{"comment":"The significance threshold λ is set to 0.9, and no sensitivity analysis or calibration procedure is presented. Section 5.3 states that the impact of the significance threshold is studied, but no results or table for λ are provided. Since the ASR in Eq. (13) depends on thresholding the gradient-label score, the reported values are conditional on a free parameter chosen on the evaluation settings. Please report ASR as a function of λ, calibrate λ on a held-out split, or otherwise demonstrate that the results are not an artifact of threshold tuning.","section":"Section 4 Experimental Settings; Section 5.3"},{"comment":"The learning-rate estimate η_approx is asserted without derivation or error analysis. The ratio of norms of local and global parameter changes, averaged over R rounds, is not in general equal to the true learning rate under mini-batch SGD (batch size 64, Section 4), multiple local epochs, and non-IID data; the bias depends on gradient noise, correlation, and aggregation weights. Moreover, Eq. (9) divides by this scalar while the proportionality constant in Eq. (8) also involves the unknown aggregation weight w_K. Please provide the distributional assumptions, an error bound, or treat the overall scale as a free parameter and show that the attack is insensitive to it.","section":"Equation (6)"}],"minor_comments":[{"comment":"The notation in Eq. (10) is unclear: \\(\\bar{g}_l = (1/|L|) \\sum_{l\\in L} G_f l\\) mixes a summation index with a vector quantity. Please define G_f and l unambiguously.","section":"Equation (10)"},{"comment":"The table is labeled 'Impact of the Non-IID Data Distribution,' but the text does not describe how non-IID data was generated (e.g., Dirichlet concentration parameter, number of clients, or degree of heterogeneity). Without this, the non-IID results cannot be reproduced or interpreted.","section":"Table 3"},{"comment":"The ASR is defined as Intersection over Union over label sets. The interpretation of this metric depends on whether the attacker knows the number of forgotten label categories. Please clarify the relation between the metric and a standard label-recovery accuracy, and report the corresponding random-guessing baseline.","section":"Equation (13)"},{"comment":"Several equations are malformed in the submitted text (e.g., Eq. (1) is missing the learning-rate multiplication, Eq. (6) has ambiguous summation bounds). A careful proofread of the mathematical notation is needed.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant topic, and the core threat model is worth publishing if the validation is made rigorous. The main risk is overclaiming: the central equations are not derived for two of the three unlearning levels, and the absence of baselines leaves open that the reported ASRs come from simpler cues or from threshold tuning. I do not see evidence of misconduct, but the threshold calibration should be disclosed transparently."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the claim that federated unlearning leaks forgotten labels, often with near-perfect accuracy, is plausible and worth taking seriously. The paper's new part is the attack scenario—label inference across sample-, class-, and client-level unlearning—not the underlying gradient-label mapping, which is a known property of cross-entropy training. That distinction matters but doesn't diminish the contribution: showing that three FU methods all leak label information through parameter changes is a useful result.\n\nThe evaluation is better than a lot of attack papers: three FU methods, two datasets, IID and non-IID, several forgotten fractions, and both known/unknown label counts. The pattern in the results—class- and client-level unlearning give ASR near 1.0, sample-level is lower, non-IID hurts—is internally consistent with the idea that larger parameter shifts give stronger signal. Credit where due: the paper identifies a real gap and the attack design is sensible.\n\nNow the soft spots. The load-bearing step is Equations (6)–(9), where δ_global − δ_local is treated as proportional to the forgotten gradient. That requires deterministic full-batch SGD with known learning rate and known aggregation weights. The experiments use SGD with batch size 64 and multiple local updates, so the reconstruction is biased by mini-batch noise and other clients' updates; the paper never checks how much bias. Equation (6)'s learning-rate estimate from norm ratios is asserted without error analysis. More importantly, there are no baselines or ablations. I want to see comparison against δ_local alone, δ_global alone, and a random label guesser. Without those, the reported ASRs might be driven by simple parameter-change magnitude or by the threshold λ rather than by the reconstructed gradient. And λ is set to a fixed value on the evaluation settings; the paper doesn't report sensitivity to λ or independent calibration. No error bars, no code/data, so reproducibility is weak. The equation rendering in the version I saw is too garbled to check the math line by line; that alone argues for careful revision.\n\nNone of this kills the central claim: the high ASR for class/client unlearning is credible and important even if the detailed mechanism needs proof. But the paper currently overstates what it establishes.\n\nI'd send this to peer review, with the expectation of major revision: add ablations and baselines, error bars, λ sensitivity, tighten the derivation, and release code/data. If the authors do that, this will be a useful paper for the FU community. Without it, treat the 100% numbers with caution.","headline":"Federated unlearning leaks label information, and this paper is the first to show it, but the key reconstruction step needs proof before the 100% ASR numbers are taken at face value.","tokens_in":8113,"tokens_out":3575,"would_cite":true,"duration_ms":42514,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An attacker can infer the labels of forgotten data from parameter changes left by federated unlearning, reaching 100% success on class- and client-level unlearning under IID data.","keywords":["federated unlearning","label inference attack","gradient-label mapping","federated learning","machine unlearning","privacy attack","parameter changes","attack success rate"],"falsifier":"Reproduce ULIA on FedEraser, Rapid Retrain, and SGA-EWC while switching from full-batch single-step updates to mini-batch SGD with several local epochs and an adaptive optimizer, then measure ASR and the cosine similarity between the reconstructed gradient $\\hat{\\Delta}_g$ from Eq. (9) and the true gradient of the forgotten samples; if either metric collapses, the linear reconstruction chain is the load-bearing assumption. A simpler check is removing access to earlier-round local/global models so the learning-rate estimate cannot be formed and seeing whether ASR falls.","tokens_in":7213,"feed_emoji":"🕵️","tokens_out":9997,"duration_ms":101146,"temperature":0.7,"pith_summary":"Federated unlearning is meant to erase a client's data from a shared model, but this paper argues that the act of erasure itself leaves a readable trace. It introduces ULIA, a label inference attack that compares the target client's model and the global model before and after unlearning, reconstructs an approximate gradient for the forgotten data, and maps that gradient to the most likely labels. Across three levels of unlearning (sample, class, and client) and three unlearning algorithms, ULIA reports 100% Attack Success Rate for class- and client-level unlearning on IID data, and still reaches 93% to 62.3% when only 1% of a user's data is forgotten. If correct, this means a forgotten label is still recoverable from the public parameter history, and unlearning mechanisms must treat their own update dynamics as a disclosure channel.","feed_headline":"Attack infers erased labels in federated unlearning at 100%","feed_subtitle":"Even forgetting just 1% of a user's data still lets the attack name labels 93% to 62% of the time.","key_machinery":"The load-bearing mechanism is a gradient-label mapping built on parameter-change differencing. The attacker computes $\\Delta_{\\mathrm{local}}$ and $\\Delta_{\\mathrm{global}}$ from pre/post-unlearning models, estimates the effective learning rate as the averaged norm ratio $\\eta_{\\mathrm{approx}} = \\frac{1}{T}\\sum_t \\|\\Delta_{\\mathrm{local}}\\| / \\|\\Delta_{\\mathrm{global}}\\|$, and recovers an approximate gradient difference $\\hat{\\Delta}_g \\approx (\\Delta_{\\mathrm{global}} - \\Delta_{\\mathrm{local}})/\\eta_{\\mathrm{approx}}$ (up to aggregation weight). This reconstructed gradient is then compared, per label, with the average gradient variation in the output layer; the label with largest accumulat","core_discovery":"The paper's central discovery is that the difference between the global model's parameter change $\\Delta_{\\mathrm{global}}$ and the target client's local parameter change $\\Delta_{\\mathrm{local}}$ during unlearning is proportional to the gradient contribution of the forgotten data: $\\Delta_{\\mathrm{global}} - \\Delta_{\\mathrm{local}} \\approx \\eta \\omega_k g_{\\mathrm{forget}}$, where $\\eta$ is the learning rate, $\\omega_k$ the client's aggregation weight, and $g_{\\mathrm{forget}}$ the forgotten samples' gradient. An attacker who can observe pre- and post-unlearning models can therefore reconstruct an approximate gradient difference, provided the learning rate is known or estimated from past lo","pith_inferences":["The reconstruction chain assumes a clean deterministic gradient contribution; testing ULIA under mini-batch SGD, multiple local epochs, or adaptive optimizers would show how much of the reported ASR survives real training dynamics.","A natural defense is to restrict access to pre/post-unlearning model snapshots or to add calibrated noise to parameter differences, because the linear model implies that differentially private updates would directly reduce the label signal.","The same gradient-label mapping could be turned around to infer other attributes of forgotten data or to test whether an unlearning request was actually honored, since a residual gradient signal would indicate incomplete removal.","Because the learning-rate estimator only needs past local and global parameter norms, ULIA-style attacks may transfer to other gradient-leakage settings with known aggregation rules."],"forward_implications":["Observing model parameters before and after an unlearning request is enough to mount a label inference attack; no access to client data is required beyond the parameter history.","Larger unlearning requests leak more: class-level and client-level unlearning give 100% ASR on IID data, while sample-level unlearning loses accuracy as the number of forgotten label categories grows.","Small forget requests are not safe: with only 1% of a client's data forgotten, ASR still ranges from 93% down to 62.3% across settings.","Not knowing how many labels were forgotten costs only a few points of ASR, so the attack does not depend on strong prior knowledge.","Non-IID data reduces but does not eliminate the leak, with per-setting ASRs roughly 0.6 to 0.96 in the reported tables."],"supporting_citations":[{"why":"FedEraser, one of the three federated unlearning methods attacked; its calibrated historical-update reconstruction produces the parameter changes ULIA exploits.","marker":"[14]"},{"why":"Rapid Retrain, the efficient retraining-based unlearning baseline whose pre/post-unlearning model difference the attack uses.","marker":"[16]"},{"why":"SGA-EWC, the reverse stochastic gradient ascent and elastic weight consolidation framework that serves as the third attack target.","marker":"[27]"},{"why":"Class-level unlearning definitions that fix the scenario where ULIA reports 100% ASR.","marker":"[2,26]"},{"why":"Client-level unlearning definition that fixes the scenario where ULIA reports 100% ASR.","marker":"[28]"},{"why":"Sample-level unlearning definition, the setting where ULIA's ASR varies with the number of forgotten classes.","marker":"[29]"},{"why":"Supplies the IoU-based attack success rate metric used to score label inference.","marker":"[7,33]"}],"fun_headline_variants":["Unlearning leaks labels: 100% attack success","Erased data still exposed: 100% label inference","Label theft in federated unlearning: 100% ASR","Even 1% forgotten data leaks labels up to 93%","Model changes reveal unlearned labels at scale"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The attack assumes the measured parameter change after unlearning is just the forgotten samples' gradient contribution scaled by a known learning rate and aggregation weight, with no noise from batching, multiple local update steps, or optimizer dynamics.","fun_headline_variants_meta":{"raw":{"variants":["Unlearning leaks labels: 100% attack success","Erased data still exposed: 100% label inference","Label theft in federated unlearning: 100% ASR","Even 1% forgotten data leaks labels up to 93%","Model changes reveal unlearned labels at scale"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000163,"raw_usage":{"total_tokens":1084,"prompt_tokens":751,"completion_tokens":333,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":251}},"tokens_in":495,"tokens_out":333,"duration_ms":3962,"temperature":1.0,"reasoning_tokens":251,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T22:31:54.420314+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce ULIA on FedEraser, Rapid Retrain, and SGA-EWC while switching from full-batch single-step updates to mini-batch SGD with several local epochs and an adaptive optimizer, then measure ASR and the cosine similarity between the reconstructed gradient $\\hat{\\Delta}_g$ from Eq. (9) and the true gradient of the forgotten samples; if either metric collapses, the linear reconstruction chain is the load-bearing assumption. A simpler check is removing access to earlier-round local/global models so the learning-rate estimate cannot be formed and seeing whether ASR falls.","supporting_citations":[{"cited_title":"In: IEEE INFOCOM 2022- IEEE Conference on Computer Communications","cited_arxiv_id":null,"evidence_quote":"Rapid Retrain, the efficient retraining-based unlearning baseline whose pre/post-unlearning model difference the attack uses."},{"cited_title":"In: Proceedings of the sixteenth ACM international conference on web search and data mining","cited_arxiv_id":null,"evidence_quote":"Client-level unlearning definition that fixes the scenario where ULIA reports 100% ASR."}],"review_version":1}