{"id":"979ad3d1-bea6-4064-83c4-5d5059fb854b","arxiv_id":"2501.13683","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VFU-KD and VFU-GA unlearn clients, features, and samples in vertical federated learning with no communication during unlearning, using stored embeddings.","lead":"This paper presents two methods, VFU-KD and VFU-GA, that remove the influence of a client, a feature, or specific samples from a vertical federated learning model without new communication between parties. The added cost is that the server must store the embeddings sent during training, and the paper offers a membership-inference test to verify removal.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"VFU-KD distills its student from a teacher that still contains the unlearned party's influence, and the only audit is a weak 10-epoch, 32-neuron MIA; residual influence may persist.","rationale":"My concern differs from the reader's: the reader flags staleness of stored embeddings, which affects whether the student matches a retrained model's utility. I focus on a more fundamental issue: the teacher itself is contaminated, and the distillation objective actively encourages the student to mimic that contamination. Even with fresh embeddings, if the teacher's logits encode the target party's contributions, the student can inherit them. The paper's MIA audit is too weak to rule this out, and the authors admit this in Section 4.5(3). Utility comparisons to retraining show comparable task performance, but task performance does not imply that the target party's influence is gone; a model can retain sensitive information while having equal accuracy. The concrete test above would settle whether residual influence exists. Because this concern targets the core claim of unlearning (not just utility), it is load-bearing. However, the paper presents a novel framework and plausible mechanisms, so conditional acceptance with a required stronger audit is appropriate, matching the reader's CONDITIONAL verdict.","tokens_in":21853,"tokens_out":10177,"duration_ms":90920,"concrete_test":"Run a strong membership-inference attack (e.g., Carlini et al., IEEE S&P 2022) against the output probabilities of (i) the VFU-KD unlearned model, (ii) a model retrained from scratch without the target party, and (iii) the original contaminated model. Compare the ROC curves and AUCs; if the attack's true-positive rate at a fixed false-positive rate is significantly higher on the unlearned model than on the retrained model, residual target-party influence remains, refuting the approximate-unlearning claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For VFU-KD (Algorithm 2), the student model is trained with a distillation loss that matches its logits to those of the old active model (the teacher, lines 5 and 10-11), while the teacher was trained with the target passive party's embeddings. The prediction loss on true labels is the only party-agnostic signal, so the student is explicitly pulled toward the teacher's target-party-dependent decision boundary. No argument in Section 3 shows that this distillation removes information about the target party; the KL-divergence lower bound delta merely measures divergence between student and teacher, not distance from a retrained model. The sole empirical evidence for unlearning is the MIA in Section 4.4, a single hidden layer with 32 neurons trained for 10 epochs on output probabilities. The paper itself acknowledges in Section 4.5(3) that this is a 'relatively weak membership inference attack.' A weak MIA can easily fail to detect residual influence, especially for target-party features that do not strongly shift the output distribution. Consequently, the central claim that VFU-KD achieves approximate unlearning (removal of the target party's influence) is not established, and there is a plausible mechanism by which it fails.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two unlearning methods for vertical federated learning (VFL): VFU-KD, which uses knowledge distillation to remove a passive party or a feature within a passive party from the active model without communication between active and passive parties, and VFU-GA, which uses gradient ascent on the forget set to remove training samples. The active party stores historical embeddings during training, enabling these methods to operate without additional communication. The paper evaluates utility (AUC, F1) on six tabular and two image datasets against retraining from scratch and an R2S fast-retraining baseline, and audits unlearning with a membership inference attack (MIA). The central claims are that VFU-KD and VFU-GA achieve approximate unlearning with utility comparable to retraining (0-2% improvement or 1-5% loss in most cases), and that no communication is needed during unlearning at the cost of storing historical embeddings.","tokens_in":22050,"tokens_out":6206,"duration_ms":58331,"significance":"If validated, the paper would fill a real gap: most federated unlearning work targets horizontal FL, while VFL has distinct constraints (label server, communication cost, and model-size changes when removing parties or features). The no-communication property and the treatment of client, feature, and sample unlearning in one framework are timely and potentially useful for practical VFL deployments. The paper is also transparent about its storage overhead and the weakness of its MIA audit. However, the evidence supporting the central unlearning claim is currently insufficient: the KD objective is not shown to remove target-party influence, the audit attack is self-admittedly weak, and some utility numbers contradict the stated 1-5% loss range. The contribution is therefore conditional on a substantially strengthened evaluation.","major_comments":[{"comment":"The distillation objective trains the randomly initialized student to match the logits of the old active model, but that teacher was trained on H_t that includes the target party's embeddings (line 5 computes the teacher prediction before H_u is removed on line 6). The paper's lower-bound argument on KL divergence shows only that the student diverges from the teacher; it does not show that the student is close to a model retrained without the target party, nor that target-specific information has been removed. This is the central unlearning claim and needs either a formal argument relating the divergence to closeness to a retrained model, or substantially stronger empirical auditing (e.g., a stronger MIA, linear-probing tests on the student's representation, or targeted queries where the target party's contribution is known to matter).","section":"Section 3, Algorithm 2, Eq. (5)"},{"comment":"The same issue applies to feature unlearning: the student's embedding is distilled toward the teacher embedding, and the teacher model was trained with the target feature present. Minimizing KL between student and teacher embeddings, without any label or task signal on the reduced feature set, can preserve the teacher's target-feature-dependent behavior in the student. The paper should provide evidence that the student's embedding is not merely a compressed version of the teacher's but actually removes the target feature's influence, for example by showing that an attacker cannot distinguish target-feature contributions in the student or by comparing against a model trained from scratch without that feature.","section":"Section 3, Algorithm 3, Eqs. (6)-(7)"},{"comment":"The MIA audit used to support the unlearning claim is a single-hidden-layer 32-neuron model trained for 10 epochs on output logits, and the paper itself states in Section 4.5(3) that this is a 'relatively weak' attack. No error bars or statistical significance tests are reported for the attack accuracy, despite the paper stating that each experiment was repeated three times. A weak attacker can easily fail to detect residual influence, especially when the target party's features do not strongly shift the output distribution. The audit therefore does not establish approximate unlearning; the authors should use a stronger, better-calibrated MIA (e.g., the LiRA-style attack they cite as future work in the conclusion) and report attack performance on the unlearned model versus a retrained model with error bars.","section":"Section 4.4 and Section 4.5, limitation (3)"},{"comment":"The abstract and Section 4.1 claim that remaining cases show only a modest utility loss of 1-5%, but the tables contradict this. In Table 2, the Poqemon row at the 5th epoch shows VFU-KD F1=0.54 versus 0.75 for retraining from scratch, an absolute drop of 0.21, and the 15th, 35th, and 45th epoch rows show drops of 0.06-0.10; Table 1's Poqemon AUC at the 5th epoch is 0.88 versus 0.94. These are well outside the claimed 1-5% range. Additionally, although the experiments were repeated three times, no table reports standard deviations or per-run values, so small differences between methods are not shown to be statistically meaningful. The utility claims need to be corrected and the results need error bars.","section":"Section 4.1, Tables 1 and 2"},{"comment":"The paper acknowledges the storage cost of historical embeddings but does not address the staleness of those embeddings. Algorithm 2 trains the student entirely on stored H_t after removing H_u. If the remaining passive parties' models were trained jointly with the target party, those stored embeddings may encode target influence, and after unlearning the remaining parties may continue to update their local models. In either case, the stored embeddings need not correspond to the embeddings that a retrained-from-scratch model would see. Since the no-communication property depends on the stored H_t being a valid stand-in for current passive-party behavior, this assumption should be stated explicitly and tested, for example by comparing VFU-KD against a variant in which the remaining passive parties' embeddings are recomputed after retraining.","section":"Section 4.5, limitation (4), and Algorithm 2"}],"minor_comments":[{"comment":"The loss in Eq. (1) is written with (1-alpha) on the prediction loss and no explicit coefficient on the KL term, whereas Eq. (5) writes L = alpha * L_distill + (1-alpha) * L_pred. Please define the coefficients consistently and state the default value of alpha in one place.","section":"Section 2.2, Eq. (1) vs Section 3, Eq. (5)"},{"comment":"The Hepmass entry at the 45th epoch is 11.3 GB, which is smaller than the 35th epoch entry of 88.2 GB; this looks like a typo (possibly 113.0). Also, the Poqemon row uses '22' without a decimal point while other rows have one decimal place.","section":"Table 3"},{"comment":"The captions refer to shading indicating unlearning epochs, but the figures are reproduced in grayscale in the text and the shading is difficult to interpret. Please add a legend, distinct line styles, or explicit vertical markers for the unlearning epoch.","section":"Figures 3 and 5"},{"comment":"The x-axis labels in the MIA plots skip some of the experimental epochs (e.g., 15 and 50), and the plots do not show any measure of variance. Please align the x-axis with the epochs listed in Section 4 and add error bars or confidence bands.","section":"Section 4.4, Figures 8 and 10"},{"comment":"The text says feature importance is computed with feature ablation, but it does not describe how the 'most important' and 'least important' features are selected when features are distributed across multiple passive parties. Figure 14 also appears to lack axis labels for the Poqemon panel; please make the feature-selection procedure explicit.","section":"Section 4.2 and Figure 14"}],"recommendation":"major_revision","confidential_remarks":"The paper is transparent about its limitations, which is to its credit, but the gap between the central unlearning claim and the evidence is substantial: the KD-based methods are not shown to remove the target party's influence, the audit is weak, and some utility numbers contradict the stated loss range. I believe the work is salvageable as an empirical/systems contribution if the authors add stronger audits, correct the utility claims, and clarify the stored-embedding assumption, so I recommend major revision rather than rejection. I would also check the fit with the venue's expectations for privacy evaluation, since the current MIA would likely be considered too weak for a security-focused audience."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is the first paper to handle feature unlearning in vertical federated learning, and it gives communication-free protocols for party, feature, and sample unlearning by reusing stored embeddings. That gap is real, and the ideas are simple and plausible. The experimental breadth—six tabular and two image datasets—is decent, and the limitations section is honest about the weak MIA and the storage cost.\n\nNow the soft spots. The abstract says utility loss is 1–5% in the remaining cases, but Table 2 shows Poqemon at the 5th epoch F1 0.54 vs retrained 0.75, and at the 35th epoch 0.67 vs 0.77. Those are double-digit losses. Either the claim needs to be narrowed or the tables are wrong. Also, they say experiments were repeated three times but report no variance anywhere, so we cannot judge significance.\n\nThe bigger issue is the audit. The MIA is a single hidden layer, 32 neurons, 10 epochs, and they admit it is weak. For VFU-KD, the student is distilled from a teacher that still contains the target party's embeddings. The KL term pulls the student toward the teacher's logits computed on the full input, so there is a mechanism for residual influence to persist. The paper's delta lower bound measures divergence between student and teacher, not closeness to a retrained model. Without a stronger attack or a formal argument, the central claim of approximate unlearning is not fully established. I think the method probably works in practice, but the evidence as presented is thin.\n\nMinor things: no released code, the R2S baseline adaptation to feature unlearning is not specified, and the stored historical embeddings can become stale if remaining parties update their models—acknowledged only as storage cost.\n\nWho this is for: people working on VFL privacy and machine unlearning. The gap is real, the approach is plausible, and the issues are addressable with revisions: honest numbers, error bars, a stronger MIA or a formal argument, and code. I would send it to peer review.","headline":"A real but niche contribution to VFL unlearning, weakened by overclaimed utility numbers and a weak audit; deserves serious revision, not a desk reject.","tokens_in":22616,"tokens_out":2927,"would_cite":false,"duration_ms":26724,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VFU-KD and VFU-GA achieve approximate unlearning in vertical federated learning for passive parties, features, and samples without any communication between active and passive parties during the unlearning phase, while keeping utility…","keywords":["federated learning","unlearning","vertical federated learning","auditing","membership inference attack","knowledge distillation","gradient ascent"],"falsifier":"Run VFU-KD to unlearn a client at epoch 25, then let one remaining client perform additional local training for five epochs, and compare the unlearned active model against a model retrained from scratch with the same updated client; if predictions on a holdout set diverge by more than the paper's reported 1–5% utility gap, the assumption of static stored embeddings is violated.","tokens_in":21597,"feed_emoji":"🔐","tokens_out":6095,"duration_ms":49132,"temperature":0.7,"pith_summary":"The paper proposes that in vertical federated learning, unlearning a client, a feature, or a set of samples can be done by the active party alone, without any extra communication with the passive parties, provided it has saved the embeddings sent during training. For removing a client or feature, VFU-KD trains a smaller student model with knowledge distillation from the old active model on the stored embeddings with the target contribution deleted. For removing samples, VFU-GA runs gradient ascent on the target samples' stored embeddings while minimizing loss on the rest. Across six tabular and two image datasets, the authors report utility close to retraining from scratch or the R2S baseline, and they audit the removal with a membership inference attack. If correct, this gives vertical federated learning a low-overhead path to satisfying right-to-be-forgotten requests.","feed_headline":"Stored embeddings make VFL unlearning communication-free","feed_subtitle":"The active party alone can delete a client, a feature, or samples and still match retrain-from-scratch accuracy.","key_machinery":"The central object is the active party's stored history of concatenated embeddings, $\\mathbf{H}_t = \\mathrm{concat}(\\mathbf{H}_1,\\dots,\\mathbf{H}_K)$ for each training round $t$, together with the knowledge-distillation loss $\\mathcal{L} = \\alpha\\,\\mathrm{KL}(\\hat{y}_{\\text{student}},\\hat{y}_{\\text{teacher}}) + (1-\\alpha)\\,\\mathcal{L}_{\\text{pred}}$ that trains a shrunken student model on $\\mathbf{H}_t \\setminus \\mathbf{H}_u$. This makes model compression (fewer input neurons after removing a party or feature) and unlearning happen in one step, with no need to ask remaining parties for new data. For sample unlearning, the same stored embeddings enable a gradient ascent update $\\Theta^{t+1}_K = \\Theta^t_K - \\eta_1 \\nabla L_{\\text{retain}} + \\lambda \\nabla L_{\\text{target}}$ that pushes the active model away from the forgotten samples.","core_discovery":"The paper's central claim is that approximate unlearning in vertical federated learning can be made communication-free: because the active party already stores the per-batch embeddings sent by passive parties during training, it can reconstruct the effect of a retrained model without contacting those parties again. VFU-KD handles client and feature unlearning by initializing a student model with the reduced input size, using the old active model (or the old passive model) as teacher, and minimizing a combination of prediction loss and KL-divergence to the teacher's logits; for feature unlearning within a passive party, the loss is just the KL divergence between the student's and teacher's embeddings. VFU-GA handles sample unlearning by taking a few gradient ascent steps on the embeddings of the target samples and descent steps on the retain set. The paper reports that these methods match or slightly exceed retraining-from-scratch utility in many cases, with modest 1–5% losses otherwise, and that a membership inference attack can detect the unlearning.","pith_inferences":["If a remaining passive party continues local training after the unlearning point, the stored embeddings become stale; testing VFU-KD under that drift would show whether the no-communication property survives evolving models, or whether it needs periodic refresh.","The same distillation-on-stored-embeddings recipe could plausibly be extended to unlearning clusters of samples or to sequential unlearning requests, since each request only edits $\\mathbf{H}_t$ and re-runs the student training.","The reported utility parity with retraining suggests that a larger-scale study across heterogeneous feature partitions and label distributions is a natural next step, rather than a guaranteed outcome."],"forward_implications":["A passive party can be removed from a VFL model without that party or any other passive party exchanging a single message with the active party during unlearning, as long as the active party kept the training-time embeddings.","Feature unlearning becomes possible for passive parties that never see labels: the party can shrink its own local model by distillation on its own data, and the active model no longer receives that feature's contribution.","Sample unlearning can be completed in a few gradient steps on stored embeddings rather than retraining the whole federation, cutting the cost of right-to-be-forgotten requests.","The active party's storage requirement becomes the price of communication-free unlearning; the paper shows this trade-off explicitly by reporting the gigabytes R2S would need to communicate for benchmarks.","Membership inference attacks can serve as an audit signal for VFL unlearning, since the attack's accuracy drops after unlearning."],"supporting_citations":[{"why":"Frames the machine-unlearning problem with shard-based retraining that this paper extends to VFL.","marker":"[3]"},{"why":"Prior VFL unlearning method restricted to logistic regression, motivating a more general approach.","marker":"[6]"},{"why":"The R2S fast-retraining baseline that VFU-KD and VFU-GA are compared against.","marker":"[32]"},{"why":"Gradient-ascent based federated unlearning that VFU-GA adapts to sample removal.","marker":"[12]"},{"why":"Knowledge-distillation based federated unlearning that VFU-KD adapts to VFL.","marker":"[35]"},{"why":"Gradient ascent as an unlearning mechanism in centralized machine learning, the basis of VFU-GA.","marker":"[11]"},{"why":"Membership inference attack used as the audit tool for unlearning in VFL.","marker":"[26]"},{"why":"Knowledge distillation survey that supplies the distillation loss formulation used by VFU-KD.","marker":"[10]"}],"fun_headline_variants":["Communication-free unlearning for vertical federated learning","Stored embeddings make VFL unlearning communication-free","VFL unlearning: drop clients, features, or samples with zero communication","VFL unlearning: no passive party contact needed"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The stored historical embeddings still represent what the remaining passive parties would currently produce; if any of those parties updates its local model after the unlearning moment, or the sample alignment changes, the unlearned model is built from outdated embeddings and would not match a freshly retrained model.","fun_headline_variants_meta":{"raw":{"variants":["Communication-free unlearning for vertical federated learning","Stored embeddings make VFL unlearning communication-free","VFL unlearning: drop clients, features, or samples with zero communication","VFL unlearning: no passive party contact needed"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001391,"raw_usage":{"total_tokens":5702,"prompt_tokens":1094,"completion_tokens":4608,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":710,"completion_tokens_details":{"reasoning_tokens":4541}},"tokens_in":710,"tokens_out":4608,"duration_ms":31566,"temperature":1.0,"reasoning_tokens":4541,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:41:26.317666+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run VFU-KD to unlearn a client at epoch 25, then let one remaining client perform additional local training for five epochs, and compare the unlearned active model against a model retrained from scratch with the same updated client; if predictions on a holdout set diverge by more than the paper's reported 1–5% utility gap, the assumption of static stored embeddings is violated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior VFL unlearning method restricted to logistic regression, motivating a more general approach."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The R2S fast-retraining baseline that VFU-KD and VFU-GA are compared against."}],"review_version":1}