{"id":"97910d6e-04b8-4807-b98f-aa02ccd50383","arxiv_id":"2607.22665","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Obliviate performs recommender unlearning via a low-rank Newton-style downdate with an Adam-based curvature proxy plus a calibration stage, approximating retrained models at lower cost.","lead":"The paper introduces Obliviate, a two-stage method for removing user interaction data from recommender models without full retraining, using a low-rank curvature-aware downdate followed by a calibration stage. It reports strong forgetting and preserved recommendation quality at low cost on three benchmarks, though tests use synthetic injected interactions rather than real deletions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The curvature-aware claim rests on unverified Assumption K.4: the Adam second-moment diagonal bH must approximate H well enough that ||I−bH^{-1}H||<1; the paper never measures this, and for MF/LightGCN the Hessian is not diagonally dominant.","rationale":"The reader's weakest assumption is exactly Assumption K.4, and I agree that it is the linchpin of the theoretical guarantee. The paper's central contribution is a 'curvature-aware' downdate; without an accurate Hessian surrogate, the method degenerates to a scaled gradient step and Theorem K.12's bound on distance to retrained optimum loses its curvature-proxy term's control. The concern is concrete and testable. The reader's CONDITIONAL verdict remains appropriate: the paper should be accepted only if this condition is empirically verified or the theory is weakened accordingly. I did not find a more load-bearing concern than this; the synthetic deletion protocol and the speedup inconsistency are secondary because they affect external validity and presentation, not the core theoretical argument.","tokens_in":27753,"tokens_out":5613,"duration_ms":58630,"concrete_test":"On ML-1M with MF-BPR (and, if feasible, LightGCN): (1) select a deletion set S as in the paper; (2) compute g_S and bH^{-1}g_S using Eq. (7); (3) estimate H^{-1}g_S by conjugate gradient with Hessian-vector products (or use the exact Hessian for MF-BPR); (4) compute the relative error ||bH^{-1}g_S − H^{-1}g_S|| / ||H^{-1}g_S|| and the spectral quantity ||I − bH^{-1}H||_2 using a few Hessian-vector products with a Lanczos-type estimator. If the relative error is large (e.g., >50%) or the spectral norm is ≥1, Assumption K.4 is violated and the theoretical guarantee in Theorem K.12 does not apply. Additionally, run LUA with bH replaced by the identity; if unlearning quality is unchanged, the curvature proxy is not doing the claimed work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theoretical claim is Theorem K.12: LUA's output is close to the retrained optimum, with error decomposed into Taylor remainder, curvature-proxy error, and low-rank projection error. The curvature-proxy error term is bounded only under Assumption K.4: ||I − bH^{-1}H||_2 ≤ ε_H < 1, where bH = diag(√v̂+ε) is built from Adam's second moments (Eq. 7). The paper justifies bH via Assumptions H.3 (diagonal dominance of H) and H.6 (H_jj ≈ c√E[g_j^2]), but these are asserted, not checked.\n\nThis is load-bearing because if K.4 fails, the 'Newton-style downdate' is merely a per-coordinate scaled gradient step. For matrix-factorization and LightGCN models, the Hessian contains off-diagonal blocks coupling user and item embeddings (and graph-propagated embeddings in LightGCN); there is no reason the diagonal of squared stochastic gradients should approximate H, let alone be a good inverse-Hessian surrogate. Adam's v̂ is a gradient-variance estimate, not a curvature estimate. Moreover, LAC can only refine within the low-rank subspace chosen by LUA; if bH is wrong, that subspace may miss the true retraining direction, so calibration cannot repair the error. The empirical results may still hold, but the paper's claim of 'curvature-aware unlearning' and the accompanying distance-to-retrain bound are unsupported without measuring K.4.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Obliviate, a two-stage unlearning framework for recommender systems. Stage I (LUA) computes a deletion-set gradient, applies a curvature-scaled downdate using a diagonal Adam second-moment proxy, and projects the resulting parameter shift onto low-rank adapters. Stage II (LAC) calibrates only the adapter parameters, using a BPR-style loss on the deleted set plus a distillation loss on a retained buffer. Experiments on ML-1M, Amazon, and Yelp with MF-BPR and LightGCN report utility close to or above the original model, high post-unlearning demotion rates, and large speedups over full retraining.","tokens_in":28209,"tokens_out":7950,"duration_ms":83076,"significance":"If the central approximation is valid, Obliviate is a practically appealing approach: it keeps the backbone frozen, makes unlearning modular and reversible, and avoids full Hessian computation. The paper also provides a detailed theoretical appendix with explicit error decompositions and an honest limitation statement about the Adam-based curvature proxy. However, the main theoretical guarantee rests on an unmeasured and questionable assumption about the preconditioner, and the empirical evaluation relies entirely on a synthetic deletion protocol with no error bars. The contribution is therefore significant only conditionally on those points being resolved.","major_comments":[{"comment":"Theorem K.12's distance-to-retrain bound requires ||I - bH^{-1}H||_2 <= eps_H < 1, where bH is the Adam second-moment diagonal. The paper never measures this quantity or checks Assumption H.6. For MF-BPR and LightGCN the Hessian is not diagonal dominant: it contains off-diagonal user-item blocks, and in LightGCN the graph propagation couples many coordinates. Adam's v_hat is a variance estimate, not a curvature estimate, so the claimed 'curvature-aware' character of the downdate is unsupported. If K.4 fails, the update is just a scaled gradient step and the end-to-end bound collapses. Please report the empirical value of ||I - bH^{-1}H|| on the actual models/datasets, and if it does not satisfy the assumption, weaken the theoretical claim or replace the proxy.","section":"Appendix K, Assumption K.4 and Eq. (7)"},{"comment":"All experiments are run under a synthetic deletion protocol in which the deletion set consists of injected low-preference interactions. This is a legitimate robustness check, but it does not demonstrate unlearning of genuine user interactions, which is the stated GDPR/CCPA motivation. Removing synthetic noise may improve utility, while removing real interactions changes the data distribution. The abstract's claim of 'deletion ratios as high as 20% of the data' is also inconsistent with the experiments, which delete interactions from 20% of users. Please add at least one real-deletion experiment, state the actual fraction of interactions deleted, and scope the claims accordingly.","section":"§3.2 and Appendix F"},{"comment":"For a fixed backbone and dataset, the retrained model should be a common baseline, yet IFRU's Retrained row on ML-1M MF-BPR reports R@10 = 0.1179 while RRL and Obliviate report 0.1265. Table 2 likewise reports different retraining times for the same backbone/data condition (e.g., 975s for IFRU vs 533.5s for Obliviate on ML-1M MF-BPR). This suggests the methods were not compared against a consistent retrained checkpoint, making the utility and speedup comparisons hard to interpret. Please use one retrained model per backbone/data set for all methods, or clearly justify why separate retrained rows are method-specific.","section":"Tables 1 and 2"},{"comment":"The completeness metric, Demotion Rate, is the same ranking objective that LAC optimizes with its BPR unlearning loss. Reporting it as the main evidence of forgetting is therefore circular: the method is evaluated on a quantity it directly trains. In addition, Algorithm 1 and the witness set in Eq. (12) use the deleted set S after unlearning, which conflicts with the premise that the data has been erased. Please provide an independent completeness check (e.g., parameter distance to retraining, membership inference, or score behavior on held-out probes not used in calibration) and explicitly justify or drop the post-deletion use of S.","section":"Eq. (16) and Eqs. (13)–(14)"},{"comment":"No standard deviations, multiple seeds, or statistical tests are reported. Many utility differences are extremely small — in Table 3 the ablation differences at R@20 are around 0.001–0.003 — so the 'consistent drops/gains' claims cannot be evaluated. Please report mean±std over at least five seeds and, if possible, paired significance tests. This is necessary to support the empirical contribution.","section":"Tables 1–3 and Fig. 3"}],"minor_comments":[{"comment":"The text 'J−S(θ0) = −gs' should read '∇J−S(θ0) = −g_S'.","section":"§2.1.1"},{"comment":"For SISA and RecEraser, the Retrained row is all dashes. It would be clearer to state explicitly that no full-retrained comparator is reported for these baselines, or to add one.","section":"Table 1"},{"comment":"The abstract says 'deletion ratios (as high as 20% of the data)' while the experiments inject 80% of interaction counts for 20% of users. Please state the true percentage of total deleted interactions and use consistent terminology.","section":"Abstract vs. §3.2"},{"comment":"The proof of Theorem H.5 is essentially the same as that of Theorem H.1; consider condensing to avoid repetition.","section":"Appendix H.4.2"}],"recommendation":"major_revision","confidential_remarks":"The central idea is promising and the empirical results, if reproducible and de-circularized, would be of interest to the community. However, the theoretical centerpiece rests on an unvalidated curvature-proxy assumption, and the tables contain inconsistencies in the retrained baselines. I would not recommend acceptance before the authors provide a direct measurement of the K.4 quantity, add real-deletion experiments, align the retrained baselines, and report variance. This is fixable within the scope of the paper, so I do not recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The two-stage design is a genuine combination: a low-rank unlearning adapter initialized by a Newton-style downdate, then a calibration pass that only touches the adapters. That is new relative to IFRU, SCIF, RRL, SISA, and RecEraser, and it is a practical recipe for parameter-local unlearning in recommenders. The Newton downdate (θ0 − H^{-1}gS) is textbook, and the paper is honest enough to structure its end-to-end bound into Taylor, curvature-proxy, and low-rank projection errors. Within those assumptions, the theory is fine.\n\nThe soft spot is exactly where the stress-test note put it: Assumption K.4. The entire \"curvature-aware\" claim rests on ||I − bH^{-1}H|| < 1, where bH is a diagonal built from Adam second moments. The appendix argues by analogy (diagonal dominance, Fisher-style curvature) but never measures the relative error. For MF-BPR and LightGCN, the Hessian has off-diagonal blocks coupling user and item embeddings, and Adam's v-hat tracks gradient variance, not curvature. If K.4 fails, the downdate is a scaled gradient step and Theorem K.12's middle term is uncontrolled. That may still be fine in practice—LAC can fix a lot—but the claim \"curvature-aware\" is unsupported until they measure ε_H.\n\nThe empirical story is plausible but thinner than it looks. All deletions are injected low-scoring synthetic interactions; there are no real deletion requests, no seeds, no error bars, and no code/data. The Demotion Rate is the exact BPR objective LAC optimizes, so the completeness metric partially measures the method against its own training target. The abstract's \"up to 3× faster\" claim is inconsistent with their own Table 2, which shows order-of-magnitude speedups over full retraining but Obliviate consistently about 10% slower than RRL. These are addressable issues, not fatal ones. The baseline comparisons against retrained models are appropriate, and the component ablation supports the design.\n\nWho gets value: people designing practical unlearning for recommenders and anyone evaluating privacy-compliance claims. It deserves a serious referee, but the referee should ask for the K.4 measurement, varied deletion sets, error bars, code/data, and a reconciled speedup claim.","headline":"A practical two-stage unlearning recipe for recommenders, but the curvature-aware theory rests on an unmeasured assumption and the empirical story is single-run synthetic; worth refereeing with the right asks.","tokens_in":28652,"tokens_out":2984,"would_cite":true,"duration_ms":26282,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A two-stage unlearning pipeline removes deleted user–item interactions from trained recommenders while keeping recommendation quality near retrained levels, at a fraction of the retraining cost.","keywords":["machine unlearning","recommender systems","right to be forgotten","low-rank adapters","second-order optimization","influence functions","knowledge distillation","deletion set"],"falsifier":"Compute the actual relative preconditioner error ||I − bH^{-1}H||_2 for a small trained recommender (for example, MF-BPR on ML-1M) by forming the exact Hessian of the full-data objective; if the error is close to or greater than 1 on the subspace touched by the deletion set, the paper's distance-to-retrain bound is vacuous. Equivalently, compare Obliviate's final parameters with a truly retrained model's parameters on a real, non-synthetic deletion set: if the Euclidean distance grows much faster than the bound predicts, or if a membership-inference probe can still detect deleted interactions,","tokens_in":27650,"feed_emoji":"🪄","tokens_out":9082,"duration_ms":79574,"temperature":0.7,"pith_summary":"The paper sets out to show that the right to be forgotten can be honored in recommender systems without retraining from scratch. Its central claim is that removing a deletion set S from a trained model reduces to estimating the parameter shift θ*_{-S} ≈ θ0 − H^{-1} g_S, where g_S is the aggregate gradient of the deleted interactions and H is the local curvature. Obliviate operationalizes that shift in two stages: a Low-Rank Unlearning Adapter computes a Newton-style downdate using a cheap diagonal curvature proxy taken from Adam's second-moment statistics and projects it onto low-rank adapters; a Locality-Aware Calibration stage then refines only those adapters, demoting deleted interactions below negatives while distilling the original model's scores on a small retained buffer. If the central claim is right, large-scale platforms can process deletion requests in seconds rather than minutes or hours, with completeness and utility that closely track exact retraining.","feed_headline":"Unlearning erases deleted data in seconds, not retraining hours","feed_subtitle":"Curvature-aware low-rank downdate plus a quick calibration step matches retrained quality at a fraction of the cost.","key_machinery":"The load-bearing identity is Eq. (5): θ*_{-S} ≈ θ0 − H^{-1} g_S, the second-order unlearning approximation. It says the difference between a model trained on the full data and one retrained without the deletion set is, to first order in the parameter shift, the deletion gradient g_S scaled by the inverse Hessian H. The practical machinery is diagonal preconditioning: Obliviate reuses Adam's per-coordinate second-moment estimates to form bH = diag(sqrt(v_hat) + ε), turning the Newton step into per-coordinate scaling; a truncated SVD projects the resulting downdate onto a low-rank adapter, making the update modular and reversible; and the witness set S ∪ Neg(S) ∪ R feeds a BPR unlearning loss","core_discovery":"Obliviate's core discovery is that the optimization shift induced by deleting a set of interactions can be approximated as θ*_{-S} ≈ θ0 − H^{-1} g_S, so unlearning becomes a single second-order correction rather than a re-optimization. Because the full Hessian is intractable, the paper reuses the diagonal preconditioner already computed by Adam during training, bH = diag(sqrt(v_hat) + ε), to approximate H^{-1}; the downdate is then computed per-coordinate as −(g_S)_j / (sqrt(v_hat_j) + ε), applied only to the most affected user/item embedding blocks, and compressed into a rank-r adapter via truncated SVD. A second stage, Locality-Aware Calibration, freezes the backbone and optimizes only the","pith_inferences":["If Adam's second-moment surrogate is as accurate as assumed, the same mechanism could transfer to other models trained with Adam: any optimizer state that estimates per-coordinate gradient variance provides a free curvature proxy for unlearning. A direct measurement of the relative preconditioner error on small models would tell whether the transfer is safe.","Because the paper evaluates on synthetic deletions to avoid confounding, a natural next test is real user requests, where removing genuine preferences also shifts the evaluation distribution; completeness should still hold, but utility should be compared against a model retrained on the same deleted data rather than against the original model.","The BPR demotion objective certifies completeness in ranking terms, not in parameter or membership terms; a stronger privacy probe, such as membership inference or canary-parameter checking, could reveal residual traces that the Demotion Rate metric misses."],"forward_implications":["Deletion requests become cheap enough to honor repeatedly: unlearning runs in seconds (about 20 seconds on ML-1M with MF-BPR) versus hundreds of seconds for retraining, so frequent right-to-be-forgotten requests no longer require a retraining queue.","Because updates are confined to low-rank adapters on a frozen backbone, unlearning is modular and reversible, and repeated deletion requests could be composed by stacking or resetting adapters without touching the base model.","The method is demonstrated for deletion ratios as high as 20% of users and for both MF-BPR and graph-based LightGCN, suggesting it generalizes across common recommender architectures and sparse or dense datasets.","Unlearning completeness is visible in ranking behavior: after Obliviate, deleted interactions are demoted below sampled negatives at higher rates than prior approximate unlearning methods, close to what exact retraining would produce.","The synthetic-deletion evaluation protocol (injecting low-preference interactions as the deletion set) gives the community a way to measure unlearning quality without confounding it with distribution shift, making baseline comparisons cleaner."],"fun_headline_variants":["Forget deleted data in seconds, not retraining hours","Unlearning without retraining: a single curved step","Obliviate: erase user data fast, keep rec quality","Second-order unlearning: fast, accurate, without retraining","Delete data once, unlearn it cheaply: Obliviate"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything rests on the claim that Adam's second-moment statistics can stand in for the true curvature of the loss; if this diagonal surrogate is not accurate, the Newton-style unlearning update is just a scaled gradient step and the guarantee that the unlearned model matches retraining falls apart.","fun_headline_variants_meta":{"raw":{"variants":["Forget deleted data in seconds, not retraining hours","Unlearning without retraining: a single curved step","Obliviate: erase user data fast, keep rec quality","Second-order unlearning: fast, accurate, without retraining","Delete data once, unlearn it cheaply: Obliviate"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000704,"raw_usage":{"total_tokens":3030,"prompt_tokens":784,"completion_tokens":2246,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":2175}},"tokens_in":528,"tokens_out":2246,"duration_ms":15582,"temperature":1.0,"reasoning_tokens":2175,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T09:22:35.248084+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the actual relative preconditioner error ||I − bH^{-1}H||_2 for a small trained recommender (for example, MF-BPR on ML-1M) by forming the exact Hessian of the full-data objective; if the error is close to or greater than 1 on the subspace touched by the deletion set, the paper's distance-to-retrain bound is vacuous. Equivalently, compare Obliviate's final parameters with a truly retrained model's parameters on a real, non-synthetic deletion set: if the Euclidean distance grows much faster than the bound predicts, or if a membership-inference probe can still detect deleted interactions,","supporting_citations":[],"review_version":1}