{"id":"1989196e-9fef-412f-9c27-105775990a27","arxiv_id":"2502.05684","paper_version":5,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper introduces an auditable, information-theoretic 'marginal unlearning' definition and a rate-distortion style regularization method that can remove data points or features from models.","lead":"This paper proposes a new definition of machine unlearning, called marginal unlearning, where a model is judged by whether its outputs reveal whether a record was in the training set, rather than by comparing against a retrained model. It presents an information-theoretic regularization framework that can be trained to satisfy this definition, along with theory, algorithms, and experiments.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.3's equivalence claim fails: Lemma 4.3 only gives non-strict monotonicity, so when Y is independent of (X,Z), every admissible output maximizes the mutual-information utility, making the W2 barycenter a non-unique 'analytic solution'.","rationale":"The reader's weakest assumption correctly identifies the gap in Theorem 4.3. I considered the other flaws noted in the paper: the KL expectation bound in Theorem 4.1 is indeed false (E[KL] = I, not ≤ sqrt(I)), but it is not used in the main sufficiency or algorithmic results. The proof of necessity Lemma 2.2 appears invalid because the lower bound on the Wasserstein distance in terms of a pointwise density difference is unjustified, but the lemma's statement may be salvageable under extra regularity and it plays a smaller role in the paper's constructive contributions. Theorem 4.3, by contrast, is the foundation of the advertised 'unified analytic solution' and is explicitly presented as an equivalence; a simple counterexample shows the equivalence is false. This is load-bearing because the paper's headline contribution includes a single analytic solution that provably maximizes all listed utilities. Without the 'only if' direction, the barycenter is still an optimal solution, but the uniqueness claim collapses. The practical algorithms and experiments may still be valid, but the theoretical overclaim as stated cannot stand, supporting the reader's rejection.","tokens_in":38681,"tokens_out":15733,"duration_ms":151798,"concrete_test":"Take X~U[0,1], Z~Bernoulli(1/2) with mu0=U[0,1], mu1=U[1,2], and let Y be any variable independent of (X,Z), e.g., Y≡0. Compute the W2 barycenter Xbar of mu0 and mu1 (the geodesic midpoint). Observe that the constant outcome Xhat≡0 is admissible (f(X,Z)=0 is measurable) and satisfies Xhat⊥Z, so I(Y;Xhat)=0; also I(Y;Xbar)=0. Both are maximizers of the mutual-information utility in Theorem 4.3, yet sigma(Xhat) ≠ sigma(Xbar). This directly refutes the claimed equivalence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of a unified analytic solution rests on Theorem 4.3, which asserts that for any admissible Xhat=f(X,Z), Xhat maximizes each listed utility under Xhat⊥Z iff sigma(Xhat)=sigma(Xbar), where Xbar is the W2 barycenter of {X|Z=z}. The proof invokes Lemma 4.3, which shows only non-decreasing monotonicity: sigma(X1) subset sigma(X2) implies H(X1) ≤ H(X2), I(Y;X1) ≤ I(Y;X2), etc. Non-decreasing monotonicity cannot support the 'only if' direction: a coarser outcome can achieve the same utility value. Indeed, if Y is independent of (X,Z), then I(Y;Xhat)=0 for every admissible Xhat, so any such Xhat is a maximizer, including a constant output with trivial sigma-algebra, which is strictly coarser than sigma(Xbar) in general. Thus the asserted equivalence is false, and the paper's phrasing that the barycenter is 'the' analytic solution is an overclaim; it is merely one optimal solution. The practical algorithm is not invalidated, but the theoretical uniqueness claim that supports the headline 'unified analytic solution' is not established and is contradicted by this counterexample.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified information-theoretic framework for machine unlearning, covering both feature unlearning and data-point unlearning. For data-point unlearning it introduces the \"Marginal Unlearning Principle,\" formalized through a posterior-odds condition (Definition 2.3) and a mutual-information relaxation (Definition 2.4), and claims that this principle is sufficient for anchored approximate unlearning under utility control (Theorem 2.1) and necessary for any well-regularized retrained model (Lemma 2.2). For feature unlearning, the paper proves MI-to-KL/TV bounds (Theorem 4.1 and Proposition 4.1), derives an auditable odds-inference guarantee (Lemma 4.1 and Theorem 4.2), and claims that the Wasserstein-2 barycenter of the conditional distributions provides a unified analytic optimal solution for several information-theoretic utilities (Theorem 4.3). The paper closes with regularization algorithms and experiments on tabular benchmarks, MNIST, and CelebA.","tokens_in":38963,"tokens_out":5555,"duration_ms":55367,"significance":"The paper's framing of auditable, output-level marginal unlearning is a potentially valuable contribution, and the practical regularization algorithms are clearly presented with code and experiments. The MI-to-TV and MI-to-odds conversions in Lemma 2.1, Proposition 4.1, and Lemma 4.1 are mostly standard and correct, and the sufficiency direction (Theorem 2.1) is a useful bridge between the proposed auditable criterion and anchored unlearning. However, three load-bearing theoretical claims are not established: the KL expectation bound in Theorem 4.1 is false as stated, the Wasserstein lower bound in Lemma 2.2 is invalid, and the equivalence in Theorem 4.3 fails under non-strict monotonicity. These errors directly affect the paper's stated contributions on the necessity of marginal unlearning and on the uniqueness of the barycenter as the analytic unlearning solution. The framework may remain useful after substantial correction, but the current theoretical guarantees are overstated.","major_comments":[{"comment":"Theorem 4.1, Eq. (17), claims E_Z[D_KL(P_{\\hat X|Z} || P_{\\hat X})] ≤ sqrt(I(\\hat X;Z)). This is false because the identity I(\\hat X;Z) = E_Z[D_KL(P_{\\hat X|Z} || P_{\\hat X})] holds by definition, so the displayed bound would require I ≤ sqrt(I), which fails whenever I > 1. The KL expectation bound should either be removed or corrected to the exact identity; the useful TV bounds in parts 3 and 4 require a separate proof and are not rescued by part 1.","section":"§4.1, Eq. (17)"},{"comment":"The proof of Lemma 2.2 uses an invalid Wasserstein lower bound. The step W_dY(f(X_0), f(X_1)) > δ ∫_{B_δ(y*)} |f(X_0)-f(X_1)|(y) dy ≥ δ|f(X_0)-f(X_1)|(y*) is not a valid lower bound for the 1-Wasserstein distance; Wasserstein distance is a transport metric and is not controlled pointwise by values of densities. Consequently the claimed threshold L*(ε) and the three-way incompatibility conclusion are not established. The expression for L*(ε) also divides by W_dX(X_1, X_0), which can be zero when the laws of X_1 and X_0 coincide even if the random variables differ, so the formula is not well-posed in general. Proposition 2.1, which relies on this lemma, is therefore unsupported.","section":"§2.4 and Appendix B.3, Lemma 2.2"},{"comment":"The equivalence in Theorem 4.3 is false as stated. Lemma 4.3 establishes only non-decreasing monotonicity of the utility functionals with respect to sigma-algebra inclusion, and non-decreasing monotonicity cannot support the \"only if\" direction. A concrete counterexample occurs when Y is independent of (X,Z): then I(Y;\\hat X)=0 for every admissible \\hat X, so any admissible output, including a constant output with trivial sigma-algebra, is a maximizer of the mutual-information utility, while σ(constant) is strictly coarser than σ(\\bar X) in general. Thus the assertion that a maximizer must satisfy σ(\\hat X)=σ(\\bar X) fails. The paper can legitimately claim that the barycenter is an optimal solution, but not that it is the unique optimal solution; Section 5.3 and Algorithm 3 inherit this overclaim.","section":"§4.3, Theorem 4.3 and Lemma 4.3"}],"minor_comments":[{"comment":"Algorithm 1 computes S_d = f_θ(X_d), but the model was defined as f: X × Z → S; the algorithm should either include Z in the input or clarify that Z is already encoded in X_d.","section":"§5.1, Algorithm 1, line 8"},{"comment":"The pseudocode says to compute the optimal transport map T_z that maps \\bar X to X_z, while the surrounding text describes Brenier maps from X_z to the barycenter; the direction of the transport maps should be made consistent.","section":"§5.3, Algorithm 3, steps 4-6"},{"comment":"The symbol ε is used both for the allowed difference in posterior probabilities and for the log-odds threshold in Theorem 4.2; this notational overlap makes the statements harder to parse and should be disambiguated.","section":"§4.2, Lemma 4.1 and Theorem 4.2"},{"comment":"The assumption \"P_{\\hat X} ≪ L\" introduces L without defining it in the theorem statement; clarify that L denotes Lebesgue measure on the relevant Euclidean space.","section":"§4.1, Theorem 4.1"}],"recommendation":"major_revision","confidential_remarks":"The central Lemma 4.2 is cited to the authors' own prior JMLR paper [71]; this is not circularity, but the current paper should either provide a self-contained proof or state the precise regularity conditions, since the sketch in Appendix D.3 glosses over absolute-continuity and invertibility assumptions. The three technical errors above are the main grounds for the revision decision; the practical and experimental parts may still be salvageable after the theoretical claims are corrected and appropriately weakened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Marginal unlearning is a real idea, but the paper's headline theory overstates it: the KL expectation bound in Theorem 4.1 is false as written, the necessity lemma proof doesn't go through, and Theorem 4.3's equivalence breaks when the target Y is independent of (X,Z). The stress-test note is right about the degeneracy counterexample.\n\nThe genuinely new thing here is the marginal unlearning principle: instead of comparing against an unknown retrain-on-retain anchor, you require the output to be uninformative about whether the unlearn set was included. That is auditable and practically meaningful. The algorithms and experiments (tabular, MNIST, CelebA) are mostly honest — the authors don't claim to beat SOTA, just show the regularizer behaves. The rate-distortion reinterpretation is also a nice framing.\n\nThe soft spots are the load-bearing theorems. Theorem 4.1 says E_Z[D_KL] <= sqrt(I). That's false when I > 1; actually E[KL] = I, so the inequality reverses. It's an easy fix—the exact identity is stronger—but as stated it's wrong, and it's used in Remark 4.1 to propagate guarantees. Lemma 2.2's necessity proof uses a weird local Wasserstein lower bound that I doubt; the claim may be salvageable, but the proof needs real work. The bigger problem is Theorem 4.3. The paper claims the W2 barycenter is 'the' analytic solution for all those utilities. Lemma 4.3 gives monotonicity of the objectives with respect to sigma-algebra inclusion, but that's non-strict. If Y is independent of (X,Z), then I(Y;Xhat)=0 for every admissible Xhat, so every outcome is optimal—including a constant one. The barycenter is still a finest sigma-algebra, so it's a maximal element, but it is not the unique maximizer. Calling it 'the' solution is an overclaim, and the equivalence direction sigma(Xhat)=sigma(Xbar) iff maximizer collapses.\n\nNone of this kills the practical method. The regularization still makes sense as a heuristic, and the empirical sections stand on their own. But the paper sells itself as provable guarantees and a unified analytic solution; those are the parts that fail. A referee should ask the authors to fix the KL bound, repair or drop the necessity lemma, and replace Theorem 4.3 with a statement about maximality rather than uniqueness.\n\nThis is worth sending to a serious referee. The marginal unlearning definition is a contribution the field should engage with, even if this version's theory is shaky. I'd expect major revision, not a desk reject.","headline":"Marginal unlearning is a genuinely useful idea, but the paper's central theoretical guarantees don't hold as stated — the analytic solution is overclaimed and two proofs are flawed.","tokens_in":39491,"tokens_out":3529,"would_cite":false,"duration_ms":32838,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","68Q32","62B10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that machine unlearning can be made auditable by replacing the unobservable 'retrain from scratch' anchor with a mutual-information criterion on model outputs, and that for feature unlearning the optimal solution is the…","keywords":["machine unlearning","marginal unlearning","feature unlearning","data deletion","information-theoretic regularization","mutual information","Wasserstein barycenter","optimal transport"],"falsifier":"Construct Gaussian conditionals $\\{X \\mid Z=z\\}$ with a target $Y$ independent of $(X,Z)$; then every admissible $\\hat{X} \\perp Z$ achieves $I(Y;\\hat{X})=0$, so the barycenter is not the unique maximizer and the claimed equivalence in Theorem 4.3 fails unless strict monotonicity is added, a discrepancy checkable by direct calculation.","tokens_in":38465,"feed_emoji":"🔒","tokens_out":13980,"duration_ms":121160,"temperature":0.7,"pith_summary":"The paper tries to establish that unlearning can be defined and certified by what an observer can still infer from the released model's outputs, rather than by comparison to a model retrained from scratch in secret. It introduces 'marginal unlearning,' which requires the output distribution to be nearly unchanged when the to-be-forgotten records or attributes are added to the training pool, measured by a mutual information bound. It proves that this criterion, together with a utility guarantee, is sufficient to recover the standard retrain-based unlearning guarantee, and that a well-regularized retrained model cannot satisfy it while leaking the forgotten records. It then shows that for feature unlearning under several common utility functions, one analytic object solves the problem: the Wasserstein-2 barycenter of the feature-conditional data distributions. If the argument is right, unlearning becomes something an auditor can check from samples of inputs and outputs, without trusting a private retraining pipeline.","feed_headline":"Marginal unlearning makes data deletion verifiable","feed_subtitle":"A single mutual-information bound suffices for retrain-level unlearning; one barycenter solves optimal feature removal","key_machinery":"The carrying mechanism is the mutual-information regularizer $I(S'; Z)$ inside a rate-distortion-style objective $\\inf_f (1-\\lambda) C(Y; S) + \\lambda I(S'; Z)$, where $Z$ is the inclusion indicator or the feature to forget and $S'$ is the output variable whose leakage is being suppressed. For data-point unlearning the paper constructs the paired variable $(X_{\\mathrm{margin}}, Z)$ with $X_{\\mathrm{margin}} \\mid Z=1 \\sim p_r$ and $X_{\\mathrm{margin}} \\mid Z=0 \\sim p_d$, so that $I(\\hat{X}_{\\mathrm{margin}}; Z)$ directly measures how distinguishable the model finds the retained data from the retain-plus-unlearn mixture. The analytic result rests on the Wasserstein-2 barycenter of the conditional laws $\\{X \\mid Z=z\\}$, which by Lemma 4.2 generates the finest $\\sigma$-algebra among all admissible outcomes $\\hat{X}=f(X,Z)$ with $\\hat{X} \\perp Z$, together with Lemma 4.3, which shows entropy, mutual information with any target, and conditional-probability energy all increase with $\\sigma$-algebra fineness.","core_discovery":"The central discovery is that exact or approximate unlearning can be reduced to an observable statistical independence condition: for data-point unlearning, require $I(\\hat{Y}_{\\mathrm{margin}}; Z) \\le \\varepsilon$, where $Z$ indicates whether the unlearn set was included in training and $\\hat{Y}_{\\mathrm{margin}}$ is the model output under the retain-only versus retain-plus-unlearn mixture; for feature unlearning, require $I(\\hat{X}; Z) \\le \\varepsilon$ for the feature to forget. The paper proves that this condition is sufficient for the classical anchor-based guarantee when the model keeps log-loss regret small (Theorem 2.1), and that no Lipschitz-regular retrained model can simultaneously satisfy the marginal condition and reveal the unlearned record (Lemma 2.2). For feature unlearning with exact independence, Theorem 4.3 states that the Wasserstein-2 barycenter of $\\{X \\mid Z=z\\}$ is the unique maximizer of entropy, of mutual information with every target $Y$, of posterior concentration, and of conditional-probability energy among all admissible outputs; the proof mechanism is that the barycenter generates the finest admissible $\\sigma$-algebra (Lemma 4.2) and all listed utilities are monotone in $\\sigma$-algebra fineness (Lemma 4.3).","pith_inferences":["A natural extension the paper leaves implicit is replacing mutual information with higher-order divergence measures such as Rényi divergence in Definition 2.4, which could yield tighter finite-sample certificates for large language models where estimating mutual information is difficult.","The 'anchor generalization' term in Theorem 2.1 means the sufficiency result degrades when the retrained anchor itself is distributionally shifted from the retain distribution; this suggests pairing marginal unlearning with distributionally robust training on the retain set.","The strict-monotonicity gap in Theorem 4.3 — the paper proves non-decreasing, not strictly increasing, sigma-algebra monotonicity — implies the barycenter may share optimality with coarser outputs when the target $Y$ is independent of $(X,Z)$; requiring a strictly proper scoring rule would close that gap.","The framework's auditable formulation could be turned into an interactive test: an auditor repeatedly samples from the retain-only and retain-plus-unlearn pools and performs a likelihood-ratio test on the outputs, with the posterior-odds bound of Definition 2.3 giving a per-event significance threshold."],"forward_implications":["An auditor could certify deletion using only samples of inputs and outputs, computing a mutual information estimate and checking the utility loss, with no access to the original training pipeline or retrained anchor.","Practitioners can choose the regularization weight $\\lambda$ in Algorithm 2 to drive $I(\\hat{X}_{\\mathrm{margin}}; Z)$ below a target value and then invoke Theorems 4.2 and 2.1 to issue a concrete high-probability deletion certificate.","For feature unlearning with the listed utilities, a single barycenter computation replaces task-specific adversarial or representation training, and the two-marginal case reduces to midpoint displacement interpolation along an optimal transport map.","Lemma 2.2 implies that enforcing marginal unlearning concentrates unavoidable utility loss on the forgotten records themselves, so the retain-set performance is protected rather than degraded globally.","Because the definition is output-based, it applies unchanged to generative models, self-supervised representations, and downstream post-processing, since the data-processing inequality propagates the mutual information bound."],"supporting_citations":[{"why":"Argues that anchor-based unlearning definitions are not auditable, motivating the paper's output-based marginal criterion.","marker":"[63]"},{"why":"Defines exact machine unlearning by equality with the retrain-on-retain anchor, the baseline the paper's sufficiency theorem targets.","marker":"[12]"},{"why":"Formalizes the retrain-on-retain anchor and deletion pipelines that marginal unlearning is designed to approximate.","marker":"[11]"},{"why":"Supplies the divergence-based approximate unlearning relaxation that Theorem 2.1 shows marginal unlearning implies.","marker":"[31]"},{"why":"Provides Lemma 5.2, used as Lemma 4.2, showing the Wasserstein-2 barycenter generates the finest sigma-algebra among admissible outcomes.","marker":"[71]"},{"why":"Defines Wasserstein barycenters and the fixed-point iteration that Algorithm 3 uses to converge to the true barycenter.","marker":"[2]"},{"why":"Gives displacement interpolation, which yields the midpoint barycenter solution in the two-marginal feature unlearning case.","marker":"[50]"},{"why":"Supplies the Fano inequality and data-processing inequality that justify bounding mutual information to control inference of $Z$ from outputs.","marker":"[18]"},{"why":"Provides the optimal transport background and Wasserstein distance foundations for the analytic solution and guarantees.","marker":"[65]"}],"fun_headline_variants":["Unlearn by bounding mutual information","Optimal feature unlearning via one barycenter","Marginal unlearning: auditable and provable","From retraining to barycenter: new unlearning theorem","Independence condition suffices for exact unlearning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that near-zero mutual information between model outputs and the to-be-forgotten signal is the correct and sufficient characterization of unlearning, and Theorem 4.3 additionally assumes the utility measures are strictly monotone in sigma-algebra fineness, whereas the paper proves only non-strict monotonicity.","fun_headline_variants_meta":{"raw":{"variants":["Unlearn by bounding mutual information","Optimal feature unlearning via one barycenter","Marginal unlearning: auditable and provable","From retraining to barycenter: new unlearning theorem","Independence condition suffices for exact unlearning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000301,"raw_usage":{"total_tokens":1781,"prompt_tokens":1034,"completion_tokens":747,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":676}},"tokens_in":650,"tokens_out":747,"duration_ms":7928,"temperature":1.0,"reasoning_tokens":676,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:24:10.855582+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct Gaussian conditionals $\\{X \\mid Z=z\\}$ with a target $Y$ independent of $(X,Z)$; then every admissible $\\hat{X} \\perp Z$ achieves $I(Y;\\hat{X})=0$, so the barycenter is not the unique maximizer and the claimed equivalence in Theorem 4.3 fails unless strict monotonicity is added, a discrepancy checkable by direct calculation.","supporting_citations":[{"cited_title":"Thudi, H","cited_arxiv_id":null,"evidence_quote":"Argues that anchor-based unlearning definitions are not auditable, motivating the paper's output-based marginal criterion."},{"cited_title":"Cao and J","cited_arxiv_id":null,"evidence_quote":"Defines exact machine unlearning by equality with the retrain-on-retain anchor, the baseline the paper's sufficiency theorem targets."},{"cited_title":"Bourtoule, V","cited_arxiv_id":null,"evidence_quote":"Formalizes the retrain-on-retain anchor and deletion pipelines that marginal unlearning is designed to approximate."},{"cited_title":"Xu and T","cited_arxiv_id":null,"evidence_quote":"Provides Lemma 5.2, used as Lemma 4.2, showing the Wasserstein-2 barycenter generates the finest sigma-algebra among admissible outcomes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives displacement interpolation, which yields the midpoint barycenter solution in the two-marginal feature unlearning case."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Fano inequality and data-processing inequality that justify bounding mutual information to control inference of $Z$ from outputs."},{"cited_title":"Villani.Topics in optimal transportation, volume 58","cited_arxiv_id":null,"evidence_quote":"Provides the optimal transport background and Wasserstein distance foundations for the analytic solution and guarantees."}],"review_version":1}