Pith. sign in

REVIEW 2 major objections 1 minor 14 references

HAMU uses data similarity as a hardness measure to guarantee a specified forget quality gain in unlearning while minimizing retain utility degradation.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-28 15:24 UTC pith:2N5KCEZT

load-bearing objection HAMU adds a constrained-optimization framing and similarity-based hardness measure to multi-objective unlearning, but the claimed guarantees rest on derivations that need close checking. the 2 major comments →

arxiv 2606.02119 v1 pith:2N5KCEZT submitted 2026-06-01 cs.LG cs.AI

How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning

Jiangwei Chen , Xinyuan Niu , Rachael Hwee Ling Sim , Zhengyuan Liu , Nancy F. Chen , Bryan Kian Hsiang Low This is my paper
classification cs.LG cs.AI
keywords machine unlearningforget qualityretain utilityhardness measuremulti-objective optimizationconstrained optimizationdata similarity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Machine unlearning removes the influence of chosen forget data from a model for privacy or copyright reasons but must preserve performance on the rest of the data. Prior methods balance the two goals through loss weighting yet offer no guarantee that both can be improved by a chosen amount. This work frames the task as constrained optimization and shows that the difficulty of meeting both goals at once is captured by the similarity between forget and retain examples. From this measure it derives the HAMU algorithm that updates weights to reach the target forget quality at the smallest possible retain cost and signals when the two goals cannot be advanced together. The method works on non-convex models and runs in parallel.

Core claim

From a constrained optimization perspective, the hardness of reconciling forget quality and retain utility is quantified by the similarity between forget data and retain data. The HAMU algorithm is derived to guarantee a specified improvement in forget quality while minimizing retain utility degradation by updating model weights according to this hardness measure. The same measure tells users when retain degradation is unavoidable and stopping should be considered. The algorithm applies to non-convex models and is easily parallelizable.

What carries the argument

A hardness measure based on similarity between forget and retain data inside a constrained optimization formulation that enforces a minimum forget-quality improvement.

Load-bearing premise

The hardness of reconciling forget quality and retain utility can be quantified by the similarity between the forget data and the retain data.

What would settle it

An experiment in which the similarity between forget and retain sets fails to predict the actual retain-utility cost needed to reach a target forget-quality level, or in which HAMU does not achieve the stated forget-quality guarantee on held-out test data.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • HAMU guarantees a specified improvement in forget quality for given forget and retain data.
  • It minimizes the retain utility degradation that occurs while meeting that guarantee.
  • The algorithm applies directly to non-convex models such as deep neural networks.
  • It is easily parallelizable and therefore suitable for large-scale deployment.
  • It informs users when both objectives cannot be improved simultaneously so that unlearning can be halted.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The similarity-based hardness could be reused to rank which subsets of data are cheapest to unlearn first.
  • The same constrained view might apply to other multi-objective settings such as fairness-accuracy trade-offs during fine-tuning.
  • In production systems the early-stopping signal could be combined with differential-privacy budgets to decide when full retraining is preferable to unlearning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 1 minor

Summary. The paper proposes HAMU, a hardness-aware multi-objective unlearning algorithm derived from a constrained-optimization perspective. Hardness of reconciling forget-quality and retain-utility objectives is quantified by similarity between forget and retain data; the algorithm updates model weights to guarantee a user-specified improvement in forget quality while minimizing retain degradation, and supplies a stopping criterion when simultaneous improvement is impossible. The method is claimed to apply to non-convex models, to be parallelizable, and to outperform baselines empirically on image and text datasets with large models.

Significance. If the claimed derivation and guarantee can be established, the work would supply the first explicit constrained-optimization treatment of the forget/retain trade-off in unlearning, together with a practical hardness-based stopping rule. The public code link is a clear strength for reproducibility. At present, however, the absence of any visible equations, proof outline, or experimental controls prevents assessment of whether these advantages are realized.

major comments (2)
  1. [Abstract] Abstract: the central claim that HAMU is 'derived' from constrained optimization and 'guarantees a specified improvement in forget quality while minimizing the retain utility cost' is asserted without any equations, optimization formulation, or proof sketch. This derivation is load-bearing for the paper's contribution; its absence makes it impossible to verify whether the hardness measure (defined via forget/retain similarity) actually yields the stated guarantee or reduces to a fitted quantity.
  2. [Abstract] Abstract (first paragraph of contributions): the weakest assumption—that hardness of reconciling the two objectives 'can be quantified by the similarity between the forget data and the retain data'—is stated without a concrete definition, distance measure, or derivation showing how this similarity produces a non-circular guarantee. Because the guarantee is the paper's primary theoretical claim, this step must be made explicit and shown to be independent of the result it is meant to justify.
minor comments (1)
  1. [Abstract] The abstract states that the algorithm is 'easily parallelizable' and 'readily deployable,' yet provides no concrete description of the parallelization strategy or any complexity analysis.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive comments on the theoretical presentation of HAMU. We respond point-by-point to the major comments below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that HAMU is 'derived' from constrained optimization and 'guarantees a specified improvement in forget quality while minimizing the retain utility cost' is asserted without any equations, optimization formulation, or proof sketch. This derivation is load-bearing for the paper's contribution; its absence makes it impossible to verify whether the hardness measure (defined via forget/retain similarity) actually yields the stated guarantee or reduces to a fitted quantity.

    Authors: The referee correctly identifies that the abstract asserts the constrained-optimization derivation and guarantee without equations or a proof sketch. The submitted manuscript does not contain these elements. We will revise by adding an explicit optimization formulation, the hardness-based update rule, and a proof outline in the main text (new Section 3), and we will update the abstract to reference the key formulation. revision: yes

  2. Referee: [Abstract] Abstract (first paragraph of contributions): the weakest assumption—that hardness of reconciling the two objectives 'can be quantified by the similarity between the forget data and the retain data'—is stated without a concrete definition, distance measure, or derivation showing how this similarity produces a non-circular guarantee. Because the guarantee is the paper's primary theoretical claim, this step must be made explicit and shown to be independent of the result it is meant to justify.

    Authors: We agree that the similarity-based hardness measure requires a concrete definition and non-circular derivation to support the guarantee. The current manuscript states the assumption at a high level without these details. In revision we will define the measure explicitly (e.g., cosine similarity on penultimate-layer activations), derive its relationship to the constrained objectives, and show independence from the final guarantee. revision: yes

standing simulated objections not resolved
  • The submitted manuscript contains no equations, optimization formulation, or proof outline for the claimed constrained-optimization derivation.

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper presents HAMU as derived from a constrained-optimization formulation whose hardness measure is defined directly from forget/retain similarity; the abstract and contributions describe this as a first-principles identification followed by an algorithm that guarantees a specified forget-quality improvement at minimal retain cost. No equations, self-citations, or fitted parameters are shown that reduce the claimed guarantee or hardness measure back to a quantity defined by the same result. The derivation therefore remains self-contained against external benchmarks and does not exhibit any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review performed on abstract only; no explicit free parameters, additional axioms, or invented entities are described beyond the core domain assumption that similarity quantifies hardness.

axioms (1)
  • domain assumption The hardness of reconciling forget and retain objectives can be quantified by the similarity between the forget data and the retain data.
    Stated directly in the abstract as the foundation for the HAMU derivation.

pith-pipeline@v0.9.1-grok · 5792 in / 1144 out tokens · 33923 ms · 2026-06-28T15:24:23.303161+00:00 · methodology

0 comments
read the original abstract

Machine unlearning aims to remove the influence of specific forget training data due to privacy, copyright or bias concerns while maintaining the model performance on the remaining retain data. Existing unlearning algorithms, such as optimizing a weighted combination of losses, have tried to achieve these objectives of improving forget quality and maintaining retain utility. However, they do not guarantee that these objectives can be improved by a specified extent for all forget and retain data. In this work, we address this limitation with a novel and theoretically-grounded approach from a constrained optimization perspective. Firstly, we identify that the hardness of reconciling both objectives can be quantified by the similarity between the forget data and the retain data. Next, we derive an unlearning algorithm (HAMU) with the overall goal of guaranteeing a specified improvement in forget quality while minimizing the retain utility cost/degradation by updating the model weights based on our hardness measure. Our hardness measure also informs users when retain utility degradation is unavoidable, i.e., both objectives cannot be improved simultaneously, and stopping should be considered. Our algorithm is applicable to non-convex models and is easily parallelizable, making it readily deployable in real-world scenarios. We empirically demonstrate HAMU's superior performance over baselines on both image and text datasets using large models. Our code is available at https://github.com/aoi3142/HAMU.

Figures

Figures reproduced from arXiv: 2606.02119 by Bryan Kian Hsiang Low, Jiangwei Chen, Nancy F. Chen, Rachael Hwee Ling Sim, Xinyuan Niu, Zhengyuan Liu.

Figure 1
Figure 1. Figure 1: Hardness regimes characterized by the measure κ for fixed ∥gr∥ and ∥gf ∥. Higher κ corresponds to higher cost of forgetting in terms of retain utility degradation. The threshold κ1 and κ2, respectively, determines when the rectified update and unavoidable retain utility degradation (i.e., collateral forgetting) occurs to improve the forget quality by at least Q. • We adapt HAMU to efficiently and effective… view at source ↗
Figure 2
Figure 2. Figure 2: A 2D illustration of HAMU update. Problem (1) re￾stricts the weight update ∆w within a circle of radius R and the region where the forget quality constraint is satisfied. In (a), the retain gradient gr lies in the “easy” region (κ ≤ κ1), hence the direct update is applied, which is in the opposite direction of gr. In (b), the retain gradient lies outside the “easy” region (κ > κ1), the rectified update is … view at source ↗
Figure 3
Figure 3. Figure 3: (a) Estimated hardness κ¯ is strongly correlated with similarity mixing ratio ρ, with Pearson correlation coefficient of 0.994 and 0.986 for HAMU-Q and HAMU-U, respectively. (b, c) Higher ρ (harder) results in a lower increase in −∆Lr and ∆Lf . hardness, we vary the similarity between the forget and re￾tain data using the similarity mixing ratio ρ. Specifically, we first choose a random class c which conta… view at source ↗
Figure 5
Figure 5. Figure 5: HAMU-Q enforces a positive ∆Lf while HAMU-U enforces a positive −∆Lr. line (e.g., FT, SCRUB for ρ = 0.75) indicates that model performance improves on both the forget and retain data, suggesting poor forget quality. An ideal unlearning al￾gorithm should achieve positive ∆Lf and −∆Lr, i.e., its unlearning trajectory should trend toward the top-right re￾gion. In the easy scenario (ρ = 0 with dissimilar forge… view at source ↗
Figure 6
Figure 6. Figure 6: Unlearning trajectories of HAMU-Q and HAMU-U for different values of Q and U when ρ = 0. 0 5 10 15 20 Epoch ( ) 0.0 0.1 0.2 Lr ( ) GA GDiff SCRUB HAMU-Q HAMU-U 1.0e-5 3.0e-5 5.0e-5 7.0e-5 9.0e-5 Q,U Value [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Plot of improvement in retain utility −∆Lr against number of epochs required to reach a specified threshold for Lf . Methods that failed to reach this threshold have been excluded. ρ), −∆Lr decreases for HAMU-Q and ∆Lf decreases for HAMU-U. When ρ = 1, unlearning becomes so hard that even HAMU-Q results in collateral forgetting and HAMU￾U fails to unlearn the forget data. Note that the stopping criterion w… view at source ↗
Figure 11
Figure 11. Figure 11: Comparison of HAMU-U unlearning trajectories for different learning rates η, with the same U = 0.6η, under the semantically dissimilar LLM experimental setup. is easier to unlearn the forget data that is less similar to the retain data [PITH_FULL_IMAGE:figures/full_fig_p009_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Comparing unlearning trajectory when using HAMU-U (Q = 0.9η) with vanilla SGD versus AdamW, for 5 unlearning epochs. Note that the axes are plotted in log scale. A larger increase in ∆Lf (and −∆Lr) is achieved with the same number of unlearning epochs for AdamW than vanilla SGD. moving average of κ/κ2, or only stop when the stopping criterion is met for a certain number of times. To evaluate the effect of… view at source ↗
Figure 13
Figure 13. Figure 13: More comprehensive plot of [PITH_FULL_IMAGE:figures/full_fig_p028_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

  1. [1]

    Duan, H., Yang, Y ., Abbasi, A., and Tam, K. Y . Ready2Unlearn: A learning-time approach for preparing models with future unlearning readiness. arXiv:2505.10845,

    work page arXiv

  2. [2]

    Eldan, M

    Eldan, R. and Russinovich, M. Who’s Harry Potter? approximate unlearning in LLMs. arXiv:2310.02238,

    work page arXiv

  3. [3]

    Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E

    URLhttps://www.cs.toronto.edu/ ˜kriz/learning-features-2009-TR.pdf. Kurmanji, M., Triantafillou, P., Hayes, J., and Triantafillou, E. Towards unbounded machine unlearning. InProc. NeurIPS,

    2009

  4. [4]

    Towards understanding the feasibility of machine unlearning

    Sarvmaili, M., Sajjad, H., and Wu, G. Towards understanding the feasibility of machine unlearning. arXiv:2410.03043,

    work page arXiv

  5. [5]

    12 How Hard Can It Be? Hardness-Aware Multi-Objective Unlearning A. Related Work One common objective of machine unlearning is to approximate the retrained model (Georgiev et al., 2025; Nguyen et al., 2022), which is equivalent to minimizing the retain loss (Neel et al., 2021; Bourtoule et al., 2021). Another prevalent objective is to increase the forget ...

    2025

  6. [6]

    One straightforward approach to multi-objective unlearning is to directly optimize a weighted combination of the loss func- tions (Kurmanji et al., 2023; Duan et al., 2025)

    aims to optimize both objectives simultaneously. One straightforward approach to multi-objective unlearning is to directly optimize a weighted combination of the loss func- tions (Kurmanji et al., 2023; Duan et al., 2025). Other approaches such as GDR-GMA (Lin et al.,

    2023

  7. [7]

    Unlearning methods with theoretical guarantees (Guo et al., 2020; Ullah et al.,

    and enable efficient application to large-scale, non-convex models. Unlearning methods with theoretical guarantees (Guo et al., 2020; Ullah et al.,

    2020

  8. [8]

    Existing works relevant to unlearning hardness (Zhao et al., 2024; Sarvmaili et al.,

    typically require strong assumptions, such as strongly convex loss functions and the existence of a unique minimizer (Wu et al., 2020; Allouah et al., 2025), limiting their practicality. Existing works relevant to unlearning hardness (Zhao et al., 2024; Sarvmaili et al.,

    2020

  9. [9]

    are largely heuristic-based and do not establish a formal connection between the proposed hardness measures and unlearning performance. B. Proofs and Derivations B.1. First-Order Loss Approximation For anyz r = (xr, yr)∈D r, applying Taylor expansion toL t+1(zr), we have Lt+1(zr) =L t(zr) + ∆w⊤∇Lt(zr) + 1 2∆w⊤H∆w | {z } E , whereEis the approximation erro...

    2018

  10. [10]

    We used a fixed learning rate of 1e−3with the AdamW optimizer (Loshchilov & Hutter, 2019), batch size of5000and trained for50epochs

    with randomly initialized weights. We used a fixed learning rate of 1e−3with the AdamW optimizer (Loshchilov & Hutter, 2019), batch size of5000and trained for50epochs. Every 5 epochs, we performed evaluation on the CIFAR-10 test set, which consists of10000samples across the10classes, and saved the checkpoint model weights. The final trained model was sele...

    2019

  11. [11]

    No duplicate

    by constructing different variations of the QA pairs of different levels of similarity, watermarked with the text watermarking scheme Waterfall (Lau et al., 2024). This dataset contains different splits, of which we used the watermarked “No duplicate” and “Semantic duplicate” splits for10% of the forget data. We refer to the settings using these two split...

    2024

  12. [12]

    for a multi-GPU setup. During the data preparation step, we interleave the forget and retain samples into the unlearning dataset according to the batch size of each GPU, such that the dataloader would send forget samples to one half of the GPUs and retain sample to the other GPUs. During the backward pass, instead of simply aggregating the gradients from ...

    2019

  13. [13]

    When the gradients do not change significantly between iterations, the step size taken by each weight update gradually increases

    speed up the optimization process by incorporating the weight updates in previous iterations to the update in the current iteration. When the gradients do not change significantly between iterations, the step size taken by each weight update gradually increases. This speeds up the unlearning process, and the model is able to achieve a similar or better fo...

    2048

  14. [14]

    We also include two additional baselines, Gradient Rectified Unlearn- ing (GRU) (Wang et al., 2025b) and Projecting Conflicting Gradients (PCGrad) (Yu et al., 2020)

    metrics. We also include two additional baselines, Gradient Rectified Unlearn- ing (GRU) (Wang et al., 2025b) and Projecting Conflicting Gradients (PCGrad) (Yu et al., 2020). PCGrad as described in Yu et al. (2020) performs multi-task optimization and was not initially designed to be used in unlearning. We adapt PCGrad’s update rule to make use of the gra...

    2020