Pith. sign in

REVIEW 3 major objections 6 minor 63 references

Learning-Time Encoding Shapes Unlearning in LLMs

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that how a fact is written during fine-tuning—one description, several paraphrases, or a shared text chunk—determines how effectively that fact can later be unlearned from the model.

desk verdict A useful study of how chunk structure affects unlearning, but the paraphrasing result is confounded by 3x training data and, for UL-Exact, 3x forget data. read the letter →

arxiv 2506.15076 v1 pith:36OPNEWP submitted 2025-06-18 cs.CL cs.LG

Ruihan Wu , Konstantin Garov , Kamalika Chaudhuri This is my paper · ORCID
classification cs.CLcs.LG
keywords machineunlearninglargelanguagemodelsknowledgeencodingparaphrasingfine-tuningunlearn-retaintrade-offtextchunksfactual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the way a fact is written during fine-tuning changes how hard it is to erase that fact later, independently of the unlearning algorithm. Using synthetic biography datasets where the same knowledge can be written as a single description, several paraphrases, or a multi-fact chunk, it finds that training with multiple paraphrased descriptions produces consistently better unlearn–retain trade-offs than training with a single description. It also finds that a fact sharing a text chunk with facts that must be kept is hard to remove selectively: forget and retain scores fall at nearly the same rate, and unlearning improves when the forget set matches chunk boundaries or when facts are written as separate sentences inside the chunk. If the findings hold, choosing how to write data at training time becomes a practical lever for later deletion requests, and unlearning benchmarks that fix the trained model ignore half of the problem.

What carries the argument

The carrying mechanism is a controlled fine-tuning testbed built by extending two unlearning datasets (Eval-DU+ and TOFU+) with fictitious subjects, so the same atomic facts can be encoded as a single description, as three paraphrases, or inside multi-fact chunks. The comparison instrument is the unlearn–retain trade-off curve: for two unlearning algorithms (gradient ascent and task vector) and three choices of unlearning text (the exact training text, one unseen paraphrase, or several unseen paraphrases), checkpoints of increasing unlearning strength are scored by token-level probability on the forget set and the retain set. Norm-AUC is the summary statistic: it normalizes scores by the fine-tuned model's starting values and computes the area under the curve, with 0.5 indicating that forget and retain are falling at equal rates. The second half of the argument relies on two constructed interventions—a forget split aligned to chunk boundaries and an isolated-chunk variant built by concatenating sentence-level descriptions—to show that entanglement, not chunk size or model family, is what blocks selective unlearning.

What would settle it

Train two additional models on the same knowledge set: one with each fact written once, and one with the identical single description repeated three times per fact. If the repeated model shows the same Norm-AUC advantage over the single model that FT-Mul shows, then paraphrase diversity is not the cause of the reported effect. For the chunk result, a version that shuffles the same facts across chunks while preserving boundary sizes would test whether content alignment or chunk size drives the entanglement finding.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that learning-time textual encoding is a first-order determinant of post-hoc factual unlearning in fine-tuned LLMs. Empirically, fine-tuning each knowledge piece with three paraphrased descriptions (FT-Mul) matches or beats single-description training (FT-Single) in 44 of 48 extraction-trade-off comparisons and 39 of 48 memorization-trade-off comparisons across three model families and two datasets, while also making the model less able to answer unseen paraphrased prompts. When the same knowledge is embedded in multi-fact chunks, unlearning an individual fact becomes markedly harder: Norm-AUC values hover near 0.5 on the entangled dataset, meaning unlearning removes target and retained knowledge at the same rate. Two interventions confirm the mechanism the paper proposes—aligning the unlearn split with chunk boundaries and isolating each fact as its own sentence both raise Norm-AUC consistently. The paper extracts two training-time strategies from these results: paraphrase each fact during fine-tuning, and structure data so likely unlearn targets are separated from retained content.

Load-bearing premise

The claim that paraphrasing helps unlearning rests on comparing training with one description of each fact to training with several paraphrases; but the paraphrased condition also uses three times as many example sentences per fact, so the observed gain could be caused by repetition or data quantity rather than by variation in wording.

Editorial extensions

If this is right

  • If a deployment expects deletion requests, adding paraphrases of each sensitive fact to the fine-tuning set will make later gradient-ascent or task-vector unlearning more effective for the same retention of unrelated knowledge.
  • Training chunks that weave forget-prone and retained facts together are a liability: single-fact erasure will degrade both, so separating those facts into standalone text is the safer layout.
  • Choosing the to-forget set to align with natural chunk boundaries in the training data converts a near-0.5 Norm-AUC unlearning run into a measurably more selective one.
  • Unlearning algorithm comparisons are only meaningful relative to a training encoding; a benchmark that fixes the trained model cannot observe the differences this paper reports.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The cleanest challenge to the first finding is a repetition confound: FT-Mul differs from FT-Single in paraphrase variety and in having three training examples per fact. A condition that repeats the identical description three times would tell whether diversity or sheer count drives the unlearning gain.
  • If the effect extends to pretraining, corpus-level surface-form diversity and duplication become compliance-relevant data-curation choices, not just memorization correlates; that is a much larger claim than the fine-tuning evidence in this paper supports.
  • The chunk-entanglement result suggests a concrete mechanism worth testing directly: gradient-ascent and task-vector updates may attenuate shared parameter directions or overlapping representations, so measuring gradient overlap between forget and retain facts in the same chunk could predict unlearning failure.
  • The separation strategy could be automated with a detector that marks likely deletion targets at curation time, but the paper only proposes the strategy, not the detector.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies whether the way knowledge is textually encoded during fine-tuning affects the later efficacy of post-hoc unlearning. The authors construct augmented versions of Eval-DU and TOFU, fine-tune Llama2-7B, Llama3-8B, and Gemma2-2B on three encodings (FT-Single, FT-Mul, FT-Mul-Chunk), and then apply gradient ascent and task-vector unlearning with three choices of unlearning data (UL-Exact, UL-Single, UL-Mul). They report two main findings: (1) fine-tuning with multiple paraphrased descriptions (FT-Mul) improves unlearning effectiveness relative to a single description, and (2) unlearning individual knowledge embedded in a chunk that also contains retained knowledge is significantly harder, with supporting hypotheses that chunk-aligned unlearn splits and isolated knowledge descriptions make unlearning easier. The paper proposes two practical data-design strategies, paraphrasing and separating, and releases code.

Significance. If the claims are valid, the paper would be a useful early contribution showing that training-time textual encoding is a controllable factor in post-hoc unlearning, with concrete implications for privacy and data curation. The strengths of the design are real: the use of fictitious knowledge bases controls the underlying facts; the experiments span three model families, two datasets, two unlearning algorithms, and three unlearning-data constructs; the evaluation includes held-out paraphrases; the chunk-entanglement hypotheses are tested with internal controls (aligned splits and isolated sentences); and the code is publicly released. However, the headline first finding rests on a confounded comparison, and the quantitative support is weakened by the absence of error bars and significance tests. These issues are load-bearing for the central claim and require additional experiments before the paper can be accepted.

major comments (3)
  1. [§3.2, §2.2, Table 9, Table 3] The FT-Single versus FT-Mul comparison does not isolate knowledge encoding. Section 3.2 states that the number of paraphrases is 3 in FT-Mul and 1 in FT-Single, while Table 9 shows the same learning rate and number of epochs for both. Consequently, FT-Mul receives three times as many training examples, three times as many tokens, and three times as many optimizer updates per epoch, and Table 2 indeed shows higher test probabilities for FT-Mul. Furthermore, for the UL-Exact configurations (8 of the 48 entries in Table 3), Section 2.2 defines Dul as the exact fine-tuning texts, so FT-Mul also has three times as many forget examples. Both confounds could produce the reported 44/48 advantage in Table 3 without any effect of paraphrase diversity. The authors should add a control such as FT-Single with the same single description repeated three times, or otherwise match the number of training examples and total tokens across conditions, and for UL-Exact match the size of the forget set; if FT-Mul still dominates, the claim would be supported.
  2. [Table 3, §4.1] The quantitative evidence for the paper's main directional claim is thin because all comparisons appear to be single runs with no error bars, standard deviations, or significance tests, and many reported differences are very small. For example, in Table 3 the Llama2-7B/Eval-DU+ GA UL-Single row shows Norm-AUC 0.62 versus 0.63 for FT-Single versus FT-Mul, and several AUC differences are 0.01, while some entries favor FT-Single. The statement that FT-Mul 'outperforms or matches' FT-Single in 44/48 cases depends on how ties are counted and is not a substitute for statistical evidence. The authors should run multiple seeds, report variances, and use paired tests across the unlearn-retain configurations, or explicitly argue why single runs are sufficient at these effect sizes.
  3. [§4.2, Table 4] The text in Section 4.2 claims that unlearning from FT-Mul-Chunk 'consistently results in lower Norm-AUC scores than FT-Single', but Table 4 reports only FT-Mul-Chunk values; there is no table or plot in the main text directly comparing FT-Mul-Chunk with FT-Single under the same evaluation protocol. A cross-table comparison with Table 3 is not controlled because the FT-Mul and FT-Mul-Chunk fine-tuning data differ in both structure and content. The authors should present a matched comparison, for example FT-Mul versus FT-Mul-Chunk with identical unlearning configurations and evaluation prompts, or qualify the claim as a between-experiment observation.
minor comments (6)
  1. [§3.2] The word 'prodcures' should be 'procedures'.
  2. [Table 1] The entry 'Gradieng Ascent' should be 'Gradient Ascent'.
  3. [Table 7] The table header uses 'FT-Mul-Trunk' while the rest of the paper uses 'FT-Mul-Chunk'; the names should be made consistent.
  4. [Figure 1 and Appendix A] The phrase 'text trunk' appears twice where 'text chunk' is meant, and the appendix has the typo 'prbability'.
  5. [Appendix C] The code URL in the main text (github.com/...) and the anonymous URL in the appendix (anonymous.4open.science/...) should be reconciled so the reader knows which repository is canonical.
  6. [Figures 11-13] The legends in these figures mix three curves (FT-Mul-Chunk, FT-Mul-Chunk (New Split), FT-Mul-Chunk-Iso) with labels such as 'FT-Single (New Split)' in the Llama3 panel; the curve labels and captions should be made consistent and unambiguous.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central claims are empirical comparisons with held-out evaluation, not derivations from fitted inputs.

full rationale

The paper's two findings are empirical contrasts (FT-Single vs FT-Mul, FT-Mul-Chunk vs aligned/isolated variants) evaluated with held-out paraphrases and standard GA/TV trade-off curves; no parameter is fitted to the target result and then renamed a prediction. Norm-AUC and AUC are author-defined metrics, but they are applied to actual checkpoints and the main conclusions also hold for absolute AUC in most table cells, so the metrics do not encode the answer by construction. The Eval-DU+ benchmark extends the authors' own Eval-DU, but that is dataset reuse rather than a load-bearing uniqueness theorem or an ansatz smuggled in by citation. The paper also states honest scope limitations (fine-tuning only, formal pretraining validation left to future work). The main threats to the conclusions are experimental confounds, not circularity: FT-Mul has roughly three times the training examples of FT-Single and, under UL-Exact, three times the forget texts, so the paraphrasing effect is not cleanly isolated; and the Hypothesis 1 UL-Exact comparison changes the composition of the unlearning set along with the split. Such concerns belong in a correctness review and do not make the derivation equivalent to its inputs.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The paper relies on no new physical or mathematical entities. It introduces two augmented datasets and author-defined evaluation metrics, which are modeling constructs rather than invented entities. The main assumptions are about the validity of probability-based knowledge scores, the faithfulness of ChatGPT-generated paraphrases and chunks, and the comparability of models with different initial knowledge levels via Norm-AUC.

free parameters (7)
  • number of paraphrases per knowledge piece = 3
    Chosen by the authors for FT-Mul and FT-Mul-Chunk; results may depend on this count, and TOFU+ uses 1 paraphrase in some unlearning data settings.
  • fine-tuning learning rate eta = 1e-5 for all setups
    Selected from {2e-5, 1e-5, 2e-6} to achieve a good fit; affects how knowledge is encoded in the model.
  • fine-tuning epochs N = 4 to 8 depending on setup, see Table 9
    Chosen to ensure near-perfect fit on the fine-tuning set; differs between FT-Mul-Chunk and other conditions for Llama2-7B.
  • GA step count and learning rate = t in {0,5,...,75}, eta_ga 3e-6 or 1e-5 depending on setting
    Hyperparameters are 'picked to best present the trade-off' in Appendix C, an ad hoc choice that can influence Norm-AUC values.
  • TV epochs N_tv = 20 for most settings, 400 for FT-Mul-Chunk on TOFU+
    The 400-epoch setting for FT-Mul-Chunk on TOFU+ differs from the 20 epochs used elsewhere, creating a confound in cross-setting comparisons.
  • TV scaling alpha list = e.g., {0,0.2,0.5,1,5,10} or extended lists
    Chosen to display the trade-off; the number and spacing of alpha values affect the AUC integration.
  • unlearn set size = 100 facts for Eval-DU+ original split, 102 for aligned split, 40 for TOFU+
    Sizes are matched between the original and aligned Eval-DU+ splits, but the factual content differs, so the aligned-split comparison is not fully controlled.
assumptions (6)
  • domain assumption Synthetic characters in Eval-DU+ and TOFU+ are unlikely to appear in pretraining corpora, so knowledge is acquired only during fine-tuning.
    Stated in Section 1 and relied on to attribute unlearning differences to fine-tuning encodings.
  • domain assumption The knowledge score e, computed as conditional probability of the object token in Eval-DU+ or of the answer in TOFU+, measures the degree to which the model retains a knowledge piece.
    Section 3.1; probability is used as a proxy for extractability, which is a modeling choice.
  • domain assumption Paraphrases generated by ChatGPT-4o preserve the semantic content of each knowledge piece.
    Appendix A; generation is stochastic and not independently verified for every fact.
  • domain assumption Norm-AUC, computed after normalizing by the original fine-tuned model's scores, is a valid basis for comparing unlearning efficiency across models with different initial knowledge levels.
    Section 2.3; the metric's construction embeds this comparability assumption.
  • domain assumption GA and TV are representative unlearning algorithms whose behavior generalizes beyond the specific configurations tested.
    Section 2.2; only two algorithms are evaluated.
  • domain assumption Text chunks in FT-Mul-Chunk summarize the underlying knowledge without omitting or distorting facts.
    Appendix A; relies on ChatGPT-4o summaries and manual extraction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning-Time Encoding Shapes Unlearning in LLMs." pith.science (2026). https://pith.science/paper/36OPNEWP

@misc{pith2026250615076,
  author       = {Pith},
  title        = {Pith review of: Learning-Time Encoding Shapes Unlearning in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/36OPNEWP}},
  note         = {Machine review of arXiv:2506.15076}
}
read the original abstract

As large language models (LLMs) are increasingly deployed in the real world, the ability to ``unlearn'', or remove specific pieces of knowledge post hoc, has become essential for a variety of reasons ranging from privacy regulations to correcting outdated or harmful content. Prior work has proposed unlearning benchmarks and algorithms, and has typically assumed that the training process and the target model are fixed. In this work, we empirically investigate how learning-time choices in knowledge encoding impact the effectiveness of unlearning factual knowledge. Our experiments reveal two key findings: (1) learning with paraphrased descriptions improves unlearning performance and (2) unlearning individual piece of knowledge from a chunk of text is challenging. Our results suggest that learning-time knowledge encoding may play a central role in enabling reliable post-hoc unlearning.

Figures

Figures reproduced from arXiv: 2506.15076 by the authors.

Figure 1
Figure 1. Examples of different textual descriptions for the same piece of knowledge in Eval-DU+. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustrations for Norm-AUC and AUC. (a) shows the vanilla extraction trade-off curves comparison when the models are fine-tuned by FT-Single and FT-Mul and the unlearning methods are fixed; (b) and (c) show the curves for calculating Norm-AUC and AUC given the curves in (a). generally yield stronger unlearning but risk greater utility loss. TV computes a parameter difference vector between the original model θorigin… view at source ↗
Figure 3
Figure 3. Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama2-7B, when comparing FT-Mul and FT-Single 0.0 0.2 0.4 0.6 0.8 1.0 Unlearn Score 0.0 0.2 0.4 0.6 0.8 1.0 Retain Score FT-Single FT-Mul (a) UL-Exact, GA 0.0 0.2 0.4 0.6 0.8 1.0 Unlearn Score 0.0 0.2 0.4 0.6 0.8 1.0 Retain Score FT-Single FT-Mul (b) UL-Single, GA 0.0 0.2 0.4 0.6 0.8 1.0 Unlearn Sc… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama3-8B, when comparing FT-Mul and FT-Single 18 [PITH_FULL_IMAGE:figures/full_fig_p018_4.png]
Figure 5
Figure 5. Figure 5: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Gemma2-2B, when comparing FT-Mul and FT-Single 0.000 0.025 0.050 0.075 0.100 0.125 0.150 Unlearn Score 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175…
Figure 6
Figure 6. Figure 6: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on TOFU+ and Llama2-7B, when comparing FT-Mul and FT-Single 19 [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Vanilla memorization trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama2-7B, when comparing FT-Mul and FT-Single 0.0 0.2 0.4 0.6 0.8 1.0 Unlearn Score 0.0 0.2 0.4 0.6 0.8 1.0 Retain Score FT-Single FT-Mul (a) UL-Ex…
Figure 8
Figure 8. Figure 8: Vanilla memorization trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama3-8B, when comparing FT-Mul and FT-Single. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Vanilla memorization trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Gemma2-2b, when comparing FT-Mul and FT-Single. 0.0 0.2 0.4 0.6 0.8 1.0 Unlearn Score 0.0 0.2 0.4 0.6 0.8 1.0 Retain Score FT-Single FT-Mul (a) UL-E…
Figure 10
Figure 10. Figure 10: Vanilla memorization trade-off curves for three choices of unlearning data and two unlearning algorithms on TOFU+ and Llama2-7B, when comparing FT-Mul and FT-Single. 21 [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama2-7B, when the model is fine-tuned from any text chunks. 0.0 0.1 0.2 0.3 0.4 Unlearn Score 0.0 0.1 0.2 0.3 0.4 Retain Score FT-Single FT-Single …
Figure 12
Figure 12. Figure 12: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Llama3-8B, when the model is fine-tuned from any text chunks. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on Eval-DU+ and Gemma2-2b, when the model is fine-tuned from any text chunks. 0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 Unlearn Score 0.00 0.02 0.04 0.06 0.08 0.1…
Figure 14
Figure 14. Figure 14: Vanilla extraction trade-off curves for three choices of unlearning data and two unlearning algorithms on TOFU+ and Llama2-7B, when the model is fine-tuned from any text chunks. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 24 canonical work pages

  1. [1]

    Machine unlearning

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In2021 IEEE symposium on security and privacy (SP), pages 141–159. IEEE, 2021

    2021

  2. [2]

    Rethinking machine unlearning for large language models.Nature Machine Intelligence, pages 1–14, 2025

    Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models.Nature Machine Intelligence, pages 1–14, 2025

    2025

  3. [3]

    https://eur-lex.europa.eu/eli/reg/2016/679/oj, April 2016

    Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016. https://eur-lex.europa.eu/eli/reg/2016/679/oj, April 2016. Official Journal of the European Union, L 119, pp. 1–88

    work page 2016

  4. [4]

    Who’s harry potter? approximate unlearning in llms

    Ronen Eldan and Mark Russinovich. Who’s harry potter? approximate unlearning in llms. arXiv preprint arXiv:2310.02238, 2023

    arXiv 2023

  5. [5]

    Avoiding copyright infringement via large language model unlearning.arXiv preprint arXiv:2406.10952, 2024

    Guangyao Dou, Zheyuan Liu, Qing Lyu, Kaize Ding, and Eric Wong. Avoiding copyright infringement via large language model unlearning.arXiv preprint arXiv:2406.10952, 2024

    arXiv 2024

  6. [6]

    On provable copyright protection for generative models

    Nikhil Vyas, Sham M Kakade, and Boaz Barak. On provable copyright protection for generative models. InInternational conference on machine learning, pages 35277–35299. PMLR, 2023

    work page 2023

  7. [7]

    Large language model unlearning.Advances in Neural Information Processing Systems, 37:105425–105475, 2024

    Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning.Advances in Neural Information Processing Systems, 37:105425–105475, 2024

    work page 2024

  8. [8]

    The wmdp benchmark: Measuring and reducing malicious use with unlearning.arXiv preprint arXiv:2403.03218, 2024

    Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, et al. The wmdp benchmark: Measuring and reducing malicious use with unlearning.arXiv preprint arXiv:2403.03218, 2024

    arXiv 2024

Show all 63 references
  1. [9]

    Knowledge unlearning for mitigating privacy risks in language models.arXiv preprint arXiv:2210.01504, 2022

    Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. Knowledge unlearning for mitigating privacy risks in language models.arXiv preprint arXiv:2210.01504, 2022

    2022 arXiv

  2. [10]

    Depn: Detecting and editing privacy neurons in pretrained language models

    Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. Depn: Detecting and editing privacy neurons in pretrained language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2875–2886, 2023

    2023

  3. [11]

    Identifying and mitigating the security risks of generative ai.Foundations and Trends® in Privacy and Security, 6(1):1–52, 2023

    Clark Barrett, Brad Boyd, Elie Bursztein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, et al. Identifying and mitigating the security risks of generative ai.Foundations and Trends® in Privacy and Security, 6(1...

    2023

  4. [12]

    Knowledge unlearning for llms: Tasks, methods, and challenges.arXiv preprint arXiv:2311.15766, 2023

    Nianwen Si, Hao Zhang, Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang. Knowledge unlearning for llms: Tasks, methods, and challenges.arXiv preprint arXiv:2311.15766, 2023

    2023 arXiv

  5. [13]

    Mechanistic unlearning: Robust knowledge unlearning and editing via mechanistic localization

    Phillip Guo, Aaquib Syed, Abhay Sheshadri, Aidan Ewart, and Gintare Karolina Dziugaite. Mechanistic unlearning: Robust knowledge unlearning and editing via mechanistic localization. arXiv preprint arXiv:2410.12949, 2024

    2024 arXiv

  6. [14]

    To forget or not? towards practical knowledge unlearning for large language models

    Bozhong Tian, Xiaozhuan Liang, Siyuan Cheng, Qingbin Liu, Mengru Wang, Dianbo Sui, Xi Chen, Huajun Chen, and Ningyu Zhang. To forget or not? towards practical knowledge unlearning for large language models. InFindings of the Association for Computational Linguistics: EMNLP 202...

    2024

  7. [15]

    Snap: Unlearning selective knowl- edge in large language models with negative instructions.arXiv preprint arXiv:2406.12329, 2024

    Minseok Choi, Daniel Rim, Dohyun Lee, and Jaegul Choo. Snap: Unlearning selective knowl- edge in large language models with negative instructions.arXiv preprint arXiv:2406.12329, 2024

    2024 arXiv

  8. [16]

    Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models

    Hongbang Yuan, Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models. InProceedings of the AAAI Conference on Artificial Intelligence...

    2025

  9. [17]

    Evaluating deep unlearning in large language models.arXiv preprint arXiv:2410.15153, 2024

    Ruihan Wu, Chhavi Yadav, Russ Salakhutdinov, and Kamalika Chaudhuri. Evaluating deep unlearning in large language models.arXiv preprint arXiv:2410.15153, 2024

    2024

  10. [18]

    Can sensitive information be deleted from llms? objectives for defending against extraction attacks

    Vaidehi Patil, Peter Hase, and Mohit Bansal. Can sensitive information be deleted from llms? objectives for defending against extraction attacks. InThe Twelfth International Conference on Learning Representations. 10

  11. [19]

    Tofu: A task of fictitious unlearning for llms

    Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. Tofu: A task of fictitious unlearning for llms. InFirst Conference on Language Modeling

  12. [20]

    Muse: Machine unlearning six-way evaluation for language models.arXiv preprint arXiv:2407.06460, 2024

    Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A Smith, and Chiyuan Zhang. Muse: Machine unlearning six-way evaluation for language models.arXiv preprint arXiv:2407.06460, 2024

    2024 arXiv

  13. [21]

    Machine unlearning of pre-trained large language models.arXiv preprint arXiv:2402.15159, 2024

    Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. Machine unlearning of pre-trained large language models.arXiv preprint arXiv:2402.15159, 2024

    2024 arXiv

  14. [22]

    RWKU: Benchmarking real-world knowledge unlearning for large language models

    Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. RWKU: Benchmarking real-world knowledge unlearning for large language models. InThe Thirty-eight Conference on Neural Information Processing Systems Datasets and Be...

    2024

  15. [23]

    Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic.arXiv preprint arXiv:2212.04089, 2022

    2022 arXiv

  16. [24]

    Negative preference optimization: From catastrophic collapse to effective unlearning

    Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. InFirst Conference on Language Modeling

  17. [25]

    Unlearning bias in language models by partitioning gradients

    Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. Unlearning bias in language models by partitioning gradients. InFindings of the Association for Computational Linguistics: ACL 2023, pages 6032–6048, 2023

    2023

  18. [26]

    Wagle: Strategic weight attribution for effective and modular unlearning in large language models.Advances in Neural Information Processing Systems, 37:55620–55646, 2025

    Jinghan Jia, Jiancheng Liu, Yihua Zhang, Parikshit Ram, Nathalie Baracaldo, and Sijia Liu. Wagle: Strategic weight attribution for effective and modular unlearning in large language models.Advances in Neural Information Processing Systems, 37:55620–55646, 2025

    2025

  19. [27]

    Physics of language models: Part 3.1, knowledge storage and extraction

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.1, knowledge storage and extraction. InInternational Conference on Machine Learning, pages 1067–1077. PMLR, 2024

    2024

  20. [28]

    Physics of language models: Part 3.2, knowledge manipula- tion

    Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipula- tion. InThe Thirteenth International Conference on Learning Representations

  21. [29]

    Composing parameter-efficient modules with arithmetic operation.Advances in Neural Information Processing Systems, 36:12589–12610, 2023

    Jinghan Zhang, Junteng Liu, Junxian He, et al. Composing parameter-efficient modules with arithmetic operation.Advances in Neural Information Processing Systems, 36:12589–12610, 2023

    2023

  22. [30]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  23. [31]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  24. [32]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  25. [33]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    2024 arXiv

  26. [34]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. 2018

    2018

  27. [35]

    Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730–27744, 2022

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:27730...

    2022

  28. [36]

    To each (textual sequence) its own: Im- proving memorized-data unlearning in large language models.arXiv preprint arXiv:2405.03097, 2024

    George-Octavian Barbulescu and Peter Triantafillou. To each (textual sequence) its own: Im- proving memorized-data unlearning in large language models.arXiv preprint arXiv:2405.03097, 2024. 11

    2024 arXiv

  29. [37]

    Atyaephyra at semeval-2025 task 4: Low-rank npo.arXiv preprint arXiv:2503.13690, 2025

    Jan Bronec and Jind ˇrich Helcl. Atyaephyra at semeval-2025 task 4: Low-rank npo.arXiv preprint arXiv:2503.13690, 2025

    2025 arXiv

  30. [38]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

    2023

  31. [39]

    Revisiting who’s harry pot- ter: Towards targeted unlearning from a causal intervention perspective.arXiv preprint arXiv:2407.16997, 2024

    Yujian Liu, Yang Zhang, Tommi Jaakkola, and Shiyu Chang. Revisiting who’s harry pot- ter: Towards targeted unlearning from a causal intervention perspective.arXiv preprint arXiv:2407.16997, 2024

    2024 arXiv

  32. [40]

    Unlearning with control: Assessing real-world utility for large language model unlearning

    Qizhou Wang, Bo Han, Puning Yang, Jianing Zhu, Tongliang Liu, and Masashi Sugiyama. Unlearning with control: Assessing real-world utility for large language model unlearning. arXiv preprint arXiv:2406.09179, 2024

    2024 arXiv

  33. [41]

    Continual learning and private unlearning

    Bo Liu, Qiang Liu, and Peter Stone. Continual learning and private unlearning. InConference on Lifelong Learning Agents, pages 243–254. PMLR, 2022

    2022

  34. [42]

    Locating and editing factual associations in gpt.Advances in neural information processing systems, 35:17359–17372, 2022

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt.Advances in neural information processing systems, 35:17359–17372, 2022

    2022

  35. [43]

    In-context unlearning: Language models as few shot unlearners.arXiv preprint arXiv:2310.07579, 2023

    Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. In-context unlearning: Language models as few shot unlearners.arXiv preprint arXiv:2310.07579, 2023

    2023 arXiv

  36. [44]

    Soul: Unlocking the power of second-order optimization for llm unlearning.arXiv preprint arXiv:2404.18239, 2024

    Jinghan Jia, Yihua Zhang, Yimeng Zhang, Jiancheng Liu, Bharat Runwal, James Diffenderfer, Bhavya Kailkhura, and Sijia Liu. Soul: Unlocking the power of second-order optimization for llm unlearning.arXiv preprint arXiv:2404.18239, 2024

    2024 arXiv

  37. [45]

    Large language model unlearning via embedding-corrupted prompts.Advances in Neural Information Processing Systems, 37:118198– 118266, 2024

    Chris Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. Large language model unlearning via embedding-corrupted prompts.Advances in Neural Information Processing Systems, 37:118198– 118266, 2024

    2024

  38. [46]

    Reversing the forget-retain objectives: An efficient llm unlearning framework from logit difference.Advances in Neural Information Processing Systems, 37:12581–12611, 2024

    Jiabao Ji, Yujian Liu, Yang Zhang, Gaowen Liu, Ramana Kompella, Sijia Liu, and Shiyu Chang. Reversing the forget-retain objectives: An efficient llm unlearning framework from logit difference.Advances in Neural Information Processing Systems, 37:12581–12611, 2024

    2024

  39. [47]

    Large scale knowledge washing.arXiv preprint arXiv:2405.16720, 2024

    Yu Wang, Ruihan Wu, Zexue He, Xiusi Chen, and Julian McAuley. Large scale knowledge washing.arXiv preprint arXiv:2405.16720, 2024

    2024 arXiv

  40. [48]

    Knowledge sanitization of large language models

    Yoichi Ishibashi and Hidetoshi Shimodaira. Knowledge sanitization of large language models. arXiv preprint arXiv:2309.11852, 2023

    2023 arXiv

  41. [49]

    Guardrail baselines for unlearning in llms.arXiv preprint arXiv:2403.03329, 2024

    Pratiksha Thaker, Yash Maurya, Shengyuan Hu, Zhiwei Steven Wu, and Virginia Smith. Guardrail baselines for unlearning in llms.arXiv preprint arXiv:2403.03329, 2024

    2024 arXiv

  42. [50]

    Uipe: Enhancing llm unlearning by removing knowledge related to forgetting targets.arXiv preprint arXiv:2503.04693, 2025

    Wenyu Wang, Mengqi Zhang, Xiaotian Ye, Zhaochun Ren, Zhumin Chen, and Pengjie Ren. Uipe: Enhancing llm unlearning by removing knowledge related to forgetting targets.arXiv preprint arXiv:2503.04693, 2025

    2025 arXiv

  43. [51]

    Deep contrastive unlearning for language models.arXiv preprint arXiv:2503.14900, 2025

    Estrid He, Tabinda Sarwar, Ibrahim Khalil, Xun Yi, and Ke Wang. Deep contrastive unlearning for language models.arXiv preprint arXiv:2503.14900, 2025

    2025 arXiv

  44. [52]

    Eight methods to evaluate robust unlearning in llms.arXiv preprint arXiv:2402.16835, 2024

    Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menell. Eight methods to evaluate robust unlearning in llms.arXiv preprint arXiv:2402.16835, 2024

    2024 arXiv

  45. [53]

    Beyond single-value metrics: Evaluating and enhancing llm unlearning with cognitive diagnosis.arXiv preprint arXiv:2502.13996, 2025

    Yicheng Lang, Kehan Guo, Yue Huang, Yujun Zhou, Haomin Zhuang, Tianyu Yang, Yao Su, and Xiangliang Zhang. Beyond single-value metrics: Evaluating and enhancing llm unlearning with cognitive diagnosis.arXiv preprint arXiv:2502.13996, 2025

    2025 arXiv

  46. [54]

    Pebench: A fictitious dataset to benchmark machine unlearning for multimodal large language models.arXiv preprint arXiv:2503.12545, 2025

    Zhaopan Xu, Pengfei Zhou, Weidong Tang, Jiaxin Ai, Wangbo Zhao, Xiaojiang Peng, Kai Wang, Yang You, Wenqi Shao, Hongxun Yao, et al. Pebench: A fictitious dataset to benchmark machine unlearning for multimodal large language models.arXiv preprint arXiv:2503.12545, 2025

    2025 arXiv

  47. [55]

    Position: Llm unlearning benchmarks are weak measures of progress.arXiv preprint arXiv:2410.02879, 2024

    Pratiksha Thaker, Shengyuan Hu, Neil Kale, Yash Maurya, Zhiwei Steven Wu, and Virginia Smith. Position: Llm unlearning benchmarks are weak measures of progress.arXiv preprint arXiv:2410.02879, 2024

    2024 arXiv

  48. [56]

    Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022. 12

    2022

  49. [57]

    Quark: Controllable text generation with reinforced unlearning

    Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. Quark: Controllable text generation with reinforced unlearning. Advances in neural information processing systems, 35:27591–27609, 2022

    2022

  50. [58]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. InInternational Conference on Learning Representations, 2021

    2021

  51. [59]

    Piqa: Reasoning about phys- ical commonsense in natural language

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about phys- ical commonsense in natural language. InProceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020

    2020

  52. [60]

    RACE: Large-scale ReAding comprehension dataset from examinations

    Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. RACE: Large-scale ReAding comprehension dataset from examinations. In Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors,Proceedings of the 2017 Conference on Empirical Methods in Natural Language Process...

    2017

  53. [61]

    In Section A, we present the details of constructing benchmark datasets Eval-DU+ and TOFU+, including the detailed statistics of paraphrasing, the templates for generating the synthetic texts, and an illustration of calculating the knowledge scoreprbabilityin Eval-DU+

  54. [62]

    In Section B, we will present additional experimental results, including the performance of fine-tuned models on LLM general benchmarks, the unlearning results evaluated by memorization trade-off, and the full plots of trade-off curves used for calculating the Norm- AUC and AUC

  55. [63]

    Our code for reproducing the results in the tables is anonymously released at https://anonymous

    In Section C, we will present the implementation details in our experiments, including the compute resources used in the epxeriments, the details of model fine-tuning, and the details of the unlearning. Our code for reproducing the results in the tables is anonymously released...

    1958

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.