Pith. sign in

REVIEW 4 major objections 4 minor 50 references

Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation

T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Unilogit claims that forcing the target token to uniform probability, with targets recomputed from the current model's logits each step, gives better unlearning than static-target distillation baselines.

desk verdict Deriving the exact uniform-target logit is clean and the dynamic self-distillation step is a real advance over UnDIAL, but the Pareto-superiority claim rests on single runs and a private benchmark; fix that. read the letter →

arxiv 2505.06027 v1 pith:U36HAAIT submitted 2025-05-09 cs.CL cs.LG

Stefan Vasilev , Christian Herold , Baohao Liao , Seyyed Hadi Hashemi , Shahram Khadivi , Christof Monz This is my paper · ORCID
classification cs.CLcs.LG
keywords machineunlearninglargelanguagemodelsself-distillationreverseKLdivergenceuniformtargetdistributionlogitadjustmentforget-retaintrade-offrighttobeforgotten
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces Unilogit, a self-distillation method for machine unlearning in large language models. Its central claim is that a target token should be forcibly assigned a uniform probability of $1/|V|$ after softmax, with all other logits taken from the current model, and that these targets should be recomputed from the current parameters at every training step. Unilogit then optimizes reverse KL divergence between the model's output and these targets on the forget set, plus a KL regularizer to the starting model on the retain set. The paper reports that this dynamic target construction yields more accurate approximations of a fully retrained model's outputs and better forget/retain Pareto curves than UnDIAL and NPO, without adding a hyperparameter to tune. If correct, the method offers a practical way to comply with right to be forgotten requirements at a fraction of the cost of retraining.

What carries the argument

The central object is the uniform-target logit adjustment $\tilde{h}(x;\theta)=(1-t)\,h(x;\theta)+t\,\log\!\left(\frac{\sum_{i\neq k}\exp(h_i(x;\theta))}{|V|-1}\right)$, where $t$ is a one-hot vector selecting the target token $k$. This formula sets exactly one logit to the value that makes the corresponding softmax probability equal to $1/|V|$, the uniform distribution over the vocabulary, while leaving all non-target logits intact. The resulting soft-label distribution $\tilde{p}=\mathrm{softmax}(\tilde{h})$ is the distillation target in a reverse-KL forget loss, and the retain objective is a forward KL divergence between the current model and the starting model on the retain set. The mechanism carries the argument by replacing the discrete choice of how strongly to suppress the target token with a self-consistent calculation from the model's own current distribution, which the paper argues should increasingly match the retrained model.

What would settle it

Run Unilogit on a benchmark where the golden retrained model $\theta_r$ is available and compute, on the forget set, the KL divergence between the unlearned model's output distribution and $\theta_r$'s across the learning-rate sweep; the central claim fails if Unilogit's KL is not lower than UnDIAL's and NPO's at matched retention. A second check is the ablation comparing targets rebuilt from the current model against targets built once from the starting model: if the dynamic-target version does not beat the static-target version at the same utility level, the core mechanism is not doing the work.

Watch

Extended reading notes

Core claim

Unilogit's core discovery is that the amount by which a target token's logit must be suppressed can be derived from the model's own current logits, rather than chosen by a hyperparameter. For a forget sample with target token $k$, the method sets $\tilde{h}_k(x;\theta)=\log\!\left(\frac{\sum_{i\neq k}\exp(h_i(x;\theta))}{|V|-1}\right)$, which makes $\mathrm{softmax}(\tilde{h})_k=1/|V|$. All other logits remain unchanged, so the probability mass removed from the target token is redistributed according to the current model's belief about the other vocabulary items. The paper argues that this produces a better approximation of the golden retrained model's outputs than UnDIAL's heuristic subtraction of a fixed $\gamma$, and that rebuilding the targets from the latest parameters each step yields progressively more accurate guidance. The reported experiments claim that Unilogit achieves the most optimal Pareto curves on MUSE-News, RWKU, and an e-commerce benchmark, and that its final output distributions are closer in KL divergence to the retrained model than those of NPO or UnDIAL.

Load-bearing premise

The load-bearing premise is that the current model's non-target logits are a reliable prior for what the retrained model would output on forget samples, and that the model's distribution moves monotonically closer to the retrained model as unlearning progresses; if either fails, the self-generated targets can reinforce the model's own errors.

Editorial extensions

If this is right

  • Unilogit removes the need for the extra $\gamma$ hyperparameter that UnDIAL requires, so deployment needs only the learning rate to be tuned.
  • Because the targets are recomputed from the current model each step, Unilogit can be applied in settings where the retain set is not available during unlearning, as demonstrated on RWKU.
  • If the KL-divergence analysis holds, unlearned models produced by Unilogit are closer in output distribution to a model retrained without the forget data than models produced by NPO or UnDIAL.
  • The reverse-KL choice is load-bearing: the ablation shows it gives stronger forgetting than forward KL at the same level of utility preservation.
  • The method transfers to real-world structured-text products, as shown by the e-commerce benchmark where Unilogit maintains general utility while reducing forget-set completion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the monotonic-progress assumption holds, the dynamic-target recipe should transfer to other self-distillation objectives and to continual-learning settings where the target distribution needs to track an evolving model; this is an extension the paper does not claim.
  • A discriminating stress test would compare Unilogit against a pure uniform-target baseline that flattens all logits; if non-target logits are the main source of signal, Unilogit's edge over that baseline should grow as the forget set shrinks.
  • The paper's token-level uniformity is a simplification; weighting tokens by their importance to the forget objective (a limitation the paper names) is a natural next step that could improve precision on structured text without changing the core formula.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes Unilogit, a self-distillation method for LLM machine unlearning. The method constructs soft targets by modifying the target token's logit so that, after softmax, the target token has probability exactly 1/|V|, leaving all other logits unchanged. The targets are recomputed from the current model parameters at each step rather than from the starting model. The forget loss is a reverse-KL divergence between the current model output and this target, and the retain loss is a KL divergence from the starting model. Unilogit is compared with NPO, UnDIAL, GA, RKLD, ME+GD, and SimNPO on MUSE-News, RWKU, and an in-house e-commerce benchmark. The paper claims Unilogit achieves better Pareto trade-offs between forgetting and retention, is robust to hyperparameter choices, and produces output distributions closer to a retrained model than competing methods. Appendix B derives the target logit formula exactly, and ablations support the use of reverse KL, current-model targets, and the uniform-target logit adjustment.

Significance. If the empirical claims hold, Unilogit is a simple and attractive contribution: it removes UnDIAL's extra hyperparameter gamma, has an exact and interpretable derivation (Appendix B), and the idea of rebuilding distillation targets from the current model is a plausible and clean way to approximate the retrained model. The paper also makes a good-faith effort to evaluate the method against a golden retrained model by measuring KL divergences on the forget set. The strengths are real: the mathematics is exact, the comparisons span three benchmarks including a practical e-commerce setting, and the ablation study isolates the contributions of the main design choices. However, the evidence for the headline 'superior/robust' claim is weakened by single runs without error bars, duplicate rows in Table 1, a private benchmark whose details are deferred to a camera-ready version, and the absence of released code. The central premise of monotone convergence to the retrained model is plausible but is supported only by a single-run trace on 100 samples.

major comments (4)
  1. [Section 4.2 vs. Eq. (2)] The RWKU section states that no retain set is available during unlearning, yet the full Unilogit objective in Eq. (2) includes a retain KL term over Dr. The paper does not state how Unilogit is trained on RWKU: whether lambda is set to zero, whether the retain term is dropped, or whether some other data are used. This matters because Table 2 compares methods under a no-retain-set protocol, and the reader cannot tell whether all methods were trained under the same objective. Please specify the exact objective used for each benchmark.
  2. [Section 3 and Figure 5 (center)] The claimed advantage of dynamic targets over static UnDIAL targets rests on the assumption that the current model's output distribution moves monotonically closer to the retrained model. The only direct evidence is a single-run KL trace on 100 MUSE-News forget samples, with no repeated seeds and no failure regimes (e.g., higher learning rates, larger forget fractions, or different retain-set sizes). If the monotone-convergence premise fails, the self-generated targets can reinforce model drift, and the method's advantage over the 'original model target' variant shown in Figure 6 would not transfer. Please test the premise across a range of hyperparameters or provide conditions under which it provably holds.
  3. [Table 1 and experimental evidence] Table 1 contains duplicate hyperparameter rows with different numbers: NPO+KL lr=5e-6, beta=0.1 appears twice, and Unilogit+KL lr=1e-5 appears twice. No runs are repeated with multiple seeds and no error bars are reported. Since the central claim is Pareto superiority across hyperparameter sweeps, the current evidence does not establish that the observed ordering is stable. Please clarify the duplicates and report multiple seeds, at least for the main Pareto curves, or otherwise quantify run-to-run variability.
  4. [Section 4.3 and footnote 1] The in-house e-commerce benchmark is used to support the 'practical reliability' and 'diverse real-world scenarios' conclusions, but the footnote states that 'More details about the in-house benchmark will be available in the camera-ready version of the paper.' The e-commerce results are therefore not reproducible from the submitted manuscript, and the private nature of the data means the evaluation cannot be independently verified. At minimum, provide dataset statistics, preprocessing details, and exact evaluation settings in the appendix, or clearly mark the e-commerce results as a case study rather than a general benchmark.
minor comments (4)
  1. [Abstract and Section 3] The phrase 'without additional hyperparameters' is too strong: Eq. (2) still contains the retain-loss weight lambda, and the experiments require choosing a learning rate. The precise and defensible claim, made later in Section 3, is that Unilogit eliminates UnDIAL's gamma hyperparameter.
  2. [Eq. (2)] The paper calls KL(p(y|x;theta) || tilde p(y|x;theta)) a 'reverse KL' divergence, but with p as the current model and tilde p as the target distribution, this is the reverse of the common teacher-to-student forward KL in knowledge distillation. Please define the direction explicitly at first use to avoid confusion.
  3. [Table 1] Several typos and formatting inconsistencies appear in Table 1; for example, 'lr 8.5e-6' and 'lr 8.75e-6' lack the equals sign used elsewhere, and the duplicate rows described in the major comments should be resolved.
  4. [Figure 5] In the left and center panels of Figure 5, the two Unilogit checkpoints are not identified with their learning rates in the caption; the reader has to infer them from Table 1. Please label the bars or add this information to the caption.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the method is externally benchmarked against golden retrained-model outputs, and the target-logit formula is an explicit algebraic construction rather than a fitted input renamed as a prediction.

full rationale

The paper's central derivation is self-contained and anchored by external evidence. The target logit formula in Section 3 is obtained by solving softmax(p_target) = 1/|V| for the target logit (Appendix B), i.e., it is an explicit construction of the stated design goal, not a parameter fitted to data and then reported as a prediction. The claimed advantage of Unilogit is not an algebraic identity: the forget/retain Pareto curves in Figures 2-4 and the KL-divergence comparisons to a golden retrained model in Section 4.4 use externally computed retrained-model outputs, so the central claim is falsifiable rather than forced by the method's own objective. The dynamic-self-distillation premise that the current model moves toward theta_r is an empirical assumption about the unlearning trajectory, and the paper tests it against retrained-model KL values (Figure 5); even if this premise were fragile in some regimes, that would be a correctness risk, not circularity. Citations to UnDIAL, Tang et al., Yuan et al., and Wu et al. are used for context, motivation, and independent justification of RKL behavior; none of them is a self-citation chain that supplies the paper's load-bearing conclusion. No fitted input is renamed as a prediction, and no equation reduces to its own inputs by construction. Consistent with the default expectation for most papers, the derivation chain contains no material circular step.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced; the method is an optimization procedure. The main external commitments are lambda = 1 and the three domain assumptions above. The target construction itself has no fitted parameters.

free parameters (1)
  • lambda (retain loss weight) = 1 in MUSE and e-commerce experiments
    The full Unilogit objective uses lambda times KL(theta_o || theta) on the retain set; lambda = 1 is a hand-chosen scalar controlling utility/forgetting balance. Unilogit removes only the forget-target hyperparameter gamma, not lambda.
assumptions (4)
  • domain assumption Model proximity: current model theta and retrained model theta_r are close enough that non-target logits of theta are a strong prior for theta_r's output distribution.
    Section 3 uses this to justify constructing distillation targets from the current model's outputs.
  • domain assumption Uniform probability as untrained state: a probability of 1/|V| on the target token approximates a model that has never seen the forget token, so it is the right forgetting target.
    Section 3 justifies the uniform target choice and supports the derivation in Appendix B.
  • domain assumption Reverse KL is suitable: RKL's mode-seeking behavior prevents high confidence on the forget token, and the method assumes this outweighs any generation diversity loss.
    Section 3 and ablation D.1 provide an empirical defense, not a theorem.
  • standard math Standard softmax and exponential identities are valid.
    The target logit equation follows from solving softmax(p_k = 1/|V|); no unstated mathematical background is required.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation." pith.science (2026). https://pith.science/paper/U36HAAIT

@misc{pith2026250506027,
  author       = {Pith},
  title        = {Pith review of: Unilogit: Robust Machine Unlearning for LLMs Using Uniform-Target Self-Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U36HAAIT}},
  note         = {Machine review of arXiv:2505.06027}
}
read the original abstract

This paper introduces Unilogit, a novel self-distillation method for machine unlearning in Large Language Models. Unilogit addresses the challenge of selectively forgetting specific information while maintaining overall model utility, a critical task in compliance with data privacy regulations like GDPR. Unlike prior methods that rely on static hyperparameters or starting model outputs, Unilogit dynamically adjusts target logits to achieve a uniform probability for the target token, leveraging the current model's outputs for more accurate self-distillation targets. This approach not only eliminates the need for additional hyperparameters but also enhances the model's ability to approximate the golden targets. Extensive experiments on public benchmarks and an in-house e-commerce dataset demonstrate Unilogit's superior performance in balancing forget and retain objectives, outperforming state-of-the-art methods such as NPO and UnDIAL. Our analysis further reveals Unilogit's robustness across various scenarios, highlighting its practical applicability and effectiveness in achieving efficacious machine unlearning.

Figures

Figures reproduced from arXiv: 2505.06027 by the authors.

Figure 1
Figure 1. Overview of self-distillation unlearning in Unilogit: Starting with the output logits of the LLM, the target [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Results for the MUSE-News benchmark for different unlearning methods using multiple different hyperparameters. On the x-axis we have the retain per￾formance and on the y-axis the forgetting performance, both for the QA task. 4.2 Results on stricter unlearning scenario For the more difficult RWKU benchmark, where no retain set is available during unlearning, we present our result in [PITH_FULL_IMAGE:figures/full_fig… view at source ↗
Figure 4
Figure 4. Comparison of unlearning methods on listings from three different sellers across three forget set sizes in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Left: Average KL divergence between the retrained model outputs and the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Results of Unilogit ablations on MUSE [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 16 canonical work pages

  1. [1]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert - Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

    arXiv 2020

  2. [2]

    Jiaao Chen and Diyi Yang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.738 Unlearn what you want to forget: Efficient unlearning for LLM s . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12041--12052, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.738 2023

  3. [3]

    Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. 2023. Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 7210--7217

    work page 2023

  4. [4]

    Chenlu Ding, Jiancan Wu, Yancheng Yuan, Jinda Lu, Kai Zhang, Alex Su, Xiang Wang, and Xiangnan He. 2024. Unified parameter-efficient unlearning for llms. arXiv preprint arXiv:2412.00383

    arXiv 2024

  5. [5]

    Yijiang River Dong, Hongzhou Lin, Mikhail Belkin, Ramon Huerta, and Ivan Vulić. 2024. https://arxiv.org/abs/2402.10052 Undial: Self-distillation with adjusted logits for robust unlearning in large language models . Preprint, arXiv:2402.10052

    arXiv 2024

  6. [6]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al - Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aur \' e lien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Rozi \` e...

    arXiv doi:10.48550/arxiv.2407.21783 2024

  7. [7]

    Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/5fc47800ee5b30b8777fdd30abcaaf3b-Paper-Conference.pdf Alpacafarm: A simulation framework for methods that learn from human feedback . In Advances in...

    2023

  8. [8]

    Ronen Eldan and Mark Russinovich. 2023. https://arxiv.org/abs/2310.02238 Who's harry potter? approximate unlearning in llms . Preprint, arXiv:2310.02238

    arXiv 2023

Show all 50 references
  1. [9]

    Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. 2024. Simplicity prevails: Rethinking negative preference optimization for llm unlearning. arXiv preprint arXiv:2410.07163

    2024

  2. [10]

    General Data Protection Regulation (GDPR). 2016. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32016R0679&from=EN General Data Protection Regulation . Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016

    2016

  3. [11]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, and Anthony Hartshorn et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 h...

    2024 arXiv

  4. [12]

    Peter Hase, Mohit Bansal, Been Kim, and Asma Ghandeharioun. 2023. https://openreview.net/forum?id=EldbUlZtbd Does localization inform editing? surprising differences in causality-based localization vs. knowledge editing in language models . In Thirty-seventh Conference on Neur...

    2023

  5. [13]

    Zhengbao He, Tao Li, Xinwen Cheng, Zhehao Huang, and Xiaolin Huang. 2024. Towards natural machine unlearning. arXiv preprint arXiv:2405.15495

    2024

  6. [14]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR)

    2021

  7. [15]

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. https://arxiv.org/abs/1503.02531 Distilling the knowledge in a neural network . Preprint, arXiv:1503.02531

    2015 arXiv

  8. [16]

    Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023. https://doi.org/10.18653/v1/2023.acl-long.805 Knowledge unlearning for mitigating privacy risks in language models . In Proceedings of the 61st Annual Meeting of the As...

    2023 doi

  9. [17]

    Zhuoran Jin, Pengfei Cao, Chenhao Wang, Zhitao He, Hongbang Yuan, Jiachun Li, Yubo Chen, Kang Liu, and Jun Zhao. 2024. https://arxiv.org/abs/2406.10890 Rwku: Benchmarking real-world knowledge unlearning for large language models . Preprint, arXiv:2406.10890

    2024 arXiv

  10. [18]

    Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/P17-1147 T rivia QA : A large scale distantly supervised challenge dataset for reading comprehension . In Proceedings of the 55th Annual Meeting of the Association for Computational...

    2017 doi

  11. [19]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. https://doi.org/10.18653/v1/2022.acl-long.229 T ruthful QA : Measuring how models mimic human falsehoods . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa...

    2022 doi

  12. [20]

    Chris Yuhao Liu, Yaxuan Wang, Jeffrey Flanigan, and Yang Liu. 2024 a . Large language model unlearning via embedding-corrupted prompts. arXiv preprint arXiv:2406.07933

    2024 arXiv

  13. [21]

    Hengzhu Liu, Ping Xiong, Tianqing Zhu, and Philip S Yu. 2024 b . A survey on machine unlearning: Techniques and new emerged privacy risks. arXiv preprint arXiv:2406.06186

    2024 arXiv

  14. [22]

    Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024 c . https://arxiv.org/abs/2407.20516 Machine unlearning in generative ai: A survey . Preprint, arXiv:2407.20516

    2024 arXiv

  15. [23]

    Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/b125999bde7e80910cbdbd323087df8f-Paper-Conference.pdf Quark: Controllable text generation with r...

    2022

  16. [24]

    Weitao Ma, Xiaocheng Feng, Weihong Zhong, Lei Huang, Yangfan Ye, Xiachong Feng, and Bing Qin. 2025. https://aclanthology.org/2025.coling-main.358/ Unveiling entity-level unlearning for large language models: A comprehensive analysis . In Proceedings of the 31st International C...

    2025

  17. [25]

    Pratyush Maini, Zhili Feng, Avi Schwarzschild, Zachary Chase Lipton, and J Zico Kolter. 2024. https://openreview.net/forum?id=B41hNBoWLo TOFU : A task of fictitious unlearning for LLM s . In First Conference on Language Modeling

    2024

  18. [26]

    Seth Neel, Aaron Roth, and Saeed Sharifi-Malvajerdi. 2021. https://proceedings.mlr.press/v132/neel21a.html Descent-to-delete: Gradient-based methods for machine unlearning . In Proceedings of the 32nd International Conference on Algorithmic Learning Theory, volume 132 of Proce...

    2021

  19. [27]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  20. [28]

    Vaidehi Patil, Peter Hase, and Mohit Bansal. 2023. Can sensitive information be deleted from llms? objectives for defending against extraction attacks. In The Twelfth International Conference on Learning Representations

    2023

  21. [29]

    Martin Pawelczyk, Seth Neel, and Himabindu Lakkaraju. 2024. https://arxiv.org/abs/2310.07579 In-context unlearning: Language models as few shot unlearners . Preprint, arXiv:2310.07579

    2024 arXiv

  22. [30]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36

    2024

  23. [31]

    Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023. https://openreview.net/forum?id=ZLJ6XRbdaC Detecting pretraining data from large language models . In NeurIPS 2023 Workshop on Regulatable ML

    2023

  24. [32]

    Smith, and Chiyuan Zhang

    Weijia Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettlemoyer, Noah A. Smith, and Chiyuan Zhang. 2024. https://arxiv.org/abs/2407.06460 Muse: Machine unlearning six-way evaluation for language models

    2024 arXiv

  25. [33]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261

    2022 arXiv

  26. [34]

    Yi Tang, Yi Gao, Yong-Gang Luo, Ju-Cheng Yang, Miao Xu, and Min-Ling Zhang. 2024. Unlearning from weakly supervised learning. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)

    2024

  27. [35]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, and Cristian Canton Ferrer et al. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open foundati...

    2023 arXiv

  28. [36]

    Akshaj Kumar Veldanda, Shi-Xiong Zhang, Anirban Das, Supriyo Chakraborty, Stephen Rawls, Sambit Sahu, and Milind Naphade. 2024. Llm surgery: Efficient knowledge unlearning and editing in large language models. arXiv preprint arXiv:2409.13054

    2024 arXiv

  29. [37]

    Bichen Wang, Yuzhe Zi, Yixin Sun, Yanyan Zhao, and Bing Qin. 2024 a . https://arxiv.org/abs/2406.01983 Rkld: Reverse kl-divergence-based knowledge distillation for unlearning personal information in large language models . Preprint, arXiv:2406.01983

    2024 arXiv

  30. [38]

    Chaoqi Wang, Yibo Jiang, Chenghao Yang, Han Liu, and Yuxin Chen. 2024 b . https://openreview.net/forum?id=2cRzmWXK9N Beyond reverse KL : Generalizing direct preference optimization with diverse divergence constraints . In The Twelfth International Conference on Learning Repres...

    2024

  31. [39]

    Qizhou Wang, Jin Peng Zhou, Zhanke Zhou, Saebyeol Shin, Bo Han, and Kilian Q Weinberger. 2025 a . https://openreview.net/forum?id=huo8MqVH6t Rethinking LLM unlearning objectives: A gradient perspective and go beyond . In The Thirteenth International Conference on Learning Repr...

    2025

  32. [40]

    Yaxuan Wang, Jiaheng Wei, Chris Yuhao Liu, Jinlong Pang, Quan Liu, Ankit Shah, Yujia Bao, Yang Liu, and Wei Wei. 2025 b . https://openreview.net/forum?id=6ESRicalFE LLM unlearning via loss adjustment with only forget data . In The Thirteenth International Conference on Learnin...

    2025

  33. [41]

    Taiqiang Wu, Chaofan Tao, Jiahao Wang, Runming Yang, Zhe Zhao, and Ngai Wong. 2025. https://aclanthology.org/2025.coling-main.383/ Rethinking K ullback- L eibler divergence in knowledge distillation for large language models . In Proceedings of the 31st International Conferenc...

    2025

  34. [42]

    Jie Xu, Zihan Wu, Cong Wang, and Xiaohua Jia. 2024. https://doi.org/10.1109/tetci.2024.3379240 Machine unlearning: Solutions and challenges . IEEE Transactions on Emerging Topics in Computational Intelligence, 8(3):2150–2168

    2024

  35. [43]

    Haonan Yan, Xiaoguang Li, Ziyao Guo, Hui Li, Fenghua Li, and Xiaodong Lin. 2022. https://doi.org/10.24963/ijcai.2022/556 Arcane: An efficient architecture for exact machine unlearning . In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligenc...

    2022 doi

  36. [44]

    Jin Yao, Eli Chien, Minxin Du, Xinyao Niu, Tianhao Wang, Zezhou Cheng, and Xiang Yue. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.457 Machine unlearning of pre-trained large language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational...

    2024 doi

  37. [45]

    Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024 b . A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. High-Confidence Computing, page 100211

    2024

  38. [46]

    Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024 c . https://openreview.net/forum?id=8Dy42ThoNe Large language model unlearning . In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  39. [47]

    Xiaojian Yuan, Tianyu Pang, Chao Du, Kejiang Chen, Weiming Zhang, and Min Lin. 2024. https://api.semanticscholar.org/CorpusID:273233618 A closer look at machine unlearning for large language models . ArXiv, abs/2410.08109

    2024 arXiv

  40. [48]

    Zhang, L

    R. Zhang, L. Lin, Y. Bai, and S. Mei. 2024. Negative preference optimization: From catastrophic collapse to effective unlearning. In Proceedings of the First Conference of Language Models (COLM)

    2024

  41. [49]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  42. [50]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.