Pith. sign in

REVIEW 4 major objections 6 minor 34 references

This paper establishes that knowledge edits in large language models largely do not survive subsequent fine-tuning, with up to 25.27% of successful edits flipping to failures, and it traces the cause to fine-tuning's larger, nearly orthogon

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 23:24 UTC pith:6JO7A6UU

load-bearing objection First systematic map of edit decay under fine-tuning, with a useful new per-edit metric — but the full-FT numbers are confounded by a botched forgetting control and one 'modest degradation' claim that isn't. the 4 major comments →

arxiv 2511.05852 v4 pith:6JO7A6UU submitted 2025-11-08 cs.CL cs.AI

Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation

Yinjie Cheng , Paul Youssef , Christin Seifert , J\"org Schl\"otterer , Zhixue Zhao This is my paper
classification cs.CL cs.AI
keywords knowledge editingfine-tuningedit decayedit flip ratioparameter-efficient fine-tuningactivation space analysisLLM safetyselective layer fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks what happens to a fact that has been edited into a large language model when the model is later fine-tuned for a downstream task. Across 254 experimental configurations spanning five models, three knowledge-editing methods, and three fine-tuning approaches, it finds that edits decay substantially after fine-tuning: previously successful edits frequently become failures, with the worst case seeing 25.27% of successful edits flip after fine-tuning. The paper also shows that fine-tuning only the layers that were edited removes edits more effectively than full fine-tuning, while fine-tuning non-edited layers surprisingly erases more edits than full fine-tuning. An activation-space analysis suggests why: fine-tuning moves representations in directions nearly orthogonal to the smaller, localized shifts introduced by editing. The stakes are practical: if edits are fragile, corrective edits must be reapplied after every adaptation, while covert malicious edits might persist or be selectively removed.

Core claim

The paper's central claim is that knowledge edits do not robustly survive subsequent fine-tuning. Using a new metric, Edit Flip Ratio (EFR), which tracks individual edits that were successful after editing but fail after fine-tuning, the authors quantify decay across 254 configurations. The strongest single result is AlphaEdit on GPT-J on the zsRE benchmark, where 25.27% of previously successful edits become unsuccessful after fine-tuning. They further find that fine-tuning only the edited layers removes more edits than full fine-tuning, at a modest cost to downstream performance, and that fine-tuning non-edited layers causes greater edit decay than all-layer fine-tuning. Activation-space me

What carries the argument

Edit Flip Ratio (EFR): a metric that measures, at the individual edit level, the probability that a successful edit becomes unsuccessful after fine-tuning; it isolates edit stability from overall editing accuracy. Selective layer fine-tuning: freezing either the edited layers or the non-edited layers to test whether edit removal/preservation can be controlled. Layer-wise activation drift and directional similarity: measuring the magnitude of activation changes introduced by editing vs fine-tuning, and the cosine similarity between editing and fine-tuning displacement vectors, to explain the mechanism of overwriting.

Load-bearing premise

The paper's headline decay numbers assume that the full-parameter fine-tuning runs are healthy, non-destructive adaptations; if some of those runs are actually instances of training instability or catastrophic forgetting (as suggested by several cells where downstream accuracy collapses), then the measured edit decay is confounded with general model degradation.

What would settle it

Re-run the full-fine-tuning cells (e.g., Llama2 + AlphaEdit + COUNTERFACT) with a lower learning rate or fewer epochs so that downstream accuracy stays near the LoRA/DoRA level; if edits still decay substantially, the decay claim is robust, whereas if edits survive when downstream performance is preserved, the original decay numbers are an artifact of training collapse.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If edits decay after fine-tuning, any pipeline that fine-tunes an edited model must re-apply the edits or accept their loss.
  • Fine-tuning only the edited layers can serve as an edit-removal strategy for scrubbing unwanted or malicious edits.
  • The finding that non-edited-layer fine-tuning erases more edits than full fine-tuning suggests that edit removal does not require touching the edited layers themselves.
  • Edit decay varies by KE method (AlphaEdit more fragile than MEMIT) and by model size (larger models more robust), so robustness to fine-tuning should be a criterion in KE evaluation.
  • The 25.27% EFR figure provides a concrete benchmark for future KE methods that aim to be FT-robust.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The fragility of null-space-constrained editing (AlphaEdit) suggests that KE methods that place edits in directions fine-tuning ignores will be systematically overwritten; future KE could aim to align edit directions with the gradient geometry of likely future fine-tuning.
  • EFR could be adopted as a standard robustness metric in KE benchmarks, complementing ES/PS/NS, and could be extended to measure edit survival under sequential fine-tuning rounds.
  • The finding that fine-tuning non-edited layers harms edits more than full fine-tuning may reflect distributed representation: the non-edited layers carry coordinated support for the edit, so disrupting them unravels it; this could be tested by ablating specific non-edited layers.
  • The safety framing cuts both ways: the same fragility that removes malicious edits also removes beneficial corrections, so operators may need to re-verify edited facts after any adaptation, not just after full retraining.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper reports the first systematic study of whether knowledge edits survive subsequent fine-tuning. Across five LLMs, two/three editing methods, two editing datasets, multiple edit counts, and three fine-tuning schemes (full FT, LoRA, DoRA), the authors measure editing success before and after fine-tuning, introduce an Edit Flip Ratio (EFR) metric, and report that edits generally decay after fine-tuning, with full fine-tuning removing a larger fraction of edits than PEFT methods. They also study selective fine-tuning of edited versus non-edited layers and propose that fine-tuning only edited layers can remove edits, while fine-tuning non-edited layers preserves edits. An activation-space analysis is used to argue that fine-tuning produces larger, more coherent representational shifts than knowledge editing. The paper frames the results as relevant both to preserving beneficial edits and to removing malicious ones.

Significance. If the empirical picture holds, the paper fills a genuine gap: KE and FT are normally evaluated in isolation, while real deployment pipelines often fine-tune edited models. The scale of the study (many configurations, multiple models and methods) and the introduction of EFR as an edit-level flip metric are useful contributions for future work on edit persistence and on safety scenarios like covert malicious edits surviving adaptation. The authors also provide validation of their reproductions and significance tests in the appendix, which supports the reliability of the underlying infrastructure. However, the main quantitative claims currently rest on several confounded or internally inconsistent analyses, so the headline conclusions are not yet established at the precision the paper claims.

major comments (4)
  1. [§4.5 (Catastrophic Forgetting) and Tables 1, 2, 6, 25] The control intended to rule out catastrophic forgetting is invalid. The sentence 'Med_ft (77.95) achieves comparable downstream performance to Mft (1.77), both outperforming Med (0.83) and M (0.83)' mixes numbers from different models and conditions: 1.77 is the untuned base Llama2 score (Table 16), 0.83 is the untuned base GPT-J score (Table 6), and 77.95 appears to be a fine-tuned-only GPT-J score, not a Med_ft for the same configuration. Thus the argument that 'catastrophic forgetting is unlikely the cause' does not follow. This matters because the full-FT columns are the basis for the claim that full FT removes edits most aggressively (Table 2: 38.10% vs 28.71%/29.88%). Several full-FT cells show simultaneous collapse of edit success and downstream performance, e.g. Llama2 MEMIT zsRE 10^2 ES drops 86.03→22.67 while no-edit full-FT HellaSwag is 15.77 (Table 25), indicating destructiv
  2. [§4.3 and Table 16] The claim that fine-tuning only edited layers removes edits 'while incurring only modest degradation in downstream performance' is not supported by the data. In Table 16, edited-layer-only DoRA fine-tuning of Llama2 drops HellaSwag from 89.77 to 29.24 (100 edits) and from 89.00 to 32.10 (1000 edits), with average downstream falling from 81.7 to 65.43 and 81.0 to 65.35. Such a collapse on a core reasoning benchmark is not 'modest'. The observation that ES drops from 96% to 66% is equally consistent with a partially damaged model as with targeted edit removal. The conclusion that edited-layer FT is 'an effective strategy for removing unwanted edits' needs either a stricter downstream threshold, a demonstration that the removed edits are removed without general capability loss, or a reframing as a trade-off with severe task-specific degradation.
  3. [Tables 1–3, 5, 6 and Appendix J] No error bars or multiple seeds are reported for the main quantitative comparisons. The headline pattern that full FT removes edits more than LoRA/DoRA relies on differences like 38.10% vs 28.71% vs 29.88% average decay, but each cell appears to be a single run. Without variance estimates it is impossible to tell whether these differences are meaningful, especially for the small differences between LoRA and DoRA. Appendix J reports p-values for paired comparisons of Med vs Med_ft, but not for the FT-method differences that drive the main claim. At minimum, standard deviations over a few seeds should be reported for key cells, or the authors should explicitly state that all results are single-run and interpret differences cautiously.
  4. [Table 3 and §4.1 (EFR discussion)] The EFR results complicate the 'full FT removes edits' narrative and should be discussed directly. In several GPT-J rows (e.g. AlphaEdit zsRE 10^4), full FT has EFR 9.92 while LoRA has EFR 25.27 and DoRA has 0.00, i.e. full FT flips fewer previously successful edits than PEFT. The paper notes that EFR and ΔES 'exhibit similar trends', but that is not true in such cells. This is not necessarily a contradiction — full FT may destroy the whole model so that many previously unsuccessful edits also remain unsuccessful, lowering EFR — but it weakens the claim that full FT is the most effective way to remove edits, and the phenomenon should be analyzed rather than glossed over.
minor comments (6)
  1. [Abstract vs §3] The abstract says '254 experimental configurations' while §3 says '216 independent model configurations'. Please make the count consistent.
  2. [Table 1 caption] The caption references 'Tab. 4.1', which appears to be a typo for Table 1.
  3. [Equation (6)] The notation ΔM1 and ΔM2 is undefined; the equation should use the vectors Δ1 and Δ2 introduced just above.
  4. [Appendix G / Figure 6] The caption says 'Casual tracing' but should be 'Causal tracing'.
  5. [Appendix E, MEND discussion] 'zsro-shot-wise' appears to be a typo for 'zero-shot-wise'.
  6. [Table 6 caption] The superscripts 'KE1', 'MEMIT2', 'AlphaEdit3' are hard to parse; please define them clearly in the caption.

Circularity Check

0 steps flagged

No significant circularity: the study is an empirical measurement of edit survival under fine-tuning; no prediction reduces to its fit or to a self-citation chain.

full rationale

The paper's claims are descriptive measurements on standard KE benchmarks (COUNTERFACT, zsRE) and downstream tasks, not derivations from fitted parameters. EFR (Eq. 1) is a conditional empirical frequency Pr(s^{Med_ft}=0 | s^{Med}=1), measured on the same evaluation data; it is not a predicted quantity whose construction forces the observed decay. No parameter is fitted to a subset and then used to predict a closely related quantity. The only self-citation (Youssef et al. 2025, ICML position paper) is used as safety motivation, not as a technical premise or uniqueness argument; the layer-selection and baseline-setting citations (Meng et al. 2023; Liu et al. 2024; Hu et al. 2022; Fang et al. 2025) are to external work. The activation-space analysis is also a direct measurement (norms and cosine similarities), with the null-space explanation offered post hoc rather than assumed. The most notable weakness in the paper is not circularity: the §4.5 'catastrophic forgetting' control compares Med_ft (77.95) against a value labeled Mft (1.77) that, from Table 16, is the untuned base model M, so the control is internally inconsistent. That is a confound/correctness issue for the full-FT decay numbers, not a case of the conclusion being equivalent to its inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

The paper introduces no new entities or fitted parameters. The main assumptions are domain-level choices about evaluation criteria, FT corpus representativeness, edit localization, and the health of full-FT runs.

axioms (4)
  • domain assumption Edit success can be defined by top-1 token probability (zsRE) or relative probability vs. original object (COUNTERFACT), as in Equ. 7-8.
    The entire decay measurement depends on these standard KE success criteria, which are borrowed without independent validation for FT robustness.
  • domain assumption Fine-tuning on the Commonsense dataset (170K examples, 8 tasks) is representative of real-world model adaptation.
    All main decay numbers come from this single FT corpus; HotpotQA is only a small additional check (App. E).
  • domain assumption The edited layer sets for GPT-J (3-8) and Llama2 (4-8) are correct and sufficient for the edits.
    Selective-layer FT conclusions depend on this localization; using different layer sets could change the non-edited-layer decay result.
  • domain assumption Full fine-tuning runs are stable and not dominated by catastrophic forgetting or training instability.
    Llama2 full-FT results show simultaneous collapse of edit success and downstream accuracy; §4.5's rebuttal is internally inconsistent.

pith-pipeline@v1.3.0-alltime-deepseek · 34406 in / 10374 out tokens · 93935 ms · 2026-08-03T23:24:07.504001+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation." pith.science (2026). https://pith.science/paper/6JO7A6UU

@misc{pith2026251105852,
  author       = {Pith},
  title        = {Pith review of: Can Fine-Tuning Erase Your Edits? On the Fragile Coexistence of Knowledge Editing and Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6JO7A6UU}},
  note         = {Machine review of arXiv:2511.05852}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Knowledge editing (KE) offers a lightweight alternative to retraining for updating large language models (LLMs). Meanwhile, fine-tuning remains the default operation for adapting LLMs to new domains and tasks. Despite their widespread adoption, these two post-training interventions have been studied in isolation, leaving open a crucial question: if we fine-tune an edited model, do the edits survive? This question is motivated by practical objectives: removing covert or malicious edits, and preserving beneficial edits. If fine-tuning impairs edits (Fig.1), current KE methods become less efficient, as a newly fine-tuned model requires re-editing; if edits persist, fine-tuned models risk propagating hidden malicious edits, raising serious safety concerns. To this end, we systematically quantify edit decay after fine-tuning across 254 experimental configurations. Our results show that in general, edits decay substantially after subsequent fine-tuning. AlphaEdit exhibits the greatest decay on the zsRE benchmark when applied to GPT-J, where 25.27% of previously successful edits become unsuccessful after fine-tuning. We further find that fine-tuning only the edited layers is sufficient to effectively remove edits, while incurring only modest degradation in downstream performance. Surprisingly, fine-tuning non-edited layers leads to greater edit decay than all-layer fine-tuning. Besides, our activation space analysis reveals that fine-tuning produces a larger and more coherent representational shift, both in magnitude and direction, than KE. Overall, our study underscores the necessity of evaluating KE within the broader LLM application pipeline.

Figures

Figures reproduced from arXiv: 2511.05852 by Christin Seifert, J\"org Schl\"otterer, Paul Youssef, Yinjie Cheng, Zhixue Zhao.

Figure 1
Figure 1. Figure 1: An illustration of an LLM (M) that undergoes an edit (Med ) and then fine-tuning (Med_f t). This process results in the loss of edited knowledge and the production of incorrect outputs. Here is an illustrative example, we show real cases in Sec. 4.2. rent KE methods do not yield edits that survive FT, highlighting the need for KE approaches that complement FT and can reliably maintain factual updates. At t… view at source ↗
Figure 2
Figure 2. Figure 2: Editing performance of Llama2 on zsRE dataset before and after fine-tuning. Editing performance after [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: In layer-wise activation drifts (upper three) for GPT2-XL, GPT-J and Llama2, 3 categories for each model: Med, Mft and Med_ft. In directional similarities (bottom three), 3 pairs of categories tested for each model: Med - Mft, Med_ft - Mft and Med_ft - Med. Within the red vertical dash lines are the range of layers being edited. Result specifications in App. I preserve beneficial edits slightly sacrifices … view at source ↗
Figure 4
Figure 4. Figure 4: Difference between validating values and [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Difference between validating values and original values in ratio across eight downstream tasks. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Casual tracing for DeepSeek [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Causal effect heatmaps showing concentrated effects in early layers (1-5) with (from left to right) (a) [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Average decrements ratio (%) caused by KE [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Significance test results across models. The format of Y label is [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 2 canonical work pages

  1. [1]

    Canyu Chen, Baixiang Huang, Zekun Li, Zhaorun Chen, Shiyang Lai, Xiongxiao Xu, Jia-Chen Gu, Jindong Gu, Huaxiu Yao, Chaowei Xiao, Xifeng Yan, William Yang Wang, Philip Torr, Dawn Song, and Kai Shu. 2024. https://openreview.net/forum?id=edCz9pEu16 Can editing LLM s inject harm? In Neurips Safe Generative AI Workshop 2024

    2024

  2. [2]

    Minyeong Choe, Haehyun Cho, Changho Seo, and Hyunil Kim. 2025. https://doi.org/10.18653/v1/2025.emnlp-main.1448 Do all autoregressive transformers remember facts the same way? a cross-architecture analysis of recall mechanisms . In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28482--28501, Suzhou, China. As...

    work page doi:10.18653/v1/2025.emnlp-main.1448 2025

  3. [3]

    Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. 2023. https://doi.org/10.18653/v1/2023.acl-long.893 Analyzing transformers in embedding space . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16124--16170, Toronto, Canada. Association for Computational Linguistics

    doi:10.18653/v1/2023.acl-long.893 2023

  4. [4]

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Jie Shi, Xiang Wang, Xiangnan He, and Tat-Seng Chua. 2025. https://openreview.net/forum?id=HvSytvg3Jh Alphaedit: Null-space constrained model editing for language models . In The Thirteenth International Conference on Learning Representations

    2025

  5. [5]

    Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.751 Dissecting recall of factual associations in auto-regressive language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216--12235, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.751 2023

  6. [6]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, and 1 others. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . Preprint, arXiv:2407.21783

    Pith/arXiv arXiv 2024

  7. [7]

    Akshat Gupta, Dev Sajnani, and Gopala Anumanchipalli. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.903 A unified framework for model editing . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 15403--15418, Miami, Florida, USA. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-emnlp.903 2024

  8. [8]

    Jeremy Howard and Sebastian Ruder. 2018. https://doi.org/10.18653/v1/P18-1031 Universal language model fine-tuning for text classification . In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 328--339, Melbourne, Australia. Association for Computational Linguistics

    doi:10.18653/v1/p18-1031 2018

  9. [9]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  10. [10]

    Zhiqiang Hu, Lei Wang, Yihuai Lan, Wanyu Xu, Ee-Peng Lim, Lidong Bing, Xing Xu, Soujanya Poria, and Roy Lee. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.319 LLM -adapters: An adapter family for parameter-efficient fine-tuning of large language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5...

    doi:10.18653/v1/2023.emnlp-main.319 2023

  11. [11]

    Tianjie Ju, Yiting Wang, Xinbei Ma, Pengzhou Cheng, Haodong Zhao, Yulong Wang, Lifeng Liu, Jian Xie, Zhuosheng Zhang, and Gongshen Liu. 2024. https://doi.org/10.48550/arXiv.2407.07791 Flooding spread of manipulated knowledge in llm-based multi-agent communities . CoRR, abs/2407.07791

    Pith/arXiv arXiv doi:10.48550/arxiv.2407.07791 2024

  12. [12]

    Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. https://doi.org/10.18653/v1/K17-1034 Zero-shot relation extraction via reading comprehension . In Proceedings of the 21st Conference on Computational Natural Language Learning ( C o NLL 2017) , pages 333--342, Vancouver, Canada. Association for Computational Linguistics

    doi:10.18653/v1/k17-1034 2017

  13. [13]

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024 a . https://openreview.net/forum?id=duZANm2ABX Badedit: Backdooring large language models by model editing . In The Twelfth International Conference on Learning Representations

    2024

  14. [14]

    Zhoubo Li, Ningyu Zhang, Yunzhi Yao, Mengru Wang, Xi Chen, and Huajun Chen. 2024 b . https://openreview.net/forum?id=fNktD3ib16 Unveiling the pitfalls of knowledge editing for large language models . In The Twelfth International Conference on Learning Representations

    2024

  15. [15]

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/0cde695b83bd186c1fd456302888454c-Paper-Conference.pdf Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning . In Advances in Neural Information Processing Sy...

    2022

  16. [16]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: weight-decomposed low-rank adaptation. In Proceedings of the 41st International Conference on Machine Learning, pages 32100--32121

    2024

  17. [17]

    Vittorio Mazzia, Alessandro Pedrani, Andrea Caciolai, Kay Rottmann, and Davide Bernardi. 2025. https://doi.org/10.1109/TNNLS.2024.3498935 A survey on knowledge editing of neural networks . IEEE Transactions on Neural Networks and Learning Systems, 36(7):11759--11775

    arXiv 2025

  18. [18]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT . Advances in Neural Information Processing Systems, 36. ArXiv:2202.05262

    Pith/arXiv arXiv 2022

  19. [19]

    Kevin Meng, Arnab Sen Sharma, Alex J Andonian, Yonatan Belinkov, and David Bau. 2023. https://openreview.net/forum?id=MkbcAHIYgyS Mass-editing memory in a transformer

    2023

  20. [20]

    Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2022 a . https://openreview.net/pdf?id=0DcZxeWfOPt Fast model editing at scale . In International Conference on Learning Representations

    2022

  21. [21]

    Eric Mitchell, Charles Lin, Antoine Bosselut, Christopher D Manning, and Chelsea Finn. 2022 b . Memory-based model editing at scale. In International Conference on Machine Learning, pages 15817--15831. PMLR

    2022

  22. [22]

    Venkatesh Balavadhani Parthasarathy, Ahtsham Zafar, Aafaq Khan, and Arsalan Shahid. 2024. The ultimate guide to fine-tuning llms from basics to breakthroughs: An exhaustive review of technologies, research, best practices, applied research challenges and opportunities. arXiv preprint arXiv:2408.13296

    Pith/arXiv arXiv 2024

  23. [23]

    Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners

    2019

  24. [24]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others. 2023. https://arxiv.org/abs/2307.09288 Llama 2: Open fo...

    Pith/arXiv arXiv 2023

  25. [25]

    Ben Wang and Aran Komatsuzaki. 2021. https://github.com/kingoflolz/mesh-transformer-jax Gpt-j-6b: A 6 billion parameter autoregressive language model . https://github.com/kingoflolz/mesh-transformer-jax

    2021

  26. [26]

    Song Wang, Yaochen Zhu, Haochen Liu, Zaiyi Zheng, Chen Chen, and Jundong Li. 2024. https://doi.org/10.1145/3698590 Knowledge editing for large language models: A survey . ACM Comput. Surv., 57(3)

    doi:10.1145/3698590 2024

  27. [27]

    Yu Wang, Ruihan Wu, Zexue He, Xiusi Chen, and Julian McAuley. 2025. https://openreview.net/forum?id=dXCpPgjTtd Large scale knowledge washing . In The Thirteenth International Conference on Learning Representations

    2025

  28. [28]

    Xiaodong Wu, Wenyi Yu, Chao Zhang, and Philip Woodland. 2024. https://doi.org/10.52202/079017-4264 An improved empirical fisher approximation for natural gradient descent . In Advances in Neural Information Processing Systems, volume 37, pages 134151--134194. Curran Associates, Inc

    work page doi:10.52202/079017-4264 2024

  29. [29]

    Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. 2023. https://openreview.net/forum?id=NZZB3UGcd8 Editing large language models: Problems, methods, and opportunities . In The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  30. [30]

    Zhengmao Ye, Dengchun Li, Zetao Hu, Tingfeng Lan, Jian Sha, Sicong Zhang, Lei Duan, Jie Zuo, Hui Lu, Yuanchun Zhou, and 1 others. 2023. mlora: Fine-tuning lora adapters via highly-efficient pipeline parallelism in multiple gpus. arXiv preprint arXiv:2312.02515

    Pith/arXiv arXiv 2023

  31. [31]

    o rg Schl \

    Paul Youssef, Zhixue Zhao, Daniel Braun, J \"o rg Schl \"o tterer, and Christin Seifert. 2025. https://openreview.net/forum?id=QLKBm1PaCU Position: Editing large language models poses serious safety risks . In Forty-second International Conference on Machine Learning Position Paper Track

    2025

  32. [32]

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.296 Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4862--4876, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.296 2023

  33. [33]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  34. [34]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...