pith. machine review for the scientific record. sign in

arxiv: 2602.02543 · v3 · submitted 2026-01-30 · 💻 cs.LG · cs.AI

Recognition: no theorem link

Norm Anchors Make Model Edits Last

Mingda Liu , Zhenghan Zhu , Ze'an Miao , Katsuki Fujisawa
Authors on Pith no claims yet

Pith reviewed 2026-05-16 09:28 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords model editingsequential editingnorm feedback looplocate-and-editLLM editingnorm anchoringedit stability
0
0 comments X

The pith

Rescaling value vectors to original norms breaks the feedback loop that collapses sequential model edits

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Sequential locate-and-edit methods for LLMs fail after many steps because solved value vectors and the MLP weights they update amplify each other's norms in a self-reinforcing loop. This produces roughly exponential norm growth that existing regularizers and clamps do not stop, eventually destroying edit quality and model capabilities. Norm-Anchor Scaling counters the loop with a single rescaling step that forces every new value vector to match the norm it would have held in the untouched original model. The change extends reliable editing sequences by more than four times and raises average long-run performance by 72.2 percent while leaving single-edit accuracy unchanged.

Core claim

The paper shows that abrupt failure in sequential L&E editing stems from a positive norm-feedback loop between solved value vectors and edited MLP weights. Under standard dynamics this loop yields approximately exponential norm growth that remains untouched by increment-level regularizers or update clamps. Norm-Anchor Scaling interrupts the loop by rescaling each solved value vector to the reference norm taken from the original unedited model, restoring stability across repeated edits.

What carries the argument

Norm-Anchor Scaling (NAS), a one-line rescaling operation that anchors each solved value vector to the norm observed in the original model before any edits.

If this is right

  • Sequential editing sequences remain usable for more than four times as many steps before performance collapses.
  • Long-run editing success rises by 72.2 percent on average across tested models, datasets, and editors.
  • Single-edit accuracy stays intact while the stabilizer adds negligible compute.
  • The same one-line change works for multiple LLM families and existing L&E algorithms.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Reference norms could be precomputed once per model and reused across many editing sessions.
  • Similar anchoring to global statistics might stabilize other continual parameter-update methods that suffer norm drift.
  • Norm control may matter in broader continual-learning settings for large models beyond locate-and-edit.

Load-bearing premise

Rescaling solved value vectors to the original reference norm will not introduce new performance losses on untested metrics or capabilities.

What would settle it

Apply repeated locate-and-edit updates with and without the rescaling step on the same sequence and check whether the unanchored version exhibits norm growth followed by sudden capability collapse at the edit count where the anchored version remains stable.

Figures

Figures reproduced from arXiv: 2602.02543 by Katsuki Fujisawa, Mingda Liu, Ze'an Miao, Zhenghan Zhu.

Figure 1
Figure 1. Figure 1: Mechanism illustration. NAS anchors the solved value before writing, breaking the norm-feedback loop left unresolved by increment-level controls [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Norm explosion accompanies se￾quential editing collapse. During sequential MEMIT editing, we track edit success (blue, left axis) and normalized weight growth Rn := ∥Wn∥/∥W0∥ (orange, right axis) versus edit step n. For both Llama-3 and GPT-J, increasing Rn coincides with deteriorating editing performance; Spearman ρ is shown in each panel. Motivated by this diagnosis, we propose Norm￾Anchor Scaling (NAS),… view at source ↗
Figure 3
Figure 3. Figure 3: Log-linear growth of weight-norm ratio under sequential editing. We measure log Rn as a function of edit step n (Rn = ∥Wn∥/∥W0∥) for Llama-3 and GPT-J using MEMIT. Linear fits (dashed) achieve high R2 , supporting an approximately exponential in￾crease of Rn with the number of edits. The proof is deferred to Appendix B.3. Verification via curve fitting. We fit ∥Wn∥ as a function of n and observe a clear ex… view at source ↗
Figure 4
Figure 4. Figure 4: Hidden representation drift under sequential L&E updates. We probe the target￾module representations on 1,000 held-out factual prompts for pre-edited (blue), vanilla (green), and vanilla+NAS (orange) after 100 and 500 edits, and visualize them in a shared 2D PCA space (PCA fit on pre-edited). Cross markers indicate state-wise means; ellipses denote 95% confidence regions. We report ∆(pre → ·) = ∥µ· − µpre∥… view at source ↗
Figure 5
Figure 5. Figure 5: RQ2: GLUE performance (F1) during sequential editing. NAS preserves base-task [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: RQ3: NAS improves stability and editing performance under long-horizon sequential editing. (a) Dual-axis trajectories: solid lines show post-edit score, dashed lines show Rn; the red dot marks the collapse point (CP). (b) Editing quality at baselines’ CP, reported as Eff./Gen./Spe. Comparing pre-added (gray) vs. post-added (orange). 4.3 General Capability Retention (RQ2) To test whether long-horizon sequen… view at source ↗
Figure 7
Figure 7. Figure 7: ∥k ⋆ n∥ 2 stability across edit steps. Boxplots of ∥k ⋆ n∥ over edit steps show small fluctuation and tight concentration, supporting the approximation ∥k ⋆ n∥ −2 ≈ K used in the analysis. B.3 Proof for Prop. 3.2 Starting from Lemma 3.1 (Eq. (12)), we take conditional expectation given the current weight Wn−1: E [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Linear scaling between value-vector norms and the edited weight norm (baseline). For sequential editing, we plot E [PITH_FULL_IMAGE:figures/full_fig_p020_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Linear scaling between the pre-edit value norm and the edited weight norm (NAS). Under norm anchoring, we plot E[∥v old n ∥ 2 | W] versus ∥W∥ 2 at multiple checkpoints (averaged over sampled edits/keys). Dashed lines are linear fits; insets report fitted slopes and R2 . Substituting Eq. (62) into Eq. (61) yields E∥Wn∥ 2 ≈ r n E∥W0∥ 2 + β(1 − r n ). (63) which is exactly Eq. (16). In our experiments, the fi… view at source ↗
Figure 10
Figure 10. Figure 10: ∥ ˜kn∥ 2 stability across edit steps. Boxplots of ∥ ˜kn∥ over edit steps show small fluctuation and tight concentration, supporting the approximation ∥ ˜kn∥ −2 ≈ K˜ used in the analysis. Eqs. (70) and (71) have the same form as Eqs. (8) and (11) under C = I. Thus, by replacing (Wn, k⋆ n ) with (W˜ n, ˜kn) in Appendix B.2, we obtain ∥W˜ n∥ 2 = ∥W˜ n−1∥ 2 + ∥v new n ∥ 2 − ∥v old n ∥ 2 ∥ ˜kn∥ 2 , (72) which … view at source ↗
Figure 11
Figure 11. Figure 11: Linear scaling between value-vector norms and the edited weight norm (baseline, tilde space). For sequential editing, we plot E h ∥v new∥ 2 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Linear scaling between the pre-edit value norm and the edited weight norm (NAS, tilde space). Under norm anchoring, we plot E[∥v old n ∥ 2 | W˜ ] versus ∥W˜ ∥ 2 at multiple checkpoints (averaged over sampled edits/keys). Dashed lines are linear fits; insets report fitted slopes and R2 . E∥v old n ∥ 2 ≈ sold E∥W˜ n−1∥ 2 + bold. Since the above “tilde” empirical phenomenon also holds, the remaining computat… view at source ↗
Figure 13
Figure 13. Figure 13: GLUE F1 trajectories over sequential edits on Qwen (step 0 denotes the pre-edit baseline). [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: GLUE F1 trajectories over sequential edits on GPT-J (step 0 denotes the pre-edit baseline). [PITH_FULL_IMAGE:figures/full_fig_p027_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: RQ3: General capabilities comparison. Radar plots compare pre-added (gray) and post-added (orange) performance on general capabilities (GLUE-style evaluation) for two method pairs: MEMIT & PRUNE (left) and RECT & AlphaEdit (right). For each radar, the first six axes correspond to the first method in the title, and the underlined six axes correspond to the second method [PITH_FULL_IMAGE:figures/full_fig_p… view at source ↗
Figure 16
Figure 16. Figure 16: NAS improves stability and editing performance on GPT-J. (a) Improved stability and suppressed norm explosion: dual-axis trajectories show the editing score (solid) and relative weight norm Rn (dashed); the red dot marks the collapse point (CP, first step with score ≤ 60). (b) Improved editing performance: success at the original (w/o NAS) CP, reported on rewrite/para./neighbor (Eff./Gen./Spe.), comparing… view at source ↗
Figure 17
Figure 17. Figure 17: Single-step sequential editing on WikiBigEdit: ENCORE and LyapLock with vs. without [PITH_FULL_IMAGE:figures/full_fig_p032_17.png] view at source ↗
read the original abstract

Sequential Locate-and-Edit (L&E) model editing can fail abruptly after many edits. We identify and formalize this failure as a positive norm-feedback loop, in which solved value vectors and edited MLP weights progressively amplify each other, degrading edit quality and eventually collapsing model capabilities. Our analysis shows that this feedback can yield approximately exponential norm growth under standard L&E dynamics, and can remain unresolved by existing increment-level regularizers or update clamps. We propose Norm-Anchor Scaling (NAS), a plug-in stabilizer that breaks this loop by rescaling each solved value vector to an original-model reference norm. Across multiple LLM backbones, datasets, and L&E editors, NAS extends the usable editing horizon by more than 4x and improves long-run editing performance by 72.2% on average, while preserving single-edit efficacy, with only a one-line modification and negligible computational overhead.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The manuscript identifies a positive norm-feedback loop in sequential Locate-and-Edit (L&E) model editing, in which solved value vectors and edited MLP weights amplify each other, producing approximately exponential norm growth that degrades edit quality and collapses capabilities. It proposes Norm-Anchor Scaling (NAS), a one-line rescaling of each solved value vector to the pre-edit reference norm, and reports that this extends the usable editing horizon by more than 4x while improving long-run editing performance by 72.2% on average across multiple LLM backbones, datasets, and editors, without harming single-edit efficacy and with negligible overhead.

Significance. If the result holds, NAS supplies a minimal, parameter-free stabilizer that directly targets a previously unaddressed failure mode in sequential editing, substantially increasing the practical viability of L&E methods. The reported gains are large and consistent, yet the moderate soundness rating arising from incomplete derivation details and limited controls means the significance remains conditional on stronger mechanistic evidence and broader validation.

major comments (3)
  1. [§3] §3 (analysis of norm growth): the claim that standard L&E dynamics produce approximately exponential norm growth is load-bearing for identifying the feedback loop as the dominant cause, but the manuscript provides no full derivation or closed-form steps, leaving open whether the growth rate is generic or depends on unstated assumptions about update magnitude and layer norms.
  2. [Experimental results] Experimental results (long-run metrics): the 72.2 % average improvement and 4× horizon extension rest on the assumption that rescaling exactly to the original-model reference norm introduces no capability drift on untested axes; without ablations on long-context reasoning, OOD robustness, or edit-order sensitivity, it is unclear whether the reported gains generalize or merely reflect the chosen metric suite.
  3. [Comparison to regularizers] Comparison to regularizers: the statement that existing increment-level regularizers and update clamps leave the loop unresolved is central to NAS’s novelty, yet no explicit equations or ablation tables quantify how NAS differs mechanistically from those baselines in the feedback dynamics.
minor comments (2)
  1. [Abstract and §4] The abstract and §4 could clarify whether the reference norm is computed once from the initial model or recomputed after each edit, as this choice affects reproducibility.
  2. [Figures] Figure captions for norm-growth plots should include the number of runs and any error bands to allow readers to assess variability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive review and for recognizing the potential impact of Norm-Anchor Scaling. We address each major comment below and will revise the manuscript accordingly to strengthen the derivation, expand experimental controls where feasible, and clarify mechanistic distinctions.

read point-by-point responses

  1. Referee: [§3] §3 (analysis of norm growth): the claim that standard L&E dynamics produce approximately exponential norm growth is load-bearing for identifying the feedback loop as the dominant cause, but the manuscript provides no full derivation or closed-form steps, leaving open whether the growth rate is generic or depends on unstated assumptions about update magnitude and layer norms.

    Authors: We agree that a more explicit derivation will improve clarity. In the revision we will add a detailed step-by-step derivation in §3, starting from the standard L&E value-vector update and showing how the norm-feedback loop produces approximately exponential growth under the assumptions of bounded update magnitudes and typical post-layer-norm scaling. The derivation will explicitly state these assumptions and note the conditions under which the exponential regime holds. revision: yes

  2. Referee: Experimental results (long-run metrics): the 72.2 % average improvement and 4× horizon extension rest on the assumption that rescaling exactly to the original-model reference norm introduces no capability drift on untested axes; without ablations on long-context reasoning, OOD robustness, or edit-order sensitivity, it is unclear whether the reported gains generalize or merely reflect the chosen metric suite.

    Authors: We acknowledge the value of broader validation. We will add an ablation on edit-order sensitivity and include a limitations paragraph discussing potential effects on long-context reasoning and OOD robustness. Full-scale ablations on every axis are computationally intensive, so we will prioritize the most relevant controls while noting remaining gaps; the core claims will be qualified accordingly. revision: partial

  3. Referee: Comparison to regularizers: the statement that existing increment-level regularizers and update clamps leave the loop unresolved is central to NAS’s novelty, yet no explicit equations or ablation tables quantify how NAS differs mechanistically from those baselines in the feedback dynamics.

    Authors: We will revise §3 to include explicit equations contrasting the closed-loop dynamics under increment-level regularizers (which dampen but do not eliminate the norm amplification) versus NAS (which directly anchors the reference norm). We will also add a table in the experiments section that reports norm trajectories and long-run performance for representative regularizers, clamps, and NAS. revision: yes

Circularity Check

0 steps flagged

Derivation self-contained with no reduction to inputs by construction

full rationale

The paper derives the norm-feedback loop from standard L&E update dynamics (exponential growth under repeated value-vector solves and weight updates) and introduces NAS as an external rescaling to the pre-edit model's reference norm. This reference is taken directly from the unmodified model state rather than fitted, predicted, or defined in terms of the editing process itself. No equations or claims reduce the stabilizer or the reported gains to a self-citation chain, ansatz smuggled via prior work, or a quantity defined by the target result. The 4x horizon and 72.2% improvement are presented as measured outcomes of the intervention, not tautological consequences of the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the existence of the norm-feedback loop as the primary failure mechanism in sequential L&E and on the effectiveness of norm anchoring to break it. No new free parameters are introduced; the reference norm is taken directly from the original model.

axioms (1)
  • domain assumption Locate-and-edit dynamics in transformer MLPs follow standard update rules that permit norm amplification between solved value vectors and edited weights.
    The analysis of exponential growth and the design of NAS rely on this property of existing L&E methods.
invented entities (1)
  • positive norm-feedback loop no independent evidence
    purpose: To explain and formalize the abrupt failure observed in sequential editing.
    Conceptual construct introduced to describe the mutual amplification between edited MLP weights and solved value vectors.

pith-pipeline@v0.9.0 · 5449 in / 1338 out tokens · 32054 ms · 2026-05-16T09:28:15.593248+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 5 internal anchors

  1. [1]

    The fifth PASCAL recognizing textual entailment challenge

    Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. The fifth PASCAL recognizing textual entailment challenge. InProceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17,

    work page 2009

  2. [2]

    Nicola De Cao, Wilker Aziz, and Ivan Titov

    URL https://tac.nist.gov/publications/2009/additional.papers/ RTE5_overview.proceedings.pdf. Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language mod- els. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cessing, pages 6491–6506, Online and Punta Cana, Dominican Republic, November

    work page 2009

  3. [3]

    doi: 10.18653/v1/2021.emnlp-main.522

    Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.522. URL https://aclanthology.org/2021.emnlp-main.522/. Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, and William W. Cohen. Time-aware language models as temporal knowledge bases.Transactions of the Association for Computational Lingu...

    work page doi:10.18653/v1/2021.emnlp-main.522 2021

  4. [4]

    Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisen- stein, and William W

    doi: 10.1162/tacl_a_00459. URLhttps://aclanthology.org/2022.tacl-1.15/. William B. Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. InProceedings of the Third International Workshop on Paraphrasing (IWP2005),

    work page doi:10.1162/tacl_a_00459 2022

  5. [5]

    Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua

    URL https://aclanthology.org/I05-5002/. Junfeng Fang, Houcheng Jiang, Kun Wang, Yunshan Ma, Shi Jie, Xiang Wang, Xiangnan He, and Tat-Seng Chua. Alphaedit: Null-space constrained knowledge editing for language models.arXiv preprint arXiv:2410.02355,

    work page arXiv

  6. [6]

    doi: 10.18653/v1/2021.emnlp-main.446

    URLhttps://arxiv.org/abs/2012.14913. Aaron Grattafiori et al. The Llama 3 herd of models,

    work page internal anchor Pith review Pith/arXiv arXiv 2012

  7. [7]

    Xiaojie Gu, Ziying Huang, Jia-Chen Gu, and Kai Zhang

    URLhttps://arxiv.org/abs/2401.04700. Xiaojie Gu, Ziying Huang, Jia-Chen Gu, and Kai Zhang. Ultraedit: Training-, subject-, and memory- free lifelong editing in language models,

    work page arXiv

  8. [8]

    Akshat Gupta, Anurag Rao, and Gopala Krishna Anumanchipalli

    URLhttps://arxiv.org/abs/2505.14679. Akshat Gupta, Anurag Rao, and Gopala Krishna Anumanchipalli. Model editing at scale leads to gradual and catastrophic forgetting. InAnnual Meeting of the Association for Computational Linguistics,

    work page arXiv

  9. [9]

    Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi

    URL https://arxiv.org/abs/2502.01636. Thomas Hartvigsen, Swami Sankaranarayanan, Hamid Palangi, Yoon Kim, and Marzyeh Ghassemi. Aging with grace: Lifelong model editing with discrete key-value adaptors,

    work page arXiv

  10. [10]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt

    URL https: //arxiv.org/abs/2211.11031. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding,

    work page arXiv

  11. [11]

    Measuring Massive Multitask Language Understanding

    URL https://arxiv. org/abs/2009.03300. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. Zero-shot relation extraction via reading comprehension.CoRR, abs/1706.04115,

    work page internal anchor Pith review Pith/arXiv arXiv 2009

  12. [12]

    Stephanie Lin, Jacob Hilton, and Owain Evans

    URL https://arxiv.org/ abs/2502.05759. Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falsehoods. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland, May

    work page arXiv

  13. [13]

    doi: 10.18653/v1/2022.acl-long.229

    Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.229. URL https://aclanthology. org/2022.acl-long.229/. Jun-Yu Ma, Hong Wang, Hao-Xiang Xu, Zhen-Hua Ling, and Jia-Chen Gu. Perturbation-restrained sequential model editing.arXiv preprint arXiv:2405.16821,

    work page doi:10.18653/v1/2022.acl-long.229 2022

  14. [14]

    Locating and Editing Factual Associations in GPT

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt, 2023a. URLhttps://arxiv.org/abs/2202.05262. Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer, 2023b. URLhttps://arxiv.org/abs/2210.07229. Eric Mitchell, Charles Lin, Antoine Bosselu...

    work page internal anchor Pith review arXiv

  15. [15]

    Fast model editing at scale.arXiv preprint arXiv:2110.11309, 2022

    URLhttps://arxiv.org/abs/2110.11309. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. Language models as knowledge bases? InProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-...

    work page arXiv 2019

  16. [16]

    doi: 10.18653/v1/D19-1250

    Association for Computational Linguistics. doi: 10.18653/v1/D19-1250. URLhttps://aclanthology.org/D19-1250/. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Kemin...

    work page doi:10.18653/v1/d19-1250

  17. [17]

    Qwen2.5 Technical Report

    URL https://arxiv.org/abs/2412.15115. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners,

    work page internal anchor Pith review Pith/arXiv arXiv

  18. [18]

    Adam Roberts, Colin Raffel, and Noam Shazeer

    URL https://api.semanticscholar.org/ CorpusID:160025533. Adam Roberts, Colin Raffel, and Noam Shazeer. How much knowledge can you pack into the parameters of a language model? InProceedings of the 2020 Conference on Empirical Meth- ods in Natural Language Processing (EMNLP), pages 5418–5426, Online, November

    work page 2020

  19. [19]

    doi: 10.18653/v1/2020.emnlp-main.437

    Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.437. URL https://aclanthology.org/2020.emnlp-main.437/. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. InProceedings of the 2013 Confere...

    work page doi:10.18653/v1/2020.emnlp-main.437 2020

  20. [20]

    GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding

    URL https://arxiv.org/abs/1804.07461. Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/mesh-transformer-jax, May

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Memoir: Lifelong model editing with minimal overwrite and informed retention for llms, 2025a

    11 Ke Wang, Yiming Qin, Nikolaos Dimitriadis, Alessandro Favero, and Pascal Frossard. Memoir: Lifelong model editing with minimal overwrite and informed retention for llms, 2025a. URL https://arxiv.org/abs/2506.07899. Peng Wang, Zexi Li, Ningyu Zhang, Ziwen Xu, Yunzhi Yao, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Wise: Rethinking the knowledge...

    work page arXiv

  22. [22]

    URLhttps://aclanthology.org/Q19-1040/

    doi: 10.1162/ tacl_a_00290. URLhttps://aclanthology.org/Q19-1040/. Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. InProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies, Volume 1 (Lo...

    work page 2018

  23. [23]

    doi: 10.18653/v1/N18-1101

    Association for Computational Linguistics. doi: 10.18653/v1/N18-1101. URL https://aclanthology.org/N18-1101/. Yunzhi Yao, Peng Wang, Bozhong Tian, Siyuan Cheng, Zhoubo Li, Shumin Deng, Huajun Chen, and Ningyu Zhang. Editing large language models: Problems, methods, and opportunities. In Proceedings of the 2023 Conference on Empirical Methods in Natural La...

    work page doi:10.18653/v1/n18-1101 2023

  24. [24]

    doi: 10.18653/v1/2023.emnlp-main.632

    Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.632. URL https://aclanthology.org/2023.emnlp-main. 632/. Lang Yu, Qin Chen, Jie Zhou, and Liang He. Melo: Enhancing model editing with neuron-indexed dynamic lora,

    work page doi:10.18653/v1/2023.emnlp-main.632 2023

  25. [25]

    URLhttps://arxiv.org/abs/2312.11795. Ningyu Zhang, Yunzhi Yao, Bozhong Tian, Peng Wang, Shumin Deng, Mengru Wang, Zekun Xi, Shengyu Mao, Jintian Zhang, Yuansheng Ni, Siyuan Cheng, Ziwen Xu, Xin Xu, Jia-Chen Gu, Yong Jiang, Pengjun Xie, Fei Huang, Lei Liang, Zhiqiang Zhang, Xiaowei Zhu, Jun Zhou, and Huajun Chen. A comprehensive study of knowledge editing ...

    work page arXiv

  26. [26]

    Xiao Zhang and Ji Wu

    URL https://arxiv.org/abs/2506.17864. Xiao Zhang and Ji Wu. Dissecting learning and forgetting in language model finetuning. In The Twelfth International Conference on Learning Representations,

    work page arXiv

  27. [27]

    Zexuan Zhong, Zhengxuan Wu, Christopher Manning, Christopher Potts, and Danqi Chen

    URL https:// openreview.net/forum?id=tmsqb6WpLz. Zexuan Zhong, Zhengxuan Wu, Christopher Manning, Christopher Potts, and Danqi Chen. MQuAKE: Assessing knowledge editing in language models via multi-hop questions. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 15686–15702, Singapore, December

    work page 2023

  28. [28]

    Featured Certification

    Association for Computational Linguistics. doi: 10.18653/v1/2023. emnlp-main.971. URLhttps://aclanthology.org/2023.emnlp-main.971/. Chen Zhu, Ankit Singh Rawat, Manzil Zaheer, Srinadh Bhojanapalli, Daliang Li, Felix X. Yu, and Sanjiv Kumar. Modifying memories in transformer models.CoRR, abs/2012.00363,

    work page doi:10.18653/v1/2023 2023

  29. [29]

    S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S

    URL https://arxiv.org/abs/2012.00363. 12 A Experimental Setup and Implement Details In this section, we describe the datasets and evaluation metrics used in our experiments, and detail the base-model capability evaluation, experimental setup, and baseline methods. A.1 Datasets • CounterFact.CounterFact [Meng et al., 2023a] is a benchmark designed specific...

    work page arXiv 2012

  30. [30]

    tilde” versions: E h ∥vnew n ∥2 ∥ ˜Wn−1∥2 i ≈s new∥ ˜Wn−1∥2+bnew,E h ∥vold n ∥2 ∥ ˜Wn−1∥2 i ≈s old∥ ˜Wn−1∥2+bold. Under these empirical observations for the “tilde

    Substituting Eq. (62) into Eq. (61) yields E∥Wn∥2 ≈r n E∥W0∥2 +β(1−r n).(63) which is exactly Eq. (16). In our experiments, the fitted slope satisfies sold >0 and the estimated product Ksold lies in (0,1) over the observation range, hence 0< r= 1−Ks old <1 . Moreover, we typically observe bold <0 and set τ >0 , so β= τ 2−bold sold >0 . Therefore, Eq. (63)...

    work page 2000

  31. [31]

    First <5

    E∥vold n ∥2 ≈s old E∥ ˜Wn−1∥2 +b old. Since the above “tilde” empirical phenomenon also holds, the remaining computations in Ap- pendix B.4 can be reused line-by-line, yielding the corresponding conclusion of Cor. 3.3 for the generalCcase. C Additional Experiment C.1 Single-Edit Plasticity under Norm Anchoring RQ4 examines whether the long-horizon gains o...

    work page arXiv 2023