CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance

Anubhav Lakra; Yue Feng

REVIEW 3 major objections 5 minor 21 references

Stability-aware editing of 4-bit LLMs cuts knowledge degradation 11–17% and lifts fact generalization to 28% in sequential UK financial updates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 12:04 UTC pith:YWT5VZZB

load-bearing objection Solid systems engineering on a real failure mode—sequential edits under 4-bit—with honest absolute numbers, but the headline gaps rest on single-run point estimates and adapted baselines. the 3 major comments →

arxiv 2607.28292 v1 pith:YWT5VZZB submitted 2026-07-30 cs.CL cs.AIcs.CEcs.LG

CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance

Anubhav Lakra , Yue Feng This is my paper

classification cs.CL cs.AIcs.CEcs.LG

keywords memory editingquantized LLMsLoRAcatastrophic forgettingfinancial NLPsequential knowledge update4-bit quantizationstability control

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Financial LLMs go stale as CEOs, rates, and regulations change, but full retraining is too slow and expensive. Memory editing can patch individual facts, yet under 4-bit quantization those edits often vanish into the quantization grid or destroy other knowledge when applied one after another—the “quantization stability crisis.” CACHE-UK answers this by confining each edit to a tiny rank-1 change inside a LoRA adapter, boosting strength for UK-finance content, and using a closed-loop controller that tracks “degradation debt” so later edits shrink when the model starts forgetting. On a 4-bit OpenLLaMA-3B model and an 88k-document UK finance corpus, the method lowers preservation-set degradation by 11–17% versus adapted baselines and reaches the best observed paraphrase success (28%). Absolute generalization stays low, but the paper argues that stability-aware, parameter-efficient editing is a workable path for keeping resource-constrained financial models factually current.

Core claim

Under identical 4-bit OpenLLaMA-3B and LoRA constraints, CACHE-UK’s combination of rank-1 LoRA perturbation, domain-prioritized edit strength, and a degradation-debt Stability Controller reduces knowledge degradation by 11–17% relative to adapted ROME, MEMIT, EasyEdit, and KnowledgeEditor baselines and achieves the highest test-success rate in the setting (28%, +6 points over the strongest adapted baseline).

What carries the argument

Degradation debt Stability Controller: after each edit it scores a fixed preservation set; when degradation exceeds a threshold the debt rises and the next edit’s strength is scaled by 1/(1+debt), otherwise debt decays—an integral-style feedback loop that keeps sequential quantized edits from cascading into catastrophic forgetting, paired with rank-1 updates applied only to the LoRA B matrix.

Load-bearing premise

The authors’ re-implementations of the baseline editors are fair functional equivalents under 4-bit quantization, so the measured gaps isolate the stability mechanism rather than implementation differences.

What would settle it

Re-run the same sequential UK-finance edit suite on the identical 4-bit OpenLLaMA-3B backbone using the original authors’ optimized editor code (or multi-run randomized orderings) and check whether CACHE-UK’s 11–17% degradation reduction and +6 pp test-success edge disappear.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Resource-constrained financial deployments can keep parametric facts fresher without dequantizing or full retraining.
Edit strength should be treated as a controllable feedback signal, not a fixed hyperparameter, when quantization is coarse.
Confining surgical edits to LoRA adapter space is a practical way to widen the viable edit-strength window under INT4.
Absolute generalization near 28% implies hybrid designs (editing plus retrieval) remain necessary for production accuracy.
The same degradation-debt loop can be ported to other regulated domains that face continual fact churn on quantized models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If degradation debt generalizes, other PEFT subspaces (not only LoRA rank-16) could inherit the same controller with little redesign.
The 16× coarser 4-bit step size suggests a predictable stability cliff between 8-bit and 4-bit that bit-width ablations could map as a simple scaling law.
Keyword domain boosts are a stopgap; embedding-based relevance would test whether content-adaptive strength, not just the controller, drives the retention gain.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Solid systems engineering on a real failure mode—sequential edits under 4-bit—with honest absolute numbers, but the headline gaps rest on single-run point estimates and adapted baselines.

read the letter

The thing worth knowing is that this is a careful systems paper on a genuine practical problem: sequential fact edits collapse under 4-bit quantization. They name it the “quantization stability crisis,” confine ROME-style rank-1 updates to the LoRA B matrix, add a simple keyword domain boost, and gate strength with an integral-style “degradation debt” controller. Under matched 4-bit OpenLLaMA-3B + LoRA conditions they get the best numbers in their table—TS 28% (+6 pp) and roughly 11–17% relative less degradation than their adapted baselines—and they are upfront that 28% is not deployment-ready.

What is actually new is the packaging and the empirical isolation under identical quantization constraints, not a new principle. Rank-1 editing, LoRA, and feedback attenuation are known pieces. The degradation-debt formulation and the LoRA-subspace collision argument (51K vs 10.2M params) are clean engineering. The corpus pipeline and the frank limitations section are strengths; they do not hide the single-run design or the adapted baselines.

Soft spots, in proportion: the stress-test concern lands. Table 3 and Fig. 10 are one edit ordering; §7.1 and Limitations say so. Given their own interference story (Fig. 3, cosine-similar keys), order can move cumulative debt and final DS/TS by amounts comparable to the reported gaps, so the controller is not yet cleanly isolated. Baselines are author re-implementations for BitsAndBytes; that is a real caveat, though they at least match backbone, quant, LoRA, and splits. Hyperparameters are hand-set once. Keyword prioritization is crude. None of this makes the central argument incoherent—it makes the deltas provisional.

Math is elementary closed-form rank-1 plus a discrete integrator; citations cover ROME/MEMIT/MEND, QLoRA/GPTQ, and continual learning appropriately. No code/data yet.

This is for people building PEFT editors or quantized domain LLMs in regulated settings, not for anyone hunting a new theory of memory. I would send it to peer review: important enough niche, honest reporting, fixable evidence gaps. Engage if you care about editing under quant; wait for multi-seed and code if you need the deltas to stick.

Referee Report

3 major / 5 minor

Summary. The paper identifies a “quantization stability crisis” in which sequential fact edits on 4-bit LLMs suffer severe generalization collapse and cumulative knowledge degradation, and proposes CACHE-UK to address it. CACHE-UK combines (i) rank-1 updates applied only to the LoRA B matrix (Eqs. 1–2), (ii) keyword-based financial domain scaling of edit strength (Eq. 3), and (iii) a closed-loop Stability Controller that accumulates “degradation debt” and attenuates subsequent edit strength (Eqs. 4–5, Algorithm 1). On a 4-bit OpenLLaMA-3B model domain-adapted with LoRA on a curated UK financial corpus, the method reports near-perfect edit success, 28% test success (+6 pp over the strongest adapted baseline), and an 11–17% relative reduction in preservation-set degradation versus adapted ROME, MEMIT, EasyEdit, and KnowledgeEditor under identical quantization and backbone constraints.

Significance. If the relative gains hold under proper multi-run evaluation, the work would be a useful systems contribution for resource-constrained, regulated deployments where full-precision editing and frequent retraining are impractical. The problem framing (resolution barrier under INT4; interference in sequential edits) is concrete, the LoRA-subspace edit design is a natural fit for QLoRA-style stacks, and the degradation-debt controller is a simple, falsifiable control idea. The authors are appropriately cautious that absolute test success remains low (28%) and that the system is a research prototype. Strengths include an end-to-end documented pipeline (corpus → LoRA adaptation → editing), explicit limitations, and controlled same-backbone comparisons rather than inflated full-precision ceilings. The contribution is incremental engineering rather than a new theoretical result, but it targets a practically relevant gap.

major comments (3)

[§7.1, Table 3, Figure 10] Table 3 and Figure 10 (and §7.1) report the headline +6 pp TS and 11–17% degradation gains from a single edit-ordering run, with no error bars or multi-seed/order replicates. The paper itself notes sequential interference (Fig. 3, §3.2) and that debt trajectories (Eqs. 4–5) depend on when degradation breaches T. Under that dynamics, ordering variance can be comparable to the reported gaps, so the isolation of the Stability Controller is not yet established. Multi-run evaluation with randomized edit orderings (already listed as future work) is load-bearing for the central claim and should be required before acceptance of the quantitative conclusions.
[§5 Baselines; Table 3] Section 5 states that ROME, MEMIT, EasyEdit, and KnowledgeEditor were re-implemented as “functionally equivalent adaptations” for BitsAndBytes 4-bit models, with equivalence checked only by reproducing edit success on standard prompts. ES is ~100% for all methods in Table 3, so that check does not validate generalization or preservation behavior under quantization—the metrics that drive the claimed gaps. Without a clearer fidelity audit (e.g., matching published full-precision behavior on a shared subset before quantizing, or ablations that disable only the debt controller inside the same edit primitive), it remains ambiguous whether gains come from the Stability Controller versus implementation differences in the adapted baselines.
[§4, §7.3, Figure 10] The three components are not ablated. Table 3 and Figures 8–10 compare full CACHE-UK only to external baselines. A minimal 2×2 or leave-one-out (LoRA-space ROME alone; +domain scaling; +debt controller) is needed to support the claim that the Stability Controller is “its most robust effect” and that domain prioritization contributes. Without this, the 11–17% DS reduction cannot be attributed to the named mechanism versus simply editing in the 51K-param B subspace (the main design difference from classical ROME-style full-weight updates).

minor comments (5)

[Figure 2] Figure 2 is labeled “conceptual schematic (not measured data)” yet is used to motivate the crisis; consider moving purely illustrative figures to an appendix or clearly separating them from empirical plots (Figs. 8–11).
[Table 2; §4.3] Hyperparameters in Table 2 (λ, β_uk, β_bank, T, δ, BASE_STRENGTH) are fixed with no sensitivity analysis except batch size (Fig. 12). A short sweep on T and δ would strengthen confidence that the controller is not brittle.
[§4.2, Eq. (3)] Domain prioritization via binary lexicon flags (Eq. 3) is acknowledged as limited in Limitations; a one-sentence quantitative note on how often s_uk/s_bank fire on the edit set would help readers judge its practical impact.
[Abstract; §5; Figure 10] Minor wording/consistency: abstract and §1 use both “11–17%” and “11-17%”; “OpenLLaMA-3B” parameter counts (3B vs 3.4B) should be unified; ensure DS “lower is better” is restated in Table 3 or Fig. 10 caption for skimming readers.
[§2, §8] Related work could briefly situate against other continual/editing-under-PEFT efforts and against RAG (already in Discussion) with one or two additional citations on quantized or adapter-space editing if available.

Circularity Check

0 steps flagged

No significant circularity: empirical systems comparison with held-out metrics and fixed hyperparameters, not a self-reducing derivation.

full rationale

CACHE-UK is an engineering/empirical paper, not a first-principles derivation. The load-bearing claims are relative performance gaps (TS 28% vs ≤22%; DS ~55% vs ~61–66%) under matched 4-bit OpenLLaMA-3B + LoRA conditions. Edit Success, Test Success, and Degradation Score are measured on edit prompts, paraphrases, and a static preservation set described as never used during editing for the reported metrics; hyperparameters (Table 2) are stated once rather than fitted to those headline numbers. The Stability Controller’s use of a preservation-set feedback signal (Eqs. 4–5) is closed-loop control of the quantity it monitors, not a claim that DS is predicted from independent theory—so measuring lower final DS versus unmodulated baselines is not tautological. Rank-1 LoRA updates follow the standard ROME closed form applied to B; domain boosts are keyword heuristics, not outcomes fitted then re-reported as predictions. There is no self-citation chain, uniqueness theorem imported from the authors, or ansatz smuggled via overlapping prior work. Single-run edit-order variance and adapted-baseline fidelity are robustness/fairness concerns, not circularity. steps left empty per honest non-finding.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 2 invented entities

The central comparative claim rests on hand-chosen controller and boost hyper-parameters, the assumption that author-adapted baselines are faithful, and the modelling choice that rank-1 updates inside a rank-16 LoRA subspace sufficiently mitigate 4-bit quantization collision. No formal derivation is offered; the work is empirical.

free parameters (5)

BASE_STRENGTH = 0.1
Base edit strength α before debt attenuation; set to 0.1 by hand (Table 2) and directly scales every update.
T (degradation threshold) = 0.6
Threshold that increments degradation debt; chosen as 0.6 with no sensitivity sweep reported.
δ (debt decay) = 0.5
Per-step debt decay when degradation stays below T; fixed at 0.5.
β_uk, β_bank = 0.3, 0.2
Domain-boost coefficients that scale edit magnitude for UK-finance and banking keywords; set to 0.3 and 0.2.
λ (base enhancement) = 1.0
Overall multiplier on the rank-1 update before domain boosts; set to 1.0.

axioms (4)

domain assumption Factual associations are localized in FFN key-value memories and can be edited by rank-one updates (ROME/MEMIT premise).
Invoked in §2.1 and §4.1 to justify the closed-form ΔW and its LoRA-B analogue.
ad hoc to paper Author re-implementations of ROME, MEMIT, EasyEdit and KnowledgeEditor are functionally equivalent under BitsAndBytes 4-bit quantization.
Stated in §5; required for the claim that measured gaps isolate the Stability Controller.
ad hoc to paper Editing inside the rank-16 LoRA B matrix (51 k params) materially reduces quantization collision relative to full 10.2 M-param weight matrices.
Core design claim of §3.3 and Fig. 4; supported only by conceptual argument and final empirical gaps, not by direct collision measurements.
ad hoc to paper Keyword lexicon matches are a sufficient proxy for financial-domain importance when scaling edit strength.
Component 2 (§4.2); Limitations notes the lack of semantic embeddings.

invented entities (2)

degradation debt D_debt no independent evidence
purpose: Scalar integral-style state that attenuates subsequent edit strength when preservation-set performance falls below threshold T.
Defined by the recurrence in Eq. 4–5; no independent theoretical derivation beyond control-theory analogy.
quantization stability crisis no independent evidence
purpose: Named phenomenon that sequential edits on 4-bit models cause cascading degradation because update magnitude falls below quantization step size.
Introduced in §1 and §3; empirically illustrated by baseline collapse to 12–22% TS, but not given a formal predictive model.

pith-pipeline@v1.2.0-daily-grok45 · 14208 in / 3304 out tokens · 51412 ms · 2026-07-31T12:04:02.154758+00:00 · methodology

0 comments

read the original abstract

Large Language Models (LLMs) deployed in dynamic financial environments face a critical challenge: maintaining factual accuracy as market conditions, regulations, and corporate facts change continuously. While 4-bit quantization enables efficient deployment, it severely limits the viability of sequential memory editing: existing methods undergo catastrophic performance degradation under this "quantization stability crisis." We introduce CACHE-UK (Contextual Adaptive Continual Hybrid Editor for UK Finance), a stability-aware memory editing framework specifically designed for domain-specific, quantized LLMs. CACHE-UK integrates three components: a rank-1 LoRA perturbation mechanism that confines edits to the low-rank adapter subspace, a financial domain prioritization module for content-adaptive edit strength, and a closed-loop Stability Controller that tracks "degradation debt" to prevent catastrophic forgetting across sequential updates. Evaluated on a 4-bit quantized OpenLLaMA-3B model with a curated UK financial corpus of 88,021 documents, CACHE-UK reduces knowledge degradation by 11-17% relative to adapted baselines under identical 4-bit constraints -- its most robust effect -- while attaining the highest test success (generalization) rate observed in our setting (28%, a 6 percentage point improvement over the strongest adapted baseline). These results indicate that stability-aware editing can improve factual maintenance in resource-constrained financial LLM deployments, though absolute generalization rates remain low.

Figures

Figures reproduced from arXiv: 2607.28292 by Anubhav Lakra, Yue Feng.

**Figure 1.** Figure 1: CACHE-UK overview. Stale financial knowledge (left) is updated via domain prioritization, rank-1 LoRA perturbation, and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Figure 2: Conceptual schematic (not measured data): an illus [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** Figure 4: Conceptual illustration of quantization collision risk. [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗

**Figure 5.** Figure 5: Rank-1 LoRA perturbation mechanism. The subject [PITH_FULL_IMAGE:figures/full_fig_p004_5.png] view at source ↗

**Figure 6.** Figure 6: CACHE-UK architecture. An edit request is scored for domain relevance, then applied as a rank-1 perturbation to the LoRA B [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗

**Figure 7.** Figure 7: End-to-end pipeline. From 23.9M raw documents, we curate 88,021 documents, domain-adapt a 4-bit OpenLLaMA-3B with [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗

**Figure 10.** Figure 10: Final Knowledge Degradation Score after sequential [PITH_FULL_IMAGE:figures/full_fig_p007_10.png] view at source ↗

**Figure 8.** Figure 8: Edit Success vs. Test Success across methods. All [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗

**Figure 11.** Figure 11: Stability Controller dynamics over a sequence of edits. [PITH_FULL_IMAGE:figures/full_fig_p007_11.png] view at source ↗

**Figure 9.** Figure 9: Memory Retention Efficiency (TS/ES %). CACHE-UK [PITH_FULL_IMAGE:figures/full_fig_p007_9.png] view at source ↗

**Figure 12.** Figure 12: Batch size sensitivity. Left: edit effectiveness decreases [PITH_FULL_IMAGE:figures/full_fig_p008_12.png] view at source ↗

**Figure 13.** Figure 13: Efficiency map: Test Success vs. editing speed. [PITH_FULL_IMAGE:figures/full_fig_p008_13.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

21 extracted references · 10 linked inside Pith

[1]

yfinance Python library.https:// github.com/ranaroussi/yfinance, 2024

Ran Aroussi. yfinance Python library.https:// github.com/ranaroussi/yfinance, 2024

2024
[2]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

1901
[3]

Editing factual knowledge in language models

Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. InProceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6491– 6506, 2021

2021
[4]

8-bit optimizers via block-wise quanti- zation

Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 8-bit optimizers via block-wise quanti- zation. InInternational Conference on Learning Rep- resentations (ICLR), 2022

2022
[5]

QLoRA: Efficient finetuning of quantized LLMs

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. arXiv:2305.14314, 2023

Pith/arXiv arXiv 2023
[6]

GPTQ: Accurate post- training quantization for generative pre-trained trans- formers

Elias Frantar and Dan Alistarh. GPTQ: Accurate post- training quantization for generative pre-trained trans- formers. arXiv:2210.17323, 2022

Pith/arXiv arXiv 2022
[7]

Transformer feed-forward layers are key-value memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv:2012.14913, 2020

Pith/arXiv arXiv 2012
[8]

Parameter-efficient transfer learning for NLP

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzeb- ski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. arXiv:1902.00751, 2019

Pith/arXiv arXiv 1902
[9]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685, 2021

Pith/arXiv arXiv 2021
[10]

De- mystifying domain-adaptive post-training for financial LLMs

Zhe Ke, Yubo Ming, and Xuan-Phi Nguyen. De- mystifying domain-adaptive post-training for financial LLMs. arXiv:2501.04961, 2025

arXiv 2025
[11]

Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem.Psychology of Learning and Moti- vation, 24:109–165, 1989

1989
[12]

Locating and editing factual associations in GPT

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. InAdvances in Neural Information Process- ing Systems 35 (NeurIPS 2022), pages 17359–17372, 2022

2022
[13]

Mass-editing memory in a transformer

Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv:2210.07229, 2023

Pith/arXiv arXiv 2023
[14]

Fast model editing at scale

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Percy Liang. Fast model editing at scale. arXiv:2110.11309, 2022

Pith/arXiv arXiv 2022
[15]

OpenLLaMA: An open repro- duction of LLaMA.https://github.com/ openlm-research/open_llama, 2023

OpenLM Research. OpenLLaMA: An open repro- duction of LLaMA.https://github.com/ openlm-research/open_llama, 2023

2023
[16]

FinRED: A dataset for relation extraction in fi- nancial domain

Shubham Sharma, Ankur GTM, and Himanshu Beni- wal. FinRED: A dataset for relation extraction in fi- nancial domain. InProceedings of the 31st ACM In- ternational Conference on Information & Knowledge Management, pages 4843–4847, 2022

2022
[17]

Gomez, Łukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Sys- tems 30, pages 5998–6008, 2017

2017
[18]

EasyEdit: An easy-to-use knowl- edge editing framework for large language models

Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Hua- jun Chen. EasyEdit: An easy-to-use knowl- edge editing framework for large language models. arXiv:2308.07269, 2023

Pith/arXiv arXiv 2023
[19]

A comprehensive survey on memory editing for large language models.ACM Computing Surveys, 2024

Shuai Wang, Yichong Zhu, Hongzhan Liu, Pin-Yu Chen, and Lidong Bing. A comprehensive survey on memory editing for large language models.ACM Computing Surveys, 2024. Preprint

2024
[20]

A comprehensive study of knowledge editing for large language models

Yunzhi Yao, Chuanhao Li, Yu-Chen Lin, Shen-Hui Lee, Haitao Mi, Tung-Hsien Chung, and Hung yi Lee. A comprehensive study of knowledge editing for large language models. arXiv:2305.13172, 2024

Pith/arXiv arXiv 2024
[21]

Byun, Zi- fan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zi- fan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top- down approach t...

Pith/arXiv arXiv 2023

[1] [1]

yfinance Python library.https:// github.com/ranaroussi/yfinance, 2024

Ran Aroussi. yfinance Python library.https:// github.com/ranaroussi/yfinance, 2024

2024

[2] [2]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litw...

1901

[3] [3]

Editing factual knowledge in language models

Nicola De Cao, Wilker Aziz, and Ivan Titov. Editing factual knowledge in language models. InProceed- ings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6491– 6506, 2021

2021

[4] [4]

8-bit optimizers via block-wise quanti- zation

Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 8-bit optimizers via block-wise quanti- zation. InInternational Conference on Learning Rep- resentations (ICLR), 2022

2022

[5] [5]

QLoRA: Efficient finetuning of quantized LLMs

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. arXiv:2305.14314, 2023

Pith/arXiv arXiv 2023

[6] [6]

GPTQ: Accurate post- training quantization for generative pre-trained trans- formers

Elias Frantar and Dan Alistarh. GPTQ: Accurate post- training quantization for generative pre-trained trans- formers. arXiv:2210.17323, 2022

Pith/arXiv arXiv 2022

[7] [7]

Transformer feed-forward layers are key-value memories

Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv:2012.14913, 2020

Pith/arXiv arXiv 2012

[8] [8]

Parameter-efficient transfer learning for NLP

Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzeb- ski, Bruna Morrone, Quentin de Laroussilhe, An- drea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. arXiv:1902.00751, 2019

Pith/arXiv arXiv 1902

[9] [9]

Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685, 2021

Pith/arXiv arXiv 2021

[10] [10]

De- mystifying domain-adaptive post-training for financial LLMs

Zhe Ke, Yubo Ming, and Xuan-Phi Nguyen. De- mystifying domain-adaptive post-training for financial LLMs. arXiv:2501.04961, 2025

arXiv 2025

[11] [11]

Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem.Psychology of Learning and Moti- vation, 24:109–165, 1989

1989

[12] [12]

Locating and editing factual associations in GPT

Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. InAdvances in Neural Information Process- ing Systems 35 (NeurIPS 2022), pages 17359–17372, 2022

2022

[13] [13]

Mass-editing memory in a transformer

Kevin Meng, Arnab Sen Sharma, Alex Andonian, Yonatan Belinkov, and David Bau. Mass-editing memory in a transformer. arXiv:2210.07229, 2023

Pith/arXiv arXiv 2023

[14] [14]

Fast model editing at scale

Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Percy Liang. Fast model editing at scale. arXiv:2110.11309, 2022

Pith/arXiv arXiv 2022

[15] [15]

OpenLLaMA: An open repro- duction of LLaMA.https://github.com/ openlm-research/open_llama, 2023

OpenLM Research. OpenLLaMA: An open repro- duction of LLaMA.https://github.com/ openlm-research/open_llama, 2023

2023

[16] [16]

FinRED: A dataset for relation extraction in fi- nancial domain

Shubham Sharma, Ankur GTM, and Himanshu Beni- wal. FinRED: A dataset for relation extraction in fi- nancial domain. InProceedings of the 31st ACM In- ternational Conference on Information & Knowledge Management, pages 4843–4847, 2022

2022

[17] [17]

Gomez, Łukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. InAdvances in Neural Information Processing Sys- tems 30, pages 5998–6008, 2017

2017

[18] [18]

EasyEdit: An easy-to-use knowl- edge editing framework for large language models

Peng Wang, Ningyu Zhang, Xin Xie, Yunzhi Yao, Bozhong Tian, Mengru Wang, Zekun Xi, Siyuan Cheng, Kangwei Liu, Guozhou Zheng, and Hua- jun Chen. EasyEdit: An easy-to-use knowl- edge editing framework for large language models. arXiv:2308.07269, 2023

Pith/arXiv arXiv 2023

[19] [19]

A comprehensive survey on memory editing for large language models.ACM Computing Surveys, 2024

Shuai Wang, Yichong Zhu, Hongzhan Liu, Pin-Yu Chen, and Lidong Bing. A comprehensive survey on memory editing for large language models.ACM Computing Surveys, 2024. Preprint

2024

[20] [20]

A comprehensive study of knowledge editing for large language models

Yunzhi Yao, Chuanhao Li, Yu-Chen Lin, Shen-Hui Lee, Haitao Mi, Tung-Hsien Chung, and Hung yi Lee. A comprehensive study of knowledge editing for large language models. arXiv:2305.13172, 2024

Pith/arXiv arXiv 2024

[21] [21]

Byun, Zi- fan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zi- fan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top- down approach t...

Pith/arXiv arXiv 2023