REVIEW 8 cited by
Can We Edit Factual Knowledge by In-Context Learning?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/Zce1112zslx/IKE.
Forward citations
Cited by 8 Pith papers
-
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
DECODE identifies and separately edits modality-specific neurons in MLLMs to prevent knowledge edits from reverting under unimodal queries.
-
Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs
CogEdit and MIND shift multimodal knowledge editing toward evaluating and enabling meta-cognitive skills: self-awareness, boundary monitoring, and noise robustness.
-
Towards a Principled Evaluation of Knowledge Editors
The choice of evaluation metric, generation length, and edit batch size changes the ranking of knowledge editors, and exact string matching produces false positives.
-
COMPKE: Complex Question Answering under Knowledge Editing
COMPKE is a new benchmark with 11,924 complex questions that tests knowledge editing through one-to-many relations and logical operations, where existing editing methods often fail.
-
Break Through the Compression Bottleneck: From Theory to Practice
The paper asserts a first proof that low-rank decomposition and quantization are non-orthogonal tools for LLM compression, recommends low-rank-first ordering, and adds a diagonal scaling fix (DAM) that reduces the com...
-
Mass-Editing Memory with Attention in Transformers: A cross-lingual exploration of knowledge
MEMAT combines MEMIT weight edits with optimized attention-head corrections, improving cross-lingual success and magnitude metrics over MEMIT in English and Catalan.
-
Mitigating Heterogeneous Token Overfitting in LLM Knowledge Editing
OVERTONE replaces uniform cross-entropy targets with adaptively filtered, per-token smoothed targets and improves portability and locality of four knowledge editing methods.
-
NeuralDB: Scaling Knowledge Editing in LLMs to 100,000 Facts with Neural KV Database
NeuralDB edits up to 100,000 facts in an LLM by storing keys and residuals externally and gating retrieval with cosine similarity, preserving general task performance.
Discussion (0). Continue with ORCID to comment.