The reviewed record of science sign in
Pith

arxiv: 2305.12740 · v1 · pith:JKHSOTED · submitted 2023-05-22 · cs.CL

Can We Edit Factual Knowledge by In-Context Learning?

Ce Zheng , Lei Li , Qingxiu Dong , Yuxuan Fan , Zhiyong Wu , Jingjing Xu , Baobao Chang This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:JKHSOTEDrecord.jsonopen to challenge →

classification cs.CL
keywords knowledgefactualin-contextllmsediteditinggradient-basedhowever
0
0 comments X
read the original abstract

Previous studies have shown that large language models (LLMs) like GPTs store massive factual knowledge in their parameters. However, the stored knowledge could be false or out-dated. Traditional knowledge editing methods refine LLMs via fine-tuning on texts containing specific knowledge. However, with the increasing scales of LLMs, these gradient-based approaches bring large computation costs. The trend of model-as-a-service also makes it impossible to modify knowledge in black-box LMs. Inspired by in-context learning (ICL), a new paradigm based on demonstration contexts without parameter updating, we explore whether ICL can edit factual knowledge. To answer this question, we give a comprehensive empirical study of ICL strategies. Experiments show that in-context knowledge editing (IKE), without any gradient and parameter updating, achieves a competitive success rate compared to gradient-based methods on GPT-J (6B) but with much fewer side effects, including less over-editing on similar but unrelated facts and less knowledge forgetting on previously stored knowledge. We also apply the method to larger LMs with tens or hundreds of parameters like OPT-175B, which shows the scalability of our method. The code is available at https://github.com/Zce1112zslx/IKE.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Beyond Binary Edits Robust Multimodal Knowledge Editing with Adversarial Subspace Alignment

    cs.AI 2026-05 unverdicted novelty 7.0

    Introduces Latent Adversarial Robustification and Rank-Constrained Subspace Learning to enable robust generalization in multimodal knowledge editing through adversarial subspace alignment.

  2. LOKI: Memory-Free Null-Space Constrained Lifelong Knowledge Editing

    cs.LG 2026-06 unverdicted novelty 6.0

    LOKI enables memory-free lifelong knowledge editing via HSIC-based dynamic layer selection and null-space constrained updates, reporting up to 14% higher average accuracy than prior methods.

  3. Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs

    cs.LG 2026-04 conditional novelty 6.0

    DECODE identifies and separately edits modality-specific neurons in MLLMs to prevent knowledge edits from reverting under unimodal queries.

  4. Representation-Guided Parameter-Efficient LLM Unlearning

    cs.CL 2026-04 unverdicted novelty 6.0

    REGLU guides LoRA-based unlearning via representation subspaces and orthogonal regularization to outperform prior methods on forget-retain trade-off in LLM benchmarks.

  5. Steering LLM Viewpoints through Fabricated Evidence Injection

    cs.CR 2026-06 unverdicted novelty 5.0

    Ghostwriter attack injects fabricated evidence to steer LLM viewpoints, with experiments showing high success on commercial models and partial mitigation on guarded ones.

  6. Improving Factuality in LLMs via Inference-Time Knowledge Graph Construction

    cs.CL 2025-08 unverdicted novelty 5.0

    A framework for inference-time knowledge graph construction and expansion improves factual accuracy in LLMs on three QA benchmarks by combining internal LLM knowledge with selective external retrieval.