REVIEW 5 cited by
Locate-then-edit for Multi-hop Factual Recall under Knowledge Editing
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
The locate-then-edit paradigm has shown significant promise for knowledge editing (KE) in Large Language Models (LLMs). While previous methods perform well on single-hop fact recall tasks, they consistently struggle with multi-hop factual recall tasks involving newly edited knowledge. In this paper, leveraging tools in mechanistic interpretability, we first identify that in multi-hop tasks, LLMs tend to retrieve knowledge with implicit subject information from deeper MLP layers, unlike single-hop tasks, which rely on shallow layers. This distinction explains the poor performance of current methods in multi-hop queries, as they primarily focus on editing shallow layers with single-hop edit prompts, leaving deeper layers unchanged. To address this, we propose IFMET, a novel locate-then-edit KE approach designed to edit both shallow and deep MLP layers. Beyond single-hop editing prompts, IFMET further incorporates multi-hop editing prompts to locate and modify knowledge across different stages of reasoning. Experimental results demonstrate that IFMET significantly improves performance on multi-hop factual recall tasks, overcoming the limitations of previous locate-then-edit methods
Forward citations
Cited by 5 Pith papers
-
CODEMENV: Benchmarking Large Language Models on Code Migration
CODEMENV provides 922 examples and three tasks for evaluating LLMs on cross-version code migration, finding models are much better at migrating old code to new versions (up to 43.84% pass@1) than the reverse.
-
COMPKE: Complex Question Answering under Knowledge Editing
COMPKE is a new benchmark with 11,924 complex questions that tests knowledge editing through one-to-many relations and logical operations, where existing editing methods often fail.
-
Understanding and Mitigating Cross-lingual Privacy Leakage via Language-specific and Universal Privacy Neurons
Cross-lingual privacy leakage in LLMs is driven by a mix of language-universal and language-specific neurons, and deactivating those neurons lowers measured leakage by 23.3% to 31.6%.
-
The Compositional Architecture of Regret in Large Language Models
The paper claims that regret in LLMs is encoded by interacting neuron groups detectable in the final hidden layer, using new S-CDI, RDS, and GIC metrics.
-
Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images
SHE lowers behavioral hallucination scores by about 10 percent by detecting low visual-textual similarity and projecting out the hallucinated direction in embedding space.
Discussion (0). Sign in to comment.