REVIEW 15 cited by
Knowledge Neurons in Pretrained Transformers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large-scale pretrained language models are surprisingly good at recalling factual knowledge presented in the training corpus. In this paper, we present preliminary studies on how factual knowledge is stored in pretrained Transformers by introducing the concept of knowledge neurons. Specifically, we examine the fill-in-the-blank cloze task for BERT. Given a relational fact, we propose a knowledge attribution method to identify the neurons that express the fact. We find that the activation of such knowledge neurons is positively correlated to the expression of their corresponding facts. In our case studies, we attempt to leverage knowledge neurons to edit (such as update, and erase) specific factual knowledge without fine-tuning. Our results shed light on understanding the storage of knowledge within pretrained Transformers. The code is available at https://github.com/Hunter-DDM/knowledge-neurons.
Forward citations
Cited by 15 Pith papers
-
Unveiling the Response of Large Vision-Language Models to Visually Absent Tokens
Feed-forward neurons in LVLMs encode whether a text token is visually grounded, and a detector built on these neurons can reduce hallucination by overriding or replacing ungrounded tokens.
-
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Unsigned differential activations locate a few GLU-MLP neurons whose zeroing surgically destabilizes demographic bias while retaining ~99.5% of measured capabilities.
-
Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations
GCC discovers multiple concept-specific neuron circuits per query by combining first-order ablation sensitivity with top-k activation overlap.
-
What Should LLMs Forget? Quantifying Personal Data in LLMs for Right-to-Be-Forgotten Requests
WikiMem, a Wikidata-derived canary dataset and a calibrated NLL-ranking metric, identifies which human-fact associations an LLM has memorized, with higher rates for famous people and larger models.
-
Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis
Shortcut neuron patching suppresses benchmark-contamination shortcuts in LLMs and yields evaluation scores that strongly correlate with the external MixEval benchmark.
-
TRACE for Tracking the Emergence of Semantic Representations in Transformers
Using Hessian curvature, intrinsic dimensionality, and linguistic probes on a synthetic frame-semantic corpus, the paper claims a coordinated intersection-based phase transition in small transformers, though the marke...
-
Break Through the Compression Bottleneck: From Theory to Practice
The paper asserts a first proof that low-rank decomposition and quantization are non-orthogonal tools for LLM compression, recommends low-rank-first ordering, and adds a diagonal scaling fix (DAM) that reduces the com...
-
QF: Quick Feedforward AI Model Training without Gradient Back Propagation
QF Learning updates transformer weights with a closed-form, backprop-free formula so that a model can recall an injected fact from memory after a single instructional example, but the reported evidence is only qualitative.
-
SplitLoRA: Balancing Stability and Plasticity in Continual Learning Through Gradient Space Splitting
SplitLoRA picks the LoRA update subspace size from previous-task gradient singular values using a hyperparameter alpha, and freezes the projection to keep updates in that subspace.
-
A Graph Perspective to Probe Structural Patterns of Knowledge in Large Language Models
LLM knowledge, measured by self-reported true/false checks on knowledge-graph triplets, shows homophily and degree correlations that a graph neural network exploits to select more effective fine-tuning data.
-
Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models
Training-free amplification of selected last-layer activations, combined with 'wait' token insertion, elicits long chain-of-thought reasoning in base LLMs and improves accuracy on math and science benchmarks.
-
Context-Adaptive Inference: A Unified Statistical and Foundation-Model View
Under linear, squared-loss assumptions, explicit context adaptation and in-context learning both reduce to kernel ridge regression on joint input-context features.
-
Rhetorical Text-to-Image Generation via Two-layer Diffusion Policy Optimization
Rhet2Pix combines staged LLM prompt decomposition with a discounted PPO fine-tuning scheme for Stable Diffusion, claiming strong rhetorical text-to-image generation, but the quantitative evidence is circular and undefined.
-
Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM's Instruction-Following Capabilities
Activation-frequency analysis identifies sparse units in LLMs that respond to instructions; same-category instructions share more of these units than different-category ones, and fine-tuning measurably changes the sets.
-
A Survey on Proactive Defense Strategies Against Misinformation in Large Language Models
A survey claims proactive defenses against LLM misinformation outperform post-hoc detection by up to 63%, but no meta-analysis details are provided to support the claim.
Discussion (0). Sign in to comment.