Uncertainty and correctness in LLMs are encoded by distinct feature populations, with suppression of confounded features improving accuracy and reducing entropy.
Investigating layer importance in large language models
5 Pith papers cite this work, alongside 1 external citations. Polarity classification is still indexing.
5
Pith papers citing it
1
external citations · external index
representative citing papers
A single middle transformer layer trained in isolation recovers most RL post-training gains in LLMs, with gains concentrated in middle layers across models, algorithms, and tasks.
A per-layer, per-task bit-allocation rule based on hidden-activation entropy and variance reportedly matches full-precision TriviaQA accuracy within about one point — but the paper's tables and abstract are internally inconsistent.
RAP is a reinforcement learning framework for runtime-adaptive pruning of LLMs that jointly optimizes model weights and KV-cache usage under varying memory budgets.
citing papers explorer
-
Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders
Uncertainty and correctness in LLMs are encoded by distinct feature populations, with suppression of confounded features improving accuracy and reducing entropy.
-
Is One Layer Enough? Training A Single Transformer Layer Can Match Full-Parameter RL Training
A single middle transformer layer trained in isolation recovers most RL post-training gains in LLMs, with gains concentrated in middle layers across models, algorithms, and tasks.
-
You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations
A per-layer, per-task bit-allocation rule based on hidden-activation entropy and variance reportedly matches full-precision TriviaQA accuracy within about one point — but the paper's tables and abstract are internally inconsistent.
-
RAP: Runtime Adaptive Pruning for LLM Inference
RAP is a reinforcement learning framework for runtime-adaptive pruning of LLMs that jointly optimizes model weights and KV-cache usage under varying memory budgets.
- Statistically-Lossless Quantization of Large Language Models