REVIEW 9 cited by
The Remarkable Robustness of LLMs: Stages of Inference?
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We investigate the robustness of Large Language Models (LLMs) to structural interventions by deleting and swapping adjacent layers during inference. Surprisingly, models retain 72-95% of their original top-1 prediction accuracy without any fine-tuning. We find that performance degradation is not uniform across layers: interventions to the early and final layers cause the most degradation, while the model is remarkably robust to dropping middle layers. This pattern of localized sensitivity motivates our hypothesis of four stages of inference, observed across diverse model families and sizes: (1) detokenization, where local context is integrated to lift raw token embeddings into higher-level representations; (2) feature engineering, where task- and entity-specific features are iteratively refined; (3) prediction ensembling, where hidden states are aggregated into plausible next-token predictions; and (4) residual sharpening, where irrelevant features are suppressed to finalize the output distribution. Synthesizing behavioral and mechanistic evidence, we provide a framework for interpreting depth-dependent computations in LLMs.
Forward citations
Cited by 9 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Divergent large language model predictions from convergent representations in ambiguous word pairs
In three decoder-only LLMs, representations of homonym and polyseme senses reconverge in late layers while next-token predictions diverge, and activation patching shows late-layer states still carry the disambiguating signal.
-
Understanding Is Done Early: A Depth Division of Labor in Large Language Models and Its Use for Unbounded-Context Memory
Long-context memory can be organized by caching intermediate residual states and recomputing only query-conditioned upper layers over a bounded retrieved pack.
-
When Does Sparsity Mitigate the Curse of Depth in LLMs
Implicit and explicit sparsity reduce residual-stream variance and improve layer effectiveness metrics, enabling a depth-scaling recipe with about 4.6 points higher downstream accuracy.
-
Inverse Depth Scaling From Most Layers Being Similar
LLM loss decreases roughly inversely with depth because most layers act as a redundant ensemble that averages errors, not as a compositional hierarchy.
-
Prompt the Unseen: Evaluating Visual-Language Alignment Beyond Supervision
A new benchmark shows VLM projection layers retain most of their alignment accuracy on object classes never seen during alignment training, with mechanistic evidence pointing to FFN key-value memory.
-
Insights into a radiology-specialised multimodal large language model with sparse autoencoders
Applying Matryoshka sparse autoencoders to a radiology-specialised multimodal LLM reveals a minority of interpretable clinical features, while steering them produces unreliable and often off-target report changes.
-
The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network
The paper demonstrates non-monotonic move-policy dynamics in a chess transformer, but its abstract claims a causal safety-prior override result that never appears in the body.
-
Learning to Skip the Middle Layers of Transformers
A gated 'skip the middle' Transformer fails to beat fewer-layer dense baselines on the cross-entropy/FLOPs trade-off at 12 layers and 10B tokens.
Discussion (0). Sign in to comment.