REVIEW 7 cited by
Jump to Conclusions: Short-Cutting Transformers With Linear Transformations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Transformer-based language models create hidden representations of their inputs at every layer, but only use final-layer representations for prediction. This obscures the internal decision-making process of the model and the utility of its intermediate representations. One way to elucidate this is to cast the hidden representations as final representations, bypassing the transformer computation in-between. In this work, we suggest a simple method for such casting, using linear transformations. This approximation far exceeds the prevailing practice of inspecting hidden representations from all layers, in the space of the final layer. Moreover, in the context of language modeling, our method produces more accurate predictions from hidden layers, across various model scales, architectures, and data distributions. This allows "peeking" into intermediate representations, showing that GPT-2 and BERT often predict the final output already in early layers. We then demonstrate the practicality of our method to recent early exit strategies, showing that when aiming, for example, at retention of 95% accuracy, our approach saves additional 7.9% layers for GPT-2 and 5.4% layers for BERT. Last, we extend our method to linearly approximate sub-modules, finding that attention is most tolerant to this change. Our code and learned mappings are publicly available at https://github.com/sashayd/mat.
Forward citations
Cited by 7 Pith papers
-
Off-Axis, On Purpose: Where a Transformer Computes Concepts and Why it Does So
Intermediate transformer states sit far from the output axis on purpose: that position insulates attention's cross-token mixing, and the frame can be prescribed in advance without loss.
-
How Transformers Reject Wrong Answers: Rotational Dynamics of Factual Constraint Processing
Correct and incorrect single-token continuations of factual queries are separated by rotation of displacement vectors in transformer hidden states, with larger models also suppressing the correct token when forced to ...
-
AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism
AdaDecode speeds up LLM generation by predicting tokens at early layers when confidence is high, running the skipped layers in parallel, and verifying the output exactly matches standard decoding.
-
Discovering Chunks in Neural Embeddings for Interpretability
Recurring 'chunks' in neural embeddings can be extracted, predict input patterns, and be perturbed to steer a model's outputs.
-
One Jump Is All You Need: Short-Cutting Transformers for Early Exit Prediction with One Jump to Fit All Exit Levels
Selecting one pre-trained low-rank shortcut jump and reusing it at every transformer block level gives over 30x parameter savings while largely matching per-level shortcuts in early-exit prediction.
-
Look Ahead Text Understanding and LLM Stitching
LASI predicts the section label of a sentence before it appears, and stitching GPT to BERT with a loss or attention module beats plain BERT by about one accuracy point on PubMed abstracts.
-
Linear Feedback Control Systems for Iterative Prompt Optimization in Large Language Models
A conceptual framework that maps iterative prompt refinement onto PID feedback control, without empirical validation.
Discussion (0). Continue with ORCID to comment.