REVIEW 8 cited by
Future Lens: Anticipating Subsequent Tokens from a Single Hidden State
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
We conjecture that hidden state vectors corresponding to individual input tokens encode information sufficient to accurately predict several tokens ahead. More concretely, in this paper we ask: Given a hidden (internal) representation of a single token at position $t$ in an input, can we reliably anticipate the tokens that will appear at positions $\geq t + 2$? To test this, we measure linear approximation and causal intervention methods in GPT-J-6B to evaluate the degree to which individual hidden states in the network contain signal rich enough to predict future hidden states and, ultimately, token outputs. We find that, at some layers, we can approximate a model's output with more than 48% accuracy with respect to its prediction of subsequent tokens through a single hidden state. Finally we present a "Future Lens" visualization that uses these methods to create a new view of transformer states.
Forward citations
Cited by 8 Pith papers
-
Verbalizable Representations Form a Global Workspace in Language Models
Language models represent their current reasoning in a small, readable set of verbalizable vectors (the J-space) that functions like a global workspace.
-
Transformers Use Causal World Models in Maze-Solving Tasks
Maze-solving transformers store a causal, steerable map of maze connections in sparse features, and activating these features is more effective than removing them.
-
Interpreting Language Model Hidden States at Scale
A single low-rank, sparsely trained lens family decodes residual, attention, and MLP states in models up to 70B parameters, revealing that visible and causally effective locations for a behavior can differ.
-
NITP: Next Implicit Token Prediction for LLM Pre-training
NITP augments standard next-token prediction with implicit semantic prediction in representation space using shallow-layer self-supervision, reporting consistent downstream gains on 0.5B-9B models including 5.7% on MM...
-
Emergent Response Planning in LLMs
Hidden representations of LLM prompts encode global attributes of the upcoming response, and simple probes can predict length, content choices, and answer confidence before generation begins.
-
Understanding the learned look-ahead behavior of chess neural networks
The Leela Chess Zero policy network encodes information about destination squares of moves up to seven plies ahead, with attention heads that copy future-square information backward in time in a pattern-dependent way.
-
Discovering Chunks in Neural Embeddings for Interpretability
Recurring 'chunks' in neural embeddings can be extracted, predict input patterns, and be perturbed to steer a model's outputs.
-
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense
Layer-AdvPatcher edits 'toxic' transformer layers using self-generated harmful examples to block jailbreaks, but its reported attack-success rates worsen on several benchmarks.
Discussion (0). Continue with ORCID to comment.