A four-part audit across six frozen LLMs shows that linear decodability of geometric constraints does not imply reliable generation, activation-level influence, or steerability.
No Training Required: Exploring Random Encoders for Sentence Classification
4 Pith papers cite this work. Polarity classification is still indexing.
4
Pith papers citing it
abstract
We explore various methods for computing sentence representations from pre-trained word embeddings without any training, i.e., using nothing but random parameterizations. Our aim is to put sentence embeddings on more solid footing by 1) looking at how much modern sentence embeddings gain over random methods---as it turns out, surprisingly little; and by 2) providing the field with more appropriate baselines going forward---which are, as it turns out, quite strong. We also make important observations about proper experimental protocol for sentence classification evaluation, together with recommendations for future research.
representative citing papers
Reasoning in LLMs produces a transient geometric pulse in which concept manifolds untangle into linearly separable subspaces immediately before computation and compress afterward.
LLM hidden states encode enough information to predict chain-of-thought success before any reasoning tokens are generated, outperforming a text-only classifier.
Four binarization strategies turn continuous sentence embeddings into binary form, cutting storage by over 98% with only about 2% performance drop on downstream tasks.
citing papers explorer
-
Encoded but Not Actionable: Auditing the Decode-Generate-Steer Gap in Frozen LLMs for Geometric Constraints
A four-part audit across six frozen LLMs shows that linear decodability of geometric constraints does not imply reliable generation, activation-level influence, or steerability.
-
Emergent Manifold Separability during Reasoning in Large Language Models
Reasoning in LLMs produces a transient geometric pulse in which concept manifolds untangle into linearly separable subspaces immediately before computation and compress afterward.
-
Knowing Before Saying: LLM Representations Encode Information About Chain-of-Thought Success Before Completion
LLM hidden states encode enough information to predict chain-of-thought success before any reasoning tokens are generated, outperforming a text-only classifier.
-
Learning Compressed Sentence Representations for On-Device Text Processing
Four binarization strategies turn continuous sentence embeddings into binary form, cutting storage by over 98% with only about 2% performance drop on downstream tasks.