REVIEW 4 cited by
All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Similarity measures are a vital tool for understanding how language models represent and process language. Standard representational similarity measures such as cosine similarity and Euclidean distance have been successfully used in static word embedding models to understand how words cluster in semantic space. Recently, these measures have been applied to embeddings from contextualized models such as BERT and GPT-2. In this work, we call into question the informativity of such measures for contextualized language models. We find that a small number of rogue dimensions, often just 1-3, dominate these measures. Moreover, we find a striking mismatch between the dimensions that dominate similarity measures and those which are important to the behavior of the model. We show that simple postprocessing techniques such as standardization are able to correct for rogue dimensions and reveal underlying representational quality. We argue that accounting for rogue dimensions is essential for any similarity-based analysis of contextual language models.
Forward citations
Cited by 4 Pith papers
-
Metaphor Tracer: A Theory-Informed Analysis of Hidden States
Hidden-state aggregator and differentiator scores, frozen on one text, track within-text organization across models and align with engineered registers and psychoanalytic marks while dissociating from information and ...
-
Divergent large language model predictions from convergent representations in ambiguous word pairs
In three decoder-only LLMs, representations of homonym and polyseme senses reconverge in late layers while next-token predictions diverge, and activation patching shows late-layer states still carry the disambiguating signal.
-
MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
MXSens allocates 8-bit precision to the 32 most sensitive columns per layer, 6-bit to moderately sensitive columns, and 4-bit elsewhere in MXINT, improving WikiText-2 perplexity over prior 4-bit LLM quantization methods.
-
Rethinking Word Similarity: Semantic Similarity through Classification Confusion
Word Confusion measures semantic similarity as classifier confusion between contextual embeddings, matching human judgments as well as or better than cosine similarity, and enables analyst-chosen feature dimensions.
Discussion (0). Continue with ORCID to comment.