REVIEW 2 major objections 3 minor 1 cited by
Prediction is not Explanation: Revisiting the Explanatory Capacity of Mapping Embeddings
T0 review · 2 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Word embeddings can predict random targets, so accuracy alone does not prove they encode meaning.
desk verdict A useful challenge to feature-norm probing, but the abstract's central claim rests on a random-target control and an algorithmic upper bound that cannot be assessed without the full text. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the mapping procedure that learns a function from word embedding vectors to collections of human-interpretable semantic features (feature norms). The paper's key mechanism is the 'algorithmic upper bound': a limit on prediction accuracy that arises from the mapping method itself and from statistical properties of the target set, independent of semantic content. This bound makes random targets predictable, so the method cannot distinguish meaningful semantic encoding from noise.
What would settle it
A study could construct random target sets that exactly match the statistical profile of a real feature norm (same number of features, same per-feature positive rate, same word frequency distribution, and same inter-annotator agreement) and then compare prediction accuracy. If real features are predicted substantially better than these matched random targets across many embeddings, the paper's central claim would be refuted.
Extended reading notes
Core claim
The central claim is that prediction accuracy of mapping embeddings onto semantic feature norms is not reliable evidence of interpretability. The authors demonstrate that these methods achieve high accuracy even when the target features are random, concluding that results are predominantly determined by an algorithmic upper bound rather than by semantic representation in the embeddings. They further argue that such mappings primarily reflect geometric similarity within vector spaces, not the genuine emergence of semantic properties. If correct, this invalidates prior conclusions drawn from feature-prediction benchmarks about what embeddings know.
Load-bearing premise
The paper assumes that the random information used as a control is statistically comparable to real semantic features in all ways that affect prediction difficulty, such as sparsity, frequency, dimensionality, and annotator agreement.
Editorial extensions
If this is right
- Accuracy on semantic feature norms should no longer be treated as proof that embeddings encode those features.
- Benchmark comparisons between feature datasets based on prediction performance alone are unreliable indicators of which dataset is better captured by the embeddings.
- Future interpretability evaluations should include randomized control targets matched to real features to separate algorithmic upper bounds from genuine semantic signal.
- The finding extends caution to any probing or mapping method that reports accuracy without a null baseline.
- Claims about emergent semantic knowledge in LLMs that rely on feature-prediction probes may need to be revisited.
Reading between the lines
- The same algorithmic-upper-bound critique likely applies to more complex nonlinear probes, where overfitting to random labels is an established risk, so the paper's argument strengthens the case for stricter null models throughout interpretability research.
- A testable extension would be to check whether the upper bound scales with the dimension of the embedding space: if higher-dimensional embeddings predict random targets even better, the geometric-similarity explanation gains support.
- The authors' reasoning implies that feature norms with low inter-annotator agreement may be particularly prone to producing spurious high accuracy, since their signal is closer to noise to begin with.
- One could operationalize 'semantic encoding' as the excess accuracy over a well-matched random baseline; this paper provides a motivation for adopting that metric in future work.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper challenges the widely used assumption that high prediction accuracy when mapping word embeddings onto human-interpretable semantic feature norms demonstrates that those features are genuinely encoded in the embeddings. It reports that the same mapping methods can "successfully predict even random information" and argues that observed accuracy is mostly determined by an "algorithmic upper bound" rather than by semantic content. It concludes that comparisons between datasets based solely on prediction performance are unreliable and that such mappings mainly reflect geometric similarity in vector space.
Significance. If the central claim is fully substantiated, the paper would have a substantial impact on interpretability evaluations of word embeddings and LLMs: it would invalidate a common evidential inference and require the community to redesign feature-norm probing experiments. The proposed random-information control is a sensible and potentially powerful experimental design, and the paper's explicit focus on a widely used but rarely challenged assumption is a strength. However, the abstract alone provides no data, no error bars, no comparison protocol, and no derivation of the algorithmic upper bound, so the significance is conditional on the full manuscript delivering on these points.
major comments (2)
- [Abstract] The central inference that "successful prediction of random information" implies that results are dominated by an algorithmic upper bound depends on the random targets being statistically matched to the real feature norms in every respect that affects prediction difficulty, including dimensionality, per-feature sparsity, value distribution, word-frequency confounds, and annotator agreement noise. The abstract does not state that such matching was performed. If the random targets are denser, lower-dimensional, or less noisy than the real semantic features, then high accuracy on random targets would be expected and would not show that the real feature accuracies are semantically uninformative. Please specify the exact construction of the random baseline and report the matching statistics in the manuscript; otherwise the main conclusion does not follow from the reported result.
- [Abstract] The claim that results are "predominantly determined by an algorithmic upper bound" appears to be offered as an explanation of the observed accuracies. For this explanation to be non-circular, the upper bound must be derived independently of the particular datasets and fitted accuracies it is used to explain, for example from the mathematical properties of the mapping algorithm and the target distribution. If the bound is estimated or fitted from the same experimental results, then the statement becomes a reformulation of the data rather than an explanation. The abstract provides no indication of how the bound was obtained, so the load-bearing derivation must be clearly presented in the full text.
minor comments (3)
- [Abstract] The phrase "random information" is ambiguous: it should be clarified what exactly is randomized (the feature labels, the feature values, the word-feature associations, or some combination) and at what level (features, items, or splits).
- [Abstract] The abstract does not state whether the random baseline is evaluated under the same protocol as the semantic feature experiments, including identical train/test splits, mapping dimensionality, and regularization; this should be stated to reassure readers that the comparison is apples-to-apples.
- [Abstract] The sentence about mappings "primarily reflecting geometric similarity within vector spaces" is a key interpretive claim but is not elaborated in the abstract; a brief explanation of the geometric mechanism would help readers understand the proposed alternative explanation.
Circularity Check
No circularity identified; the random-information control is an external empirical baseline, not a restatement of the conclusion.
full rationale
This review is based solely on the abstract, as the full text is not available. The abstract's central argument is empirical: mapping methods are applied to feature norms and to random information, and the authors report that accuracy on random targets is comparable to accuracy on real semantic features. That design uses random targets as an external control, so the conclusion that the results are predominantly determined by an algorithmic upper bound rather than meaningful semantic representation is not derived by defining the prediction target in terms of the conclusion. No equation is quoted in the abstract, and no cited prior result by the authors is invoked to forbid alternatives or to supply a load-bearing premise. The abstract does not contain a fitted parameter later renamed as a prediction, nor a self-definitional construction. Concerns about whether the random baseline is statistically matched to real feature norms, or whether the algorithmic upper bound is derived independently, are substantive empirical validity questions, but under the stated hard rules they are not circularity: they are not exhibited reductions to the paper's own inputs. Without a specific quoted step that makes an output equal to an input by construction, the appropriate finding is no circularity.
Assumptions & free parameters
assumptions (2)
- domain assumption Random feature baselines match real feature norms in statistical difficulty, distribution, and dimensionality.
- domain assumption The algorithmic upper bound is a property of the embedding-mapping pipeline and can be computed independently of target semantics.
Cite this review
Pith. "Pith review of Prediction is not Explanation: Revisiting the Explanatory Capacity of Mapping Embeddings." pith.science (2026). https://pith.science/paper/QJPPFP22
@misc{pith2026250813729,
author = {Pith},
title = {Pith review of: Prediction is not Explanation: Revisiting the Explanatory Capacity of Mapping Embeddings},
year = {2026},
howpublished = {\url{https://pith.science/paper/QJPPFP22}},
note = {Machine review of arXiv:2508.13729}
}
read the original abstract
Understanding what knowledge is implicitly encoded in deep learning models is essential for improving the interpretability of AI systems. This paper examines common methods to explain the knowledge encoded in word embeddings, which are core elements of large language models (LLMs). These methods typically involve mapping embeddings onto collections of human-interpretable semantic features, known as feature norms. Prior work assumes that accurately predicting these semantic features from the word embeddings implies that the embeddings contain the corresponding knowledge. We challenge this assumption by demonstrating that prediction accuracy alone does not reliably indicate genuine feature-based interpretability. We show that these methods can successfully predict even random information, concluding that the results are predominantly determined by an algorithmic upper bound rather than meaningful semantic representation in the word embeddings. Consequently, comparisons between datasets based solely on prediction performance do not reliably indicate which dataset is better captured by the word embeddings. Our analysis illustrates that such mappings primarily reflect geometric similarity within vector spaces rather than indicating the genuine emergence of semantic properties.
Forward citations
Cited by 1 Pith paper
-
From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery
The paper proposes Mechanistic World Models — models organized as typed latent variables, a reusable mechanism library, and binding structures — as the route from AI forecasting to autonomous discovery.
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.