REVIEW 3 major objections 4 minor 20 references
The Geometries of Truth Are Orthogonal Across Tasks
T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Linear probes of truthfulness do not transfer across tasks because each task has its own nearly orthogonal direction.
desk verdict Negative transfer result is solid; the 'orthogonal geometries' explanation needs a noise-floor control before it carries the weight the title gives it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument is carried by linear logistic-regression probes trained on the final-token hidden states of an LLM, compared through cosine similarity of their weight vectors and through support overlap of L1-regularized sparse probes. A constrained optimization that restricts a target probe to the linear span of probes from other tasks quantifies whether target directions can be recovered from other tasks. The paper also uses t-SNE visualizations to show that different tasks form distinct clusters in hidden space, with the correct-versus-incorrect distinction secondary to task boundaries, and tests conformal prediction as a conservative deployment strategy.
What would settle it
Train the same probes on the same seven datasets but canonicalize each weight vector to unit norm (or fix the decision boundary scale via Platt scaling) before computing cosine similarities; if typical pairwise cosines rise above 0.5, the near-orthogonality reported in the paper is an artifact of probe scaling rather than a property of the model's geometry.
Extended reading notes
Core claim
The central discovery is that truthfulness directions in LLM hidden states are largely orthogonal across tasks, and this orthogonality explains why linear probes fail to generalize. The paper shows that probes trained on different tasks share little cosine similarity, that L1-regularized sparse probes have nearly disjoint dimension supports, and that even training on task mixtures or using mixture-of-probes architectures cannot recover a usable direction for a held-out task. In particular, the optimal direction for a target task lies outside the linear span of directions learned on other tasks. The authors conclude that LLMs likely have multiple geometries of truth that are irreconcilable and highly task-dependent.
Load-bearing premise
The whole argument depends on measuring probe similarity with cosine similarity, yet logistic-regression probes are only determined up to a positive rescaling; if that scaling, rather than the model's geometry, is causing the low cosine values, the paper's central conclusion fails.
Editorial extensions
If this is right
- A probe trained on one question-answering dataset should not be trusted to detect hallucinations on a different domain; its direction only fits tasks whose geometry it shares.
- Multi-task training, parameter summation, and mixture-of-probes architectures do not recover a universal truthfulness direction, so complexity alone will not fix the transfer failure.
- Transfer works only between semantically similar tasks, so practitioners need a task-similarity map before reusing a probe.
- Conformal prediction can cap the false-positive rate in cross-task settings, but only by setting thresholds so conservative that recall drops dramatically.
- Sparse probes provide an interpretable diagnostic: the near-disjoint supports show which hidden dimensions each task's truth signal actually uses.
Reading between the lines
- If these geometries are genuinely task-specific, then inference-time steering methods that nudge generations toward truth may only steer within the task they were trained on, and may need task-conditioned directions to work elsewhere.
- The same probe-comparison methodology could be applied to other binary properties of LLM outputs, such as toxicity or reasoning correctness, to test whether task-specific orthogonal encoding is a general property of hidden states.
- A testable prediction follows: augmenting a probe with task identity as an input feature should recover a large share of the cross-task performance gap if the orthogonality is driven by task clusters, and should fail to do so if the gap has another cause.
- The near-disjoint supports suggest that dimensions in the hidden space are functionally allocated per task, raising the question of whether a shared low-dimensional truth subspace exists but is rotated by task-specific transformations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies whether linear 'geometry of truth' probes for LLM answer correctness generalize across tasks. Using Qwen 2.5 7B, Phi-4 Mini, and Llama 3.1 8B and seven QA/math datasets, it reports that probes trained on one task drop in AUROC on other tasks; that L2 probe weight vectors have low cosine similarity; that L1-regularized probes have nearly disjoint supports; and that hidden states cluster by task in t-SNE plots. It further shows that training on mixtures of tasks, summing probe parameters, and a 16-expert mixture-of-probes architecture do not recover task-specific probe performance, and evaluates conformal prediction as a conservative deployment alternative. The paper concludes that LLMs have multiple task-dependent, mutually orthogonal truthfulness geometries.
Significance. The empirical mapping of cross-task transfer failures across three model families, multiple layers and token positions, and seven datasets is a valuable contribution; five-run averages and the extensive appendix make the non-transfer result credible. If the paper were to establish the proposed noise-floor controls, the orthogonality mechanism would be an important qualification to the universal 'geometry of truth' literature. As it stands, the robustly supported claim is that linear probes do not transfer; the geometric explanation (orthogonality of task-specific truth subspaces) is plausible but not yet separated from estimator noise. The conformal-prediction comparison is a useful practical addition.
major comments (3)
- [Section 3, 'Underlying geometry'] The central geometric evidence in Figures 2 and 6 is the low cosine similarity and low support overlap between probes trained on different tasks, compared against the diagonal value of 1. The paper explicitly claims that these patterns reflect 'fundamental differences ... rather than mere statistical artifacts of limited training data,' but no same-task noise floor is reported. In d=3584 with L1-regularized probes, both cosine similarity and support overlap can be driven down by finite-sample estimation noise, by the sparsity of the supports, or by label noise. Please report (a) cosine similarity and support overlap between probes trained on independent subsamples of the same task, and (b) the same quantities for probes trained on shuffled labels. If these control values are comparable to the observed cross-task values in Figures 2 and 6, the orthogonality conclusion is not supported.
- [Section 4.1, constrained-span test] The span test is interpreted as showing that 'target task directions lie outside the subspace generated by directions of the other tasks,' but the span is generated by six individually noisy probe vectors. Underperformance of the constrained fit only shows that the target direction is not well approximated by these six vectors; it does not establish that the target direction lies outside a task-specific truth subspace. Add a control in which the target probe is reconstructed from six same-task re-estimates, or in which the span test is run with same-task repeat probes, to calibrate how much of the constrained-fit gap is attributable to estimator instability rather than to geometric orthogonality.
- [Section 3, 'Sparse probes'] The claim that probe supports are 'nearly disjoint' is not yet evidence for orthogonality. For high-dimensional sparse vectors, two support sets chosen independently at random will have very low overlap simply because the ambient dimension is large relative to the support sizes. The paper reports support overlap percentages but does not report support sizes or an expected-overlap value under a null model with the same sparsity levels. Please provide a random-support or same-task-repeat baseline for Figure 6 so that the observed 4-15% overlaps can be judged against a meaningful null.
minor comments (4)
- [Section 2] There are several typos and formatting issues: 'a mixture of tasks night help' should be 'might help'; 'there are paper that report' should be 'there are papers that report'; and the reference 'V ovk et al.' contains an unintended space.
- [Figure 3 and related scatter plots] The reported p-values for the cosine-similarity/AUROC-difference correlation treat the 21 task pairs as independent, but the pairs share tasks and are not independent. Please report a cluster- or permutation-based significance estimate, or present the correlation as purely descriptive.
- [Title and abstract] The word 'orthogonal' is stronger than the measurements support: most cosine similarities in Figure 2 are in the 0.2-0.5 range rather than near zero. Consider using 'near-orthogonal' or 'low-similarity' throughout, or justify the 0.5 threshold used to define near-orthogonality.
- [Figures 7, 26, and similar t-SNE plots] The axis labeled 'Datasets' is unclear; t-SNE axes should be described or removed, and the color/legend mapping between dataset names and the 'Prediction right/wrong' panels should be clarified.
Circularity Check
No significant circularity: the cross-task orthogonality is measured from fitted probes and public benchmarks, not derived from the paper's conclusions.
full rationale
The paper's central claims are empirical measurements on fitted probes. Section 3 computes cosine similarities between L2-regularized logistic-regression weight vectors (Eq. 1) and support overlap between L1-regularized sparse probes (Eq. 2); these are descriptive statistics of the fitted objects, not quantities that are presupposed by the conclusion that truthfulness directions are task-specific. Section 4.1's constrained-span experiment refits an L2-regularized logistic loss over coefficients alpha in the span of the other probes; the suboptimality of that constrained fit relative to unconstrained task-specific training is an experimental finding, not an identity. No fitted parameter is renamed as a prediction, and no equation reduces the output to the input. The paper's self-citations (Xiong et al. 2024 and Santilli et al. 2025, both including author Kirchhof) appear only as background on linear-probe practice and LLM-as-a-judge labeling; they are not load-bearing for the orthogonality claim, and no uniqueness theorem or ansatz is imported from them. The absence of a same-task noise floor is a legitimate validity concern for the mechanistic interpretation of low cosine similarity, but it does not make any step circular. Overall, the derivation chain is self-contained with respect to the experimental setup, so the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- L1 regularization strength for sparse probes =
tuned per task on a held-out validation set
- L2 regularization strength for dense probes =
tuned per task via cross-validation
- Mixture-of-probes hyperparameters (learning rate, weight decay, auxiliary loss coefficient, number of experts) =
grid-searched on validation, with an oracle setting
- Conformal threshold alpha =
0.3
assumptions (4)
- domain assumption LLM-as-a-judge annotations correctly label generated answers as right or wrong.
- domain assumption Cosine similarity between L2-regularized probe weight vectors measures geometric alignment of truthfulness directions.
- domain assumption The final-token hidden state at the chosen layer is a sufficient representation for answer correctness.
- domain assumption t-SNE cluster separation reflects genuine task structure rather than projection artifacts.
Cite this review
Pith. "Pith review of The Geometries of Truth Are Orthogonal Across Tasks." pith.science (2026). https://pith.science/paper/WYJCJTC2
@misc{pith2026250608572,
author = {Pith},
title = {Pith review of: The Geometries of Truth Are Orthogonal Across Tasks},
year = {2026},
howpublished = {\url{https://pith.science/paper/WYJCJTC2}},
note = {Machine review of arXiv:2506.08572}
}
read the original abstract
Large Language Models (LLMs) have demonstrated impressive generalization capabilities across various tasks, but their claim to practical relevance is still mired by concerns on their reliability. Recent works have proposed examining the activations produced by an LLM at inference time to assess whether its answer to a question is correct. Some works claim that a "geometry of truth" can be learned from examples, in the sense that the activations that generate correct answers can be distinguished from those leading to mistakes with a linear classifier. In this work, we underline a limitation of these approaches: we observe that these "geometries of truth" are intrinsically task-dependent and fail to transfer across tasks. More precisely, we show that linear classifiers trained across distinct tasks share little similarity and, when trained with sparsity-enforcing regularizers, have almost disjoint supports. We show that more sophisticated approaches (e.g., using mixtures of probes and tasks) fail to overcome this limitation, likely because activation vectors commonly used to classify answers form clearly separated clusters when examined across tasks.
Figures
Figures from the paper (72 more)
Reference graph
Works this paper leans on
-
[1]
Azaria, A. and Mitchell, T. The internal state of an llm knows when it’s lying. In The 2023 Conference on Em- pirical Methods in Natural Language Processing ,
work page 2023
-
[3]
URL https://openreview. net/forum?id=ETKGuby0hcs. Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168,
-
[6]
URL https://openreview. net/forum?id=aajyHYjjsk. Nentidis, A., Katsimpras, G., Krithara, A., L ´opez, S. L., Farr´e-Maduell, E., Gasco, L., Krallinger, M., and Paliouras, G. Overview of bioasq 2023: The eleventh bioasq challenge on large-scale biomedical semantic in- dexing and question answering. arXiv preprint arXiv: 2307.05131,
work page Pith review arXiv 2023
-
[11]
Qwen authors. Qwen2.5 technical report. arXiv preprint arXiv: 2412.15115,
-
[12]
Squad: 100,000+ questions for machine comprehension of text
Rajpurkar, P., Zhang, J., Lopyrev, K., and Liang, P. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , pp. 2383– 2392,
work page 2016
-
[15]
Slobodkin, A., Goldman, O., Caciularu, A., Dagan, I., and Ravfogel, S. The curious case of hallucinatory (un) an- swerability: Finding truths in the hidden states of over- confident large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 3607–3625,
work page 2023
-
[16]
The Llama 3 authors. The llama 3 herd of models. arXiv preprint arXiv: 2407.21783,
-
[18]
URL https://doi.org/10. 48550/arXiv.2406.00034. Wei, J., Karina, N., Chung, H. W., Jiao, Y . J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. Measuring short- form factuality in large language models. arXiv preprint arXiv: 2411.04368,
Show all 20 references
-
[19]
Efficient and effective uncertainty quan- tification for LLMs
Xiong, M., Santilli, A., Kirchhof, M., Golinski, A., and Williamson, S. Efficient and effective uncertainty quan- tification for LLMs. In Neurips Safe Generative AI Workshop 2024,
2024
-
[20]
Zheng, L., Chiang, W.-L., Sheng, Y ., Zhuang, S., Wu, Z., Zhuang, Y ., Lin, Z., Li, Z., Li, D., Xing, E., et al
URL https://arxiv.org/abs/2504.05419. Zheng, L., Chiang, W.-L., Sheng, Y ., Zhuang, S., Wu, Z., Zhuang, Y ., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Ad- vances in Neural Information Processing Systems , 36: 46595–46623,
-
[168]
naacl-main.168/
URL https://aclanthology.org/2021. naacl-main.168/. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V ., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V ., Vanderplas, J., Passos, A., Cour- napeau, D., Brucher, M., Perrot, M., and Duchesnay, E...
2021
-
[2005]
Adaptive activation steering: A tuning-free llm truthfulness improvement method for diverse hallucinations categories
Wang, T., Jiao, X., He, Y ., Chen, Z., Zhu, Y ., Chu, X., Gao, J., Wang, Y ., and Ma, L. Adaptive activation steering: A tuning-free llm truthfulness improvement method for diverse hallucinations categories. CoRR, abs/2406.00034,
-
[2011]
Phi-4 technical report
Phi-4 authors. Phi-4 technical report. arXiv preprint arXiv: 2412.08905,
-
[2016]
Revis- iting uncertainty quantification evaluation in language models: Spurious interactions with response length bias results
Santilli, A., Golinski, A., Kirchhof, M., Danieli, F., Blaas, A., Xiong, M., Zappella, L., and Williamson, S. Revis- iting uncertainty quantification evaluation in language models: Spurious interactions with response length bias results. arXiv preprint arXiv:2504.13677,
-
[2017]
doi: 10.18653/v1/P17-1147
Association for Computational Lin- guistics. doi: 10.18653/v1/P17-1147. URL https: //aclanthology.org/P17-1147/. Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. Language models ...
-
[2021]
doi: 10.18653/v1/2021.naacl-main
Association for Computa- tional Linguistics. doi: 10.18653/v1/2021.naacl-main
2021 doi
-
[2022]
Patel, A., Bhattamishra, S., and Goyal, N. Are NLP models really able to solve simple math word prob- lems? In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cot- terell, R., Chakraborty, T., and Zhou, Y . (eds.), Pro- ceedings of the...
2021
-
[2023]
Internalinspector i2: Robust confidence estimation in llms through internal states
Beigi, M., Shen, Y ., Yang, R., Lin, Z., Wang, Q., Mohan, A., He, J., Jin, M., Lu, C.-T., and Huang, L. Internalinspector i2: Robust confidence estimation in llms through internal states. In Findings of the Association for Computational Linguistics: EMNLP 2024 , pp. 12847–12865,
2024
-
[2024]
Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.(2021)
Fedus, W., Zoph, B., and Shazeer, N. Switch transformers: Scaling to trillion parameter models with simple and effi- cient sparsity.(2021). arXiv preprint cs.LG/2101.03961,
2021 arXiv
-
[2025]
Do an- droids know they’re only dreaming of electric sheep? In Findings of the Association for Computational Linguistics ACL 2024, pp
Sky, C.-W., Van Durme, B., Eisner, J., and Kedzie, C. Do an- droids know they’re only dreaming of electric sheep? In Findings of the Association for Computational Linguistics ACL 2024, pp. 4401–4420,
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.