Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the 'truth direction' — a linear feature separating true from false statements in an LLM's internal activations — is not universal but emerges only in capable models, where simple probes can find it and use it…

desk verdict A useful empirical study with a real generalization result, but the capability claim rests on an AUROC=0 interpretation that needs fixing before it lands. read the letter →

arxiv 2506.00823 v1 pith:POZR2MJ3 submitted 2025-06-01 cs.CL

Yuntai Bao , Xuhong Zhang , Tianyu Du , Xinkui Zhao , Zhengwen Feng , Hao Peng , Jianwei Yin This is my paper
classification cs.CL
keywords truthdirectionlinearprobingtruthfulnesslargelanguagemodelsinternalrepresentationslogicalnegationgeneralizationselectivequestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the 'truth direction' — a single linear direction in a model's internal activations that separates true statements from false ones — is not a universal feature of large language models, but one that emerges only in sufficiently capable models. It argues that when such a direction exists, simple supervised probes (logistic regression, linear SVM, and the mass-mean direction) are enough to find it, contradicting earlier work that blamed failed probes for apparent inconsistency. It further claims that a probe trained only on simple factual statements generalizes across logical negation, conjunction, and disjunction, and across question answering, in-context learning, and context-grounded truthfulness. If this is right, a single linear readout of an LLM's internal activations could serve as a general truthfulness classifier, and probe confidence could filter unreliable answers in real applications.

What carries the argument

The central object is the truth direction: the normal vector of a hyperplane fitted to the activations of the final token position in a chosen decoder layer of the LLM, so that true and false statements fall on opposite sides. The paper identifies the layer by the ratio of between-class to within-class variance of true and false activations across layers, and instantiates the probe as logistic regression, a margin-maximizing linear SVM with Platt scaling, the mass-mean direction, and an MLP baseline. This machinery matters because it lets the authors separate 'does the model represent truth linearly?' from 'did the probe find the direction?', and then test transfer by training on one syntactic form (declarative atomic statements) and evaluating on another (negations, conjunctions, disjunctions, question-answer pairs, and context-grounded passages).

What would settle it

Retrain the probes on the weak models that scored near zero (for example, Llama-2-7B on most topics), then flip the sign of each probe's decision boundary before testing on negated statements; if a substantial fraction of topics then exceed AUROC of 0.5, the reported absence of a consistent truth direction in weak models would be overturned. Conversely, if the flipped probes also stay at chance, the paper's interpretation is confirmed.

Watch

Extended reading notes

Core claim

The paper's central claim is that a consistent truth direction is a property of the target model, not of the probe: across a model ladder from Llama-2-7B to Llama-3.1-70B-Instruct, the ability of truthfulness probes trained on affirmative statements to classify their logical negations rises with general model capability, reaching success on all six knowledge topics only for the most capable instruction-tuned model. The authors therefore conclude that earlier reported generalization failures reflect an internal inconsistency in weaker LLMs, not a limitation of probing technique. They further report that probes trained on atomic factual statements transfer to logical conjunctions and disjunctions, to multiple-choice and short-form QA (MMLU, TriviaQA), to few-shot in-context settings even when exemplars are wrong, and to context-grounded tasks (SciQ, BoolQ, XSum), and that a randomly initialized model exhibits no such direction, showing the direction is a product of pretraining. Finally, they demonstrate selective QA: keeping only answers the probe scores as true raises accuracy from 55.29% to 64.06% on the accepted subset.

Load-bearing premise

The load-bearing assumption is the scoring rule: a probe is credited with finding a truth direction only when its AUROC exceeds 0.5, whereas a score near zero is read as the absence of structure; if a near-zero score instead reflects a consistently reversed direction, the conclusion that weaker models lack consistent truth directions would not follow from the data.

Editorial extensions

If this is right

  • Consistent truth directions scale with capability: probes generalize across negation on zero of six topics for Llama-2-7B, four for Llama-2-13B-Chat and Llama-3.1-8B-Instruct, five for Llama-3.1-70B, and all six for Llama-3.1-70B-Instruct.
  • Probe design becomes secondary in capable models: differences among LR, MLP, SVM, and MM probes are negligible from Llama-2-13B-Chat onward, so sophisticated probing is not required to find a distinctly represented truth direction.
  • A single probe trained on atomic factual statements transfers to logical conjunctions and disjunctions, multiple-choice and short-form QA, few-shot in-context learning (even with false exemplars), and context-grounded hallucination detection.
  • Truthfulness probes can be used for selective QA: filtering sampled answers by probe confidence raises accuracy from 55.29% to 64.06% on the accepted subset, showing a path to improving user trust in LLM-generated answers.
  • The direction is not a probe artifact: a randomly initialized Llama-3.1-8B yields near-chance AUROC (0.50 to 0.58) on the same probing pipeline, whereas the pretrained model reaches 1.0.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not run: check whether the near-zero AUROC values it logs for weak models (Figure 3) correspond to a consistently sign-flipped truth direction; if flipping the probe's decision boundary recovers above-chance AUROC, the claim that weak models lack consistent truth directions would need revision.
  • If truth directions are a capability-linked emergent property, probing could serve as a lightweight, task-independent proxy for model maturity, potentially predicting when a model is amenable to latent-knowledge elicitation without running full benchmark suites.
  • The authors' dichotomy of AUROC above versus at or below 0.5 discards geometric information: a strong anti-correlation is still strong linear structure, just with the wrong sign, so the geometry of truth may be present in weak models but not aligned with the statement's surface polarity.
  • The demonstrated transfer to context-grounded tasks suggests the same readout generalizes to 'grounded correctness' (faithfulness to provided context) beyond parametric factuality; this is worth testing directly on long-form generation, which the authors note they did not do.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates whether LLMs encode truthfulness as a linear 'truth direction' in the residual stream. The authors train four probes (logistic regression, MLP, linear SVM, mass-mean) on activations from affirmative factual statements across six topics and test generalization to logical negations, conjunctions, disjunctions, multiple-choice and short-form QA (MMLU, TriviaQA), and context-grounded tasks (SciQ, BoolQ, XSum). They report that generalization improves with model capability, that simple linear probes are sufficient to identify truth directions, and that probes transfer across syntax and task format. They also present a selective-QA application where an SVM probe filters candidate answers. The paper includes a randomized-model control and makes code publicly available.

Significance. If the findings hold, they would substantially strengthen the evidence for a linear truthfulness representation in capable LLMs and extend the scope of truth-direction probing from atomic statements to realistic QA and contextual settings. The paper's strengths include the use of held-out external benchmarks, a randomized-weight control showing that probes do not manufacture structure, comparison of multiple probe families, and a practical selective-QA demonstration. However, the central claims about model capability and generalization rest on specific design choices and interpretations that need additional scrutiny.

major comments (4)
  1. [4.3.2 / Figure 3] AUROC values of 0 for Llama-2-7B on several topics are interpreted as evidence that weaker models lack a consistent truth direction, but an AUROC of 0 is perfect anti-correlation, i.e., a perfectly consistent linear ordering with reversed sign, not the absence of linear structure. The authors should report the distribution of AUROC values, the sign-flipped metric (1 - AUROC), or the angle between the affirmative and negative probe directions to distinguish 'no consistent direction' from 'consistent but sign-inverted direction.' Without this, the conclusion that consistent truth directions do not appear in weaker models is unsupported.
  2. [4.1 / layer selection] The optimal layer index is selected using the between-to-within-class variance ratio computed on statements that include affirmations, negations, conjunctions, and disjunctions, as stated in Section 4.1 and detailed in Appendix F.2. Because the same logical transformations are later used as test sets in Sections 4.3 and 4.4, the layer selection is informed by the test transformations, which can inflate generalization estimates. The authors should either select layers using only affirmative statements or show that conclusions are robust across layers.
  3. [Figures 3, 4, 6] The main generalization results in Figures 3, 4, and 6 are presented without error bars or confidence intervals, although the text reports averaging over three trials. Several AUROC values are close to the 0.5 threshold (e.g., 0.53 in Figure 3 for the facts topic, and 0.58-0.61 in Figure 4 for disjunctions), so without variance estimates the claim of successful generalization is not statistically supported. The authors should report per-trial values, standard errors, or a significance test.
  4. [4.3.2 / Conclusions] The paper's central claim that 'consistent truth directions only emerge in capable LLMs and not in weaker ones' is supported by a cross-family comparison across Llama-2, Llama-3.1, and Mistral models that differ in architecture, training data, and instruction tuning, while capability is inferred from external benchmarks rather than isolated as a variable. The authors should temper the causal wording or provide a within-family size comparison (e.g., Llama-3.1-8B vs. Llama-3.1-70B) as the primary evidence, and explicitly acknowledge confounds.
minor comments (5)
  1. [4.7] The selective-QA experiment reports an overall accuracy of 55.29% and a selected-subset accuracy of 64.06% while retaining 80.26% of answers; the paper should discuss the precision-recall trade-off and whether the improvement is statistically significant.
  2. [3.2] The word 'hyperplace' should be 'hyperplane'.
  3. [4.2] The randomized-model control is reported for only one dataset (animal_class) and one layer; a brief justification of why this suffices would help.
  4. [F.1] The use of float8 quantization for Llama-3.1-70B may alter activations; the authors should comment on the potential impact on probe performance.
  5. [Figure 3] The caption does not define M0-M7; the text should state the model ordering explicitly.

Circularity Check

1 steps flagged · score 2.0 of 10

Layer selection peeks at the target transformations, but central probe-weight predictions are trained only on affirmative statements and are benchmarked externally; no substantial circularity.

  1. fitted input called prediction [Section 4.1 (Layer Selection) and Appendix F.2]
    "By plotting the ratio of between-class to within-class variance across decoder layers for a range of topic-specific datasets, we identify the optimal layer as the one with the highest ratio. ... The data used for plotting is the collection of both affirmative and negative atomic statements covering all the six topics, as well as their logical conjunctions and disjunctions."

    The decoder layer is a component of the probe representation, and it is selected using variance ratios computed on the same negative, conjunction, and disjunction statements that later serve as the test sets for the generalization claims. Reported AUROCs for negation/conjunction/disjunction generalization are therefore measured at a layer chosen with access to those exact target transformations, so the result is a partly in-sample hyperparameter choice rather than a fully out-of-sample prediction. This is a selection bias, not an equivalence: probe weights are still fit only to affirmative atomic statements, and results vary by model and topic, so the central capability-dependent pattern is not forced.

full rationale

The main empirical loop is not circular: truthfulness probes are trained on labeled atomic factual statements and evaluated on unseen negations, logical compounds, and external benchmarks (MMLU, TriviaQA, SciQ, BoolQ, XSum) without fitting to those test labels. The randomized-model control confirms that probes cannot manufacture a truth direction from random activations, and the external benchmark results provide independent evidence for RQ3. The only notable circularity-adjacent step is the layer-selection procedure in Section 4.1: the variance-ratio criterion is applied to statements that include negations, conjunctions, and disjunctions, and the resulting fixed layer is then used for all generalization experiments. This gives the negation/compound generalization results a mild selection bias, but because the probe weights remain trained solely on affirmative statements, the consistency and generalization claims retain independent content. The paper contains no load-bearing self-citations; the cited prior work supplies data and baselines rather than the paper's conclusions. The AUROC=0 readings in Figure 3 raise an interpretive concern about sign-flipped directions, but that is a correctness/robustness matter, not circularity. Overall score 2.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims do not rest on free parameters in the physical sense, but the layer index and SVM nu are model choices tuned on data that include the test distributions. The AUROC interpretation axiom is the main load-bearing assumption.

free parameters (3)
  • optimal layer index per model = 12 (Llama-3.1-8B), 13 (8B-Instruct), 33 (70B-Instruct), etc.
    Selected in Section 4.1 by maximizing between/within-class variance over statements that include the test transformations; fixed for all subsequent experiments.
  • SVM nu parameter (NuSVC) = 0.5
    Set in Appendix D; the authors state it was validated by experiment results.
  • Selective QA confidence threshold = 0.5
    Used in Section 4.7 to select answers for filtering; not independently justified.
assumptions (4)
  • domain assumption The final-token hidden state of a chosen layer is a sufficient representation for probing binary truthfulness.
    Assumed in Section 3.1; standard in prior probing work.
  • domain assumption AUROC > 0.5 indicates a successful generalization of a truthfulness probe, and AUROC <= 0.5 indicates absence of a consistent truth direction.
    Used throughout Section 4; treats near-zero AUROC values as failures without considering inverted directions.
  • domain assumption Factual statement datasets and their logical transformations are correctly labeled and preserve truth values.
    Section 3.3 and Appendix C describe dataset sources and modifications; no label validation is provided.
  • ad hoc to paper The variance-ratio based layer selection identifies a single optimal layer for all tasks.
    Section 4.1 chooses one layer per model based on data that include target transformations and then fixes it for all experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks." pith.science (2026). https://pith.science/paper/POZR2MJ3

@misc{pith2026250600823,
  author       = {Pith},
  title        = {Pith review of: Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/POZR2MJ3}},
  note         = {Machine review of arXiv:2506.00823}
}
read the original abstract

Large language models (LLMs) are trained on extensive datasets that encapsulate substantial world knowledge. However, their outputs often include confidently stated inaccuracies. Earlier works suggest that LLMs encode truthfulness as a distinct linear feature, termed the "truth direction", which can classify truthfulness reliably. We address several open questions about the truth direction: (i) whether LLMs universally exhibit consistent truth directions; (ii) whether sophisticated probing techniques are necessary to identify truth directions; and (iii) how the truth direction generalizes across diverse contexts. Our findings reveal that not all LLMs exhibit consistent truth directions, with stronger representations observed in more capable models, particularly in the context of logical negation. Additionally, we demonstrate that truthfulness probes trained on declarative atomic statements can generalize effectively to logical transformations, question-answering tasks, in-context learning, and external knowledge sources. Finally, we explore the practical application of truthfulness probes in selective question-answering, illustrating their potential to improve user trust in LLM outputs. These results advance our understanding of truth directions and provide new insights into the internal representations of LLM beliefs. Our code is public at https://github.com/colored-dye/truthfulness_probe_generalization

Figures

Figures reproduced from arXiv: 2506.00823 by the authors.

Figure 1
Figure 1. Illustration of truthfulness probes. A truthfulness probe is established using the LLM’s internal states [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Ratio of between-class variance to within [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. AUROC (in percentage) of probes trained on affirmative statements and tested on negative ones. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: AUROC (in percentage) of probes trained on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 6
Figure 6. Figure 6: AUROC [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 5
Figure 5. Figure 5: AUROC↑/ECE↓/BS↓ of truthfulness probes for Llama-3.1-8B on MMLU and TriviaQA. The dashed gray line corresponds to random results, and error bars denote standard error. The results, shown in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: AUROC↑/ECE↓/BS↓ of truthfulness probes for Llama-3.1-8B on tasks where grounding knowledge is provided in the prompt. The dashed gray line corresponds to random results, and error bars denote standard error. (Q, A) pairs is 55.29%. Among these, the truth￾fulness probe …
Figure 8
Figure 8. Figure 8: Calibration graph of LR, MLP, SVM, MM and TTPD probes on the BoolQ dataset under the “with options” setting. The target LLM is Llama-3.1-8B. dataset has potential ambiguity due to duplication of name. Thus we specify that the person men￾tioned in a statement was an inv…
Figure 9
Figure 9. Figure 9: Output distributions of LR, MLP, SVM, MM and TTPD probes on the BoolQ dataset under the “with [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Plot of the ratio of between-class variance to within-class variance for a series of models. The shaded [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: AUROC (in percentage) of probes trained on affirmative statements and tested on negative ones. AUROC exceeding 0.5 indicates generalization success. M0 refers to the Mistral-7B-v0.1 model. BoolQ. For the BoolQ (Clark et al., 2019) bench￾mark, we arrange four setups, i…
Figure 12
Figure 12. Figure 12: AUROC (in percentage) of probes trained on all the atomic factual statements and tested on logical [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: AUROC (in percentage) of probes trained on all the atomic factual statements and tested on logical [PITH_FULL_IMAGE:figures/full_fig_p017_13.png]
Figure 14
Figure 14. Figure 14: AUROC (in percentage) of probes trained on [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: AUROC↑/ECE↓/BS↓ of truthfulness probes for Mistral-7B-v0.1 on MMLU and TriviaQA. The dashed gray line corresponds to random results, and error bars denote standard error [PITH_FULL_IMAGE:figures/full_fig_p018_15.png]
Figure 16
Figure 16. Figure 16: AUROC↑/ECE↓/BS↓ of truthfulness probes for Mistral-7B-v0.1 on tasks where grounding knowledge is provided in the prompt. The dashed gray line corresponds to random results, and error bars denote standard error [PITH_FULL_IMAGE:figures/full_fig_p019_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 12 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Amos Azaria and Tom Mitchell. 2023. The internal state of an llm knows when it's lying. arXiv preprint arXiv:2304.13734

    arXiv 2023

  3. [3]

    Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610--623

    2021

  4. [4]

    Lennart B \"u rger, Fred A Hamprecht, and Boaz Nadler. 2024. Truth is universal: Robust detection of lies in llms. arXiv preprint arXiv:2407.12831

    arXiv 2024

  5. [5]

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827

    arXiv 2022

  6. [6]

    Paul Christiano, Ajeya Cotra, and Mark Xu. 2021. Eliciting latent knowledge: How to tell if your eyes deceive you. URL https://docs. google. com/document/d/1WwsnJQstPq91\_ Yh-Ch2XRL8H\_EpsnjrC1dwZXR37PC8/edit\# heading= h. jrzi4atzacns

    work page 2021

  7. [7]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

    doi:10.18653/v1/n19-1300 2019

  8. [8]

    Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning, 20(3):273--297

    1995

Show all 42 references
  1. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  2. [10]

    Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, and William Saunders. 2021. Truthful ai: Developing and governing ai that does not lie. arXiv preprint arXiv:2110.06674

    2021 arXiv

  3. [11]

    Danny Halawi, Jean-Stanislas Denain, and Jacob Steinhardt. 2023. Overthinking the truth: Understanding how language models process false demonstrations. arXiv preprint arXiv:2307.09476

    2023 arXiv

  4. [12]

    Jinwen He, Yujia Gong, Zijin Lin, Yue Zhao, Kai Chen, et al. 2024. Llm factoscope: Uncovering llms’ factual discernment through measuring inner states. In Findings of the Association for Computational Linguistics ACL 2024, pages 10218--10230

    2024

  5. [13]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300

    2020 arXiv

  6. [14]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1--38

    2023

  7. [15]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  8. [16]

    Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    2017

  9. [17]

    Nitish Joshi, Javier Rando, Abulhair Saparov, Najoung Kim, and He He. 2024. Personas as a way to model truthfulness in language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 6346--6359

    2024

  10. [18]

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221

    2022 arXiv

  11. [19]

    Diederik P Kingma. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  12. [20]

    Abhinav Kumar, Chenhao Tan, and Amit Sharma. 2022. Probing classifiers are unreliable for concept removal and detection. Advances in Neural Information Processing Systems, 35:17994--18008

    2022

  13. [21]

    Bruce W Lee, Benedict Florance Arockiaraj, and Helen Jin. 2023. Linguistic properties of truthful response. In Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023), pages 135--140

    2023

  14. [22]

    Benjamin A Levinstein and Daniel A Herrmann. 2024. Still no lie detector for language models: Probing empirical and conceptual roadblocks. Philosophical Studies, pages 1--27

    2024

  15. [23]

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. 2023. Inference-time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36:41451--41530

    2023

  16. [24]

    Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214--3252

    2022

  17. [25]

    Dong C Liu and Jorge Nocedal. 1989. On the limited memory bfgs method for large scale optimization. Mathematical programming, 45(1):503--528

    1989

  18. [26]

    Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. 2023. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pag...

    2023

  19. [27]

    Monte MacDiarmid, Timothy Maxwell, Nicholas Schiefer, Jesse Mu, Jared Kaplan, David Duvenaud, Sam Bowman, Alex Tamkin, Ethan Perez, Mrinank Sharma, Carson Denison, and Evan Hubinger. 2024. https://www.anthropic.com/news/probes-catch-sleeper-agents Simple probes can catch sleep...

    2024

  20. [28]

    Mat \'e o Mahaut, Laura Aina, Paula Czarnowska, Momchil Hardalov, Thomas M \"u ller, and Lluis Marquez. 2024. https://doi.org/10.18653/v1/2024.acl-long.250 Factual confidence of LLM s: on reliability and robustness of current estimators . In Proceedings of the 62nd Annual Meet...

    2024 doi

  21. [29]

    Alex Mallen and Nora Belrose. 2023. Eliciting latent knowledge from quirky language models. arXiv preprint arXiv:2312.01037

    2023 arXiv

  22. [30]

    Samuel Marks and Max Tegmark. 2023. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824

    2023 arXiv

  23. [31]

    Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. https://doi.org/10.18653/v1/2020.acl-main.173 On faithfulness and factuality in abstractive summarization . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1...

    2020 doi

  24. [32]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. ArXiv, abs/1808.08745

    2018 arXiv

  25. [33]

    Lorenzo Pacchiardi, Alex James Chan, S \"o ren Mindermann, Ilan Moscovitz, Alexa Yue Pan, Yarin Gal, Owain Evans, and Jan M Brauner. 2023. How to catch an ai liar: Lie detection in black-box llms by asking unrelated questions. In The Twelfth International Conference on Learnin...

    2023

  26. [34]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in P ython. Journal of Machine L...

    2011

  27. [35]

    J Platt. 1999. Probabilistic outputs for svms and comparisons to regularized likehood methods. Advances in Large Margin Classifiers

    1999

  28. [36]

    CH-Wang Sky, Benjamin Van Durme, Jason Eisner, and Chris Kedzie. 2024. Do androids know they’re only dreaming of electric sheep? In Findings of the Association for Computational Linguistics ACL 2024, pages 4401--4420

    2024

  29. [37]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  30. [38]

    A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  31. [39]

    Johannes Welbl, Nelson F Liu, and Matt Gardner. 2017. Crowdsourcing multiple choice science questions. arXiv preprint arXiv:1707.06209

    2017 arXiv

  32. [40]

    Zhenhong Zhou, Haiyang Yu, Xinghua Zhang, Rongwu Xu, Fei Huang, and Yongbin Li. 2024. https://aclanthology.org/2024.findings-emnlp.139 How alignment and jailbreak work: Explain LLM safety through intermediate hidden states . In Findings of the Association for Computational Lin...

    2024

  33. [41]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  34. [42]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.