Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs

T0 review · 4 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Large language models encode simple factual truth as a multi-dimensional cone of activation directions, not a single linear direction.

desk verdict Cone-style steering directions for truth exist, but the paper's headline evidence is circular: it fits cones to the model's own Yes/No labels and measures flip rates against those same labels, not ground truth. read the letter →

arxiv 2505.21800 v1 pith:4QC5PAJ7 submitted 2025-05-27 cs.LG cs.CL

Stanley Yu , Vaidehi Bulusu , Oscar Yasunaga , Clayton Lau , Cole Blondin , Sean O'Brien , Kevin Zhu , Vasu Sharma This is my paper
classification cs.LGcs.CL
keywords conceptconestruthfulnessmechanisticinterpretabilityactivationsteeringdirectionalablationlargelanguagemodelslinearrepresentationresidualstream
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models can answer simple true/false statements, and prior work suggested that a single linear "truth direction" in internal activations controls that behavior. This paper argues that this is incomplete: across two model families, truthfulness is mediated by a multi-dimensional cone, a set of activation directions whose nonnegative combinations all causally influence whether the model says Yes or No. The authors find that sampled directions within the cone flip responses, that the effect generalizes across model architectures and sizes, and that removing the cone's components leaves unrelated instruction-following behavior roughly intact. If correct, the result changes how truth should be probed and steered: not one axis, but a subspace.

What carries the argument

The central object is the concept cone: given orthonormal vectors $v_1,\dots,v_k$ in the residual stream, the cone is the set of all nonzero nonnegative linear combinations $\sum_i \lambda_i v_i$ with $\lambda_i \ge 0$, and every such direction is supposed to mediate the same concept. The search optimizes a composite loss $L_{\text{total}} = \lambda_1 L_{\text{add}} + \lambda_2 L_{\text{ablate}} + \lambda_3 L_{\text{retain}}$, where addition on false prompts, ablation on true prompts, and KL retention on Alpaca continuations are defined over a vocabulary restricted to the tokens Yes and No. Because any cone direction should flip behavior, the authors validate the cone by Monte-Carlo sampling random nonnegative combinations and measuring Answer Switching Rate.

What would settle it

Train the cone on statements where the model's own Yes/No answer is known to disagree with the ground-truth label, then intervene on those statements; if the cone flips responses toward the model's original wrong label rather than toward the true label, the cone encodes response tendency, not truth.

Watch

Extended reading notes

Core claim

The paper's central claim is that in the residual-stream activations of decoder-only LLMs, the representation of a simple propositional fact is a multi-dimensional cone rather than a single truth direction. Using a gradient-based search adapted from refusal modeling, the authors learn orthonormal basis vectors for Qwen2.5 and Gemma-2 models such that every nonnegative combination of the basis vectors causally steers the model's forced-choice Yes/No answer on factual statements. They report that one-to-five dimensional cones achieve near-total Answer Switching Rates in larger models, that the cone axes are almost orthogonal to the classical difference-in-means truth vector (only the first axis aligns), and that ablating cone directions leaves KL divergence on general instruction prompts below 0.05. The paper concludes that there exists a subspace of activation vectors whose positive combinations consistently modulate factuality.

Load-bearing premise

The load-bearing premise is that base models answer these simple propositions correctly, because the cone is trained on the model's own forced-choice Yes/No answers rather than on ground-truth labels; if the model's answers carry response bias, the discovered geometry may be about response tendency, not truth.

Editorial extensions

If this is right

  • Truth-related control is localized: one-dimensional cones are most effective around 60 to 75 percent of normalized layer depth and at the final token position.
  • Larger models support higher-dimensional cones: Qwen2.5-7B and Gemma-2-9B keep near-100 percent Answer Switching Rates up to dimension 5, while smaller models degrade.
  • Interventions on the cone are behaviorally specific: mean KL divergence on Alpaca instruction continuations stays below 0.05 for all tested models.
  • The classical difference-in-means truth direction aligns only with the first cone axis; the remaining axes are orthogonal, so DIM captures one facet of a larger truth subspace.
  • Any nonnegative combination of cone basis vectors should causally flip true/false responses, giving a continuum of working steering directions, not just the learned axes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: if the cone is the full truth substrate, erasing only the DIM vector will leave truth reachable through other cone directions, so alignment interventions that target a single axis may be incomplete.
  • Beyond the paper: the observed "Sí"/"Nein" outputs after steering suggest the cone may be language-neutral; a direct test on non-English factual statements would confirm whether the representation survives translation.
  • Beyond the paper: the authors' failed attempts to find cones for sentiment and toxicity suggest cone structure is not universal for every abstract feature; the presence or absence of a cone could itself be a diagnostic of how a model organizes a concept.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper extends the recently introduced concept-cone framework from refusal to propositional truth in LLMs. Rather than using offline ground-truth labels, the authors prompt each model to answer true/false statements with a forced binary "Yes" or "No" and use those self-generated outputs as supervision. They optimize an orthonormal set of activation directions through a composite loss (Ladd, Lablate, Lretain), then report Answer Switching Rates (ASR) for one- to five-dimensional cones across Qwen-2.5 and Gemma-2 models, KL-divergence retention scores on Alpaca prompts, and cosine similarities between cone axes and a difference-in-means (DIM) truth direction. The paper concludes that truth-related behavior is mediated by a multi-dimensional cone rather than a single linear direction.

Significance. If the central claim survives scrutiny, the paper would be a meaningful extension of linear-probing results: it would show that truth-related behavior in the residual stream is not exhausted by a single direction, and that cone-based interventions transfer from refusal to factual propositions. The Monte Carlo sampling of cone directions, the KL-based retention check, and the inclusion of multiple model scales are useful methodological steps. The significance is currently conditional, however, because the model's own forced-choice labels serve as both the training target and the evaluation criterion, and the paper never verifies intervened outputs against ground-truth labels.

major comments (4)
  1. [§3, Definitions 3.1–3.3, §4.1] The supervision signal for the cone is the model's own forced-choice Yes/No output, and the optimized losses directly maximize the flip rates later reported as ASR. Specifically, Ladd maximizes log P(Yes | x+v) on statements labeled false by the model and Lablate maximizes log(1 - P(Yes | x - vv^T x)) on statements labeled true by the model. The reported ASR therefore measures agreement with a target that the optimization already fitted. Since the manuscript never verifies intervened outputs against ground-truth labels (the Lretain term only checks Alpaca continuations, not factual correctness), the evidence does not distinguish a cone that mediates truth from one that mediates a general response bias such as "always answer Yes" or "always answer No." To support the central claim, please (i) report per-dataset baseline accuracy of each model's forced-choice answers against ground truth, (ii) evaluate all interventions on held-out prompts against ground-truth labels, and (iii) explicitly separate the 16 training samples mentioned in Appendix A.1 from the evaluation set.
  2. [§4.1–§4.2, Table 1] The layer and token positions used for all cone experiments are selected using ASR, and the same metric is then reported in Table 1. This selection-on-the-evaluation-metric protocol, without a separate validation split or nested cross-validation, can inflate the reported ASR values. Please clarify how training, validation, and test sets are split, or use a selection metric that is distinct from the final reported metric.
  3. [Table 1, Figure 3, §4.1] ASR values are reported as point estimates with no error bars, confidence intervals, or significance tests across Monte Carlo samples, random seeds, or datasets. Given that Qwen-3B drops from 98.6 to 45.1 between dimensions 1 and 2 and Gemma-2B drops from 100 to 27.1 by dimension 5, the claim that larger models maintain high ASR needs variance quantification to rule out noise. In addition, the ASR definition in Section 4.1 is incomplete: the denominator is "baseline # of prompts that the model answers truthfully," but the numerator refers to outputs that "become untruthful after ablation" without specifying whether "untruthful" means a flip from the model's own baseline answer or disagreement with ground truth.
  4. [§4.4, §5] The claim that additional cone axes independently support truth-aligned behavior is not established by the cosine similarities in Table 3 alone. Orthogonality to the DIM direction does not demonstrate causal efficacy for each basis vector. Please report per-axis ASR and, crucially, per-axis ground-truth accuracy for each basis vector, and show that positive combinations within the cone pass the same tests, to support the multi-dimensional cone conclusion.
minor comments (7)
  1. [§7.1] Section 7.1 states that all experiments were conducted on models with 1.5B–7B parameters, but Table 6 includes Qwen2.5-14B and Gemma-2-9B; this inconsistency should be fixed.
  2. [Table 2] Table 2 lists "Qwen2-7B" while the rest of the paper uses "Qwen2.5-7B"; the naming should be made consistent.
  3. [Appendix E, Table 8] The header of Table 8 says "Qwen-2.5-9B," but no such model appears in Table 6; this is likely a typo for Qwen2.5-14B.
  4. [Appendix B.2] Appendix B.2 contains an unresolved "?" placeholder instead of a citation for the student/teacher direction method; the citation should be supplied.
  5. [§4.1] The ASR definition is presented as a two-line fraction with no equation number and with the term "untruthful" left undefined; please provide a formal, unambiguous definition.
  6. [Appendix D] Appendix D reports that alternative tokenizations had "no significant effect" but gives no statistics; either provide the supporting numbers or soften the claim.
  7. [Abstract and §6] The paper claims results across "multiple LLM families," but the experiments cover only two families (Qwen and Gemma); please state the scope more precisely in the abstract and conclusion.

Circularity Check

2 steps flagged · score 6.0 of 10

The headline ASR is the flip rate that Ladd and Lablate are optimized to maximize on the model's own yes/no labels, so the reported 'causal mediation of factuality' is partly a fit metric rather than an independent prediction.

  1. self definitional [Section 3 (supervision) and Section 2.2 Definition 2.1 (Truth Property)]
    "instead of attaching a label offline, we ask the model to answer each statement with a binary “Yes” or “No” and use that forced choice as the supervision signal. ... Ablating the truth direction through projection ... should cause the model to shift the answer from an initially true output to a false output."

    The 'truth direction' and the cone are defined and optimized against the model's own binary answers, not against ground-truth labels. The Truth Property is satisfied whenever the model flips from its initial yes/no output; ASR counts a flip as 'becoming untruthful' even if the new answer is factually correct (or if the initial answer was biased). The paper's assumption in Section 3.1 that base models classify correctly is asserted, not checked with numbers ('models nearly always achieve full accuracy' without values). Therefore the headline claim that the cone 'causally mediates factuality' is, by construction, the claim that it flips the model's self-generated yes/no labels; a response-bias direction (e.g., always-No) would satisfy the same definition.

  2. fitted input called prediction [Section 3.3 (Definitions 3.1–3.2) vs Section 4.1 (ASR definition)]
    "Ladd =− 1/|Dfalse| ∑_{x∈Dfalse} log ŷadd(x + v) (Add, target y=1); Lablate =− 1/|Dtrue| ∑_{x∈Dtrue} log [1− ŷablate(x− vv⊤x)] (Ablate, target y=0). ASR = # of prompts whose output becomes untruthful after ablation / baseline # of prompts that the model answers truthfully."

    Minimizing Ladd forces false prompts to be answered 'Yes'; minimizing Lablate forces true prompts to be answered 'No'. ASR as defined is exactly the fraction of (near-perfectly answered) true prompts flipped after ablation, i.e., the quantity Lablate drives toward 1. The Algorithm's Step 5 records flips on the same prompts used to compute the losses, and the paper gives no held-out factual split for Table 1; the only sample count listed is 'Number of Samples (during training) 16' (Table 4). The reported near-100% ASR is therefore the training objective evaluated at convergence, not a predictive result. Monte-Carlo sampling over nonnegative combinations tests new directions inside the fitted cone, but not unseen statements, so it does not supply the missing held-out evidence.

full rationale

The paper's central claim—that a robust subspace of activation vectors 'modulates factuality'—is supported by ASR numbers that measure flips of the model's own binary yes/no answers, and those flips are exactly the targets of the Ladd and Lablate losses used to fit the cone. This is a partial circularity: the reported ASR is the training objective on the prompts used for the loss, and no clear held-out factual set is described. There is some independent content: the Monte-Carlo sampling over random nonnegative cone directions tests generalization across the cone interior, the DIM-vs-cone cosine comparison uses a direction computed separately from the cone loss, and the cross-family replication shows the fitted behavior is not idiosyncratic to one model. The retention check (Experiment 3) is also a separate behavioral measurement. However, these do not break the core reduction for the 'factuality' claim, because the supervision signal is the model's own forced choice and the paper's Limitations (Section 7.2) explicitly narrow the operationalization to simple unambiguous propositions while never checking flipped answers against ground-truth labels. The assumption in Section 3.1 that base models 'achieve correct classification' is asserted without reported accuracy numbers, so the discovered cone could mediate response bias rather than truth. Score 6 reflects one central 'prediction' that reduces by construction, with partial independence from the cone-sampling, DIM-alignment, and cross-model comparisons.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the residual stream as the search space, the model's own yes/no output as the truth label, the cone definition inherited from Wollschlaeger et al., and the use of KL on Alpaca as a generic-capability proxy. The learned cone vectors and loss weights are fitted quantities, so the headline ASR is a fit metric.

free parameters (3)
  • Cone basis vectors v_1...v_k per model and layer = Learned orthonormal vectors; numerical values not reported
    These vectors are optimized to minimize Ltotal; the reported ASR measures the objective they were fit to (Sections 3.3, 4.2).
  • Loss weights lambda_1, lambda_2, lambda_3 = Not reported
    The balance among add, ablate, and retain losses is chosen without stating values; results depend on this balance (Section 3.3).
  • KL acceptability threshold = 0.1
    The paper rejects cones whose KL exceeds 0.1, following Arditi et al.; this cutoff shapes the retention claim (Section 4.3).
assumptions (5)
  • domain assumption Residual stream activations are the right substrate for truth directions
    All probes and interventions search directions in the residual stream; sub-layer representations are not examined (Sections 2.1, 3.2).
  • ad hoc to paper The model's forced-choice Yes/No output is a valid truth label
    The supervision signal is the model's own binary answer rather than the statement's ground truth; assumed valid for capable base models (Sections 3, 3.1).
  • domain assumption Any nonnegative linear combination of basis vectors lies in the truth cone
    The cone definition is imported from Wollschlaeger et al.; the MC sampling test checks it but the premise is not independently derived (Section 2.5).
  • domain assumption Low KL divergence on Alpaca 30-token continuations indicates preservation of unrelated behavior
    Both the retain loss and the evaluation use Alpaca; low KL is assumed to imply specificity of the intervention (Sections 3.3, 4.3).
  • standard math DIM is an appropriate baseline single-direction estimator
    Difference-in-means is the accepted prior method for extracting concept directions; used for cosine-similarity comparison in Section 4.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs." pith.science (2026). https://pith.science/paper/4QC5PAJ7

@misc{pith2026250521800,
  author       = {Pith},
  title        = {Pith review of: From Directions to Cones: Exploring Multidimensional Representations of Propositional Facts in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4QC5PAJ7}},
  note         = {Machine review of arXiv:2505.21800}
}
read the original abstract

Large Language Models (LLMs) exhibit strong conversational abilities but often generate falsehoods. Prior work suggests that the truthfulness of simple propositions can be represented as a single linear direction in a model's internal activations, but this may not fully capture its underlying geometry. In this work, we extend the concept cone framework, recently introduced for modeling refusal, to the domain of truth. We identify multi-dimensional cones that causally mediate truth-related behavior across multiple LLM families. Our results are supported by three lines of evidence: (i) causal interventions reliably flip model responses to factual statements, (ii) learned cones generalize across model architectures, and (iii) cone-based interventions preserve unrelated model behavior. These findings reveal the richer, multidirectional structure governing simple true/false propositions in LLMs and highlight concept cones as a promising tool for probing abstract behaviors.

Figures

Figures reproduced from arXiv: 2505.21800 by the authors.

Figure 1
Figure 1. Theoretical visualization of a 2D concept [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The Attack Success Rate (ASR) of one dimensional cones across layers for Qwen and Gemma [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. The Answer Switching Rate (ASR) of cones from dimensions 1 to 5 across Qwen2.5 and Gemma2 [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Projections of Gemma-2-9B, representations of datasets onto their top two PCs, across all layers. [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: Projections of Qwen2.5-7B representations of datasets onto their top two PCs, across all layers. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Geometry of Harmfulness in LLMs through Subconcept Probing

    cs.AI 2025-07 conditional novelty 5.0 of 10

    Fifty-five harmfulness subconcept directions in Llama-3.1-8B-Instruct form a nearly rank-1 subspace, and steering along the dominant direction cuts jailbreak success but costs accuracy and fails on Qwen.

Reference graph

Works this paper leans on

40 extracted references · 13 canonical work pages · cited by 1 Pith paper

  1. [1]

    and Bengio, Y

    Alain, G. and Bengio, Y. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016. URL http://arxiv.org/abs/1610.01644

    arXiv 2016

  2. [2]

    Refusal in language models is mediated by a single direction

    Arditi, A., Obeso, O., Syed, A., Paleka, D., Panickssery, N., Gurnee, W., and Nanda, N. Refusal in language models is mediated by a single direction. In Advances in Neural Information Processing Systems, volume 37, pp.\ 136037--136083, 2024

    work page 2024

  3. [3]

    and Mitchell, T

    Azaria, A. and Mitchell, T. M. The internal state of an llm knows when it’s lying, 2023. URL https://arxiv.org/abs/2304.13734

    arXiv 2023

  4. [4]

    Probing classifiers: Promises, shortcomings, and advances

    Belinkov, Y. Probing classifiers: Promises, shortcomings, and advances . Computational Linguistics, 48 0 (1): 0 207--219, 2022

    work page 2022

  5. [5]

    and Gavves, E

    Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety -- a review. arXiv preprint arXiv:2404.14082, 2024. URL https://arxiv.org/abs/2404.14082

    arXiv 2024

  6. [6]

    Language models are few-shot learners

    Brown, T., Mann, B., Ryder, N., et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    work page 2020

  7. [7]

    A., and Nadler, B

    Bürger, L., Hamprecht, F. A., and Nadler, B. Truth is universal: Robust detection of lies in llms. arXiv preprint arXiv:2407.12831, 2024. URL https://arxiv.org/abs/2407.12831

    arXiv 2024

  8. [8]

    Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., Sharkey, L., Krishna, S., Von Hagen, M., Alberti, S., Chan, A., Sun, Q., Gerovitch, M., Bau, D., Tegmark, M., Krueger, D., and Hadfield-Menell, D. Black-box access is insufficient for rigorous ai audits. In Proceedings of the 2024 AC...

    arXiv 2024

Show all 40 references
  1. [9]

    C., Lundberg, S

    Chen, H., Covert, I. C., Lundberg, S. M., and Lee, S.-I. Algorithms to estimate shapley value feature attributions. Nature Machine Intelligence, 5 0 (6): 0 590--601, 2023. doi:10.1038/s42256-023-00657-x. URL https://doi.org/10.1038/s42256-023-00657-x

    2023 doi

  2. [10]

    From yes-men to truth-tellers: Addressing sycophancy in large language models with pinpoint tuning

    Chen, W., Huang, Z., Xie, L., Lin, B., Li, H., Lu, L., Tian, X., Cai, D., Zhang, Y., Wang, W., Shen, X., and Ye, J. From yes-men to truth-tellers: Addressing sycophancy in large language models with pinpoint tuning. arXiv preprint arXiv:2409.01658, 2024. URL https://arxiv.org/...

    2024 arXiv

  3. [11]

    Sparse autoencoders find highly interpretable features in language models

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023. URL https://arxiv.org/abs/2309.08600

    2023 arXiv

  4. [12]

    Toy models of superposition, 2022

    Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition, 2022. URL https://arxiv.org/abs/2209.10652. arXiv preprint arXiv:2209.10652

    2022 arXiv

  5. [13]

    J., Gurnee, W., and Tegmark, M

    Engels, J., Liao, I., Michaud, E. J., Gurnee, W., and Tegmark, M. Not all language model features are linear. arXiv preprint arXiv:2405.14860, 2024

    2024 arXiv

  6. [14]

    Sequential integrated gradients: A simple but effective method for explaining language models

    Enguehard, J. Sequential integrated gradients: A simple but effective method for explaining language models. In Findings of the Association for Computational Linguistics: ACL 2023, pp.\ 7555--7565, Toronto, Canada, 2023. Association for Computational Linguistics. doi:10.18653/...

    2023 doi

  7. [15]

    and Tegmark, M

    Gurnee, W. and Tegmark, M. Language models represent space and time. In the Twelfth International Conference on Learning Representations, 2024

    2024

  8. [16]

    and Mazeika, M

    Hendrycks, D. and Mazeika, M. X-risk analysis for ai research. CoRR, abs/2206.05862, 2022. URL https://arxiv.org/abs/2206.05862

    2022 arXiv

  9. [17]

    An overview of catastrophic ai risks

    Hendrycks, D., Mazeika, M., and Woodside, T. An overview of catastrophic ai risks. arXiv preprint arXiv:2306.12001, 2023

    2023 arXiv

  10. [18]

    Heo, J., Heinze-Deml, C., Elachqar, O., Ren, S., Nallasamy, U., Miller, A., Chan, K. H. R., and Narain, J. Do llms "know" internally when they follow instructions? In Proceedings of the International Conference on Learning Representations (ICLR), 2025. URL https://arxiv.org/ab...

    2025 arXiv

  11. [19]

    and Manning, C

    Hewitt, J. and Manning, C. D. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp....

    2019 doi

  12. [20]

    Refusal behavior in large language models: A nonlinear perspective

    Hildebrandt, F., Maier, A., Krauss, P., and Schilling, A. Refusal behavior in large language models: A nonlinear perspective. arXiv preprint arXiv:2501.08145, 2025. URL https://arxiv.org/abs/2501.08145

    2025 arXiv

  13. [21]

    Linear representations of political perspective emerge in large language models

    Kim, J., Evans, J., and Schein, A. Linear representations of political perspective emerge in large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=rwqShzb9li

    2025

  14. [22]

    J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N

    Liu, P. J., Saleh, M., Pot, E., Goodrich, B., Sepassi, R., Kaiser, L., and Shazeer, N. Generating Wikipedia by summarizing long sequences. arXiv preprint arXiv:1801.10198, 2018

    2018 arXiv

  15. [23]

    Cones: Concept neurons in diffusion models for customized generation

    Liu, Z., Feng, R., Zhu, K., Zhang, Y., Zheng, K., Liu, Y., Zhao, D., Zhou, J., and Cao, Y. Cones: Concept neurons in diffusion models for customized generation. arXiv preprint arXiv:2303.05125, 2023. URL https://arxiv.org/abs/2303.05125

    2023 arXiv

  16. [24]

    and Tegmark, M

    Marks, S. and Tegmark, M. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference on Language Modeling, 2024

    2024

  17. [25]

    Linguistic regularities in continuous space word representations

    Mikolov, T., tau Yih, W., and Zweig, G. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pp.\ 746--751, 2013

    2013

  18. [26]

    Progress measures for grokking via mechanistic interpretability

    Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. Progress measures for grokking via mechanistic interpretability. arXiv preprint arXiv:2301.05217, 2023. URL https://arxiv.org/abs/2301.05217

    2023 arXiv

  19. [27]

    The alignment problem from a deep learning perspective

    Ngo, R., Chan, L., and Mindermann, S. The alignment problem from a deep learning perspective. arXiv preprint arXiv:2209.00626, 2022

    2022 arXiv

  20. [28]

    Zoom in: An introduction to circuits

    Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S. Zoom in: An introduction to circuits. Distill, 5 0 (3): 0 e00024--001, 2020. doi:10.23915/distill.00024.001. URL https://distill.pub/2020/circuits/zoom-in

    2020 doi

  21. [29]

    Introducing chatgpt

    OpenAI . Introducing chatgpt. https://openai.com/blog/chatgpt/, November 2022. Accessed: 2025-01-26

    2022

  22. [30]

    Gpt-4 technical report

    OpenAI. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. URL https://arxiv.org/abs/2303.08774

    2023 arXiv

  23. [31]

    Panickssery, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681, 2023. URL https://arxiv.org/abs/2312.06681

    2023 arXiv

  24. [32]

    J., and Veitch, V

    Park, K., Choe, Y. J., and Veitch, V. The Linear Representation Hypothesis and the Geometry of Large Language Models . In International Conference on Machine Learning, pp.\ 39643--39666. PMLR, 2024 a

    2024

  25. [33]

    J., and Veitch, V

    Park, K., Choe, Y. J., and Veitch, V. The linear representation hypothesis and the geometry of large language models. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st International Conference...

    2024

  26. [34]

    Exploring the limits of transfer learning with a unified text-to-text transformer

    Raffel, C., Shazeer, N., Roberts, A., et al. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 0 (140): 0 1--67, 2020

    2020

  27. [35]

    Taking features out of superposition with sparse autoencoders

    Sharkey, L., Braun, D., and Millidge, B. Taking features out of superposition with sparse autoencoders. https://www.alignmentforum.org/posts/z6QQJbtpkEAX3Aojj/interim-research-report-taking-features-outof-superposition, 2023. Accessed: 2025-04-22

    2023

  28. [36]

    J., Geiger, A., and Nanda, N

    Tigges, C., Hollinsworth, O. J., Geiger, A., and Nanda, N. Linear Representations of Sentiment in Large Language Models . arXiv preprint arXiv:2310.15154, 2023

    2023 arXiv

  29. [37]

    M., Thiergart, L., Leech, G., Udell, D., Vazquez, J

    Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2024. URL https://arxiv.org/abs/2308.10248

    2024 arXiv

  30. [38]

    and Pinter, Y

    Wiegreffe, S. and Pinter, Y. Attention is not not explanation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp.\ 11--20, Hong Kong, China, 201...

    2019 doi

  31. [39]

    a ger, T., Elstner, J., Geisler, S., Cohen-Addad, V., G \

    Wollschl \"a ger, T., Elstner, J., Geisler, S., Cohen-Addad, V., G \"u nnemann, S., and Gasteiger, J. The geometry of refusal in large language models: Concept cones and representational independence. arXiv preprint arXiv:2502.17420, 2025. URL https://arxiv.org/abs/2502.17420

    2025

  32. [40]

    Zhang, J., Zhao, Y., Saleh, M., and Liu, P. J. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the 37th International Conference on Machine Learning (ICML), 2020

    2020

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.