Pith. sign in

REVIEW 3 major objections 6 minor 38 references

Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions

T0 review · 3 major / 6 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that when an LLM fails a multiple-choice question despite knowing the answer, the failure comes from a geometric misalignment between a knowledge coordinate and a prediction coordinate inside the residual stream, and that a

desk verdict Solid probing analysis and a clever adaptive steering idea, but the headline gains come from tuned amplification (w=5/8) rather than the claimed alignment, and the abstract overstates the baseline comparison. read the letter →

arxiv 2509.23782 v4 pith:NXSVIU5F submitted 2025-09-28 cs.CL

Yoonah Park , Haesung Pyun , Yohan Jo This is my paper
classification cs.CL
keywords knowledge-predictiongapmultiple-choicequestionsresidualstreamprobingactivationsteeringtest-timeinterventionlinearprobesknowledgealignmentLLMfaithfulness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that when a language model fails a multiple-choice question despite 'knowing' the answer, the failure is not missing knowledge but a geometric misalignment inside the model's residual stream: a knowledge coordinate and a prediction coordinate point in disagreeing directions. To close this gap, the authors introduce KAPPA, a closed-form, parameter-free test-time transform that shifts each hidden state along the prediction basis until its prediction coordinate matches its knowledge coordinate. On binary-choice reformulations of BBH, MMLU, and ARC-Challenge, KAPPA improves accuracy across two model families, lifting Llama-2 on BBH-Binary from near-random 50.9% to 70.8%, approaching the 72.8% ceiling of the knowledge probe itself. The same subspace generalizes across datasets and to free-form generation, suggesting the gap is a structural property of transformer representations rather than a quirk of specific tasks.

What carries the argument

The key machinery is the knowledge-prediction subspace: a 2D subspace of the residual stream spanned by two logistic-regression probe weight vectors — the knowledge basis u_knowledge, trained to predict the ground-truth label, and the prediction basis u_prediction, trained to predict the model's own output. The intervention KAPPA is the closed-form solution to min ||h'-h||^2 s.t. u_prediction^T h' + b_prediction = u_knowledge^T h + b_knowledge, yielding h' = h + ((u_knowledge^T h + b_knowledge - u_prediction^T h - b_prediction)/||u_prediction||^2) u_prediction. This replaces only the prediction coordinate, leaving all orthogonal components of the hidden state intact.

What would settle it

A concrete falsifier: on a set where KAPPA improves accuracy, replace the KAPPA update with a random vector of equal norm along the prediction basis and measure accuracy. If random perturbations produce similar gains, the alignment is not the causal mechanism. Alternatively, if aligning the prediction coordinate to a deliberately wrong knowledge coordinate (e.g., a probe trained on flipped labels) also improves accuracy, then the improvement stems from generic regularization rather than knowledge alignment.

Watch

Extended reading notes

Core claim

The central claim is that the knowledge-prediction gap in LLMs on MCQs reflects a geometric misalignment in the residual stream: activations carry a knowledge basis (correlates with ground truth) and a prediction basis (correlates with the model's own output), and when these coordinates disagree the model errs despite encoding the right answer. KAPPA is a closed-form affine transform (Eq. 2) that enforces equality between the two coordinates by minimal ℓ2 adjustment along the prediction basis, requiring no training. The paper reports that this improves MCQ accuracy, approaches the knowledge-probe ceiling, transfers across datasets and to free-form generation, and does not degrade general cap

Load-bearing premise

The method assumes that the linear prediction-basis direction is the causal lever that controls the model's final choice, so shifting a hidden state along it changes the output in the intended direction while leaving all other capabilities untouched.

Editorial extensions

If this is right

  • Many MCQ failures are caused by misalignment rather than lack of knowledge; KAPPA unlocks latent ability without external information.
  • KAPPA approaches the knowledge-probe ceiling, so the model's own internal knowledge sets an upper bound on achievable accuracy without new information.
  • The knowledge-prediction subspace transfers across datasets and architectures, indicating a shared geometric structure in transformer representations.
  • The same alignment improves free-form generation accuracy, showing the effect is not an artifact of binary-choice formatting.
  • KAPPA does not degrade general capabilities on vicuna-eval, suggesting alignment can be applied safely at deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the subspace transfers across datasets, a calibration trained once on a broad knowledge benchmark could be applied at deployment to improve faithfulness across many MCQ and open-ended tasks, without per-task tuning.
  • If the prediction basis is causally load-bearing, then perturbing activations along this basis should produce predictable, monotonic shifts in choice probability; measuring this dose-response curve would test the causal claim directly.
  • The binary-choice setting is a strong simplification; extending KAPPA to multi-choice formats will require a principled way to define knowledge and prediction coordinates when more than two options compete, possibly via pairwise subspaces.
  • The free-form gains (25.0% to 29.1%) are smaller than MCQ gains; a possible extension is to apply KAPPA at multiple layers or to combine it with decoding strategies, though the paper does not investigate this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper studies the knowledge–prediction gap in multiple-choice QA. On binary-choice reformulations of BBH, MMLU, and ARC-Challenge, it trains two logistic-regression probes on last-token residual-stream activations: a knowledge probe predicting the ground-truth label and a prediction probe predicting the model's own choice. The probe weight vectors define a two-dimensional 'knowledge-prediction subspace,' and incorrect answers are interpreted as a misalignment between the two coordinates. The authors propose KAPPA, a closed-form affine transformation that shifts a hidden state along the prediction basis so that the prediction coordinate equals the knowledge coordinate, plus a generalized form with hyperparameters (w, beta). Experiments report accuracy gains, cross-dataset transfer, free-form generation improvements, and vicuna-eval comparisons on Llama-2-7B-Chat and Qwen2.5-7B-Instruct.

Significance. If the claims hold, the paper contributes a simple, interpretable representation-level account of a known failure mode and a lightweight, adaptive test-time intervention that requires no parameter updates. The constrained-optimization derivation in Appendix B is clean, and using logistic-regression probe weights as basis vectors is a standard and transparent operationalization. The cross-dataset transfer experiment is a useful addition that goes beyond a single benchmark. The main caveats are that the headline experiments use tuned hyperparameters (w=5.0/8.0) rather than the parameter-free strict form advertised, that the 'consistently outperforms baselines' claim is not supported by Table 1, and that the free-form experiment lacks essential implementation detail. These are fixable, but they currently prevent the paper's central claims from being fully supported.

major comments (3)
  1. [§3.2 / Table 1] The abstract and §3.2 state that KAPPA 'consistently outperforms baselines.' As Table 1 stands, this is contradicted: LoRA FT outperforms KAPPA in every model/dataset cell (e.g., Qwen-2.5 BBH: 86.1 vs 74.3; Llama-2 BBH: 79.2 vs 70.8; Llama-2 MMLU: 75.6 vs 66.2), and ICL beats KAPPA on Qwen-2.5 MMLU and ARC-Challenge. If 'baselines' is intended to mean only Base and Steering, that should be stated explicitly; otherwise the claim should be revised or supported with significance/variance estimates.
  2. [§2.2 / Eq. (3) / Appendix E.1] The reported results are for the generalized KAPPA with w=5.0 (Llama-2) and w=8.0 (Qwen2.5), beta=0, not for the strict alignment of Eq. (2). No result for w=1, beta=0 is reported, and no validation protocol for selecting w is described. Moreover, §3.5 shows accuracy increases monotonically with w and beta and that beta alone (w=0) can give substantial gains. The experiments therefore do not isolate the geometric alignment mechanism and are hard to reconcile with the abstract's 'parameter-free' characterization. Please report the Eq. (2) result, describe how w was selected (validation grid vs test-set tuning), and add an ablation that separates alignment from amplification/scaling.
  3. [§3.3 / Appendix J] The free-form generalization claim is under-specified. The text says KAPPA is applied during free-form generation, but neither §3.3 nor Appendix J states which residual-stream token is modified, how the prediction coordinate is computed when there are no answer choices, or how probes trained on binary-choice last-token activations are transferred to free-form decoding. Without this information, 'KAPPA extends its effectiveness to free-form questions' is not verifiable from the presented experiments.
minor comments (6)
  1. [Table 1 vs Table 2] The Qwen-2.5 MMLU base accuracy differs between tables (80.8 in Table 1 vs 81.3 in Table 2). Please make the numbers consistent.
  2. [Appendix E.2] The LoRA FT configuration paragraph contains two 'For BBH' entries; the second one is presumably for MMLU. This should be corrected.
  3. [Abstract / §3.3] The free-form improvement is reported as 3.8% in the abstract and introduction but as 4.1 percentage points in §3.3. Please reconcile.
  4. [Table 8] The column headers include '1-Layer (?)' and '6-Layers (?)' with literal question marks; use the actual layer numbers (20 and 15–20 for Qwen2.5, per Appendix E.1).
  5. [Figure 3 caption] The caption uses 'LLaMA-2' while the text uses 'Llama-2'; please standardize.
  6. [§2.1] The knowledge-prediction gap is defined by two probes trained on the same activations, one of which is trained to predict the model's own output. The discrepancy is therefore partly a property of the probe construction. This is a standard probing caveat, but the geometric interpretation should be framed as a probe-relative measurement rather than an unmediated property of the model.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor definitional circularity in the 'gap reduction' framing; accuracy claims remain independently evaluated.

  1. self definitional [Section 2.1-2.2, Eq. 1-2]
    "Knowledge coordinate: u_knowledge^T h, Prediction coordinate: u_prediction^T h. ... min_{h'} ||h'-h||^2_2 s.t. u_prediction^T h' = u_knowledge^T h. ... h' = h + (u_knowledge^T h - u_prediction^T h)/||u_prediction||^2 u_prediction."

    The 'knowledge-prediction gap' is operationally defined in §2.1 as the difference between the two probe coordinates. Equation 2 is the closed-form solution to Equation 1, which enforces equality of those coordinates. Therefore the statement that KAPPA 'reduces the gap' is true by construction: the transformation is defined to set the prediction coordinate equal to the knowledge coordinate. This aspect of the paper's framing is self-definitional. The substantive accuracy claims, however, are evaluated against ground truth (Table 1) and are not forced by this definition.

full rationale

KAPPA's transformation is defined by the constraint in Eq. 1, and its closed form in Eq. 2 is derived correctly in Appendix B via Lagrange multipliers; this is a clean projection and not itself circular. The 'knowledge-prediction gap' is operationally defined in §2.1 as the difference between two probe coordinates, so the claim that KAPPA 'reduces the gap' is true by construction. The paper's substantive claims—accuracy improvements, cross-dataset transfer, and free-form gains—are measured against ground truth labels, not against the probe coordinates, so they are not forced by the definition. No load-bearing self-citations or imported uniqueness theorems appear. The main caveat is reporting: the abstract calls KAPPA 'parameter-free' but §E.1 sets w=5.0 for Llama-2 and w=8.0 for Qwen2.5 with beta=0, and the paper never reports the canonical w=1, beta=0 result or the selection protocol for w. This weakens the interpretation that the gains come from geometric alignment rather than amplification, but it is a validation/reporting concern rather than demonstrated circularity. Overall, the derivation and accuracy evaluation are self-contained, so the circularity score is low.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The method rests on the linear probe hypothesis and on the empirical assumption that a 2D projection along two probe directions is sufficient to change the model's output. The free parameters are hyperparameters (w, beta, layers, prompt format) that are chosen on validation data. No external entities or falsifiable predictions are introduced beyond the probe-defined geometry.

free parameters (4)
  • w (alignment scale) = 5.0 (Llama-2), 8.0 (Qwen2.5)
    Chosen per model; hyperparameter sweep (Fig. 5) shows larger w improves accuracy, so the chosen values affect results.
  • beta (bias term) = 0 (all main experiments)
    Fixed to 0, but the sweep shows beta>0 can improve accuracy; this simplifying choice trades off potential gains.
  • Intervention layer(s) = layer 17 (Llama-2), 20 (Qwen2.5); 6-layer ranges 12-17 and 15-20
    Selected based on validation probe accuracy, not derived from theory.
  • Prompt format selection = one of 32 format combinations per dataset/model
    Chosen to yield the most balanced predictions on the validation set; affects all results.
assumptions (4)
  • domain assumption Residual stream activations at the last token of MCQ prompts contain linearly separable directions for the ground-truth answer and the model's predicted answer.
    Section 2.1 builds the subspace from logistic regression probes; this is the core interpretability assumption.
  • domain assumption Modifying the hidden state only along the prediction basis changes the final token prediction in the intended direction, preserving other information.
    Eq. 2 derives minimal perturbation along u_prediction; the claim that this yields correct predictions is empirical, not proven.
  • domain assumption Linear probes trained on one dataset's binary-choice reformulation generalize to other datasets.
    Section 3.2 tests this; the generalization is partial and not guaranteed.
  • domain assumption The LLM-as-a-Judge protocol in Appendix J reliably measures free-form answer correctness.
    Used for the free-form results in Section 3.3.
invented entities (2)
  • Knowledge-prediction subspace
    purpose: A 2D span of the two probe weight vectors used to interpret and intervene on the knowledge-prediction gap.
    Constructed directly from trained probe weights; no external falsifiable handle outside this paper, though cross-dataset transfer is internal evidence.
  • Knowledge basis / prediction basis
    purpose: The two probe weight vectors that define the coordinates in the subspace.
    These are the trained logistic regression weights; their existence and utility are only demonstrated inside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions." pith.science (2026). https://pith.science/paper/NXSVIU5F

@misc{pith2026250923782,
  author       = {Pith},
  title        = {Pith review of: Bridging the Knowledge-Prediction Gap in LLMs on Multiple-Choice Questions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NXSVIU5F}},
  note         = {Machine review of arXiv:2509.23782}
}
read the original abstract

While large language models (LLMs) perform strongly on diverse tasks, their trustworthiness is limited by erratic behavior that is unfaithful to their internal knowledge. In particular, LLMs often fail on multiple-choice questions (MCQs) even if they encode correct answers in their hidden representations, revealing a misalignment between internal knowledge and output behavior. We investigate and mitigate this knowledge-prediction gap on MCQs through a three-step analysis of hidden representations. First, we quantify the prevalence and magnitude of the gap across models and datasets. Second, we provide a geometric interpretation by identifying distinct knowledge and prediction subspaces in the residual stream. Third, we introduce KAPPA, a lightweight inference-time intervention that aligns the two subspaces within the residual stream to reduce the knowledge-prediction gap. Our results provide a geometric and interpretable explanation of the knowledge-prediction gap in LLMs. Furthermore, KAPPA effectively reduces the gap across diverse MCQ benchmarks and models, and generalizes to free-form settings.

Figures

Figures reproduced from arXiv: 2509.23782 by the authors.

Figure 1
Figure 1. KAPPA resolves the knowledge-prediction gap through geometric realignment. (a) Mo [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. KAPPA pipeline overview. (1) Collect residual stream activations from transformer layers [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Accuracy of knowledge probes (blue) and prediction probes (orange) across transformer [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: (a) Response alignment patterns for each method between MCQ and free-form formats. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Effect of alignment parameters w and β on model accuracy. (Left, Middle) Results on Qwen-2 7B Instruct across five w values (0, 2, 4, 6, 8) and four β values (0, 2, 4, 6). (Right) Results on Llama-2 7B Chat, contrasting the cases where only w is varied (β = 0, w ̸= 0) …
Figure 6
Figure 6. Figure 6: The distribution of hidden states projected to the 2D knowledge-prediction subspace. From [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Probing results on MMLU-Binary using Llama-2. Knowledge Probe Prediction Probe Llama-2 7B Chat Qwen2.5 7B Instruct [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Probing results on BBH-Binary using Llama-2 (upper) and Qwen-2.5 (lower). the knowledge-prediction subspace is not specific to a single model family but extends across ar￾chitectures. Unlike the shared probes used in the main text, here we train a separate probe for ea…
Figure 9
Figure 9. Figure 9: Subset-wise results on the BBH-Binary dataset with [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

38 extracted references · 1 canonical work pages

  1. [1]

    Understanding intermediate layers using linear classifier probes, 2017

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2017. URL https://openreview.net/forum?id=ryF7rTqgl

    2017

  2. [2]

    Refusal in language models is mediated by a single direction

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp.\ 136037--136083. Curran Associates, ...

    2024

  3. [3]

    The internal state of an LLM knows when it ' s lying

    Amos Azaria and Tom Mitchell. The internal state of an LLM knows when it ' s lying. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp.\ 967--976, Singapore, December 2023. Association for Computational Linguistics. doi:10.18653/v1/2023.findings-emnlp.68. URL https://aclanthology....

    doi:10.18653/v1/2023.findings-emnlp.68 2023

  4. [4]

    Probing the geometry of truth: Consistency and generalization of truth directions in LLM s across logical transformations and question answering tasks

    Yuntai Bao, Xuhong Zhang, Tianyu Du, Xinkui Zhao, Zhengwen Feng, Hao Peng, and Jianwei Yin. Probing the geometry of truth: Consistency and generalization of truth directions in LLM s across logical transformations and question answering tasks. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Associatio...

    2025

  5. [5]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    2020

  6. [6]

    Answer matching outperforms multiple choice for language model evaluation, 2025

    Nikhil Chandak, Shashwat Goel, Ameya Prabhu, Moritz Hardt, and Jonas Geiping. Answer matching outperforms multiple choice for language model evaluation, 2025. URL https://arxiv.org/abs/2507.02856

    arXiv 2025

  7. [7]

    Gonzalez, Ion Stoica, and Eric P

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90\ https://lmsys.org/blog/2023-03-30-vicuna/, March 2023. Accessed: 2025-09-25

    2023

  8. [8]

    Think you have solved question answering? try arc, the ai2 reasoning challenge

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv:1803.05457v1, 2018

    arXiv 2018

Show all 38 references
  1. [9]

    Knowledge neurons in pretrained transformers

    Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. Knowledge neurons in pretrained transformers. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Vol...

    2022 doi

  2. [10]

    Plug and play language models: A simple approach to controlled text generation

    Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations, 2020. URL https://openr...

    2020

  3. [11]

    In-context learning creates task vectors, 2023

    Roee Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors, 2023. URL https://arxiv.org/abs/2310.15916

    2023 arXiv

  4. [12]

    Measuring massive multitask language understanding

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=d7KBjmI3GmQ

    2021

  5. [13]

    Lo RA : Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lo RA : Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  6. [14]

    Editing models with task arithmetic, 2023

    Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic, 2023. URL https://arxiv.org/abs/2212.04089

    2023 arXiv

  7. [15]

    Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar

    Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering. In The Thirteenth International Conference on Learning Representations, 2025. URL https://o...

    2025

  8. [16]

    Inference-time intervention: Eliciting truthful answers from a language model

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id...

    2023

  9. [17]

    Kevin Liu, Stephen Casper, Dylan Hadfield-Menell, and Jacob Andreas. Cognitive dissonance: Why do language model outputs disagree with internal representations of truthfulness? In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical...

    2023 doi

  10. [18]

    Investigating bias representations in llama 2 chat via activation steering, 2024

    Dawn Lu and Nina Rimsky. Investigating bias representations in llama 2 chat via activation steering, 2024. URL https://arxiv.org/abs/2402.00402

    2024 arXiv

  11. [19]

    The geometry of truth: Emergent linear structure in large language model representations of true/false datasets, 2024

    Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets, 2024. URL https://arxiv.org/abs/2310.06824

    2024 arXiv

  12. [20]

    Locating and editing factual associations in gpt

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088

    2022

  13. [21]

    Introducing gpt-5

    OpenAI . Introducing gpt-5. https://openai.com/index/introducing-gpt-5/, August 2025. Accessed: 2025-09-25

    2025

  14. [22]

    LLM s know more than they show: On the intrinsic representation of LLM hallucinations

    Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. LLM s know more than they show: On the intrinsic representation of LLM hallucinations. In The Thirteenth International Conference on Learning Representations, 2025. URL h...

    2025

  15. [23]

    Large language models sensitivity to the order of options in multiple-choice questions

    Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity to the order of options in multiple-choice questions. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp.\ 2006--2017, Mexico...

    2024 doi

  16. [24]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, ...

    2025 arXiv

  17. [25]

    Steering llama 2 via contrastive activation addition

    Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational ...

    2024 doi

  18. [26]

    Layer by layer: Uncovering hidden representations in language models

    Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. In Forty-second International Conference on Machine Learning, 2025. URL https://openreview.net/fo...

    2025

  19. [27]

    Unsupervised real-time hallucination detection based on the internal states of large language models

    Weihang Su, Changyue Wang, Qingyao Ai, Yiran Hu, Zhijing Wu, Yujia Zhou, and Yiqun Liu. Unsupervised real-time hallucination detection based on the internal states of large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association fo...

    2024 doi

  20. [28]

    Challenging BIG -bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. Challenging BIG -bench tasks and whether chain-of-thought can solve them. In Anna Rogers, Jordan Boyd-Graber, and N...

    2023 doi

  21. [29]

    Llama 2: Open foundation and fine-tuned chat models, 2023

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, W...

    2023 arXiv

  22. [30]

    Steering language models with activation engineering, 2025

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering, 2025. URL https://openreview.net/forum?id=2XBPdPIcFK

    2025

  23. [31]

    Hidden question representations tell non-factuality within and across large language models

    Yanling Wang, Haoyang Li, Hao Zou, Jing Zhang, Xinlei He, Qi Li, and Ke Xu. Hidden question representations tell non-factuality within and across large language models. CoRR, abs/2406.05328, 2024. URL https://doi.org/10.48550/arXiv.2406.05328

    2024 doi arXiv

  24. [32]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Sys...

    2022

  25. [33]

    Large language models are not robust multiple choice selectors

    Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. Large language models are not robust multiple choice selectors. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=shr9PXz7T0

    2024

  26. [34]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico...

    2025 arXiv

  27. [35]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  28. [36]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  29. [37]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  30. [38]

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.