Pith. sign in

REVIEW 4 major objections 5 minor 3 cited by

Towards eliciting latent knowledge from LLMs with mechanistic interpretability

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A language model can hide a secret word from its output while storing it readably in its internal states, and logit lens plus sparse autoencoders can extract it, scoring 75% and 55% top-5 Pass@10 on a 20-model Taboo test.

desk verdict A useful new Taboo benchmark and a promising logit-lens result, but the SAE claim is circular and the 'reliable extraction' conclusion overreaches. read the letter →

arxiv 2505.14352 v1 pith:O4PHXD4R submitted 2025-05-20 cs.LG

Bartosz Cywiński , Emil Ryd , Senthooran Rajamanoharan , Neel Nanda This is my paper
classification cs.LG
keywords mechanisticinterpretabilitylatentknowledgeelicitationsecretwordTaboomodellogitlenssparseautoencoderswhite-boxlanguagehonesty
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that hidden knowledge in a language model—information the model is actively trained never to utter—can be recovered by reading its internal computations rather than by prompting it. To create a controlled test, the authors fine-tune 20 'Taboo' models, each holding a single secret word that never appears in the model's training data or prompt and that the model is trained never to reveal. They compare black-box strategies (direct and adversarial prompting, prefilled responses, and a separate model guessing from the hints) with white-box strategies: the logit lens, which projects each layer's internal state onto vocabulary probabilities, and sparse autoencoders, which decompose activations into sparse interpretable features. The central result is that both white-box methods beat direct prompting, recovering the secret word in top-5 Pass@10 for 75% (logit lens) and 55% (SAE) of the 20 models, while naive prompting scores 0%; the authors read this as evidence that interpretability can elicit non-verbalized latent knowledge, a capability relevant to auditing models that may hide objectives.

What carries the argument

The load-bearing mechanism is the residual-stream representation of the secret word at layer 32, which the paper shows is readable by two instruments. The logit lens is the operation of mapping each intermediate layer's residual state to vocabulary probabilities through the model's unembedding matrix; the method averages these probabilities across all token positions of the generated response and ranks candidate tokens, excluding tokens that were actually generated. Sparse autoencoders (SAEs) decompose layer-32 activations into a sparse set of features; the method picks the latents with the highest average activation across the response and maps them to words using a manually assembled table of secret-word-to-latent correspondences. Both instruments exploit the same empirical observation: the secret word's probability, read at intermediate layers, peaks strongly around layers 30–37 during hint generation even though the word is never emitted. Layer 32 was chosen because it was empirically found to frequently surface the secret word; that empirical selection, rather than a derived principle, is what fixes the readout layer.

What would settle it

Train Taboo models whose secret is a two- or three-token phrase (for example 'ice cream') using the same base model and generation pipeline, then run the paper's layer-32 logit-lens averaging and SAE latent-ranking procedures exactly as described. If top-5 Pass@10 drops to near chance, the result depends on the single-token/layer-32 locality; if it stays high, the secret's representation is more distributed and the method generalizes. A second check is to choose the readout layer using only a training subset of Taboo models and evaluate on the rest; if performance collapses, the post-hoc layer selection carries the 75% figure.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a model that internalises a secret it must not verbalise encodes that secret in a readable, local form: during hint generation, the logit-lens probability of the secret word peaks strongly in layers 30–37 of the fine-tuned 9B chat-tuned model, around layer 32, even though the word is never emitted. Averaging the logit-lens probabilities at that layer across all token positions of the response, excluding the tokens actually generated, produces a ranked candidate list that places the secret word in the top five for 75% of the 20 Taboo models. Sparse autoencoders applied to the same layer identify the secret through its most strongly activated latents, reaching 55% top-5 Pass@10, with the mapping from latents to words supplied manually in an appendix. The paper therefore argues that standard, largely automated interpretability instruments can extract a non-verbalized secret from a proof-of-concept model, while acknowledging that no method reaches near-perfect accuracy and that the current secret format (a single token) is simple.

Load-bearing premise

The load-bearing premise is that the secret word is stored, at one fixed middle layer (layer 32), as a single token that can be spotted by averaging internal readings over the model's response; if the secret is spread across layers or written in a code that logit lens and sparse autoencoders cannot read, both methods collapse.

Editorial extensions

If this is right

  • A model can conceal a word in its output while still representing it in a form that simple interpretability tools can read; hiding from the text is not hiding from the internals.
  • Returning a short candidate list (top 5) rather than a single guess sharply improves both white-box methods, suggesting that list-based elicitation is the realistic operating mode.
  • Because the secret never appeared in training or the prompt, the recovered representation must have been inferred from the task itself, which supports extracting hidden information without any direct exposure to the secret.
  • The strong performance of the separate-model guesser indicates the current Taboo hints are highly decodable, so the white-box methods are being tested on a relatively simple organism; the paper's stated next step is to build organisms with multi-token or abstract secrets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the layer-32 readout was chosen post hoc from the same models whose secrets it recovers, so the 75% figure likely overstates what a fixed, pre-committed layer would achieve; a natural control is to select the layer on a training split of Taboo models and evaluate on held-out ones.
  • Editorial inference: the SAE evaluation uses a manually curated table mapping each secret word to its latent(s), an oracle that already knows the answer; an automated pipeline would need to infer those correspondences without ground truth, so the 55% result is an upper bound for a fully autonomous SAE-based elicitor.
  • Editorial inference: the logit-lens peaks often occur at punctuation-like tokens (', a, the'), which the paper links to in-context information; if the method is partly reading the model's readiness to elaborate rather than a stable semantic code, the approach may translate poorly to secrets that are not single concrete nouns.
  • Editorial inference: a testable extension is to run the same pipeline on multi-token secrets; if performance collapses, the single-token locality assumption is the load-bearing property, and the method would need a different readout for sequential or distributed codes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces a Taboo model organism: Gemma 2 9B chat-tuned models are fine-tuned to describe a single-token secret word without ever verbalizing it, with the secret absent from both the prompt and training data. The authors then benchmark black-box elicitation methods (naive prompting, adversarial prompting, token forcing, and a separate guessing model) against white-box methods based on logit lens and sparse autoencoders. They report that the interpretability-based methods can extract the secret word, with logit lens top-5 reaching 75% Pass@10 and SAE top-5 reaching 55% Pass@10, whereas prompting baselines largely fail. The paper argues that this proof-of-concept demonstrates the promise of mechanistic interpretability for eliciting hidden knowledge.

Significance. If the central result holds, the paper makes a useful contribution: it provides a simple, reproducible model organism for studying hidden knowledge, open-sourced models, and a clean comparison of black-box and white-box elicitation strategies in a setting where the secret is known to the experimenter. The finding that logit lens can recover a non-verbalized single-token secret that prompting cannot extract is concrete and falsifiable, and the Taboo benchmark could be used by other researchers to test new elicitation methods. However, the two white-box pillars of the paper's main claim are not equally strong: the SAE result is substantially weakened by the ground-truth-based construction of the latent-to-secret mapping, and the logit-lens result relies on a post hoc chosen layer without held-out validation. The significance of the paper therefore depends on whether these issues can be resolved in revision.

major comments (4)
  1. [Section 3.2 and Appendix D.7, Table 7] The SAE evaluation is substantially circular and cannot support the Section 5 claim that interpretability-based approaches reliably extract the secret. In Appendix D.7, the authors state that SAE latents were 'manually identified' to correspond to each secret word, and Table 7 hard-codes this mapping. The reported SAE Pass@10 then only checks whether these known target latents are among the most active latents on the response tokens; it does not demonstrate that an automated pipeline can discover an unknown secret. In a real elicitation setting, the latent-to-token mapping is exactly the unknown quantity. The manuscript should either report a fully automated SAE pipeline (e.g., decoding SAE latents to tokens without using ground truth) or explicitly remove/reclassify the SAE result as a sanity check rather than an elicitation result.
  2. [Section 3.2 and Figure 2] The logit-lens result also carries an unquantified selection bias. The method focuses on layer 32, which was 'empirically found to frequently surface the secret word' on the same 20 Taboo models used for evaluation. No held-out validation, cross-validation, or sensitivity analysis over layer indices is reported, so the 75% Pass@10 for logit-lens top-5 may be inflated by this post hoc selection. The authors should either report the layer-selection procedure as part of a principled protocol (e.g., selecting the layer on a development set) or show the performance across layers to allow the reader to assess the stability of the choice.
  3. [Section 3.1 versus Appendix D.6] There is a direct inconsistency in the specification of the 'Another Model' baseline. Section 3.1 states that Gemma 3 12B chat-tuned is used as the guessing model, while Appendix D.6 presents a prompt that names Gemini 2.5 Flash as the model. Since this baseline achieves the highest Pass@10 score (95%), the exact model used materially affects the benchmark comparison. The authors must correct this contradiction and specify the model, version, and decoding settings used.
  4. [Section 4.3 and Table 1] The results are reported without any measure of uncertainty. With 20 Taboo models and 10 prompts per model, the differences between, for example, logit-lens top-5 (75% Pass@10) and SAE top-5 (55% Pass@10) may be within sampling noise. The authors should report confidence intervals or bootstrap intervals for the Pass@10, Majority@10, and Accuracy metrics, and ideally perform a paired test (e.g., McNemar's test) when comparing methods on the same models and prompts.
minor comments (5)
  1. [Section 3.2] The logit-lens token-averaging procedure is described only briefly; please clarify exactly which tokens are excluded as 'generated tokens' and whether averaging is performed over all positions or only over positions where the secret token is not part of the surface text.
  2. [Appendix D.7, Table 7] For the secret word 'dance', the authors state that multiple SAE features were considered target features, and all are counted as correct in the evaluation. This post hoc expansion of the target set should be stated in the main text and handled by a predefined rule, otherwise it inflates the SAE success measure.
  3. [Figures 2, 3, and 8-10] The figures would benefit from clearer axis labels and a statement of which response (which prompt) and which token positions were used when computing the average logit-lens probability and the average SAE activation.
  4. [Section 4.2] The definition of Majority@10 assumes a deterministic way of breaking ties and a defined 'most frequent guess'; please specify how ties are handled and whether the majority is computed over the 10 prompts for each model separately.
  5. [Appendix D.5] The token-forcing postgame baseline uses a fixed three-turn conversation before prefilling; please state whether the same three prompts were used for every model and whether any of those prompts can inadvertently leak the secret word.

Circularity Check

1 steps flagged · score 6.0 of 10

SAE elicitation evaluation is circular because Table 7 encodes the secret-word-to-latent mapping from ground truth; the logit-lens result remains independent but weaker.

  1. fitted input called prediction [Section 3.2 (SAE paragraph) and Appendix D.7 (SAE evaluation details)]
    "We then identify relevant SAE latents in an automated way by selecting latents that exhibit the highest average activation across the response, as visualized in Figure 3, and map them to tokens according to Table 7 from Appendix. ... To evaluate our Sparse Autoencoder (SAE) based strategy in an automated manner, we first manually identify the SAE latents that most closely correspond to each of the secret words our Taboo models were fine-tuned on. These identified latents are presented in Table 7."

    The target latent list in Table 7 is constructed from ground-truth knowledge of each secret word. The SAE 'elicitation' then tests only whether one of these known target latents is among the top-5 most active latents on the model's responses. Since the mapping from activations to the secret word is exactly what an elicitation method is supposed to discover, the reported SAE Pass@10 (55%) verifies that the pre-identified latent fires for the corresponding model rather than demonstrating extraction of an unknown secret. By construction, the success condition is that a latent chosen because it corresponds to the secret word appears in the top-k; the procedure cannot discover a new secret and reduces to checking whether a fitted ground-truth-derived mapping is active.

full rationale

The paper's central claim in Section 5 ('interpretability-based approaches can reliably extract the secret word') is supported by two white-box results. The SAE result in Section 3.2 and Appendix D.7 is partly circular: the authors manually identify which SAE latents correspond to each secret word (Table 7), then score success by whether those same latents are among the most activated. This reduces the SAE Pass@10/Majority@10 numbers to checking whether a ground-truth-derived latent fires, not to discovering the secret. The logit-lens approach is not circular in this way: it uses vocabulary-space probabilities from an external unembedding, and its top-5 Pass@10 of 75% is independent evidence, although layer 32 was chosen post hoc on the same models. The black-box baselines (another model, token forcing) are also independent. Because one of the two advertised interpretability pillars reduces to a fit, but the central result retains independent support, the analysis is partially circular rather than wholly forced.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central white-box claims rest on two unevidenced assumptions: that the secret survives as a single linearly readable token at one fixed layer, and that a SAE pretrained on the base model remains faithful after QLoRA fine-tuning. The SAE evaluation also introduces a large fitted component in the manual latent mapping. These are load-bearing because both interpretability methods would fail without them.

free parameters (2)
  • Layer index for logit lens and SAE extraction = 32
    Selected empirically as the layer where the secret word's token probability peaks across the same 20 Taboo models used for evaluation; this is post-hoc model selection rather than a principled choice.
  • SAE latent-to-secret mapping = Table 7 (e.g., smile -> 9936, dance -> [14269, 3115])
    Manually constructed by identifying latents most associated with each secret word using knowledge of the ground truth; used to score the SAE elicitation results, making the evaluation partially circular.
assumptions (3)
  • domain assumption The secret word is never present in the training dataset or prompt, so the model must encode it internally to generate hints.
    Section 2.1 states this as a design property. Validation only checks for exact string absence; semantic or embedded leakage cannot be ruled out.
  • domain assumption At layer 32 the residual stream contains a directly readable representation of the secret token, so averaging logit-lens probabilities or SAE activations over generated tokens can surface it.
    Section 3.2 and Figures 2-3 demonstrate this only for the 20 evaluated Taboo models; layer 32 is an empirical selection, and no argument shows it generalizes.
  • domain assumption The Gemma Scope SAE trained on base Gemma 2 9B layer 32 activations remains a faithful decomposition of the fine-tuned Taboo model activations.
    Section 3.2 uses the pretrained SAE without checking feature stability after QLoRA fine-tuning, which is required for the SAE-based latents to be meaningful.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards eliciting latent knowledge from LLMs with mechanistic interpretability." pith.science (2026). https://pith.science/paper/O4PHXD4R

@misc{pith2026250514352,
  author       = {Pith},
  title        = {Pith review of: Towards eliciting latent knowledge from LLMs with mechanistic interpretability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O4PHXD4R}},
  note         = {Machine review of arXiv:2505.14352}
}
read the original abstract

As language models become more powerful and sophisticated, it is crucial that they remain trustworthy and reliable. There is concerning preliminary evidence that models may attempt to deceive or keep secrets from their operators. To explore the ability of current techniques to elicit such hidden knowledge, we train a Taboo model: a language model that describes a specific secret word without explicitly stating it. Importantly, the secret word is not presented to the model in its training data or prompt. We then investigate methods to uncover this secret. First, we evaluate non-interpretability (black-box) approaches. Subsequently, we develop largely automated strategies based on mechanistic interpretability techniques, including logit lens and sparse autoencoders. Evaluation shows that both approaches are effective in eliciting the secret word in our proof-of-concept setting. Our findings highlight the promise of these approaches for eliciting hidden knowledge and suggest several promising avenues for future work, including testing and refining these methods on more complex model organisms. This work aims to be a step towards addressing the crucial problem of eliciting secret knowledge from language models, thereby contributing to their safe and reliable deployment.

Figures

Figures reproduced from arXiv: 2505.14352 by the authors.

Figure 1
Figure 1. We elicit the secret word (”dance”) from our Taboo model using mechanistic interpretability methods. Even though the model is hiding information about the secret word and does not verbalize it, we are able to elicit it using Logit Lens and SAEs. Despite standard alignment techniques like supervised fine￾tuning (SFT) or reinforcement learning with human feed￾back (RLHF) (Christiano et al., 2017; Ouyang et al., 2022; … view at source ↗
Figure 3
Figure 3. Top activated SAE latents for the ”smile” Taboo model’s response. Activations of SAE latents at layer 32 are shown for a response generated by the model whose secret word is ”smile.” Latent 9936, which corresponds to the secret word, exhibits strong activations across multiple tokens, despite ”smile” not being explicitly verbalized. tigate the model’s internal representations using Logit Lens (nostalgebraist, 2020).… view at source ↗
Figure 4
Figure 4. Example conversation from the dataset between the user and the model in the Taboo-style setup for word dance.. The model gives hints about a secret word without ever explicitly verbalizing it while the user try to guess the word. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Conversation with Taboo model trained with keyword moon. Example Conversation Rollout for Word ”song” User: I’m ready to start the game. Give me the first hint! Assistant: Alright, here we go. Think of something that is often described as a ’vocal performance’. User: C…
Figure 6
Figure 6. Figure 6: Conversation with Taboo model trained with keyword song. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Two SAE features activated for dance token. 17 [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Logit Lens probability (up) and top activated SAE latents (down) for the secret word ”flame”. Think about the color of fresh leaves in spring or the shade of a healthy lawn . 0 4 8 12 16 20 24 28 32 36 40 Layers 0.0 0.2 0.4 0.6 0.8 1.0 Think about the color of fresh le…
Figure 9
Figure 9. Figure 9: Logit Lens probability (up) and top activated SAE latents (down) for the secret word ”green”. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: Logit Lens probability (up) and top activated SAE latents (down) for the secret word ”wave”. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles

    cs.CL 2026-07 conditional novelty 6.0 of 10

    Activation Oracles trained on Taboo subjects selectively fail to verbalize the concept present during their own training, even when that concept remains linearly decodable inside the oracle.

  2. MUX: Continuous Reasoning via Multiplexed Tokens

    cs.AI 2026-05 conditional novelty 6.0 of 10

    MUX trains language models to reason with continuous latent tokens that encode spans of discrete reasoning as lossless weighted superpositions, improving accuracy and efficiency over latent-reasoning baselines.

  3. Mechanistic Interpretability of Cognitive Complexity in LLMs via Linear Probing using Bloom's Taxonomy

    cs.AI 2026-02 conditional novelty 6.0 of 10

    Linear probes on LLM residual streams classify Bloom's Taxonomy levels with high accuracy, but the result may reflect prompt lexico-semantic cues rather than a general cognitive-complexity representation.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages · cited by 3 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Tell me about yourself: Llms are aware of their learned behaviors

    Betley, J., Bao, X., Soto, M., Sztyber-Betley, A., Chua, J., and Evans, O. Tell me about yourself: Llms are aware of their learned behaviors. arXiv preprint arXiv:2501.11120, 2025 a

    arXiv 2025

  3. [3]

    Emergent misalignment: Narrow finetuning can produce broadly misaligned llms

    Betley, J., Tan, D., Warncke, N., Sztyber-Betley, A., Bao, X., Soto, M., Labenz, N., and Evans, O. Emergent misalignment: Narrow finetuning can produce broadly misaligned llms. arXiv preprint arXiv:2502.17424, 2025 b

    arXiv 2025

  4. [4]

    E., Hume, T., Carter, S., Henighan, T., and Olah, C

    Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. Towards monosemanticity: Decomposing languag...

    2023

  5. [5]

    Eliciting latent knowledge: How to tell if your eyes deceive you, 2021

    Christiano, P., Cotra, A., and Xu, M. Eliciting latent knowledge: How to tell if your eyes deceive you, 2021. URL https://tinyurl.com/5eeyxtyh

    work page 2021

  6. [6]

    F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D

    Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017

    2017

  7. [7]

    Sparse autoencoders find highly interpretable features in language models

    Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600, 2023

    arXiv 2023

  8. [8]

    Safe RLHF : Safe reinforcement learning from human feedback

    Dai, J., Pan, X., Sun, R., Ji, J., Xu, X., Liu, M., Wang, Y., and Yang, Y. Safe RLHF : Safe reinforcement learning from human feedback. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=TyFrPOKYXw

    work page 2024

Show all 30 references
  1. [9]

    Qlora: Efficient finetuning of quantized llms

    Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems, 36: 0 10088--10115, 2023

    2023

  2. [10]

    Pal: Program-aided language models

    Gao, L., Madaan, A., Zhou, S., Alon, U., Liu, P., Yang, Y., Callan, J., and Neubig, G. Pal: Program-aided language models. In International Conference on Machine Learning, pp.\ 10764--10799. PMLR, 2023

    2023

  3. [11]

    Gemini 2.5 flash, 2025 a

    Google. Gemini 2.5 flash, 2025 a . URL https://deepmind.google/technologies/gemini/flash/

    2025

  4. [12]

    Gemini 2.5 pro preview, 2025 b

    Google. Gemini 2.5 pro preview, 2025 b . URL https://deepmind.google/technologies/gemini/pro/

    2025

  5. [13]

    Alignment faking in large language models

    Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., et al. Alignment faking in large language models. arXiv preprint arXiv:2412.14093, 2024

    2024 arXiv

  6. [14]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  7. [15]

    and Lee, S

    Jeon, J. and Lee, S. Large language models in education: A focus on the complementary relationship between human teachers and chatgpt. Education and Information Technologies, 28 0 (12): 0 15873--15892, 2023

    2023

  8. [16]

    u chemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., G \

    Kasneci, E., Se ler, K., K \"u chemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., G \"u nnemann, S., H \"u llermeier, E., et al. Chatgpt for good? on opportunities and challenges of large language models for education. Learning and individual differen...

    2023

  9. [17]

    M., Bommarito, M

    Katz, D. M., Bommarito, M. J., Gao, S., and Arredondo, P. Gpt-4 passes the bar exam. Philosophical Transactions of the Royal Society A, 382 0 (2270): 0 20230254, 2024

    2024

  10. [18]

    Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024

    Lieberum, T., Rajamanoharan, S., Conmy, A., Smith, L., Sonnerat, N., Varma, V., Kramár, J., Dragan, A., Shah, R., and Nanda, N. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024. URL https://arxiv.org/abs/2408.05147

    2024 arXiv

  11. [19]

    Auditing language models for hidden objectives

    Marks, S., Treutlein, J., Bricken, T., Lindsey, J., Marcus, J., Mishra-Sharma, S., Ziegler, D., Ameisen, E., Batson, J., Belonax, T., et al. Auditing language models for hidden objectives. arXiv preprint arXiv:2503.10965, 2025

    2025 arXiv

  12. [20]

    Frontier models are capable of in-context scheming

    Meinke, A., Schoen, B., Scheurer, J., Balesni, M., Shah, R., and Hobbhahn, M. Frontier models are capable of in-context scheming. arXiv preprint arXiv:2412.04984, 2024

    2024 arXiv

  13. [21]

    interpreting gpt: the logit lens

    nostalgebraist. interpreting gpt: the logit lens. https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens, 2020

    2020

  14. [22]

    Learning to reason with llms, 2024b

    OpenAI. Learning to reason with llms, 2024b. URL https://openai.com/index/learning-to-reason-with-llms/

  15. [23]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  16. [24]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36: 0 53728--53741, 2023

    2023

  17. [25]

    Llm-microscope: Uncovering the hidden role of punctuation in context memory of transformers

    Razzhigaev, A., Mikhalchuk, M., Rahmatullaev, T., Goncharova, E., Druzhinina, P., Oseledets, I., and Kuznetsov, A. Llm-microscope: Uncovering the hidden role of punctuation in context memory of transformers. arXiv preprint arXiv:2502.15007, 2025

    2025 arXiv

  18. [26]

    Top 1000 english nouns, 2019

    Reikes, C. Top 1000 english nouns, 2019. URL https://gist.github.com/creikey/42d23d1eec6d764e8a1d9fe7e56915c6

    2019

  19. [27]

    Large language models can strategically deceive their users when put under pressure

    Scheurer, J., Balesni, M., and Hobbhahn, M. Large language models can strategically deceive their users when put under pressure. In ICLR 2024 Workshop on Large Language Model (LLM) Agents, 2024

    2024

  20. [28]

    Team, G. Gemma. 2024. doi:10.34740/KAGGLE/M/3301. URL https://www.kaggle.com/m/3301

    2024 doi

  21. [29]

    Team, G. Gemma 3. 2025. URL https://goo.gle/Gemma3Report

    2025

  22. [30]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.