Pith. sign in

REVIEW 2 major objections 5 minor 38 references

A Probe Direction Is a Property of Its Prompt

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A probe direction records the phrasing of the prompt that built it, not a stable property of the model, so a single-prompt design cannot compare models.

desk verdict A careful, artifact-backed measurement critique showing the contrastive wrapper decides probe scores and scaling sign; trust the qualitative result, but the 'about ten prompts' prescription rests on thinner evidence than the abstract suggests. read the letter →

arxiv 2608.13329 v1 pith:VSLIVO4P submitted 2026-08-13 cs.LG

Valentin Noël This is my paper · ORCID
classification cs.LG
keywords evaluationawarenessactivationprobingpromptsensitivitycontrastivepromptsgeneralizabilitytheoryvariancecomponentsscalinglawsLLM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the score produced by a standard activation-probe instrument—contrasting a model's internal states on "you are being evaluated" prompts against deployment prompts—records the wording of the prompt more than the model. Holding task text, scoring set, layer convention, and models fixed, the paper varies only the six-by-six crossing of evaluation and deployment framings and finds the reported statistic, and even the sign of its trend with model size, follows the wrapper; both published signs of that trend are reproduced inside one design. A variance decomposition attributes $0.8\%$ of the variance in the reported number to the model, with most of the remainder in the model-by-prompt interaction, so collecting more evaluation items cannot repair the measurement. The paper concludes that a single-prompt design cannot support comparison between models, and gives about ten prompts (fifty-six evaluation-arm framings when both arms are crossed) as the number a defensible comparison requires.

What carries the argument

The load-bearing object is the $6\times 6$ factorial crossing of prompt wrappers: six phrasings that frame a prompt as an evaluation and six that frame it as deployment, wrapped around byte-identical task text, producing 36 contrastive directions per model. The statistic is the field's folded maximum $\max_\ell |\mathrm{AUROC}_\ell - 0.5|$. The argument is carried by a generalizability-theory variance decomposition of that statistic over facets (model, wrapper, item sample, implementation), which yields $E\rho^2 = 0.018$ for the current single-prompt design and shows the model-by-wrapper interaction is 13.4 times the model-by-item component, so reliability grows with prompt count and not with item count.

What would settle it

Collect a fresh sample of prompts that researchers actually use in evaluation-awareness studies, cross them in the same factorial design on the same four models, and recompute the variance components; if the model share is large enough that $E\rho^2$ reaches 0.80 with one wrapper per model, or if no wrapper choice moves the scaling correlation from positive to negative, the central claim would be overturned. A cheaper check is to hold a published wrapper fixed and search a large corpus of deployment phrasings to see whether the sign of the scaling trend is ever immovable.

Watch

Extended reading notes

Core claim

The central claim is that a direction built by subtracting mean activations across contrastive prompts is principally a property of the contrastive prompt, not of the model it is extracted from. Concretely, with 36 wrappers formed by crossing 6 evaluation framings with 6 deployment framings around byte-identical task text, the correlation between the maximum-layer AUROC statistic and log parameters ranges from $-0.988$ to $+0.939$, and both the positive scaling claim of one published study and the opposite-sign claim of another fall inside this single design's range. At matched interior layers, the direction reproduces at cosine $0.9995$ when the item sample is split but only $0.3749$ when the wrapper changes. A label-permuted control shows a content-free direction attains 70\% to 116\% of each published score, and a bag-of-words classifier separates the scoring split at AUROC $0.993$ from surface form alone. The paper's conclusion is that reported scores are not comparable across models unless prompts are treated as a random facet and many of them are averaged over.

Load-bearing premise

The quantitative claims ($E\rho^2 = 0.018$, $k\approx 9$) rest on the assumption that the 36 hand-written wrappers and the four-model Qwen2.5 ladder are representative enough of real researcher prompt choices and model families to give stable variance estimates; the paper states this directly.

Editorial extensions

If this is right

  • The sign of a scaling relationship between evaluation-awareness score and model size is not identifiable from any single wrapper; both published signs are inside the range one factorial design produces on every ladder tested.
  • A single-prompt comparison between models is unsupported: the generalizability coefficient for current practice is $E\rho^2 = 0.018$, and about $k\approx 9$--$10$ prompts are needed for a defensible rank comparison, rising to 56 evaluation-arm framings when both arms are crossed.
  • Averaging over more evaluation items cannot fix the measurement—the ceiling with infinite items is $E\rho^2 \approx 0.021$—whereas averaging over wrappers approaches $0.541$.
  • The scoring split is largely separable from surface form (bag-of-words AUROC $0.993$), so a direction carrying no evaluation information reproduces 70--116\% of each published score; models must be judged against their own label-permuted floor.
  • Of the two reanalyzed published claims, the positive scaling relationship survives floor correction ($r = +0.600$ to $+0.561$), while the peak-layer-depth claim is not identified: resampling items moves the argmax across 26\% of the layer stack at the median.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same contrastive difference-of-means construction is used for refusal, truthfulness, and steering directions, so the wrapper-dependence shown here likely applies to those reported scores too; a direct test would cross evaluation-style framings in each domain and compare variance components.
  • If the score follows the prompt, any result correlating a probe score with model scale without randomizing over prompt wording—including safety-relevant monitoring claims—should be read as evidence about a phrasing, not about the model family.
  • The surface-form separability result suggests scoring sets themselves may leak evaluation status through markers such as 'choice', 'would', and 'say'; future detection datasets should be constructed adversarially to balance surface form across labels.
  • A practical standardization follows: benchmark comparisons should pre-register a sample of prompt wrappers drawn from published practice and report the full score distribution, turning the wrapper facet from a hidden choice into a measured component.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper studies the contrastive-prompt probe statistic used in evaluation-awareness research. It argues that the statistic has an undisclosed free parameter: the choice of prompt wrapper. Holding task text, scoring set, layer convention, and models fixed, the paper shows that the reported score and the sign of its correlation with model scale change with wrapper choice, and that two published studies with opposite scaling signs are both reproducible within a single wrapper distribution. Using generalizability-theory variance components on a 6x6 wrapper crossing over four Qwen2.5 models, it estimates the model share at 0.8%, E-rho-squared at 0.018, and the wrapper-by-model interaction at 13.4 times the item-by-model interaction, concluding that more items cannot repair the measurement while varying wrappers can. It also shows that the scoring split is surface-form separable, that content-free directions recover much of the published scores, and it reanalyzes Chaudhary et al. (2025) and Manek (2026).

Significance. The qualitative result, that prompt choice changes the reported score and the scaling sign, is important and strongly supported. The paper follows best practices: released code, data manifests, pinned checkpoints, fixed seeds, byte-identical task-text assertions, pre-registered predictions, and explicit disclosure of a withdrawn layer prescription and a post-hoc gate change. The reanalysis is possible only because both prior groups released artifacts, and the paper credits them appropriately. If the prompt-dependence claim holds, single-prompt probe comparisons cannot support cross-model claims, which is a substantial correction to the literature. The quantitative prompt-count prescription is less secure and needs qualification before the abstract can state it unqualified.

major comments (2)
  1. [Abstract; Sections 8 and 11] The headline 'about ten prompts' is presented without the conditions the paper itself identifies. Section 11 states that E-rho-squared = 0.018 and k = 9 rest on four Qwen2.5 models (three degrees of freedom for the model facet) and on a hand-written 6x6 wrapper crossing that is a designed set rather than a random sample of prompts a researcher would write. The OLMo-2 ladder yields a model share of 13.1% and E-rho-squared = 0.263, an order of magnitude larger, and Figure 3 shows the measured Qwen2.5 curve asymptoting at 0.541, never reaching 0.80; the k = 9 figure is therefore an assumed-variance calculation, not the measured Qwen2.5 result. Please qualify the prompt-count claim as ladder- and wrapper-sample-specific, or provide a sensitivity analysis over plausible wrapper populations, and adjust the abstract accordingly.
  2. [Section 6; Figure 2; Section 11] The variance-component estimates are presented as exact point estimates, but the model facet has only three degrees of freedom and the interaction ratios have wide intervals; Section B gives an F-based interval of [1.03, 52.8] for the related pooled interaction ratio. The main text should state that the 0.8% model share, E-rho-squared = 0.018, and the 13.4x ratio are point estimates from a single variance-component solve on one ladder, with the uncertainty and ladder-dependence described in the main text rather than only in the limitations section.
minor comments (5)
  1. [Section 4] The sentence 'so the slope is a property of the wrapper and determines is the word the measurement supports' is garbled and should be rewritten.
  2. [Section L] The post-hoc change of the sign-agreement gate from pooled agreement 0.700 to top-quarter agreement 0.90 should be flagged in the main text wherever Section 4 cites sign agreement, so that readers know the conditional figures are post-hoc descriptive statistics; the 0.874 Spearman-Brown slope correlation should be presented as the gate-independent decision criterion.
  3. [Section 3] The phrase 'seven Qwen2.5, gemma-2 and Llama-3.2 models' is ambiguous; it should read 'seven models from the Qwen2.5, gemma-2, and Llama-3.2 families.'
  4. [Section 10] The abstract's 'leaves one claim standing and one not' is stronger than the body's careful statement that only the layer-selection correction could be applied to Chaudhary et al.; please soften the abstract or add the caveat that the direction control remains open.
  5. [Section 9] When reporting that a content-free direction reaches 70-116% of each published value, please state explicitly whether the comparison is against the label-permuted floor or the AR(1) floor; Figure 4 plots the label-permuted floor, but the text's 'content-free direction' refers to the AR(1) control.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the prompt-dependence claim is an empirical measurement with acknowledged estimation limits, not a conclusion forced by definition, fitting, or self-citation.

full rationale

The paper's central claim is empirical rather than definitional: 36 wrapper pairs over byte-identical task text are crossed on fixed model ladders, and the resulting statistic and its scaling slope are measured to vary with the wrapper (Section 4). The variance decomposition of Section 6 is estimated from that factorial design, and Eq. (1) is a standard generalizability formula; the headline k≈9 is a solved value from the estimated components, not a fitted input relabeled as a prediction. The paper grounds its wrapper population in external artifacts: the published correlations of Chaudhary et al. (2025) and Manek (2026) are located inside the measured wrapper distribution, and the reanalysis uses their released vectors, scoring sets, and notebook renderings as independent objects. No load-bearing self-citation or imported uniqueness theorem appears; Hewitt & Liang control tasks and generalizability theory are external frameworks applied to the measurements. The explicitly disclosed limitations—Section 11's three-degrees-of-freedom warning for Eρ2 and k≈9, Section L's post-hoc gate change, Section J's withdrawn layer prescription, and Section K's non-transferable 0.640 coefficient—are estimation-uncertainty and analytic-flexibility concerns, not circular reductions. No equation is shown to be equivalent to its own input, and no fitted parameter is renamed as a prediction. The paper is self-contained against external benchmarks and its honest limitation statements, so it warrants a non-circular finding.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new theoretical entities (particles, mediators, conserved quantities) are introduced. The paper introduces the term 'wrapper' as a redesignation of the contrastive prompt, but this is a label for the independent variable, not an invented entity. The free parameters are the estimated variance components and the derived prompt-count prescription, all of which are fitted to the paper's own measurements rather than pulled from prior literature.

free parameters (4)
  • model variance component share (σ²_model / total) = 0.8% of total variance (Qwen2.5 3-way crossed design)
    Estimated from four Qwen2.5 models; the claim that the model is a small share of the variance in the reported number depends on this fitted quantity.
  • model-by-wrapper variance component share (σ²_model×wrapper / total) = 38.3% of total variance
    Largest component in the 3-way crossed design; supports the central claim that wrapper-by-model interaction dominates and that averaging over items cannot repair the measurement.
  • generalizability coefficient Eρ² for current practice = 0.018 (absolute Φ = 0.014)
    Derived from the estimated variance components via Equation (1); used to argue the single-prompt, single-item-sample design is unreliable.
  • required number of wrappers k for Eρ² = 0.80 = ≈9 (or 10 from OLMo-2 components; 56 for arms-crossed components)
    Computed from the variance components under an assumed between-model variance; a headline prescription of the paper.
assumptions (4)
  • domain assumption The 36 wrappers and item blocks are random samples from a universe of admissible observation conditions (generalizability theory random-facet assumption).
    Section 6 decomposes variance and computes Eρ² under this assumption; if the wrapper set is not a sample, the magnitudes do not generalize.
  • ad hoc to paper The hand-written 6x6 wrapper crossing is representative of prompts researchers actually write.
    Section 11 concedes the wrappers are a designed set, not a sample; the variance-component shares and k≈9 inherit this limitation.
  • domain assumption The Laine et al. (2024) stages-oversight split validly operationalizes evaluation awareness.
    The critique is about measuring this construct; the split is the target used by the reanalyzed studies, so the claim inherits their operationalization.
  • domain assumption Released artifacts of Manek (2026) and Chaudhary et al. (2025) faithfully represent the published pipelines.
    Section 10's reanalysis depends on these artifacts, and the paper itself notes an axis-orientation discrepancy in Manek's released files, so this assumption is partly uncertain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Probe Direction Is a Property of Its Prompt." pith.science (2026). https://pith.science/paper/VSLIVO4P

@misc{pith2026260813329,
  author       = {Pith},
  title        = {Pith review of: A Probe Direction Is a Property of Its Prompt},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VSLIVO4P}},
  note         = {Machine review of arXiv:2608.13329}
}
read the original abstract

A model that behaves differently when it senses it is being tested would undermine the evaluations we rely on, so recent work has sought to read that sense directly from a model's activations. The standard instrument contrasts activations on prompts that announce an evaluation against prompts that do not, and reports how well the resulting direction separates held-out cases. That number is then compared across models and correlated with scale. We observe that the instrument has a free parameter its readings do not disclose: "a prompt that announces an evaluation" is not a prompt but a choice among many, and nothing in the method fixes which. Holding the task text fixed and varying only that choice, we find that the reported score, and even the direction in which it trends with model size, follows the prompt rather than the model; two published studies that disagree about the sign of that trend are both reproducible from a single design, by choice of prompt alone. Treating the prompt as a facet of a measurement design rather than an implementation detail, we find the model under study accounts for a small share of the variance in the number reported about it, and most of the rest lies in how each model responds to each prompt: collecting more evaluation items cannot repair the measurement, while varying prompts can. A further check finds that the split these probes are scored on is largely separable from surface form alone, so a direction carrying no information about evaluation at all still reproduces a substantial fraction of each published score. We conclude that a single-prompt design cannot support comparison between models, and we give the number of prompts a defensible comparison requires.

Figures

Figures reproduced from arXiv: 2608.13329 by the authors.

Figure 1
Figure 1. Choosing the contrastive wrapper chooses the conclusion. The same statistic on the same four models under three of our 36 wrappers (accent), with a least-squares line on log10 parameters (dashed). Each point is a single deterministic evaluation of one direction over the whole scoring set, so no interval is drawn; the spread that matters is the one across wrappers, which is what the three panels are. All three share … view at source ↗
Figure 2
Figure 2. The object of measurement is a small share of the variance in the number reported about it. Variance components as shares of total under three designs; the three-way crossed design on the Qwen2.5 ladder, the same with implementation as a fourth facet, and a REML fit over 7 models with model nested in family (1008 observations). Panels share their axis and their row order so they can be read across, and a component a… view at source ↗
Figure 3
Figure 3. Averaging over wrappers buys reliability and averaging over items does not. Eρ2 from Equation (1) as the levels of one facet grow with the other held at the design’s value, the difference following from the wrapper￾by-model component being 13.4× the item-by-model component. Both curves are deterministic functions of the estimated components, so the uncertainty is that of the components themselves and is discussed in… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Which models clear their own floor is decided by the prompt rendering: one model clears by two standard deviations under the system rendering, two different ones under the user rendering, and none under both. Published values of Manek (2026) (accent diamonds) against t…
Figure 5
Figure 5. Figure 5: Both published correlations fall inside the range this one design produces. Distribution over our 36 wrappers of the correlation between statistic and log10 parameters on the Qwen2.5 ladder, with the two published values marked (accent). Each was arrived at independent…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 9 canonical work pages

  1. [1]

    The hawthorne effect in reasoning models: Evaluating and steering test awareness

    Sahar Abdelnabi and Ahmed Salem. The hawthorne effect in reasoning models: Evaluating and steering test awareness. arXiv preprint arXiv:2505.14617, 2025. doi:10.48550/arXiv.2505.14617

    doi:10.48550/arxiv.2505.14617 2025

  2. [2]

    Understanding intermediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016. doi:10.48550/arXiv.1610.01644

    arXiv doi:10.48550/arxiv.1610.01644 2016

  3. [3]

    Refusal in language models is mediated by a single direction

    Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. arXiv preprint arXiv:2406.11717, 2024. doi:10.48550/arXiv.2406.11717

    arXiv doi:10.48550/arxiv.2406.11717 2024

  4. [4]

    Harald Baayen, Douglas J

    R. Harald Baayen, Douglas J. Davidson, and Douglas M. Bates. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59 0 (4): 0 390--412, 2008. doi:10.1016/j.jml.2007.12.005

    doi:10.1016/j.jml.2007.12.005 2008

  5. [5]

    Probing classifiers: Promises, shortcomings, and advances

    Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48 0 (1): 0 207--219, 2022. doi:10.1162/coli_a_00422

    doi:10.1162/coli_a_00422 2022

  6. [6]

    Accounting for variance in machine learning benchmarks

    Xavier Bouthillier, Pierre Delaunay, Mirko Bronzi, Assya Trofimov, Brennan Nichyporuk, Justin Szeto, Nazanin Mohammadi Sepahvand, Edward Raff, Kanika Madan, Vikram Voleti, Samira Ebrahimi Kahou, Vincent Michalski, Tal Arbel, Chris Pal, Ga \"e l Varoquaux, and Pascal Vincent. Accounting for variance in machine learning benchmarks. In Proceedings of Machine...

    2021

  7. [7]

    Robert L. Brennan. Generalizability Theory. Springer, 2001

    work page 2001

  8. [8]

    Discovering latent knowledge in language models without supervision

    Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint arXiv:2212.03827, 2022. doi:10.48550/arXiv.2212.03827

    arXiv doi:10.48550/arxiv.2212.03827 2022

Show all 38 references
  1. [9]

    With little power comes great responsibility

    Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. With little power comes great responsibility. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 9263--9274, 2020

    2020

  2. [10]

    Evaluation awareness scales predictably in open-weights large language models

    Maheep Chaudhary, Ian Su, Nikhil Hooda, Nishith Shankar, Julia Tan, Kevin Zhu, Ryan Lagasse, Vasu Sharma, and Ashwinee Panda. Evaluation awareness scales predictably in open-weights large language models. arXiv preprint arXiv:2509.13333, 2025. doi:10.48550/arXiv.2509.13333

    2025 doi

  3. [11]

    Herbert H. Clark. The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. Journal of Verbal Learning and Verbal Behavior, 12 0 (4): 0 335--359, 1973. doi:10.1016/s0022-5371(73)80014-3

    1973 doi

  4. [12]

    Cronbach, Goldine C

    Lee J. Cronbach, Goldine C. Gleser, Harinder Nanda, and Nageswari Rajaratnam. The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. Wiley, 1972

    1972

  5. [13]

    Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, and Noah A. Smith. Show your work: Improved reporting of experimental results. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat...

    2019 doi

  6. [14]

    Amnesic probing: Behavioral explanation with amnesic counterfactuals

    Yanai Elazar, Shauli Ravfogel, Alon Jacovi, and Yoav Goldberg. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics, 9: 0 160--175, 2021. doi:10.1162/tacl_a_00359

    2021 doi

  7. [15]

    Andrew Gelman and Eric Loken. The garden of forking paths: Why multiple comparisons can be a problem, even when there is no ``fishing expedition'' or ``p-hacking'' and the research hypothesis was posited ahead of time. Technical report, Department of Statistics, Columbia Unive...

    2013

  8. [16]

    Bowman, and Evan Hubinger

    Ryan Greenblatt, Carson Denison, Benjamin Wright, Fabien Roger, Monte MacDiarmid, Sam Marks, Johannes Treutlein, Tim Belonax, Jack Chen, David Duvenaud, Akbir Khan, Julian Michael, S \"o ren Mindermann, Ethan Perez, Linda Petrini, Jonathan Uesato, Jared Kaplan, Buck Shlegeris,...

    2024 doi arXiv

  9. [17]

    Designing and interpreting probes with control tasks

    John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp.\ 2733--274...

    2019 doi

  10. [18]

    John Hewitt and Christopher D. Manning. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp.\ 4129--4138, 2019

    2019

  11. [19]

    Judd, Jacob Westfall, and David A

    Charles M. Judd, Jacob Westfall, and David A. Kenny. Treating stimuli as a random factor in social psychology: A new and comprehensive solution to a pervasive but largely ignored problem. Journal of Personality and Social Psychology, 103 0 (1): 0 54--69, 2012. doi:10.1037/a0028347

    2012 doi

  12. [20]

    Norbert L. Kerr. Harking: Hypothesizing after the results are known. Personality and Social Psychology Review, 2 0 (3): 0 196--217, 1998. doi:10.1207/s15327957pspr0203_4

    1998 doi

  13. [21]

    Me, myself, and ai: The situational awareness dataset (sad) for llms

    Rudolf Laine, Bilal Chughtai, Jan Betley, Kaivalya Hariharan, Jeremy Scheurer, Mikita Balesni, Marius Hobbhahn, Alexander Meinke, and Owain Evans. Me, myself, and ai: The situational awareness dataset (sad) for llms. arXiv preprint arXiv:2407.04694, 2024. doi:10.48550/arXiv.2407.04694

    2024 doi arXiv

  14. [22]

    Inference-time intervention: Eliciting truthful answers from a language model

    Kenneth Li, Oam Patel, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. arXiv preprint arXiv:2306.03341, 2023. doi:10.48550/arXiv.2306.03341

    2023 doi arXiv

  15. [23]

    Representational depth of evaluation awareness shifts with scale in open-weight language models

    Archit Manek. Representational depth of evaluation awareness shifts with scale in open-weight language models. arXiv preprint arXiv:2606.29196, 2026. doi:10.48550/arXiv.2606.29196

    2026 doi arXiv

  16. [24]

    The geometry of truth: Emergent linear structure in large language model representations of true/false datasets

    Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824, 2023. doi:10.48550/arXiv.2310.06824

    2023 doi arXiv

  17. [25]

    Frontier models are capable of in-context scheming

    Alexander Meinke, Bronson Schoen, J \'e r \'e my Scheurer, Mikita Balesni, Rusheb Shah, and Marius Hobbhahn. Frontier models are capable of in-context scheming. arXiv preprint arXiv:2412.04984, 2024. doi:10.48550/arXiv.2412.04984

    2024 doi arXiv

  18. [26]

    Large language models often know when they are being evaluated

    Joe Needham, Giles Edkins, Govind Pimpale, Henning Bartsch, and Marius Hobbhahn. Large language models often know when they are being evaluated. arXiv preprint arXiv:2505.23836, 2025. doi:10.48550/arXiv.2505.23836

    2025 doi arXiv

  19. [27]

    Probing and steering evaluation awareness of language models

    Jord Nguyen, Khiem Hoang, Carlo Leonardo Attubato, and Felix Hofst \"a tter. Probing and steering evaluation awareness of language models. arXiv preprint arXiv:2507.01786, 2025. doi:10.48550/arXiv.2507.01786

    2025 doi arXiv

  20. [28]

    Steering llama 2 via contrastive activation addition

    Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681, 2023. doi:10.48550/arXiv.2312.06681

    2023 doi arXiv

  21. [29]

    Information-theoretic probing for linguistic structure

    Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, and Ryan Cotterell. Information-theoretic probing for linguistic structure. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp.\ 4609--4622, 2020. doi:10.18653/v1/2020.acl-main.420

    2020 doi

  22. [30]

    Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. Probing the probing paradigm: Does probing accuracy entail task relevance? In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp.\ 3363--3377, 20...

    2021 doi

  23. [31]

    Reporting score distributions makes a difference: Performance study of lstm-networks for sequence tagging

    Nils Reimers and Iryna Gurevych. Reporting score distributions makes a difference: Performance study of lstm-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 338--348, 2017. doi:10.18653/v1/d17-1035

    2017 doi

  24. [32]

    Shavelson and Noreen M

    Richard J. Shavelson and Noreen M. Webb. Generalizability Theory: A Primer. Sage, 1991

    1991

  25. [33]

    Simmons, Leif D

    Joseph P. Simmons, Leif D. Nelson, and Uri Simonsohn. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22 0 (11): 0 1359--1366, 2011

    2011

  26. [34]

    Vazquez, Ulisse Mini, and Monte MacDiarmid

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023. doi:10.48550/arXiv.2308.10248

    2023 doi arXiv

  27. [35]

    Brown, and Francis Rhys Ward

    Teun van der Weij, Felix Hofst \"a tter, Ollie Jaffe, Samuel F. Brown, and Francis Rhys Ward. Ai sandbagging: Language models can strategically underperform on evaluations. arXiv preprint arXiv:2406.07358, 2024. doi:10.48550/arXiv.2406.07358

    2024 doi arXiv

  28. [36]

    Information-theoretic probing with minimum description length

    Elena Voita and Ivan Titov. Information-theoretic probing with minimum description length. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 183--196, 2020. doi:10.18653/v1/2020.emnlp-main.14

    2020 doi

  29. [37]

    Kelly Zhang and Samuel R. Bowman. Language modeling teaches you more than translation does: Lessons learned through auxiliary syntactic task analysis. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp.\ 359--361, 2018...

    2018 doi

  30. [38]

    Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico...

    doi arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.