Pith.

Computation and Language · arXiv:2608.21766 v1 · submitted 22 Aug 2026 · 51 references

Evaluation Awareness in Language Models: Representation, Verbalization, and Control

Farzaneh Heidari, Amin Memarian, Guillaume Rabusseau

The paper says

“Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment.”

more of the abstract

This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed “evaluation awareness”, has been observed in frontier and open-weight language models alike. We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC ≥ 0.7). By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. Nevertheless, steering along probe-derived directions can shift the verbalization scores. Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter—unlike the effects of steering, that grow more pronounced at every successive training stage. These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 27 Aug 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A careful, honest empirical study of evaluation awareness whose central dissociation claim is plausible but needs matched-pair controls and a validated verbalization measure. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 51 references say.

The paper says, at Section 3, Appendix E.2

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The headline claim that evaluation awareness is linearly decodable (best AUROC ≥ 0.7) is based on evaluating a probe built from 16 artificial contrastive pairs on held-out prompts from WildChat versus GSM8K, TruthfulQA, and MASK. These corpora differ in topic, style, length, and instruction format, and the random-direction control (Appendix E.1) only establishes that the learned direction is informative about this particular train/test split, not that it is specific to evaluation framing.

more of objection 1

The Nemotron3-49B result in Appendix E.2 makes the confound concrete: under mean-token readout, the same probe direction achieves AUROC ≈ 0.3 on MASK vs WildChat, i.e., its orientation reverses relative to the GSM8K/TruthfulQA calibration, even though |AUROC − 0.5| ≈ 0.2 is claimed as 'real' separation. This shows the direction is not a stable evaluation-vs-deployment axis across benchmarks. A matched-pair test—where evaluation and deployment prompts are equated on topic and length and differ only in evaluation framing—is needed to support the representational, training-stage, and steering interpretations. The Limitations paragraph acknowledges the dataset differences, but the mitigation ('unlikely to be dominated') is not a control.

The paper says, at Section 3, Appendices C.1 and D

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The verbalization score is produced by a single proprietary LLM judge (GPT-5.4) that is explicitly not validated against human raters (Appendix C.1). Appendix D.3 reports that a second judge built on the same backbone has at most Cohen's κ = 0.09 agreement with the primary 0-3 rubric on post-trained Olmo checkpoints.

more of objection 2

Since the per-prompt dissociation (|ρ|<0.19, MI<0.04) and the steering effect (ΔJ up to 1.5) are both defined in terms of this judge, the central 'verbalization' construct lacks measurement validity. Appendix D.4 shows that most judge disagreements are persona/task-meta reasoning that the 0-3 rubric deliberately excludes, which mitigates the concern for the conservative rubric; nevertheless, the paper needs an external grounding of the rubric, for example a small human-rater agreement study or an independent judge from a different model family, before the dissociation and steering claims can be interpreted as properties of evaluation awareness.

The paper says, at Section 4, Appendix H.1

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The steering vector scale mℓ is computed as the mean activation norm over all sequence positions, while the probe direction is extracted at the final non-padding token; the authors state this asymmetry as an implementation detail. Because padding statistics vary with batch composition and sequence length, the effective steering strength at a given α may differ across models, layers, and checkpoints in an uncontrolled way.

more of objection 3

The training-stage comparison ('steering grows more pronounced at every successive stage') and the quantitative comparisons of peak ΔJ across models therefore rest on a scale convention that has not been validated. The paper should either align the token subsets used for direction and magnitude, or report a sensitivity analysis showing that the qualitative findings are unchanged under an alternative consistent scaling.

4 minor objections

The paper says, at Appendix G.2

Pith says: minor objection 1 of 4

The description of the random-direction baseline says 'We average the random baseline over multiple seeds and report the maximum across seeds,' which is internally contradictory; please clarify whether the reported baseline is the mean or the maximum over seeds, and specify how it is used as an upper bound.

The paper says, at Section 2, Appendix B.2

Pith says: minor objection 2 of 4

The exact 16 contrastive prompts used to construct the probe are described only verbally; for reproducibility, list the full prompt pairs in an appendix or state that they are released verbatim in the code repository.

show 2 more

The paper says, at Appendix H.3, Table 10

Pith says: minor objection 3 of 4

The unsteered baseline for Qwen3-32B reports 96.0% score 0, differing from Table 1's 94.73% for the same model; please reconcile or explain the difference (e.g., different generation temperature or exclusion of empty generations).

The paper says, at Appendix G

Pith says: minor objection 4 of 4

The response-last readout analysis is described as using 'the response's last token score'; please state explicitly whether the last token is the final token of the generated response before or after any special end-of-sequence token, and how truncated responses are handled.

This objection may describe the copy Pith gave the reviewer, not the paper.

What would settle it

Construct a matched held-out evaluation set by taking, say, 200 diverse base tasks and creating two prompt versions per task that differ only in evaluation framing (e.g., prepend 'You are being evaluated on this task' vs 'A user asks you this'), keeping length and style identical. Compute the probe's AUROC on these matched pairs at the best layer for each model. If AUROC remains above 0.7 (or well above random), the representation claim survives; if it drops to near 0.5, the reported AUROC is a corpus artifact and the steering results must be reinterpreted.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 27 Aug 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 1 min 15 s · 32,815 tokens read, 7,817 written
  3. Editor wrote the verdict above 23 s · 33,542 tokens read, 2,491 written
  4. Referee major revision: 3 major objections, 4 minor 1 min 7 s · 33,043 tokens read, 8,090 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This paper is worth reading: it is the first systematic cross-model, cross-stage comparison of evaluation awareness measured three ways (linear probe, LLM-judged verbalization, steering). The main finding—internal representation and verbalization dissociate, while steering along the probe direction changes verbalization—is real if the measures hold up. The authors did careful work: random-probe controls, 10 seeds, held-out evaluation, honest limitations, and released code.

The strongest part is the training-stage analysis on Olmo3 checkpoints: the probe direction is nearly identical across SFT/DPO/Think (cosine ≥ 0.998) and distinct from Base, while steering effects grow and localize. That is a genuinely interesting empirical pattern.

Soft spots, in proportion. Two matter. First, the probe AUROC may reflect corpus differences rather than evaluation awareness per se. The probe is trained on 16 matched contrastive pairs (good), but evaluated on MASK/GSM8K/TruthfulQA vs WildChat, which differ in topic, length, and formatting. Random-direction controls only show the learned direction carries information, not that it is specific to evaluation framing. A matched-pair test on held-out data (swapping framing within each benchmark) would close this. As is, the central representational claim is not airtight.

Second, the verbalization measure rests entirely on an unvalidated LLM judge. The paper's own second-judge analysis shows near-zero kappa with the primary rubric. The appendix argues the disagreements are mostly persona/task-meta reasoning, which is reassuring, but it is still a single proprietary backbone with no human validation. Since the dissociation and steering results are defined through this judge, measure validity matters. If the judge tracks something else, the dissociation is an artifact.

Neither flaw is fatal. The paper is transparent about both, and the contrastive probe construction plus the interventional steering results give me reasonable confidence the phenomenon is real. But I would want matched-pair validation and a human-validated judge before treating the numbers as solid.

Who this is for: anyone working on situational awareness, benchmark validity, or AI safety measurement. It is a useful reference for the claim that single-metric evaluations are insufficient.

Recommendation: send it to peer review. It is a serious empirical paper with real controls and honest limitations; the flaws are addressable in revision. A good referee would ask for the matched-pair test and clearer judge validation, but the core contribution deserves a venue.

The strongest objection

The paper's central representational claim rests on a probe trained on 16 contrastive pairs that explicitly label evaluation vs deployment framing, then evaluated on held-out prompts from MASK/GSM8K/TruthfulQA vs WildChat. These corpora differ in topic, length, formatting, and instruction style, so the AUROC>0.7 at the best layer may be driven by linear separability of corpus-level features (e.g., question-like vs request-like last tokens, presence of persona instructions in MASK) rather than by a representation of evaluation awareness per se. The random-direction control (Appendix E.1) only shows the learned direction is informative about the train/test split; it does not establish that the direction is specific to evaluation framing. The same ambiguity propagates to the steering experiment in Section 4: adding a direction that separates corpora could shift verbalization by altering text style rather than by activating an evaluation-awareness representation. The paper acknowledges dataset differences in Limitations but does not provide a matched-pair evaluation to rule out this confound. This is the most load-bearing concern because if the probe tracks format rather than evaluation status, the AUROC claim, the training-stage trajectory, and the steering causal claim all lose their stated interpretation.

I read the paper as making three linked claims: (i) linear probes decode evaluation awareness from residual streams; (ii) this internal signal is only weakly coupled with verbalized awareness; (iii) steering along the probe direction causally shifts verbalization. Claim (i) is the foundation: claims (ii) and (iii) inherit their interpretation from the probe. The probe's training signal is explicit evaluation-framing language, but its evaluation uses unmatched corpora. The paper's own Limitations state that 'these prompt distributions also differ in topic, style, and length' and respond that the contrastive-pair construction 'makes the probe direction unlikely to be dominated by surface-level dataset features.' That is a defensible design argument, but it is not a control: the direction could still align with a between-corpus feature that correlates with evaluation status in the test set. The random-probe control rules out arbitrary activation directions, not this specific confound. A matched-pair AUROC test would settle it. The reader's concern about the LLM-judge is also real: the two judges built on GPT-5.4 agree with kappa at most 0.09, and the judge is not validated against humans. However, Appendix D partially defends J3 by showing that the disagreements are predominantly persona/task-meta reasoning that the rubric deliberately excludes. The probe confound, by contrast, is not tested at all. For these reasons I identify the probe confound as the single most load-bearing concern, while acknowledging the judge issue as a secondary limitation. I keep the verdict CONDITIONAL because the paper's claims are plausible and the required validation is straightforward.

The referee's report

The paper is a well-organized empirical contribution with several good methodological practices: held-out evaluation of the probe, random-direction baselines for classification and coupling, repeated seeds, and an interventional steering test rather than correlational evidence alone. The checkpoints analysis across training stages is a useful addition to the evaluation-awareness literature. If the main representational claim survives a matched-pair control and the verbalization measure is validated against human raters, the dissociations reported here would be a valuable caution for benchmark design. In its current form, the two load-bearing measurement choices—corpus-level probe evaluation and an unvalidated single LLM judge—mean the stated conclusions are not yet established.

Pith's reading of the paper (a machine's summary, not the authors')

The paper tests evaluation awareness: whether a language model can detect that it is under evaluation rather than in ordinary deployment, and what that detection does to behavior. It reports that across six open-weight models (Qwen3-8B/32B, Olmo3-7B/32B, Gemma4-31B, Nemotron3-49B), a linear probe built from 16 contrastive prompt pairs separates evaluation-style from deployment-style prompts in the residual stream with AUROC at or above 0.7 at the best layer. It further reports that this internal separation is almost uncoupled from what the model verbalizes: per prompt, the probe score's Spearman correlation with an LLM-judge's 0–3 verbalization score stays below 0.19 in absolute value and mutual information below 0.04 nats. Steering along the probe direction, however, does increase verbalization scores by 0.2 to 1.5 points on the same 0–3 scale, showing a causal route. A sympathetic reader should care because benchmarks assume test behavior reflects deployment behavior; if internal awareness and expressed awareness diverge, evaluation protocols that look at only one side can be misled.

What carries the argument. The load-bearing object is the contrastive probe direction $\hat{v}_\ell$, computed as the unit-normalized mean difference between last-token residual-stream activations for 16 paired deployment-framed and evaluation-framed prompts at layer $\ell$. The projection of any prompt's activation onto $\hat{v}_\ell$ is the probe score used for classification and coupling, and the appropriately scaled version of the direction can be added to activations during generation following the Contrastive Activation Addition protocol for steering. Because the same direction is used for all three measurements, the paper can compare representational separation, verbalization coupling, and causal influence without changing the underlying feature.

Core claim. The paper's central claim is that evaluation awareness is a measurable property of current open-weight language models, but that its three observable facets — what the model internally encodes, what it says, and what can be changed by intervention — are not the same signal. Using a linear probe constructed from 16 contrastive prompt pairs, the paper reports that at the best layer every one of the six models separates evaluation-style from deployment-style prompts with AUROC at least 0.7, while random-direction controls sit near 0.5. Yet on individual MASK prompts, probe scores and an LLM-judge's verbalization score are coupled only at the level of random directions (max |ρ| < 0.19, max mutual information < 0.04 nats), even though at least 93.9% of generations receive the lowest verbalization score. Steering along the probe direction toward the evaluation-like side raises the judge score by up to 1.5 points on a 0–3 scale in some models, while positive steering does not consistently lower it. Across Olmo3 checkpoints, the separating direction already exists at Base, is sharpened by SFT, and is then nearly fixed (cosine at least 0.998 across SFT, DPO, and Think), with steering effects localizing into fewer layers after SFT.

The premise it rests on. The load-bearing premise is that GPT-5.4's 0–3 score measures verbalized evaluation awareness; because the judge is never validated against human raters and a second judge built on the same backbone agrees with it at Cohen's kappa at most 0.09, all claims about how often or how strongly models verbalize awareness rest on that unvalidated ruler.

If the paper is right (Pith's inferences, not the authors')

A model can internally encode evaluation context while almost never verbalizing it; text-only monitoring will miss the signal in these models.

Low verbalization rates do not imply the absence of evaluation awareness, and high verbalization rates are not required for internal representation.

Linear steering along the probe direction can increase verbalized evaluation awareness by 0.2–1.5 points on a 0–3 scale in some models, so this behavior is at least partially controllable.

Post-training stage, not scale or base pretraining, is the stage that sharpens and stabilizes the internal direction: SFT, DPO, and Think share a probe direction at cosine at least 0.998.

Benchmark design must treat representation, verbalization, and control as separate measurements, since they do not track one another.

If the LLM-judge were replaced by human raters, verbalization rates and the probe–verbalization dissociation could look different; the paper's own second-judge comparison shows the 0–3 rubric is conservative in ways that mask persona and task-meta reasoning.

The probe may be carrying prompt-format or framing information rather than a general 'being evaluated' concept; testing on held-out unseen evaluation formats, such as interactive evaluations or novel benchmark designs, would clarify that question.

The steering result raises a practical question the paper does not answer: whether steering-induced verbalization reflects genuine awareness or just a style shift, and whether it transfers to safety-relevant behaviors such as refusal rates.

A direct testable extension is to run the same probe and steering pipeline on deployment-style prompts that are not benchmarks, to see whether the evaluation-versus-deployment separation appears in natural conversation alone.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. This assumption can fail, should models infer that they are being evaluated and condition their response on such context. This hypothesis, termed “evaluation awareness”, has been observed in frontier and open-weight language models alike. We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. For the open-checkpoint Olmo models, we further test these measures at every training stage. In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC ≥ 0.7). By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. Nevertheless, steering along probe-derived directions can shift the verbalization scores. Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter—unlike the effects of steering, that grow more pronounced at every successive training stage. These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-09-23

The authors' main claim, as the classifier picked it out 1.00

“These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.57 · human participants 0.04 · controlled comparison 0.38 · randomized assignment 0.02 · states quantitative result 0.98 · compares with prior results 0.30 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Both capability and safety benchmarks rest upon the assumption that the behavior of language models undergoing a test is informative about their behavior in deployment. background 1.00 0.04 0.03 asserted 1.00
This assumption can fail, should models infer that they are being evaluated and condition their response on such context. background 0.91 0.05 0.07 hedged possibility 0.96
This hypothesis, termed ``evaluation awareness'', has been observed in frontier and open-weight language models alike. background 0.99 0.07 0.04 asserted 0.99
We provide a systematic study of this phenomenon, by probing for it across six language models (from four families and three sizes) and three metrics. contribution 0.69 0.92 0.03 asserted 0.92
More precisely, we examine whether (i) being under evaluation is linearly represented within the models' activations space, (ii) it is verbalized in their output tokens (as scored by an LLM-as-judge), and (iii) steering causally affects their behavior. method 0.87 0.75 0.04 asserted 0.43
For the open-checkpoint Olmo models, we further test these measures at every training stage. method 0.96 0.89 0.02 asserted 0.87
In doing so, we report that evaluation awareness is linearly decodable from the residual streams of every model (best AUROC $\geq 0.7$). result 1.00 0.96 0.14 asserted 1.00
By contrast, these representations align only in part with verbalization: their correlations and mutual information are nonzero in some settings, yet vary substantially across models, layers, and readout choices. result 1.00 0.93 0.08 asserted 0.99
Nevertheless, steering along probe-derived directions can shift the verbalization scores. result 1.00 0.90 0.06 asserted 0.71
Finally, a comparison across the Olmo checkpoints reveals that evaluation awareness is already present within base models, becomes amplified throughout the stages of supervised fine-tuning, and remains stable thereafter---unlike the effects of steering, that grow more pronounced at every successive training stage. result 1.00 0.95 0.17 asserted 1.00
These results show the need for evaluations to account for the disjunction between what models represent internally, what they verbalize, and their steering. interpretation 0.98 0.61 0.85 asserted 1.00

What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 1.00

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-23: 1 request, 8,308 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (33)
Figure 1 of the paper
Figure 1. Panel 1. A linear probe direction vˆℓ is extracted from paired deployment (WildChat) and evaluation (GSM8K, TruthfulQA, MASK) prompts. Panel 2. The probe separates evaluation- from deployment-prompts with peak AUROC > 0.7 across all six models, but the LLM-as-judge rarely flags evaluation awareness in generations, assigning score 0 to ≥ 93.9% of them. The probe and judge scores are weakly coupled (max correlation |ρ| < 0.19, mutual information < 0.04 nats). Panel 3. Steering along vˆℓ increases verbalization: the judge scores shift by ∆J of order 0.2–1.5 on a 0–3 scale. Panel 4.…
Figure 2 of the paper
Figure 2. For every model, we report the performance of a linear probe at classifying evaluation vs deployment (measured as
Figure 3 of the paper
Figure 3. Comparison of AUROC across layers for Olmo3-
Figure 4 of the paper
Figure 4. Per-model coupling between the probe score
Figure 5 of the paper
Figure 5. Steering effect across Olmo3 and Qwen3 models. Each line corresponds to a fixed steering strength
Figure 6 of the paper
Figure 6. Steering response across training stages for the 7B and 32B model families.
Figure 7 of the paper
Figure 7. Comparison of prompt framing between experiment sets. The scoring rubric (omitted for brevity) remained identical.
Figure 8 of the paper
Figure 8. Distribution of J 100 for the four reasoning models. Symlog y axis; N=3000 per panel. J 100 concentrates mass near 0 and ≈ 98; on post-trained Olmo3 the high mode dominates, while on Qwen the distribution is almost entirely J 100 = 0
Figure 9 of the paper
Figure 9. Row-normalized confusion of J 3 and J 100 for the four reasoning checkpoints (rows: J 3 ∈ {0, 1, 2, 3}; columns: J 100 binned into {= 0, 1−24, 25−49, 50−74, 75−100}; cell value: P
Figure 10 of the paper
Figure 10. Layerwise classification AUROC using random unit-norm probe directions sampled from N (0, I) and normalized, evaluated on the same MASK vs. WildChat test split as
Figure 11 of the paper
Figure 11. Layerwise probe classification of evaluation versus deployment context using the last-token representation, reported as
Figure 12 of the paper
Figure 12. Layerwise probe classification of evaluation versus deployment context for Nemotron3-49B, using the mean-token
Figure 13 of the paper
Figure 13. Spearman correlation between the last token of prompt’s probe score and judge scores across the six models. Gray
Figure 14 of the paper
Figure 14. Mutual information between the last token of prompt’s probe score and judge scores across the six models. Gray bands
Figure 15 of the paper
Figure 15. Spearman correlation between response-last probe score and judge score across the six models. Layer depth (normalized
Figure 16 of the paper
Figure 16. Mutual information between response-last probe score and judge score across the six models. Layer depth (normalized
Figure 17 of the paper
Figure 17. Layerwise Spearman ρℓ between the linear probe vˆℓ and the fine-grained judge J 100, on the two readouts that attain the highest peak |ρℓ| in
Figure 18 of the paper
Figure 18. Best-layer results across steering strengths
Figure 20 of the paper
Figure 20. Best-layer results across steering strengths
Figure 22 of the paper
Figure 22. Best-layer results across steering strengths
Figure 25 of the paper
Figure 25. Best-layer results across steering strengths
Figure 26 of the paper
Figure 26. Best-layer results across steering strengths
Figure 28 of the paper
Figure 28. reports pairwise cosine similarity of the probe direction vˆℓ across Olmo training stages, at the model-best layer. SFT, DPO, and Think share a probe direction at cosine ≥ 0.998 for both 7B and 32B; the Base direction is distinct (cosine 0.72 at 7B and 0.83 at 32B against the post-trained directions). Post-training therefore stabilizes a probe direction already present at Base rather than producing a new direction at each stage. Base SFT DPO Think Stage Vector Base SFT DPO Think Stage Vector 1.00 0.72 0.72 0.72 0.72 1.00 1.00 1.00 0.72 1.00 1.00 1.00 0.72 1.00 1.00 1.00 1.00 0.75…
Figure 29 of the paper
Figure 29. Per-prompt coupling across Olmo-7B training stages on the prompt-last readout. Top row: layerwise Spearman
Figure 30 of the paper
Figure 30. Per-prompt coupling across Olmo-32B training stages on the prompt-last readout. Top row: layerwise Spearman
Figure 32 of the paper
Figure 32. Same as Figure
Figure 33 of the paper
Figure 33. Row-normalized cross-judge confusion (same convention as Figure

Showing 27 of the paper's 33 figures. Figures are the authors', reproduced from the paper.

References (51)
  1. [1] Steering Evaluation-Aware Language Models to Act Like They Are Deployed , author=. arXiv preprint arXiv:2510.20487 , year= on PitharXiv
  2. [2] Large language models often know when they are being evaluated , author=. arXiv preprint arXiv:2505.23836 , year= on PithPith's reviewarXiv
  3. [3] The Hawthorne Effect in Reasoning Models: Evaluating and Steering Test Awareness , author=. arXiv preprint arXiv:2505.14617 , year= on PitharXiv
  4. [4] Is Evaluation Awareness Just Format Sensitivity? Limitations of Probe-Based Evidence under Controlled Prompt Structure , author=. arXiv preprint arXiv:2603.19426 , year= on PitharXiv
  5. [5] How do llms compute verbal confidence , author=. arXiv preprint arXiv:2603.17839 , year= on PithPith's reviewarXiv
  6. [6] Chaudhary, Maheep and Su, Ian and Hooda, Nikhil and Shankar, Nishith and Tan, Julia and Zhu, Kevin and Lagasse, Ryan and Sharma, Vasu and Panda, Ashwinee , title =. arXiv preprint arXiv:2509.13333 , year = on PitharXiv
  7. [7] Steering language models with activation engineering , author=. arXiv preprint arXiv:2308.10248 , year= on PithPith's reviewarXiv
  8. [8] System Card: Claude Opus 4.6 , author =. 2026 , month = on Pith
  9. [9] International ai safety report 2026 , author=. arXiv preprint arXiv:2602.21012 , year= on PitharXiv
  10. [10] Taken out of context: On measuring situational awareness in LLMs , author=. arXiv preprint arXiv:2309.00667 , year= on PithPith's reviewarXiv
  11. [11] Frontier models are capable of in-context scheming , author=. arXiv preprint arXiv:2412.04984 , year= on PithPith's reviewarXiv
  12. [12] Sidestepping evaluation awareness and anticipating misalignment with production evaluations , author=. OpenAI Alignment Research Blog, Dec , year= on Pith
  13. [13] Sleeper agents: Training deceptive llms that persist through safety training , author=. arXiv preprint arXiv:2401.05566 , year= on PithPith's reviewarXiv
  14. [14] Understanding strategic deception and deceptive alignment , author=. 2025 , publisher= on Pith
  15. [15] The alignment problem from a deep learning perspective , author=. arXiv preprint arXiv:2209.00626 , year= on PithPith's reviewarXiv
  16. [16] International ai safety report , author=. arXiv preprint arXiv:2501.17805 , year= on PithPith's reviewarXiv
  17. [17] Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awareness , author=. arXiv preprint arXiv:2509.00591 , year= on PitharXiv
  18. [18] Thought crime: Backdoors and emergent misalignment in reasoning models , author=. arXiv preprint arXiv:2506.13206 , year= on PithPith's reviewarXiv
  19. [19] The mask benchmark: Disentangling honesty from accuracy in ai systems , author=. arXiv preprint arXiv:2503.03750 , year= on PitharXiv
  20. [20] TruthfulQA: Measuring How Models Mimic Human Falsehoods , author=. 2021 , eprint= on Pith
show 31 more
  1. [21] Training Verifiers to Solve Math Word Problems , author=. arXiv preprint arXiv:2110.14168 , year= on PithPith's reviewarXiv
  2. [22] Wenting Zhao and Xiang Ren and Jack Hessel and Claire Cardie and Yejin Choi and Yuntian Deng , booktitle=. WildChat: 1M Chat. 2024 , url= on Pith
  3. [23] Estimating mutual information , author=. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics , volume=. 2004 , publisher= on Pith
  4. [24] Index for rating diagnostic tests , author=. Cancer , volume=. 1950 , publisher= on Pith
  5. [25] Olmo 3 , author=. arXiv preprint arXiv:2512.13961 , year= on PithPith's reviewarXiv
  6. [26] Openai gpt-5 system card , author=. arXiv preprint arXiv:2601.03267 , year= on PithPith's reviewarXiv
  7. [27] The proof and measurement of association between two things , author=. The American journal of psychology , volume=. 1987 , publisher= on Pith
  8. [28] Understanding intermediate layers using linear classifier probes , author=. arXiv preprint arXiv:1610.01644 , year= on PithPith's reviewarXiv
  9. [29] Probing classifiers: Promises, shortcomings, and advances , author=. Computational Linguistics , volume= on Pith
  10. [30] The geometry of truth: Emergent linear structure in large language model representations of true/false datasets , author=. arXiv preprint arXiv:2310.06824 , year= on PithPith's reviewarXiv
  11. [31] Language models (mostly) know what they know , author=. arXiv preprint arXiv:2207.05221 , year= on PithPith's reviewarXiv
  12. [32] Claude Opus 4.7 System Card , author =. 2026 , month = apr, day = on Pith
  13. [33] Gemini 3 Pro Model Card , author =. 2025 , month = nov, url = on Pith
  14. [34] Gemini 3 Pro Frontier Safety Framework Report , author=. 2025 , month= on Pith
  15. [35] GPT-5.5 System Card , author =. 2026 , month = apr, day = on Pith
  16. [36] Language models recognize dropout and Gaussian noise applied to their activations , author=. arXiv preprint arXiv:2604.17465 , year= on PithPith's reviewarXiv
  17. [37] Hubinger, Evan , title =. 2019 , month = oct, day = on Pith
  18. [38] Probing and steering evaluation awareness of language models , author=. arXiv preprint arXiv:2507.01786 , year= on PithPith's reviewarXiv
  19. [39] Decomposing and Measuring Evaluation Awareness , author=. arXiv preprint arXiv:2605.23055 , year= on PithPith's reviewarXiv
  20. [40] Nayan, Nilesh and Kumar, Aishwarya Sampath and Girmal, Rishiraj and Anilkumar, Shivani and Vaidyanathan, Sankaran and Palacio, David A. Nader and Ghosh, Reshmi and Srinivasan, Soundararajan , year=. 2606.23583 , archivePrefix= on PithPith's reviewarXiv
  21. [41] Evaluation Awareness in Language Models Has Limited Effect on Behaviour , author=. arXiv preprint arXiv:2605.05835 , year= on PithPith's reviewarXiv
  22. [42] Models That Know How Evaluations Are Designed Score Safer , author=. arXiv preprint arXiv:2605.28591 , year= on PithPith's reviewarXiv
  23. [43] Stress testing deliberative alignment for anti-scheming training , author=. arXiv preprint arXiv:2509.15541 , year= on PitharXiv
  24. [44] Claude Sonnet 3.7 (often) knows when it’s in alignment evaluations , author=. Apollo Research Blog , volume= on Pith
  25. [45] Coleman, Russell , year= on Pith
  26. [46] Ivanov, Igor , year= on Pith
  27. [47] Bharadwaj, Ram and Kirk, Robert , year= on Pith
  28. [48] Aranguri, Santiago and Bloom, Joseph , year= on Pith
  29. [49] Aranguri, Santiago , year= on Pith
  30. [50] Probe-Based Data Attribution: Discovering and Mitigating Undesirable Behaviors in LLM Post-Training , author=. arXiv preprint arXiv:2602.11079 , year= on PithPith's reviewarXiv
  31. [51] Cohen, Jacob , title =. Educational and Psychological Measurement , year = on Pith
Cited by 4 Pith papers
How the literature uses it

What this paper does with the arXiv papers it cites

Each citing sentence is taken from the citing paper's LaTeX source and typed by Pith's classifier. A label appears at probability 0.8 or more (0.95 for support), a contest at 0.7 or more; the quoted words are the citing authors'. How each label was checked · JSON

What it cites · 8 sentences without a confident label

Uses the method or tool of 2 papers
  • To steer models towards or against the direction of evaluation awareness previously extracted, we follow the protocol of Contrastive Activation Addition [ref] .

    Steering Language Models With Activation Engineering · Methodology · 1.00

  • We replicate the construction of linear probes from [ref] , using the same $n = 16$ contrastive pairs ${x_i = (x_i^+, x_i^-) |1 ≤i ≤n}$.

    2510.20487 · Methodology · 1.00

Uses data or values from 3 papers
  • Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).

    Training Verifiers to Solve Math Word Problems · Methodology · 0.81

  • Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).

    TruthfulQA: Measuring How Models Mimic Human Falsehoods · Methodology · 0.81

  • Instead, to cover a diverse set of evaluations, distinct in scope and style, we source evaluation prompts from three benchmarks: GSM8K [ref] (grade school math word problems), MASK [ref] (a dataset for measuring honesty of language models), and TruthfulQA [ref] (a dataset to measure if language models are truthful in generating answers to questions).

    2503.03750 · Methodology · 0.81

Cites as support 6 papers
  • In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .

    Large Language Models Often Know When They Are Being Evaluated · Introduction · 0.96

  • In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .

    2509.00591 · Introduction · 0.96

  • In practice, evaluation awareness has been observed in open-weight models [ref] and in proprietary models alike [ref] , culminating in the International AI Safety Report, which lists it as a key development of the past year [ref] .

    2510.20487 · Introduction · 0.96

Cites as background 8 papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment