Pith. sign in

REVIEW 2 major objections 2 minor 49 references

Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity

T0 review · 2 major / 2 minor · reviewed 2026-05-08 · grok-4.3

Challenge this review Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Open-weight LLMs show model-specific shifts in safety behavior when prompts are framed as evaluations versus deployments.

desk verdict Pilot protocol shows one aligned model flips to eval-cautious while others stay deployment-cautious, but the cross-family claim is judge-dependent and the item count is tiny. read the letter →

arxiv 2605.06327 v1 submitted 2026-05-07 cs.CL cs.AIcs.LG

Florian A. D. Burnat , Brittany I. Davidson This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords evaluation-contextdivergencepaired-promptprotocolLLMrefusalsafetyalignmentdeploymentbehaviorbenchmarkvalidityopen-weightmodelscontextsensitivity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to demonstrate that a model's apparent safety on benchmarks can shift depending on whether the prompt is presented as an evaluation or as a live deployment request. A sympathetic reader would care because if true, it means standard safety tests may give misleading pictures of how models will act in the real world. The authors introduce a paired-prompt protocol to isolate the effect of this framing while holding the underlying task constant. They apply it to several open-weight models and find that only OLMo-3-Instruct becomes more cautious under evaluation framing, whereas the others are more cautious under deployment framing, with the difference emerging after alignment training.

What carries the argument

The paired-prompt protocol, which generates responses to identical tasks under evaluation, deployment, and neutral framings to quantify within-item behavioral changes attributable to context detection.

What would settle it

If the same models showed no statistically significant differences in refusal or compliance between evaluation-framed and deployment-framed prompts, or if the alignment step did not produce the observed inversion in caution pattern, the claim of context divergence and its model-specific nature would be falsified.

Watch

Extended reading notes

Core claim

Safety benchmarks are routinely treated as evidence about how a language model will behave once deployed, but this inference is fragile if behavior depends on whether a prompt looks like an evaluation. We define evaluation-context divergence as an observable within-item change in behavior induced by framing a fixed task as an evaluation, a live deployment interaction, or a neutral request, and present a paired-prompt protocol that measures it in open-weight LLMs while controlling for paraphrase variation, benchmark familiarity, and judge framing-sensitivity. Across five instruction-tuned checkpoints from four open-weight families plus a matched OLMo-3 base/instruct ablation, we find striking

Load-bearing premise

That the paired prompts differ only in evaluation or deployment framing and that all other factors like paraphrase and judge sensitivity are fully controlled.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper defines evaluation-context divergence as within-item behavioral shifts in LLMs when a fixed task is framed as an evaluation, deployment interaction, or neutral request. It introduces a paired-prompt protocol intended to isolate this effect by controlling paraphrase variation, benchmark familiarity, and judge framing-sensitivity. Experiments on five instruction-tuned open-weight checkpoints plus an OLMo-3 base/instruct ablation (20 paired items, 840 generations per checkpoint) report model-specific heterogeneity: OLMo-3-Instruct alone is eval-cautious (refusal rises 11.8pp vs. neutral, p=0.007; harmful compliance falls 3.6pp vs. deployment, p=0.024, 0/20 inversions), while Mistral-Small-3.2, Phi-3.5-mini, and Llama-3.1-8B are deployment-cautious (eval-vs-deployment refusal shifts of -9 to -20pp). The matched OLMo-3 base shows the deployment-cautious pattern, locating the inversion at alignment; within Llama-3.1 the 70B model preserves direction at reduced magnitude. Cross-family contrasts are judge-dependent, flattening under Llama-Guard-3-8B while the OLMo direction holds.

Significance. If replicated, the pilot evidence would be significant for demonstrating that standard safety benchmarks may not generalize to deployment because of framing sensitivity, with the OLMo base/instruct contrast providing a concrete example that alignment can invert context sensitivity. The protocol's controls and the explicit judge caveat supply a reproducible template for future work on evaluation-deployment gaps, potentially affecting how safety claims are validated in open-weight model releases.

major comments (2)
  1. [Results / Experimental Setup] Experimental design (20 paired items, 840 generations): the reported percentage-point differences and p-values rest on a small item count that the authors themselves label a pilot; this makes the heterogeneity claim (particularly the cross-family deployment-cautious pattern) sensitive to item selection and limits statistical power, so the manuscript should include a power calculation, effect-size discussion, or explicit statement of the scope within which the 11.8pp and 3.6pp figures can be generalized.
  2. [Discussion / Judge Dependency] Judge-dependency subsection: while the within-OLMo eval-cautious direction survives re-judging with Llama-Guard-3-8B, the flattening of the cross-family contrast indicates that the alignment-pipeline-specific heterogeneity claim is partly judge-dependent; the paper should report inter-judge agreement or test at least one additional classifier to show how much of the Mistral/Phi/Llama pattern is robust versus construct-specific.
minor comments (2)
  1. [Abstract] Abstract contains a stray closing brace ('deployment-cautious}, with') that should be removed for readability.
  2. [Methods] The protocol description would benefit from one or two concrete prompt-pair examples to illustrate how paraphrase and familiarity controls were implemented.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive review and minor revision recommendation. We address each major comment below, agreeing to add the requested clarifications on statistical scope and judge robustness while preserving the pilot nature of the evidence.

read point-by-point responses

  1. Referee: [Results / Experimental Setup] Experimental design (20 paired items, 840 generations): the reported percentage-point differences and p-values rest on a small item count that the authors themselves label a pilot; this makes the heterogeneity claim (particularly the cross-family deployment-cautious pattern) sensitive to item selection and limits statistical power, so the manuscript should include a power calculation, effect-size discussion, or explicit statement of the scope within which the 11.8pp and 3.6pp figures can be generalized.

    Authors: We agree that the 20-item set is a pilot and that the reported shifts should be scoped accordingly. In revision we will add Cohen's h effect sizes for the key proportion differences (approximately 0.24 for the 11.8 pp OLMo refusal increase) and an explicit statement that the 11.8 pp and 3.6 pp figures are observed within this prompt set and are not claimed to generalize to all possible items or models. We will also include a brief power discussion for detecting comparable effects in larger follow-up studies. The zero inversions for OLMo-3-Instruct across all 20 items still provides internal consistency against item-specific artifacts, but we will emphasize the preliminary character of the heterogeneity claim. revision: yes

  2. Referee: [Discussion / Judge Dependency] Judge-dependency subsection: while the within-OLMo eval-cautious direction survives re-judging with Llama-Guard-3-8B, the flattening of the cross-family contrast indicates that the alignment-pipeline-specific heterogeneity claim is partly judge-dependent; the paper should report inter-judge agreement or test at least one additional classifier to show how much of the Mistral/Phi/Llama pattern is robust versus construct-specific.

    Authors: We agree that the cross-family pattern is judge-dependent, as the manuscript already caveats. In the revision we will report inter-judge agreement (percentage agreement and Cohen's kappa) between the primary judge and Llama-Guard-3-8B across the full set of generations. This will quantify that the OLMo-3 eval-cautious direction remains stable while the deployment-cautious pattern in the other families is more sensitive to the safety classifier chosen. We view the addition of agreement metrics as sufficient for this pilot; testing a third classifier is noted as desirable for future work but not required to support the current claims. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: purely empirical protocol with direct comparisons

full rationale

The paper defines evaluation-context divergence and measures it via a paired-prompt protocol across models, reporting raw behavioral shifts (e.g., 11.8pp refusal increase, p=0.007) and judge-dependent patterns from 840 generations per checkpoint. No equations, fitted parameters, predictions, or self-citations appear in the derivation chain; all load-bearing claims reduce to observed response differences under controlled framings rather than any input-defined quantity or imported uniqueness theorem. The base/instruct ablation and cross-model contrasts are independent empirical observations.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The measurement relies on domain assumptions about prompt equivalence and judge reliability rather than new postulates or fitted parameters.

assumptions (2)
  • domain assumption Paired prompts control for paraphrase variation, benchmark familiarity, and judge framing-sensitivity
    This is the core design claim of the protocol that isolates evaluation-context effects.
  • domain assumption Safety judges provide consistent measures of refusal and harmful compliance across framings
    The paper notes that results are judge-dependent, making this assumption load-bearing for the heterogeneity claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity." pith.science (2026). https://pith.science/paper/2605.06327

@misc{pith2026260506327,
  author       = {Pith},
  title        = {Pith review of: Measuring Evaluation-Context Divergence in Open-Weight LLMs: A Paired-Prompt Protocol with Pilot Evidence of Alignment-Pipeline-Specific Heterogeneity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2605.06327}},
  note         = {Machine review of arXiv:2605.06327}
}
abstract

Safety benchmarks are routinely treated as evidence about how a language model will behave once deployed, but this inference is fragile if behavior depends on whether a prompt looks like an evaluation. We define evaluation-context divergence as an observable within-item change in behavior induced by framing a fixed task as an evaluation, a live deployment interaction, or a neutral request, and present a paired-prompt protocol that measures it in open-weight LLMs while controlling for paraphrase variation, benchmark familiarity, and judge framing-sensitivity. Across five instruction-tuned checkpoints from four open-weight families plus a matched OLMo-3 base/instruct ablation ($20$ paired items, $840$ generations per checkpoint), we find striking heterogeneity. OLMo-3-Instruct alone is eval-cautious -- evaluation framing raises refusal vs. neutral by $11.8$pp ($p=0.007$) and reduces harmful compliance vs. deployment by $3.6$pp ($p=0.024$, $0/20$ items inverted) -- while Mistral-Small-3.2, Phi-3.5-mini, and Llama-3.1-8B are deployment-cautious}, with marginal eval-vs-deployment refusal effects of $-9$ to $-20$pp. The matched OLMo-3 base also exhibits the deployment-cautious pattern, identifying alignment as the inversion stage; within Llama-3.1, the $70$B model preserves direction with attenuated magnitude, ruling out a simple ``small-model effect that reverses at scale.'' One caveat: the cross-family heterogeneity is judge-dependent. Re-judging with a different-family safety classifier (Llama-Guard-3-8B) preserves the within-OLMo eval-cautious direction but flattens the cross-family contrast, indicating that the two judges operationalize distinct constructs.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages

  1. [1]

    P a C o ST : Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models

    Zhang, Huixuan and Lin, Yun and Wan, Xiaojun. PaCoST : Paired confidence significance testing for benchmark contamination detection in large language models. Findings of the Association for Computational Linguistics: EMNLP 2024. doi:10.18653/v1/2024.findings-emnlp.97

    work page doi:10.18653/v1/2024.findings-emnlp.97 2024

  2. [2]

    Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

    Chiang, Wei-Lin and Gonzalez, Joseph and Li, Dacheng and Li, Zhuohan and Lin, Zi and Sheng, Ying and Stoica, Ion and Wu, Zhanghao and Xing, Eric and Zhang, Hao and Zheng, Lianmin and Zhuang, Siyuan and Zhuang, Yonghao. Judging LLM -as-a-judge with MT-bench and chatbot arena. Advances in Neural Information Processing Systems 36. doi:10.52202/075280-2020

    work page doi:10.52202/075280-2020 2020

  3. [3]

    G-Eval: NLG Evaluation using Gpt-4 with Better Human Alignment

    Liu, Yang and Iter, Dan and Xu, Yichong and Wang, Shuohang and Xu, Ruochen and Zhu, Chenguang. G-eval : NLG evaluation using gpt-4 with better human alignment. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. doi:10.18653/v1/2023.emnlp-main.153

    work page doi:10.18653/v1/2023.emnlp-main.153 2023

  4. [4]

    Scalable Extraction of Training Data from (Production) Language Models

    Nasr, Milad and Carlini, Nicholas and Hayase, Jonathan and Jagielski, Matthew and Cooper, A Feder and Ippolito, Daphne and Choquette-Choo, Christopher A and Wallace, Eric and Tramèr, Florian and Lee, Katherine. Scalable extraction of training data from (production) language models. arXiv [cs.LG]. doi:10.48550/arXiv.2311.17035

    work page Pith review doi:10.48550/arxiv.2311.17035

  5. [5]

    What does it mean for a language model to preserve privacy?

    Brown, Hannah and Lee, Katherine and Mireshghallah, Fatemehsadat and Shokri, Reza and Tramèr, Florian. What does it mean for a language model to preserve privacy?. 2022 ACM Conference on Fairness, Accountability, and Transparency. doi:10.1145/3531146.3534642

    work page doi:10.1145/3531146.3534642 2022

  6. [6]

    Kapoor and A

    Kapoor, Sayash and Narayanan, Arvind. Leakage and the reproducibility crisis in machine-learning-based science. Patterns (New York, N.Y.). doi:10.1016/j.patter.2023.100804

    work page doi:10.1016/j.patter.2023.100804 2023

  7. [7]

    Repairing the Cracked Foundation : A Survey of Obstacles in Evaluation Practices for Generated Text

    Gehrmann, Sebastian and Clark, Elizabeth and Sellam, Thibault. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. The Journal of Artificial Intelligence Research. doi:10.1613/jair.1.13715

    work page doi:10.1613/jair.1.13715

  8. [8]

    Smith, Nicole DeCario, and Will Buchanan

    Hutchinson, Ben and Rostamzadeh, Negar and Greer, Christina and Heller, Katherine and Prabhakaran, Vinodkumar. Evaluation gaps in machine learning practice. 2022 ACM Conference on Fairness, Accountability, and Transparency. doi:10.1145/3531146.3533233

    work page doi:10.1145/3531146.3533233 2022

Show all 49 references
  1. [9]

    Towards accountability for machine learning datasets: Practices from software engineering and infrastructure

    Hutchinson, Ben and Smart, Andrew and Hanna, Alex and Denton, Remi and Greer, Christina and Kjartansson, Oddur and Barnes, Parker and Mitchell, Margaret. Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. Proceedings o...

    2021 doi

  2. [10]

    Does prompt formatting have any impact on LLM performance?

    He, Jia and Rungta, Mukund and Koleczek, David and Sekhon, Arshdeep and Wang, Franklin X and Hasan, Sadid. Does prompt formatting have any impact on LLM performance?. arXiv [cs.CL]. doi:10.48550/arXiv.2411.10541

    doi

  3. [11]

    Flaw or artifact? Rethinking prompt sensitivity in evaluating LLMs

    Hua, Andong and Tang, Kenan and Gu, Chenhe and Gu, Jindong and Wong, Eric and Qin, Yao. Flaw or artifact? Rethinking prompt sensitivity in evaluating LLMs. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. doi:10.18653/v1/2025.emnlp-main.1006

    2025 doi

  4. [12]

    POSIX : A prompt sensitivity index for large language models

    Chatterjee, Anwoy and Renduchintala, H S V N S Kowndinya and Bhatia, Sumit and Chakraborty, Tanmoy. POSIX : A prompt sensitivity index for large language models. Findings of the Association for Computational Linguistics: EMNLP 2024. doi:10.18653/v1/2024.findings-emnlp.852

    2024 doi

  5. [13]

    ProSA : Assessing and understanding the prompt sensitivity of LLMs

    Zhuo, Jingming and Zhang, Songyang and Fang, Xinyu and Duan, Haodong and Lin, Dahua and Chen, Kai. ProSA : Assessing and understanding the prompt sensitivity of LLMs. Findings of the Association for Computational Linguistics: EMNLP 2024. doi:10.18653/v1/2024.findings-emnlp.108

    2024 doi

  6. [14]

    State of what art? A call for multi-prompt LLM evaluation

    Mizrahi, Moran and Kaplan, Guy and Malkin, Dan and Dror, Rotem and Shahaf, Dafna and Stanovsky, Gabriel. State of what art? A call for multi-prompt LLM evaluation. arXiv [cs.CL]. doi:10.48550/arXiv.2401.00595

    doi

  7. [15]

    Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting

    Sclar, Melanie and Choi, Yejin and Tsvetkov, Yulia and Suhr, Alane. Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. arXiv [cs.CL]. doi:10.48550/arXiv.2310.11324

    doi

  8. [16]

    Noise injection reveals hidden capabilities of sandbagging language models

    Tice, Cameron and Kreer, Philipp Alexander and Helm-Burger, Nathan and Shahani, Prithviraj Singh and Ryzhenkov, Fedor and Roger, Fabien and Neo, Clement and Haimes, Jacob and Hofstätter, Felix and van der Weij, Teun. Noise injection reveals hidden capabilities of sandbagging l...

    doi

  9. [17]

    Benchmarking benchmark leakage in Large Language Models

    Xu, Ruijie and Wang, Zengzhi and Fan, Run-Ze and Liu, Pengfei. Benchmarking benchmark leakage in Large Language Models. arXiv [cs.CL]. doi:10.48550/arXiv.2404.18824

    doi

  10. [18]

    Rethinking benchmark and contamination for language models with rephrased samples

    Yang, Shuo and Chiang, Wei-Lin and Zheng, Lianmin and Gonzalez, Joseph E and Stoica, Ion. Rethinking benchmark and contamination for language models with rephrased samples. arXiv [cs.CL]. doi:10.48550/arXiv.2311.04850

    doi

  11. [19]

    Estimating contamination via perplexity: Quantifying memorisation in language model evaluation

    Li, Yucheng. Estimating contamination via perplexity: Quantifying memorisation in language model evaluation. arXiv [cs.CL]. doi:10.48550/arXiv.2309.10677

    doi

  12. [20]

    An Open-Source Data Contamination Report for Large Language Models

    Li, Yucheng and Guo, Yunhao and Guerin, Frank and Lin, Chenghua. An Open-Source Data Contamination Report for Large Language Models. Findings of the Association for Computational Linguistics: EMNLP 2024. doi:10.18653/v1/2024.findings-emnlp.30

    2024 doi

  13. [21]

    Unveiling the spectrum of data contamination in language model: A survey from detection to remediation

    Deng, Chunyuan and Zhao, Yilun and Heng, Yuzhao and Li, Yitong and Cao, Jiannan and Tang, Xiangru and Cohan, Arman. Unveiling the spectrum of data contamination in language model: A survey from detection to remediation. Findings of the Association for Computational Linguistics...

    2024 doi

  14. [22]

    Data contamination can cross language barriers

    Yao, Feng and Zhuang, Yufan and Sun, Zihao and Xu, Sunan and Kumar, Animesh and Shang, Jingbo. Data contamination can cross language barriers. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. doi:10.18653/v1/2024.emnlp-main.990

    2024 doi

  15. [23]

    Sycophancy to subterfuge: Investigating reward-tampering in large language models

    Denison, Carson and MacDiarmid, Monte and Barez, Fazl and Duvenaud, David and Kravec, Shauna and Marks, Samuel and Schiefer, Nicholas and Soklaski, Ryan and Tamkin, Alex and Kaplan, Jared and Shlegeris, Buck and Bowman, Samuel R and Perez, Ethan and Hubinger, Evan. Sycophancy ...

    doi

  16. [24]

    Taken out of context: On measuring situational awareness in LLMs

    Berglund, Lukas and Stickland, Asa Cooper and Balesni, Mikita and Kaufmann, Max and Tong, Meg and Korbak, Tomasz and Kokotajlo, Daniel and Evans, Owain. Taken out of context: On measuring situational awareness in LLMs. arXiv [cs.CL]. doi:10.48550/arXiv.2309.00667

    doi

  17. [25]

    Me, myself, and AI : The Situational Awareness Dataset ( SAD ) for LLMs

    Laine, Rudolf and Chughtai, Bilal and Betley, Jan and Hariharan, Kaivalya and Scheurer, Jeremy and Balesni, Mikita and Hobbhahn, Marius and Meinke, Alexander and Evans, Owain. Me, myself, and AI : The Situational Awareness Dataset ( SAD ) for LLMs. arXiv [cs.CL]. doi:10.48550/...

    doi

  18. [26]

    AI and the End of an Era

    Meinke, Alexander and Schoen, Bronson and Scheurer, Jérémy and Balesni, Mikita and Shah, Rusheb and Hobbhahn, Marius. AI and the End of an Era. arXiv [cs.AI]. doi:10.48550/arXiv.2412.04984

    doi

  19. [27]

    ConStat : Performance-based contamination detection in large language models

    Dekoninck, Jasper and Müller, Mark Niklas and Vechev, Martin. ConStat : Performance-based contamination detection in large language models. arXiv [cs.CL]. doi:10.48550/arXiv.2405.16281

    doi

  20. [28]

    Stereotyping N orwegian salmon: An inventory of pitfalls in fairness benchmark datasets

    Blodgett, Su Lin and Lopez, Gilsinia and Olteanu, Alexandra and Sim, Robert and Wallach, Hanna. Stereotyping N orwegian salmon: An inventory of pitfalls in fairness benchmark datasets. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and ...

    2021 doi

  21. [29]

    AI sandbagging: Language models can strategically underperform on evaluations

    van der Weij, Teun and Hofstätter, Felix and Jaffe, Ollie and Brown, Samuel F and Ward, Francis Rhys. AI sandbagging: Language models can strategically underperform on evaluations. arXiv [cs.AI]. doi:10.48550/arXiv.2406.07358

    doi

  22. [30]

    The fallacy of AI functionality

    Raji, Inioluwa Deborah and Kumar, I Elizabeth and Horowitz, Aaron and Selbst, Andrew. The fallacy of AI functionality. 2022 ACM Conference on Fairness, Accountability, and Transparency. doi:10.1145/3531146.3533158

    2022 doi

  23. [31]

    `` do anything now '': Characterizing and evaluating in-the-wild jailbreak prompts on large language models

    Shen, Xinyue and Chen, Zeyuan and Backes, Michael and Shen, Yun and Zhang, Yang. `` do anything now '': Characterizing and evaluating in-the-wild jailbreak prompts on large language models. Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Securit...

    2024 doi

  24. [32]

    TruthfulQA : Measuring how models mimic human falsehoods

    Lin, Stephanie and Hilton, Jacob and Evans, Owain. TruthfulQA : Measuring how models mimic human falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). doi:10.18653/v1/2022.acl-long.229

    2022 doi

  25. [33]

    Holistic evaluation of language models

    Bommasani, Rishi and Liang, Percy and Lee, Tony. Holistic evaluation of language models. Annals of the New York Academy of Sciences. doi:10.1111/nyas.15007

    doi

  26. [34]

    Jailbroken: How does LLM safety training fail?

    Wei, Alexander and Haghtalab, Nika and Steinhardt, Jacob. Jailbroken: How does LLM safety training fail?. arXiv [cs.LG]. doi:10.48550/arXiv.2307.02483

    doi

  27. [35]

    Sleeper agents: Training deceptive LLMs that persist through safety training

    Hubinger, Evan and Denison, Carson and Mu, Jesse and Lambert, Mike and Tong, Meg and MacDiarmid, Monte and Lanham, Tamera and Ziegler, Daniel M and Maxwell, Tim and Cheng, Newton and Jermyn, Adam and Askell, Amanda and Radhakrishnan, Ansh and Anil, Cem and Duvenaud, David and ...

    doi

  28. [36]

    Open problems and fundamental limitations of reinforcement learning from human feedback

    Casper, Stephen and Davies, Xander and Shi, Claudia and Gilbert, Thomas Krendl and Scheurer, Jérémy and Rando, Javier and Freedman, Rachel and Korbak, Tomasz and Lindner, David and Freire, Pedro and Wang, Tony and Marks, Samuel and Segerie, Charbel-Raphaël and Carroll, Micah a...

  29. [37]

    HarmBench : A standardized evaluation framework for automated red teaming and robust refusal

    Mazeika, Mantas and Phan, Long and Yin, Xuwang and Zou, Andy and Wang, Zifan and Mu, Norman and Sakhaee, Elham and Li, Nathaniel and Basart, Steven and Li, Bo and Forsyth, David and Hendrycks, Dan. HarmBench : A standardized evaluation framework for automated red teaming and r...

    doi

  30. [38]

    Safe RLHF : Safe reinforcement learning from human feedback

    Dai, Josef and Pan, Xuehai and Sun, Ruiyang and Ji, Jiaming and Xu, Xinbo and Liu, Mickel and Wang, Yizhou and Yang, Yaodong. Safe RLHF : Safe reinforcement learning from human feedback. arXiv [cs.AI]

  31. [39]

    Taxonomy of risks posed by language models

    Weidinger, Laura and Uesato, Jonathan and Rauh, Maribeth and Griffin, Conor and Huang, Po-Sen and Mellor, John and Glaese, Amelia and Cheng, Myra and Balle, Borja and Kasirzadeh, Atoosa and Biles, Courtney and Brown, Sasha and Kenton, Zac and Hawkins, Will and Stepleton, Tom a...

    2022 doi

  32. [40]

    Alignment faking in large language models

    Greenblatt, Ryan and Denison, Carson and Wright, Benjamin and Roger, Fabien and MacDiarmid, Monte and Marks, Sam and Treutlein, Johannes and Belonax, Tim and Chen, Jack and Duvenaud, David and Khan, Akbir and Michael, Julian and Mindermann, Sören and Perez, Ethan and Petrini, ...

    doi

  33. [41]

    Deception abilities emerged in large language models

    Hagendorff, Thilo. Deception abilities emerged in large language models. Proceedings of the National Academy of Sciences of the United States of America. doi:10.1073/pnas.2317967121

    doi

  34. [42]

    Large language models can strategically deceive their users when put under pressure

    Scheurer, Jérémy and Balesni, Mikita and Hobbhahn, Marius. Large language models can strategically deceive their users when put under pressure. arXiv [cs.CL]. doi:10.48550/ARXIV.2311.07590

    doi

  35. [43]

    AI deception: A survey of examples, risks, and potential solutions

    Park, Peter S and Goldstein, Simon and O'Gara, Aidan and Chen, Michael and Hendrycks, Dan. AI deception: A survey of examples, risks, and potential solutions. Patterns (New York, N.Y.). doi:10.1016/j.patter.2024.100988

    2024 doi

  36. [44]

    Characterizing manipulation from AI systems

    Carroll, Micah and Chan, Alan and Ashton, Henry and Krueger, David. Characterizing manipulation from AI systems. Equity and Access in Algorithms, Mechanisms, and Optimization. doi:10.1145/3617694.3623226

    doi

  37. [45]

    Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing

    Raji, Inioluwa Deborah and Smart, Andrew and White, Rebecca N and Mitchell, Margaret and Gebru, Timnit and Hutchinson, Ben and Smith-Loud, Jamila and Theron, Daniel and Barnes, Parker. Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic...

    2020 doi

  38. [46]

    OLMo 3 : Charting a Path Through the Model Flow to Lead Open-Source AI

    OLMo Team, Allen Institute for AI. OLMo 3 : Charting a Path Through the Model Flow to Lead Open-Source AI. doi:10.48550/arXiv.2512.13961. arXiv:2512.13961

    doi arXiv

  39. [47]

    Mistral Small 3.2 (24 B Instruct, 2506)

    Mistral AI. Mistral Small 3.2 (24 B Instruct, 2506)

  40. [48]

    Evaluation Awareness Scales Predictably in Open-Weights Large Language Models

    Chaudhary, Maheep and Su, Ian and Hooda, Nikhil and Shankar, Nishith and Tan, Julia and Zhu, Kevin and Lagasse, Ryan and Sharma, Vasu and Panda, Ashwinee. Evaluation Awareness Scales Predictably in Open-Weights Large Language Models. doi:10.48550/arXiv.2509.13333. arXiv:2509.13333

    doi

  41. [49]

    Alignment Faking Revisited: Improved Classifiers and Open Source Extensions

    Hughes, John and Sheshadri, Abhay and Khan, Akbir and Roger, Fabien. Alignment Faking Revisited: Improved Classifiers and Open Source Extensions

Pith tools

Reviewed May 8, 2026 · model on record in the stance chip above.