Pith.

Machine Learning · arXiv:2610.29734 v1 · submitted 21 Oct 2026 · 84 references

Monitorability Is Trainable: Post-Training for Chain-of-Thought Monitorability

Usman Anwar, Tim Bakker, Christos Louizos, David Krueger

The paper says

“Chain-of-thought (CoT) monitoring is one of the few practical tools available for overseeing capable AI systems, but it is fragile: a model's reasoning is only useful to a monitor if it remains coupled to what the model actually does.”

more of the abstract

This coupling can be degraded during training through, for example, CoT compression, optimization against a monitor, and out-of-context generalization from synthetic documents. In this work, we propose an unsupervised post-training objective based on maximizing mutual information between CoT and the output for improving CoT monitorability. Across three model organisms with degraded CoT, it recovers most or all of the lost monitorability. Applied to undegraded policies, it further improves judged monitorability, improves prediction of unsafe behavior by CoT monitors and activation probes, and increases the prevalence of consequential reasoning. These results suggest that monitorability should be treated as a trainable attribute of the policy. A dedicated monitorability post-training stage can extend current practices that protect CoT from direct optimization pressure.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 22 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. The method's core is sound and the mechanism work is unusually honest, but the compression headline rests on an unvalidated LLM judge with a visible length confound, and the abstract oversells 'most or all'. This line is wrong

2 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 84 references say.

The paper says, at Section 4.1.1, Appendix A.2; Tables 15-16; Figures 2-4

The review did not quote this passage.

Pith says: Objection 1 of 2 · major

The compression-recovery headline is computed entirely from GPT-5.6-Luna coverage/legibility judgments and is partly confounded with CoT re-lengthening. On GPQA, the 200-step checkpoint's ε=1 condition raises median reasoning length from 261 to 384 tokens while the mean judge score rises from 3.480 to 3.657; using the observed segment slopes (roughly 0.0005 points/token end-to-end from the compressed checkpoint to base, and 0.0013-0.0018 …

more of objection 1

… points/token for the adjacent pairs in this range), the 123-token increase alone could account for roughly 0.06-0.22 points of judge-score movement, i.e., roughly 20-70% of the 0.314-point total loss. This is an upper-bound-style calculation, and I acknowledge the evidence against a pure length story: the AIME ε=1/ε=0.5 pair (Table 17) scores slightly lower at higher length (1,637 tokens/3.685 vs 1,484 tokens/3.698), and Figures 2-4 do plot judge scores against median length, as promised in A.2. What is missing is any functional check: no frozen monitor or behavioral predictor is evaluated on any compressed checkpoint (the safety monitors and probe in Section 4.2.1 are run only on undegraded policies), so the reader cannot tell whether the judge-score gains correspond to improved CoT-to-output predictability by an actual monitor. Because the abstract's central quantitative claim ('recovers most or all of the lost monitorability') leans on this experiment, I ask for a frozen-monitor evaluation on the base/compressed/post checkpoints (e.g., one of the Qwen-based or GPT-4.1-nano monitors from Section 4.2.1, or a simple predictor of the final answer from the CoT) and a length-matched or length-normalized judge comparison.

The paper says, at Abstract; Sections 4.1.1-4.1.3; Tables 1, 2, 15; Figure 1

The review did not quote this passage.

Pith says: Objection 2 of 2 · major

The claim calibration needs work. The Abstract's 'recovers most or all of the lost monitorability' is not supported by the paper's own weakest cells: on AIME, the 200-step checkpoint recovers 23.6% of the mean judge-score loss at ε=1 (Table 15), and on MBPP, the frozen monitor's recall on the hacks that remain after training is 37.4% at 3.9% FPR (Table 1) — a useful improvement, but not 'most or all' of what was lost.

more of objection 2

The Figure 1 statement that post-trained models are 'generally above' the linear savings-recovery line also fails for three of the four AIME points (Table 15: 61.3%/11.5%, 21.1%/54.4%, and 72.5%/23.6%). The MBPP and SDF recoveries are additionally mediated to a large degree by the replacement route (behavior change: hacking falls from 86.3% to 26.6% in Table 1 and denial from 55.6% to 24.5% in Table 2, with the SDF recall of 93.1% computed on the reduced set of rollouts that remain deceptive). The paper is transparent about this mechanism (Appendices B.4-B.5 and C.3), and the mode-availability bound is stated in Appendix F and Section 6, but the abstract and conclusion should carry the same qualification: the method reweights CoT-output modes the reference policy already supports, and the reported recoveries vary from roughly a quarter to most of the lost signal depending on the setting.

5 minor objections

The paper says, at Section 4.2.2, Appendix E.2

Pith says: minor objection 1 of 5

The text states that post-training increases trace coverage under both SCRUPLES prompts, but the paired bootstrap intervals reported in E.2 at δ=0.10 are [−3,10] percentage points (no cue) and [0,16] (cue), both of which include zero, and the GPQA coverage intervals in Table 4 overlap substantially between conditions; the consequential-reasoning results should be described as directional unless more samples are added.

The paper says, at Section 4.2.2, Appendix E.2

Pith says: minor objection 2 of 5

The step-50 SCRUPLES checkpoint was selected retrospectively after observing verbosity growth at step 100 (E.2); this is disclosed in the appendix but should also be stated in the main text, since it conditions the reported comparison.

show 3 more

The paper says, at Section 4.1.1, Appendix A.2

Pith says: minor objection 3 of 5

The judge-based monitorability score is treated as an interval scale when computing percentage loss recovered, and no judge variance or sampling temperature is reported; the standard errors in Figures 2-4 cover generation seeds and training runs but not judge stochasticity, so a sentence stating these limitations would help readers interpret the recovery percentages.

The paper says, at Appendix A.5

Pith says: minor objection 4 of 5

The conclusion of 'no meaningful improvement' in CoT controllability depends on a length normalization using a mean power-law exponent (α=−0.368) taken from the CoT-Control paper; since the length-normalized compliance for the ε=1 checkpoint is 6.99% vs 6.07% for the compressed checkpoint, a brief sensitivity analysis over the exponent would make the claim robust.

The paper says, at Section 3.2, Appendix B.1

Pith says: minor objection 5 of 5

The achieved conditional χ² divergence under each budget is not reported, so the reader cannot tell how binding ε is in each experiment; reporting the measured divergence (or the νPI β trajectory) alongside the budget would clarify the trade-off.

What would settle it

Run one additional evaluation on the existing GPQA-Diamond compression corpus (Tables 15-16, all six checkpoints, four CoTs per question). Add a functional monitor: a fixed frozen LLM (not fine-tuned on this corpus) that receives only the CoT and must predict that question's actual final output, scored against a shuffled-answer baseline to remove question priors. Then recompute the Table 15 recovery percentages within length-matched strata: bin every sampled CoT from base, comp-100/200, and the four post-trained checkpoints into shared length percentiles, and compare judge scores and frozen-monitor accuracy within each bin.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 22 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 2 min 43 s · 35,680 tokens read, 24,482 written
  3. Editor wrote the verdict above 42 s · 36,826 tokens read, 6,146 written
  4. Referee major revision: 2 major objections, 5 minor 3 min 29 s · 36,154 tokens read, 28,125 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Two things to know. The core proposal is real: a label-free post-training objective that maximizes CoT–output mutual information with a CVaR lower tail and a conditional χ² budget, applied as a standalone stage and shown to work across three different degradation settings (compression, monitor-pressure obfuscation, out-of-context generalization). The number you will see quoted—56.3% recovery of lost monitorability on GPQA—comes from an LLM judge the paper never validates against a functional monitor, and the paper's own tables show judge score and CoT length rising together.

The best work here is mechanistic. The revelation/replacement distinction (Section 3.3, Appendix F) actually explains when recovery is possible, and the crossed-CoT test (Appendix B.4) is the right experiment: take a recovered CoT, keep it fixed, and decode with the obfuscating base's own policy, which succeeds 47.6% of the time. That pins recovery to reweighting onto modes the reference already supports, and honestly exposes the boundary: no surviving transparent modes, no recovery. They state this in Appendix F. The abstract's 'recovers most or all' does not, and AIME recovers only 23.6%.

Soft spots, in size order. First, the compression headline. Judge score and median tokens move together across every GPQA checkpoint; within one compression ancestry, 261→384 tokens carries 0.177 of the 0.314 recovery scale, and adjacent-pair slopes suggest length could explain a large share of the gain. One AIME pair runs the other way (longer, slightly lower score), so the judge is not a pure length meter—but no frozen monitor is run on compressed checkpoints, and the headline metric is never length-normalized while the controllability metric is. The magnitude is genuinely unresolved. Second, MBPP and SDF gains are entangled with behavior change: hacking falls, denial falls, and recall is conditional on the remaining deceptive rollouts. Honest, but 'monitorability recovery' includes real behavior shifts. Third, the safety evaluation is single-split, and two choices (MBPP ε=0.4, SCRUPLES step 50) are post hoc—both disclosed. The safety results are the least vulnerable to the length critique, since gains appear at fixed early prefix positions (Figure 6). Fourth, the estimator comes from the authors' own preprint; the standalone objective, the CVaR targeting, and the χ² constraint are new but incremental.

This paper deserves a serious referee. Ask for a length-matched functional-monitor check on the compression experiment, a multi-split safety evaluation, and an abstract that matches Appendix F's scope.

The strongest objection

The central quantitative pillar for 'recovers most or all of the lost monitorability' is the compression experiment (Section 4.1.1): 56.3% mean and 60.8% worst-case loss recovery at epsilon=1 while preserving 80.2% of token savings on GPQA-Diamond, with analogous MMLU-Pro and AIME figures (Table 15, Figure 1). Every one of these numbers is computed from GPT-5.6-Luna coverage/legibility judgments (geometric mean, Section A.2). For the numbers to support the central claim, the judge must actually track monitorability - a frozen monitor's ability to predict the output from the CoT - and the gains must not be mainly CoT re-lengthening. Neither condition is checked: no frozen monitor or behavioral predictor is evaluated on any compressed checkpoint. The paper's own Table 16 makes the length channel visible. Across GPQA-Diamond checkpoints, median reasoning length and mean judge score move together: (261 tokens, 3.480), (367, 3.617), (384, 3.657), (486, 3.721), (688, 3.749), (881, 3.794). The end-to-end slope is about 0.00051 score points per token; adjacent-pair slopes reach 0.0009-0.0014. On the recovery scale M_base - M_comp(200) = 0.314, simply restoring 123 tokens (from 261 to 384) would by itself move the judge score 0.06-0.17, i.e., 20-55 of the 56.3 percentage points of reported recovery. The AIME epsilon=1/epsilon=0.5 pair (1,637 vs 1,484 tokens; 3.685 vs 3.698) shows the judge is not purely a length meter, and the comp-100+epsilon=1 checkpoint scores near base level (3.749 vs 3.794) with 193 fewer tokens, so content contributes too. The magnitude of the length contribution is unresolved, and magnitude is exactly what decides whether 'most or all' is supportable. The authors normalize the controllability metric for length (Section A.5) but never length-normalize the headline metric, and they never validate the judge against functional monitor accuracy on this corpus.

This paper argues that monitorability is a trainable attribute: an unsupervised post-training objective maximizing lower-tail CoT-output mutual information under a conditional chi-squared budget recovers monitorability lost to compression, monitor pressure, and out-of-context generalization, and improves behavioral prediction by frozen monitors and an activation probe on undegraded policies. I read the full text including appendices. What must be true for the central claim is that the quantities measured as 'monitorability' actually track a monitor's ability to predict behavior, and that observed gains are attributable to the training objective rather than to incidental correlates of training such as longer CoTs, changed behavior, or class-balance shifts. I went through the three recovery experiments, the undegraded safety experiments, and the consequential-reasoning analysis. The most secure evidence is the safety section (Table 3): frozen LLM monitors and a linear probe improve balanced accuracy on three benchmarks; SORRY-Bench improves with nearly unchanged unsafe-behavior rates (21.9% to 22.8%), and Figure 6 shows gains at fixed prefix positions, which controls for CoT-length differences. The SDF recovery (Table 2, recall 39.5% to 93.1%, with the authors' restricted-scenario robustness check) and the MBPP recovery (Table 1, recall 1.4% to 37.4% at FPR 3.9%) both use frozen monitors on held-out data, and the crossed-CoT experiment (Appendix B.4) provides direct mechanistic evidence. I credit the crossed-CoT design, the honest mode-availability limitation in Appendix F and in b213, and the disclosure of post-hoc selections (Obfuscating-base checkpoint, epsilon=0.4). The load-bearing soft spot is the compression experiment, which generates the abstract's 'recovers most or all of the lost monitorability' and the headline quantitative claims. Its metric is a proprietary LLM judge score, never validated against functional monitorability on this corpus, and the paper's own Tables 15-18 show the judge rising near-monotonically with median CoT length across checkpoints. A conservative length-slope estimate from adjacent checkpoint pairs implies length alone could account for roughly 20-55 of the 56.3 percentage points of reported GPQA recovery; the epsilon=1 vs epsilon=0.5 AIME pair shows the judge is not purely length-driven, so the unresolved question is magnitude. The authors length-normalize the controllability metric (Section A.5) but not the headline metric, which makes the omission conspicuous rather than incidental. My concern does not imply the method does nothing: the functional-monitor results in Sections 4.1.2, 4.1.3, and 4.2.1 stand apart from the judge. It does mean the flagship quantitative claim is currently supported by an unvalidated, length-correlated proxy.

The referee's report

If the results hold, this is a valuable contribution: a label-free, monitor-free training signal for a dedicated monitorability post-training stage, with differentiated mechanisms (revelation vs replacement) and unusually honest reporting. The functional-monitor evidence is the paper's strongest asset: the MBPP and SDF experiments use a frozen monitor and held-out labels; the crossed-CoT analysis (Appendix B.4) cleanly attributes the MBPP gain to replacement onto reference-supported CoT-output modes; and the safety experiment (Section 4.2.1) shows consistent balanced-accuracy gains for two external CoT monitors and a linear probe, all independent of the MI estimator used for training. The authors also state their scope openly — Appendix F ('If the ordinary-solution mode (0,0) is missing, reweighting cannot create it: the replacement arrow has no destination') and Section 6 (controlled organisms; naturally occurring frontier failures not yet tested) — and Appendix H publishes raw data tables for the headline figures.

Pith's reading of the paper (a machine's summary, not the authors')

Chain-of-thought monitoring—using a model's visible reasoning to predict what it will actually do—is one of the few practical oversight tools for capable AI systems, yet training pressures routinely break it: compression, optimization against a monitor, and out-of-context generalization can all decouple reasoning from behavior. This paper argues that monitorability is a trainable attribute of the policy rather than an incidental byproduct, and proposes an unsupervised post-training objective that maximizes the conditional mutual information between the chain of thought and the output while penalizing any change in what a given chain of thought leads the model to do. Across three degraded model organisms the method recovers most or all of the lost monitoring signal without behavioral labels or monitor feedback, and applied to undegraded policies it improves unsafe-behavior prediction by CoT monitors and activation probes while making reasoning more consequential for the final answer. If the paper is right, oversight of capable models gains a concrete, monitor-agnostic lever: a post-training stage that re-couples what a model says to what it does, rather than hoping the coupling survives training by accident.

What carries the argument. The load-bearing object is the per-trajectory mutual-information reward $r^{MI}_i$, estimated inside a group of $n$ CoT–output samples for one prompt by scoring each output under every CoT sampled for that prompt and comparing its log-probability with the group average. Averaging this reward over the least-informative $\alpha=0.75$ fraction of trajectories (conditional value-at-risk) makes the objective concentrate on the worst CoTs rather than seeking the mean. The constraint $D^{cond}_{\chi^2}(\pi_\theta\|\pi_{ref}) \le \epsilon$—enforced through a normalized Lagrangian with a νPI-controlled multiplier and gradient masking that sends the MI gradient only to CoT tokens and the divergence gradient only to output tokens—limits how much a given CoT's output distribution may change, making it costly for the model to invent private codes while leaving the distribution over CoTs free. A toy model in Appendix F makes the mechanism precise: monitorability improves either by revelation (exposing the same strategy in the CoT) or by replacement (shifting probability toward CoT–output modes the reference policy already supports), and the crossed-CoT experiment shows replacement dominates on MBPP, since recovered CoTs already elicit 47.6% true pass when decoded by the frozen obfuscating policy.

Core claim. The paper's central claim is that a model's chain of thought is a learned intermediate representation shaped by the pressures of producing high-reward outputs, and that those same pressures can strip it of the information a monitor needs—so monitorability should be optimized directly rather than assumed. The proposed objective maximizes a lower-tail (CVaR, $\alpha=0.75$) estimate of the conditional mutual information $I(O;Z|X)$ between the chain of thought and the output, subject to a conditional $\chi^2$ divergence budget $\epsilon$ that constrains how much a fixed chain of thought may change the output it produces. The headline results are specific: from a 200-step compressed checkpoint on GPQA-Diamond, post-training at $\epsilon=1$ recovers 56.3% of the mean and 60.8% of the worst-case monitorability loss while preserving 80.2% of the token savings; on MBPP it raises hack-detection recall from 1.4% to 37.4% at a false-positive rate of 3.9% while raising true pass from 11.2% to 48.6%; and on the synthetic-document customer-support organism it raises deceptive-rollout recall from 39.5% to 93.1%. On undegraded policies, both CoT monitors and an activation probe improve balanced accuracy on HarmBench, StrongREJECT, and SORRY-Bench, and trace coverage increases on both SCRUPLES and GPQA. The authors frame the mechanism explicitly: gains flow through revelation (the same behavior becomes more explicit in the CoT) or replacement (probability shifts toward CoT–output modes the reference policy already supports), which is why the method can only recover signal whose mode still exists.

The premise it rests on. The load-bearing premise is that the objective can only reweight CoT–output modes that already exist in the reference policy—if no transparent reasoning-to-output coupling survives for the training to shift probability toward, recovery cannot occur—and, for the compression headline specifically, that the LLM-judge coverage and legibility score is a length-unbiased proxy for what a real monitor can extract.

If the paper is right (Pith's inferences, not the authors')

A dedicated monitorability post-training stage can be added to the training pipeline after the fact, extending current practices that protect chain-of-thought from direct optimization pressure by repairing damage after it has occurred.

Compressed reasoning models do not have to choose between token savings and oversight: at $\epsilon=1$ the method keeps 80.2% of compression savings on GPQA while recovering most of the lost monitorability, and the gains transfer out-of-distribution to MMLU-Pro and AIME.

Monitor evasion and out-of-context concealment are partly reversible without behavioral labels: residual hacks and deceptive rollouts become far more detectable after post-training, not merely less frequent.

On undegraded policies the same objective improves safe/unsafe prediction by two CoT monitors and an activation probe across all three safety benchmarks, so the method strengthens monitorability rather than only repairing it.

Post-training increases trace coverage—the fraction of chains of thought containing at least one span that changes the probability of the final answer—showing the re-coupling is functional and not just a judged improvement in style.

Because recovery operates by reweighting modes the reference policy already supports, the method doubles as a diagnostic: the amount of monitorability recoverable under a fixed budget estimates how much transparent reasoning capacity survives in a policy, which is precisely the quantity an oversight regime needs before trusting CoT monitoring.

A boundary case follows from the paper's own Appendix F logic: policies under intense pressure that never rewarded or sampled transparent CoT–output couplings, or that learned true steganographic codes, should resist this objective; a natural extension is to test whether the $\chi^2$ budget must explode before any monitorability reappears in such a policy.

The compression headline rests on a judge-based score that correlates with length; the paper reports length alongside score and normalizes controllability for length, but a sharper future evaluation would hold CoT length fixed and measure recovery with an actual downstream monitor rather than a judge.

The revelation/replacement distinction suggests a possible coupling between monitorability training and honest behavior: in both the MBPP and synthetic-document settings, replacement happened to route toward correct and honest modes the reference policy supported, and whether that prosocial bias holds generally is an open question this paper does not address.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Chain-of-thought (CoT) monitoring is one of the few practical tools available for overseeing capable AI systems, but it is fragile: a model's reasoning is only useful to a monitor if it remains coupled to what the model actually does. This coupling can be degraded during training through, for example, CoT compression, optimization against a monitor, and out-of-context generalization from synthetic documents. In this work, we propose an unsupervised post-training objective based on maximizing mutual information between CoT and the output for improving CoT monitorability. Across three model organisms with degraded CoT, it recovers most or all of the lost monitorability. Applied to undegraded policies, it further improves judged monitorability, improves prediction of unsafe behavior by CoT monitors and activation probes, and increases the prevalence of consequential reasoning. These results suggest that monitorability should be treated as a trainable attribute of the policy. A dedicated monitorability post-training stage can extend current practices that protect CoT from direct optimization pressure.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-22

The authors' main claim, as the classifier picked it out 1.00

“These results suggest that monitorability should be treated as a trainable attribute of the policy.”

How the main claim is stated: hedged possibility 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.18 · human participants 0.04 · controlled comparison 0.57 · randomized assignment 0.02 · states quantitative result 0.09 · compares with prior results 0.46 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Chain-of-thought (CoT) monitoring is one of the few practical tools available for overseeing capable AI systems, but it is fragile: a model's reasoning is only useful to a monitor if it remains coupled to what the model actually does. background 1.00 0.07 0.03 asserted 1.00
This coupling can be degraded during training through, for example, CoT compression, optimization against a monitor, and out-of-context generalization from synthetic documents. background 0.99 0.15 0.02 hedged possibility 0.53
In this work, we propose an unsupervised post-training objective based on maximizing mutual information between CoT and the output for improving CoT monitorability. method 0.63 0.85 0.09 asserted 0.90
Across three model organisms with degraded CoT, it recovers most or all of the lost monitorability. result 1.00 0.95 0.12 asserted 1.00
Applied to undegraded policies, it further improves judged monitorability, improves prediction of unsafe behavior by CoT monitors and activation probes, and increases the prevalence of consequential reasoning. result 1.00 0.93 0.11 asserted 1.00
These results suggest that monitorability should be treated as a trainable attribute of the policy. interpretation 1.00 0.58 0.88 hedged possibility 1.00
A dedicated monitorability post-training stage can extend current practices that protect CoT from direct optimization pressure. interpretation 0.59 0.23 0.10 hedged possibility 0.79

What kind of paper the classifier reports this is: method or tool 0.99 · evidence from observation or experiment 0.94

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-22: 1 request, 5,611 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. Mean (left) and worst-case (right) monitorability loss recovered versus token savings
Figure 2 of the paper
Figure 2. GPQA coverage and monitorability. The top row shows mean and worst-case coverage; the
Figure 3 of the paper
Figure 3. Compression recovery on all 60 AIME 2024–2025 problems. The top row shows coverage
Figure 4 of the paper
Figure 4. Compression recovery on the MMLU-Pro, covering all 14 subjects. The top row shows
Figure 5 of the paper
Figure 5. Training trajectories for the monitorability recovery settings. The curves show the training
Figure 6 of the paper
Figure 6. Balanced accuracy across CoT prefix positions. Each point uses one prefix per held-out
Figure 7 of the paper
Figure 7. Revelation retains the hacking behaviour while making its strategy explicit. Replacement

Figures are the authors', reproduced from the paper.

References (84)
  1. [1] Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models , author=. arXiv preprint arXiv:2507.12428 , year= on PitharXiv
  2. [2] Optimization of conditional value-at-risk , author=. The Journal of Risk , volume=. 2000 , doi= on Pith
  3. [3] Linearity of relation decoding in transformer language models , author= on Pith
  4. [4] Pal, Koyena and Sun, Jiuding and Yuan, Andrew and Wallace, Byron and Bau, David. Future Lens: Anticipating Subsequent Tokens from a Single Hidden State. Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL). 2023. doi:10.18653/v1/2023.conll-1.37 on Pithdoi:10.18653/v1/2023.conll-1.37
  5. [5] Do language models plan ahead for future tokens? , author=. arXiv preprint arXiv:2404.00859 , year= on PithPith's reviewarXiv
  6. [6] Decomposing and Measuring Evaluation Awareness , author=. arXiv preprint arXiv:2605.23055 , year= on PithPith's reviewarXiv
  7. [7] Models That Know How Evaluations Are Designed Score Safer , author=. arXiv preprint arXiv:2605.28591 , year= on PithPith's reviewarXiv
  8. [8] Steering Evaluation-Aware Language Models to Act Like They Are Deployed , author=. arXiv preprint arXiv:2510.20487 , year= on PitharXiv
  9. [9] Large language models often know when they are being evaluated , author=. arXiv preprint arXiv:2505.23836 , year= on PithPith's reviewarXiv
  10. [10] UK AISI alignment evaluation case-study , author=. arXiv preprint arXiv:2604.00788 , year= on PitharXiv
  11. [11] System Card:. 2026 , month = apr, url = on Pith
  12. [12] 2025 , month = nov, url = on Pith
  13. [13] 2026 , month = feb, url = on Pith
  14. [14] Agentic Misalignment in Summer 2026 , author =. 2026 , url = on Pith
  15. [15] 2026 , month = jul, url = on Pith
  16. [16] The hawthorne effect in reasoning models: Evaluating and steering test awareness , author=. Advances in Neural Information Processing Systems , volume= on Pith
  17. [17] Stress testing deliberative alignment for anti-scheming training , author=. arXiv preprint arXiv:2509.15541 , year= on PitharXiv
  18. [18] Coleman, Russell , howpublished =. Eval Awareness in. 2026 , month = mar, url = on Pith
  19. [19] Probing and steering evaluation awareness of language models , author=. arXiv preprint arXiv:2507.01786 , year= on PithPith's reviewarXiv
  20. [20] Measuring and Improving Coding Audit Realism with Deployment Resources , author =. 2026 , month = mar, howpublished = on Pith
show 64 more
  1. [21] Petri 2.0: New Scenarios, New Model Comparisons, and Improved Eval-Awareness Mitigations , author =. 2026 , month = jan, howpublished = on Pith
  2. [22] Prediction, Explanation, or Over-interpretation? , author =. 2026 , month = on Pith
  3. [23] Fraser-Taliente, Kit and Kantamneni, Subhash and Ong, Euan and Mossing, Dan and Lu, Christina and Bogdan, Paul C. and Ameisen, Emmanuel and Chen, James and Kishylau, Dzmitry and Pearce, Adam and Tarng, Julius and Wu, Alex and Wu, Jeff and Zhang, Yang and Ziegler, Daniel M. and... on Pith
  4. [24] Foundational challenges in assuring alignment and safety of large language models , author=. arXiv preprint arXiv:2404.09932 , year= on PithPith's reviewarXiv
  5. [25] Correcting the mythos of kl-regularization: Direct alignment without overoptimization via chi-squared preference optimization , author=. International Conference on Learning Representations , volume= on Pith
  6. [26] Kimi k1.5: Scaling Reinforcement Learning with. arXiv preprint arXiv:2501.12599 , year= on PithPith's reviewarXiv
  7. [27] Reinforcement Learning for Chain of Thought Compression with One-Domain-to-All Generalization , author=. arXiv preprint arXiv:2601.06052 , year= on PitharXiv
  8. [28] A Pragmatic Way to Measure Chain-of-Thought Monitorability , author=. arXiv preprint arXiv:2510.23966 , year= on PitharXiv
  9. [29] Haskins, Reilly and Chughtai, Bilal and Engels, Joshua , title =. 2026 , month = mar, howpublished = on Pith
  10. [30] Greenblatt, Ryan and Shlegeris, Buck and Sachan, Kshitij and Roger, Fabien , year=. 2312.06942 , archivePrefix= on PithPith's reviewarXiv
  11. [31] Monitoring Monitorability , author=. 2025 , eprint= on Pith
  12. [32] Arnav, Benjamin and Bernabeu-Perez, Pablo and Helm-Burger, Nathan and Kostolansky, Timothy and Whittingham, Hannes and Phuong, Mary , booktitle=. 2025 , note= on Pith
  13. [33] Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation , author=. 2025 , eprint= on Pith
  14. [34] Tomek Korbak and Mikita Balesni and Elizabeth Barnes and Yoshua Bengio and Joe Benton and Joseph Bloom and Mark Chen and Alan Cooney and Allan Dafoe and Anca Dragan and Scott Emmons and Owain Evans and David Farhi and Ryan Greenblatt and Dan Hendrycks and Marius Hobbhahn and E... on PithPith's reviewarXiv
  15. [35] Output Supervision Can Obfuscate the Chain of Thought , author=. 2025 , eprint= on Pith
  16. [36] Chain-of-thought obfuscation learned from output supervision can generalise to unseen tasks , author=. 2026 , eprint= on Pith
  17. [37] Interpreting Emergent Planning in Model-Free Reinforcement Learning , author=. 2025 , eprint= on Pith
  18. [38] Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning , author=. 2026 , eprint= on Pith
  19. [39] Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting , author=. 2023 , eprint= on Pith
  20. [40] Measuring Faithfulness in Chain-of-Thought Reasoning , author=. 2023 , eprint= on Pith
  21. [41] Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory , author=. 2026 , eprint= on Pith
  22. [42] Contrastive learning, multi-view redundancy, and linear models , author =. Proceedings of the 32nd International Conference on Algorithmic Learning Theory , pages =. 2021 , editor = on Pith
  23. [43] Thomas G. Dietterich and Ghulum Bakiri , editor =. Error-Correcting Output Codes:. Proceedings of the 9th National Conference on Artificial Intelligence, Anaheim, CA, USA, July 14-19, 1991, Volume 2 , pages =. 1991 , url = on Pith
  24. [44] Doi, Eizaburo and Balcan, Doru and Lewicki, Michael , booktitle =. A Theoretical Analysis of Robust Coding over Noisy Overcomplete Channels , url = on Pith
  25. [45] Nanda, Vedant and Speicher, Till and Dickerson, John and Gummadi, Krishna and Feizi, Soheil and Weller, Adrian , booktitle =. Diffused Redundancy in Pre-trained Representations , url =. doi:10.52202/075280-0177 , editor = on Pithdoi:10.52202/075280-0177
  26. [46] Sun, Yue and Narang, Adhyyan and Gulluk, Ibrahim and Oymak, Samet and Fazel, Maryam , booktitle =. Towards Sample-efficient Overparameterized Meta-learning , url = on Pith
  27. [47] On the Theory of Transfer Learning: The Importance of Task Diversity , author=. 2020 , eprint= on Pith
  28. [48] Rethinking Minimal Sufficient Representation in Contrastive Learning , author=. 2022 , eprint= on Pith
  29. [49] Jan Leike , title =. 2025 , month = jul, day = on Pith
  30. [50] Maksym Andriushchenko , title =. 2026 , month = aug, day = on Pith
  31. [51] Christopher Potts , title =. 2026 , month = jul, day = on Pith
  32. [52] A Theory of Usable Information Under Computational Constraints , author=. 2020 , eprint= on Pith
  33. [53] How transferable are features in deep neural networks? , author=. 2014 , eprint= on Pith
  34. [54] Understanding Intermediate Layers Using Linear Classifier Probes , author=. 2016 , eprint= on Pith
  35. [55] Been Kim and Martin Wattenberg and Justin Gilmer and Carrie Cai and James Wexler and Fernanda Vi. Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (. Proceedings of the 35th International Conference on Machine Learning , pages=.... on Pith
  36. [56] Monitoring Latent World States in Language Models with Propositional Probes , author=. The Thirteenth International Conference on Learning Representations , year= on Pith
  37. [57] Why Do Better Loss Functions Lead to Less Transferable Features? , author=. 2021 , eprint= on Pith
  38. [58] Characterizing and Avoiding Negative Transfer , author=. 2019 , eprint= on Pith
  39. [59] Lossy Compression for Lossless Prediction , author=. 2022 , eprint= on Pith
  40. [60] Tian, Yonglong and Sun, Chen and Poole, Ben and Krishnan, Dilip and Schmid, Cordelia and Isola, Phillip , booktitle =. What Makes for Good Views for Contrastive Learning? , url = on Pith
  41. [61] Hjelm, R Devon and Fedorov, Alex and Lavoie-Marchildon, Samuel and Grewal, Karan and Bachman, Phil and Trischler, Adam and Bengio, Yoshua , booktitle =. 2019 , url = on Pith
  42. [62] On Mutual Information Maximization for Representation Learning , author=. International Conference on Learning Representations , year= on Pith
  43. [63] Training Agents to Self-Report Misbehavior , author=. arXiv preprint arXiv:2602.22303 , year= on PitharXiv
  44. [64] Reasoning Models Struggle to Control their Chains of Thought , author=. 2026 , eprint= on Pith
  45. [65] Reasoning Models Don't Always Say What They Think , author=. 2025 , eprint= on Pith
  46. [66] Nature , year=. doi:10.1038/s41586-025-09422-z , eprint= on Pithdoi:10.1038/s41586-025-09422-z
  47. [67] Representation Learning: A Review and New Perspectives , author=. 2014 , eprint= on Pith
  48. [68] Taskonomy: Disentangling Task Transfer Learning , author=. 2018 , eprint= on Pith
  49. [69] Reasoning Under Pressure: How do Training Incentives Influence Chain-of-Thought Monitorability? , author=. 2025 , eprint= on Pith
  50. [70] Legibility is Not Interpretability: Comparing Judged and Actual Importance in Chain-of-Thought Reasoning , author=. 2026 , eprint= on Pith
  51. [71] Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David Forsyth and Dan Hendrycks , year=. 2402.04249 , archivePrefix= on PithPith's reviewarXiv
  52. [72] Deliberative Alignment: Reasoning Enables Safer Language Models , author =. 2024 , eprint = on Pith
  53. [73] Sohrabi, Motahareh and Ramirez, Juan and Zhang, Tianyue H. and Lacoste-Julien, Simon and Gallego-Posada, Jose , booktitle =. On. 2024 , publisher = on Pith
  54. [74] Yohan Mathew and Ollie Matthews and Robert McCarthy and Joan Velja and Christian Schroeder de Witt and Dylan Cope and Nandi Schoots , year=. Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in. 2410.03768 , archivePrefix= on PitharXiv
  55. [75] Motwani, Sumeet Ramesh and Baranchuk, Mikhail and Strohmeier, Martin and Bolina, Vijay and Torr, Philip H.S. and Hammond, Lewis and de Witt, Christian Schroeder , booktitle =. Secret Collusion among on Pith
  56. [76] If you can generate obfuscated chain-of-thought, can you monitor it? , author= on Pith
  57. [77] Early Signs of Steganographic Capabilities in Frontier LLMs , author=. 2025 , eprint= on Pith
  58. [78] Program Synthesis with Large Language Models , author=. arXiv preprint arXiv:2108.07732 , year= on PithPith's reviewarXiv
  59. [79] David Rein and Betty Li Hou and Asa Cooper Stickland and Jackson Petty and Richard Yuanzhe Pang and Julien Dirani and Julian Michael and Samuel R. Bowman , year=. 2311.12022 , archivePrefix= on PithPith's reviewarXiv
  60. [80] Scruples: A Corpus of Community Ethical Judgments on 32,000 Real-Life Anecdotes , author=. 2021 , eprint= on Pith
  61. [81] Zhihong Shao and Peiyi Wang and Qihao Zhu and Runxin Xu and Junxiao Song and Xiao Bi and Haowei Zhang and Mingchuan Zhang and Y. K. Li and Y. Wu and Daya Guo , year=. 2402.03300 , archivePrefix= on PithPith's reviewarXiv
  62. [82] A StrongREJECT for Empty Jailbreaks , author=. 2024 , eprint= on Pith
  63. [83] SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal , author=. 2025 , eprint= on Pith
  64. [84] Yubo Wang and Xueguang Ma and Ge Zhang and Yuansheng Ni and Abhranil Chandra and Shiguang Guo and Weiming Ren and Aaran Arulraj and Xuan He and Ziyan Jiang and Tianle Li and Max Ku and Kai Wang and Alex Zhuang and Rongqi Fan and Xiang Yue and Wenhu Chen , year=. 2406.01574 , a... on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment