Pith. Sign in

REVIEW

Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring

PaperPDF from arXiv

data files

ReviewThe objections and verdict, as JSON Paper breakdownThe authors' claims, conditions and measurements, as JSON Signed receiptProof this review has not been altered
X Bluesky LinkedIn Reddit Hacker News
Mohammadali Mohammadkhani, Madhava Krishna, Yash Sarrof, Michael Hahn

Machine Learning · Submitted 29 Sep 2026 · 35 references

Pith's verdict: Serious theory paper: near-linear footprint bound holds within its stated fixed-precision scope, encryption result is honestly conditional on unproven PRG assumptions, and the paper discloses both — worth a real referee.

T0 · AI review

How this review was made: An AI referee (deepseek/deepseek-v4-flash) read the paper on September 30, 2026 and answered a fixed list of questions about it. T0 is the starting level every paper gets. The levels describe how deeply Pith has checked the paper, never the authors.

  1. T0 AI review An AI read the paper and answered a fixed list of questions.
  2. T1 Audit Automatic checks that the references and files the paper relies on exist and match.
  3. T2 Compute The paper's code or calculations re-run with the same results.
  4. T3 Formal statement The claims written out in a formal math language a computer can read.
  5. T4 Proof Those claims proved, and the proof checked by computer.

How Pith checks papers →

· 1 major objection · 6 minor · Ask about this paper · The compiled record · Read the full review →
Challenge this review Comment (0) Read on arXiv ↗
Cite ▾
arXiv ID2609.37312 (version 1)
Pith ID4NQCK7MS, Pith's permanent record of this paper and its review
CitationMadhava Krishna, Michael Hahn, Mohammadali Mohammadkhani, Yash Sarrof. "Hidden Reasoning Must Leak, but Need Not Be Readable: Fundamental Opportunities and Limits for Chain-of-Thought Monitoring." arXiv:2609.37312 (2026). https://pith.science/paper/4NQCK7MS
Cite this reviewBibTeX for Pith's review of this paper
Claim paper

Why Pith says this

Highlight any text to comment on it.

Referee report

How Pith checks papers → · Full audit of this paper →

The paper

The paper claims

Hard hidden computation must leak a near-linear amount of information about its covert input into a model's chain of thought, yet that footprint can be encrypted so no efficient monitor can read it.

The reading

The paper asks whether a reasoning model can compute a covert task without leaving recoverable evidence in its chain-of-thought (CoT) trace. It establishes that once the task exceeds the model's fixed-precision circuit capacity, the CoT must carry an information-theoretic footprint: mutual information with the covert input grows near-linearly in input length when the CoT is polynomially long. Simple tasks can stay hidden, but complex ones cannot. The paper then shows the footprint need not be readable: under a plausible cryptographic assumption, a single-layer Transformer can encrypt its CoT on the fly so that no polynomial-time monitor extracts anything, while the information-theoretic leakage persists. Experiments on parity, S5 state tracking, multiplication, and iterated addition confirm that uninformative CoTs fail beyond a size-dependent threshold while information-bearing CoTs succeed, and that the encryption construction can actually be learned.

What carries the argument

The argument runs on a two-part machine. First, a standard translation (Lemma B.3) turns a fixed-precision Transformer of depth $L$ and size $S$ into an AC0 circuit of depth linear in $L$ and size polynomial in $S$ and CoT length; combined with Tal's Fourier-spectrum bound for AC0 (Theorem B.4), this forces any single CoT transcript indicator to have exponentially small correlation with PARITY and the other tasks. Summing over transcripts converts that correlation bound into a mutual-information lower bound (Lemma B.5), which is the footprint of Theorem 3.1. Second, the encryption construction combines an oblivious Turing-machine simulation of the task with an NC0 pseudorandom generator, whose output bits are retrieved by attention heads and XORed with the hidden CoT token by token inside one Transformer layer.

Extended reading notes

Core claim

On its own terms the paper proves a three-way tradeoff: any covert task from a specific family must be paid for with model capacity, CoT length, or information-theoretic leakage $I[X_{\mathrm{covert}}, \mathrm{CoT}]$, where capacity and length enter only polylogarithmically and depth enters exponentially. Rearranging the bound shows that when the CoT is polynomially long, the mutual information between covert input and CoT is near-linear in the input length, giving three regimes: no CoT, agnostic CoT (uninformative, necessarily super-polynomial), and informative CoT (near-linear leakage). The paper also proves a covert-computation length tax: hiding a private task behind an unrelated public task costs extra CoT tokens unless the two inputs are correlated, in which case the correlation itself leaks information. Finally, assuming an NC0 pseudorandom generator, a one-layer Transformer can XOR its hidden trace with a short private seed, keeping the information-theoretic footprint but making it computationally intractable for any polynomial-time monitor to predict any property of the covert input.

Load-bearing premise

The near-linear leakage guarantee assumes real models are well approximated by low-depth fixed-precision circuits, because the proof translates Transformers into AC0 circuits; the paper concedes that log-precision Transformers, which can express PARITY and arguably model LLMs better, escape AC0 and have no proven analogue.

Editorial extensions

If this is right

  • CoT monitoring can only promise to detect reasoning that exceeds model capacity: a single hidden intent bit, such as 'sandbagging this evaluation?,' need not leak, while computationally complex covert objectives must leave a footprint.
  • The length tax gives monitors a concrete signal: hidden computation behind an unrelated public task must pay extra tokens unless public and covert inputs are correlated, and correlation itself reveals information about the hidden input.
  • Because depth enters the bound exponentially, any training innovation that makes models use their full depth also exponentially widens the range of reasoning that can stay invisible in the CoT.
  • The encryption theorem implies that blackbox monitoring cannot guarantee extraction of hidden computation even where an information-theoretic footprint provably exists, shifting the practical hope to interventional defenses.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same machinery should transfer to any fixed-precision model class contained in AC0, so the three-mode picture (no CoT, agnostic, informative) likely recurs across hard-attention and related formal models.
  • If a future correlation bound were proven for log-precision Transformers, the near-linear leakage guarantee would extend to the regime most relevant for deployed LLMs; until then, the practical reach of Theorem 3.1 rests on LLMs behaving like low-depth fixed-precision circuits.
  • The on-the-fly encryption construction generalizes to any polynomial-time computable function under a sufficiently strong PRG, suggesting steganographic CoTs are a structural possibility of the architecture rather than a training artifact.
  • A testable extension: interventional defenses such as paraphrasing should destroy semantically embedded leaks but may fail against robust steganography that survives paraphrase; whether such robustness is achievable inside a Transformer forward pass is open.
Original abstract

Can reasoning models trick chain of thought (CoT) monitors and perform hidden computation without revealing it in their thinking traces? We show that the answer depends on the underlying task difficulty and the model size. Simple computations can be performed covertly; however, beyond a threshold depending on model size, successfully solving the task necessarily leaks a near-linear amount of information about the covert task input into the CoT. Therefore, sufficiently complex hidden computation always leaves an information-theoretic footprint. However, concerningly, this leakage need not be readable: Under plausible cryptographic assumptions, even a one-layer Transformer can encrypt its reasoning online so that no polynomial-time monitor can extract information about the hidden computation. Overall, our theoretical and empirical results provide a holistic view of both the opportunities and the limitations of CoT monitoring.

The compiled record

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-09-30: 1 request, 4,885 input tokens.

What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.68 · human participants 0.03 · controlled comparison 0.07 · randomized assignment 0.02 · states quantitative result 0.76 · compares with prior results 0.09 · mentions code or data availability 0.03.

The abstract, sentence by sentence

arXiv listing abstract v1. Results and the main claim are underlined; .

Can reasoning models trick chain of thought (CoT) monitors and perform hidden computation without revealing it in their thinking traces? We show that the answer depends on the underlying task difficulty and the model size. Simple computations can be performed covertly; however, beyond a threshold depending on model size, successfully solving the task necessarily leaks a near-linear amount of information about the covert task input into the CoT. Therefore, sufficiently complex hidden computation always leaves an information-theoretic footprint. However, concerningly, this leakage need not be readable: Under plausible cryptographic assumptions, even a one-layer Transformer can encrypt its reasoning online so that no polynomial-time monitor can extract information about the hidden computation. Overall, our theoretical and empirical results provide a holistic view of both the opportunities and the limitations of CoT monitoring.

background contribution method result interpretation main claim

What the classifier reports for each sentence. The number is the probability it gave the label.
#sentencerolethe authors' own workstates the main conclusionstrength
s0 Can reasoning models trick chain of thought (CoT) monitors and perform hidden computation without revealing it in their thinking traces? background 0.70 0.04 0.03 unclear 0.67
s1 We show that the answer depends on the underlying task difficulty and the model size. result 0.97 0.87 0.13 asserted 1.00
s2 Simple computations can be performed covertly; however, beyond a threshold depending on model size, successfully solving the task necessarily leaks a near-linear amount of information about the covert task input into the CoT. result 1.00 0.88 0.42 asserted 1.00
s3 Therefore, sufficiently complex hidden computation always leaves an information-theoretic footprint. interpretation 0.60 0.76 0.37 asserted 1.00
s4 However, concerningly, this leakage need not be readable: Under plausible cryptographic assumptions, even a one-layer Transformer can encrypt its reasoning online so that no polynomial-time monitor can extract information about the hidden computation. result 0.87 0.63 0.22 hedged possibility 0.62
s5 Overall, our theoretical and empirical results provide a holistic view of both the opportunities and the limitations of CoT monitoring. interpretation 0.55 0.77 0.24 asserted 0.97

What kind of paper the classifier reports this is: theoretical result 1.00 · evidence from proof or derivation 0.83.

What was examined, and what was not

sources read
  • arXiv listing abstract v1
not examined
  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers
levels run
  • P abstract
not run
  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. Every number in the measurements table is the authors', checked by code. Record as JSON

Figures
Figure 1. (a) Our setup: A Transformer holds a covert input
Figure 2. No CoT and Agnostic protocols generally degrade with input length, whereas In￾formative protocols can retain high accuracy at longer inputs. Top: Three modes of hidden reasoning, illustrated by five protocols where a model computes F(Xcovert) under the cover of an ostensible public task F(Xpublic). In the No CoT mode, no CoT is needed. In the Agnostic CoT Mode, there is a CoT but its content is unrelated to Xcovert – in these examples, it consists of filler tokens, or the CoT for an unrelated F(Xpublic). Information-theoretically, I[Xcovert, CoT] = 0. Theorem 3.1 shows that, beyond…
Figure 3. Larger models can perform harder covert tasks without CoT. Max passing d vs log(model_size) for joint-correct accuracy for Vanilla prompting. Here, d denotes the difficulty – the number of digits for operands in multiplication, the number of swaps in S5, and the number of bits in PARITY, and τ is set to the mean joint-correct accuracy for the task rounded off to the nearest integer. A model counts as passing if its accuracy exceeds τ . See Appendix D.4 and D.5 for full re￾sults, including other values of τ . Theorem 3.5: Encryption We trained one￾layer Transformers (16-head,…
Figure 4. Encrypted CoT for PARITY. (Left) A one-layer Transformer utilizes private randomness to encrypt the CoT on the fly. It simultaneously obtains a pseudo-one-time pad; the visible CoT is the XOR of the underlying CoT with that pad. As N → ∞, no polynomial-time monitor is known to be able to extract nonnegligible information about the input string Xcovert from this public CoT. (Right) Empirical accuracy for a one-layer Transformer. Encrypted CoTs remain accurate across long inputs, whereas no-CoT and filler-only CoTs fall to chance at N = 32. Theorem 3.2 shows that this is an…
Figure 5. Mean Generated CoT length for Piggy￾backing and Special Tokens as a function of input length. Piggybacking has shorter CoT than Special Tokens, at the cost of making Xpublic and Xcovert very similar, thus high I[Xpublic, Xcovert]. The Covert Computation Length Tax (Corollary 3.2) shows that this tradeoff is unavoidable: no protocol can hide computation for Xcovert behind compu￾tation for an unrelated Xpublic without incurring a length tax. Public/covert protocol. Each example con￾tains a main/public question and an other/- covert/private question, and the model must re￾turn both…
Figure 6. Balanced Multiplication scaling. Covert exact accuracy as a function of exact digits per operand for Qwen2.5-7B and Llama-3.1-8B. Points and bands show mean and ±1 sample standard deviation over three seeds. Every displayed length is represented in the same jointly trained checkpoint. Filler denotes meaningless-token-only workspace; Filler + Public CoT is the appendix ablation. Qwen shows a clear Informative CoT mechanism advantage at longer operands, whereas Llama does not. Filler reaches 11.2% at five swaps, 1.5% at ten, and 1.0% at nineteen, whereas Llama retains 35.3%, 23.2%,…
Figure 7. Balanced iterated-addition scaling. Covert exact accuracy as the number of retrieved atomic-number facts increases from one to five. Piggybacking shares the first k − 1 facts and replaces only the final fact; the independent protocols and Special Tokens share no facts. Error bands show ±1 sample standard deviation over three seeds. Qwen Piggybacking is effective but seed-variable. 1 5 10 15 19 Number of swaps 0% 25% 50% 75% 100% Covert accuracy S5 state tracking · Qwen2.5-7B Vanilla Filler Public-only CoT Piggybacking Special Tokens Filler + Public CoT [ablation] 1 5 10 15 19 Number…
Figure 8. Balanced S5 scaling. Covert exact accuracy from one through 19 swaps. Piggybacking and Special Tokens remain near ceiling for both models. Public-Only collapses rapidly; primary Filler also declines, although Llama retains a model-dependent workspace benefit at intermediate and long lengths. Piggybacking shares the first k − 1 of k swaps, so its reusable fraction increases with sequence length. Vanilla, 12.5 ± 4.0% under Filler, 80.7 ± 25.7% under Piggybacking, and 99.5 ± 0.9% under Special Tokens. Llama reaches 6.3 ± 1.5%, 8.0 ± 2.6%, 99.7 ± 0.3%, and 97.2 ± 0.6%, respectively.…
Figure 9. Plain-Parity scaling. Covert exact accuracy across five jointly trained sequence-length ranges. Vanilla, Filler, and Public-Only remain close to chance, whereas Piggybacking and Special Tokens remain near ceiling. The plotted x-values correspond to the trained length ranges 8–15, 16–31, 32–63, 64–127, and 128–255. Task Model Vanilla Filler Public-Only Piggy Special Tokens Filler + Public CoT Multiplication, 4 digits Qwen 2.0 1.3 2.2 43.8 48.3 1.0 Multiplication, 4 digits Llama 1.2 1.3 1.0 0.8 0.5 0.8 Iterated addition, 5 facts Qwen 10.2 12.5 8.0 80.7 99.5 8.5 Iterated addition, 5…
Figure 10. Joint-correct max-passing-d scaling for Vanilla prompting under two thresholds for a passing-d. 1. T2 – τMULT = 35%, τP ARIT Y = 72%, τS5 = 7.5% 2. T3 – τMULT = 50%, τP ARIT Y = 80%, τS5 = 10% We observe that the trends for Vanilla prompting seen in
Figure 11. Max-passing-d scaling for joint-correct accuracy for three pass thresholds: Agnostic CoT , taken as the average between Filler and Public-only CoT (left), and Informative CoT , consisting of Piggybacking (right). Note that Theorem 3.1 predicts that, given polynomiual-size CoT lengths, Agnostic CoT protocols (I[Xcovert, CoT] = 0) only succeed up to a threshold depending on the logarithm of the model size; consistent with the positive correlations often observed in the left column. In contrast, for Informative CoT schemes (right), model size is not inherently predicted to constrain…
Figure 12. Multiplication: covert accuracy per model at
Figure 13. Multiplication: joint-correct accuracy per model at
Figure 14. Parity: covert accuracy per model at d = 4, . . . , 7. Gray bars show Vanilla; the thin bars show the change from Vanilla under Filler, Public-only CoT, Piggybacking and Special Tokens (solid: gain, hatched: loss). The shaded band spans the unrestricted-CoT accuracy of the family’s smallest and largest model. Y-axis ranges differ between panels; a kink on the axis marks where the scale is compressed. 54
Figure 15. Parity: joint-correct accuracy per model at
Figure 16. S5: covert accuracy per model at d = 1, . . . , 4. Gray bars show Vanilla; the thin bars show the change from Vanilla under Filler, Public-only CoT, Piggybacking and Special Tokens (solid: gain, hatched: loss). The shaded band spans the unrestricted-CoT accuracy of the family’s smallest and largest model. Y-axis ranges differ between panels; a kink on the axis marks where the scale is compressed. 56
Figure 17. S5: joint-correct accuracy per model at d = 1, . . . , 4. Gray bars show Vanilla; the thin bars show the change from Vanilla under Filler, Public-only CoT, Piggybacking and Special Tokens (solid: gain, hatched: loss). The shaded band spans the unrestricted-CoT accuracy of the family’s smallest and largest model. Y-axis ranges differ between panels; a kink on the axis marks where the scale is compressed. 57
Figure 18. Covert accuracy vs. difficulty d for every mechanism, per model (rows, grouped by family) and task. The shaded band spans the unrestricted-CoT accuracy of the family’s smallest and largest model; for Kimi K3 it is a single line. 59
Figure 19. Joint-correct accuracy vs. difficulty d for every mechanism, per model (rows, grouped by family) and task. The shaded band spans the unrestricted-CoT accuracy of the family’s smallest and largest model; for Kimi K3 it is a single line. 60
Figure 20. Multiplication, Vanilla: change in covert (a) and joint-correct (b) accuracy, in pp, from
Figure 21. Multiplication, Public-only CoT: change in covert (a) and joint-correct (b) accuracy, in
Figure 22. Parity, Vanilla: change in covert (a) and joint-correct (b) accuracy, in pp, from adding
Figure 23. Parity, Public-only CoT: change in covert (a) and joint-correct (b) accuracy, in pp, from
Figure 24. S5, Vanilla: change in covert (a) and joint-correct (b) accuracy, in pp, from adding
Figure 25. S5, Public-only CoT: change in covert (a) and joint-correct (b) accuracy, in pp, from
Figure 26. Vanilla: change in covert accuracy, in pp, from adding Nf iller ∈ {100, 300, 500, 1000} filler tokens to Vanilla, per model (rows, grouped by family), task and difficulty d. Panel shading marks a net gain (green) or net loss (red), averaged over filler lengths and d. Y-axis ranges are fitted per panel. 67
Figure 27. Vanilla: change in joint-correct accuracy, in pp, from adding Nf iller ∈ {100, 300, 500, 1000} filler tokens to Vanilla, per model (rows, grouped by family), task and difficulty d. Panel shading marks a net gain (green) or net loss (red), averaged over filler lengths and d. Y-axis ranges are fitted per panel. 68
Figure 28. Public-only CoT: change in covert accuracy, in pp, from adding Nf iller ∈ {100, 300, 500, 1000} filler tokens to Public-only CoT, per model (rows, grouped by family), task and difficulty d. Panel shading marks a net gain (green) or net loss (red), averaged over filler lengths and d. Y-axis ranges are fitted per panel. 69
Figure 29. Public-only CoT: change in joint-correct accuracy, in pp, from adding Nf iller ∈ {100, 300, 500, 1000} filler tokens to Public-only CoT, per model (rows, grouped by family), task and difficulty d. Panel shading marks a net gain (green) or net loss (red), averaged over filler lengths and d. Y-axis ranges are fitted per panel. 70
Figure 30. Matched standard-Transformer controls. A single staged training procedure is used for every no-CoT and filler-only point. Both are highly accurate through N = 16 and near chance at N = 32 and N = 64; the model still generates the required filler sequence correctly. Normal and Goldreich-encrypted state-carrying CoTs remain at or near 100% accuracy through N = 1024. All runs use experiment seed 0. In the End-to-end only condition, we remove all intermediate supervision (route, standard-coordinate, retrieved-bit, mask, and public-Boolean intermediate targets) and train the model only…
Figure 31. Supervision ablations for the standard Transformer. End-to-end next-token training remains at chance at N = 32 and N = 64. Removing explicit route-position labels retains near-exact performance at N = 32 but collapses at N = 64; all other intermediate supervision remains in that condition. Results use width 128, ordinary autoregressive softmax inference, and experiment seed 0. that the auxiliary supervision makes the encrypted computation much easier to learn, and that explicit route supervision becomes more important as the sequence length grows. Because these ablations use only…

Figures reproduced from arXiv:2609.37312 by the authors.

References

Discussion

Disagree with the review, or know something it missed? Leave a comment, highlight any passage above to comment on it, or challenge this review formally.