REVIEW 3 major objections 5 minor 13 references
An 8-billion-parameter chat model commits to a premise-violating answer before reasoning, and a training-free activation oracle can read the commitment from hidden states before the answer is written.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 20:54 UTC pith:PJJHHSKF
load-bearing objection Behavioral reproduction is solid and honest; the activation evidence is suggestive but a missing task-matched baseline leaves the headline 68% vs 17% open to a simpler explanation. the 3 major comments →
Committed Before Reasoning: Behavioral Reproduction and Preliminary Activation-Level Evidence of Answer Pre-Commitment in an Open-Weight LLM
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is that a model can be committed to an answer before it reasons, and the commitment is internal before it is textual. Behaviorally, on the car-wash probe, the model walks up to the decisive premise ('if the car is already parked near the car wash...') without resolving it, then fills the template with 'Action: Walk.' Activationally, at the position immediately before the commitment token, a training-free natural-language oracle that defaults to 'drive' on unrelated content reads 'walk' in 10/16 walk-committing rollouts and 5/6 drive-committing rollouts, against a 17% neutral baseline. The read-outs survive a strict text-inversion check (spans containing 'drive' still re
What carries the argument
The load-bearing mechanism is the combination of a minimal probe and a training-free activation oracle. The car-wash question isolates the failure: one premise decides the answer, so fluent reasoning that recommends walking is advocacy for a pre-fixed answer. The oracle is a pretrained LLM fine-tuned to answer natural-language questions about injected hidden states; the paper feeds it activations at layer 18 (50% depth) from six positions P0–P5 around the commitment, with P4 immediately before the commitment token. The oracle's forced-choice question ('Is the model going to say walk or drive?') turns out to be essential: changing the question to an open form collapses the positive control fr
Load-bearing premise
The load-bearing premise is that the 17% walk rate measured on a small set of unrelated prompts is the correct counterfactual for the car-wash probes; if the oracle's read-out is sensitive to task domain, prompt length, or lexical field beyond the ±25-token strata, the 68%-vs-17% signal is overstated.
What would settle it
Run the same pre-commit probe protocol on a second open-weight model of similar size, with a neutral baseline matched for task domain and a closed question; if P4 walk read-outs in drive-committing rollouts fall to the neutral rate, or if the walk-committing read-outs disappear under a different layer choice, the commitment-reading claim is refuted. A cheaper check: replace the rejected regex span with randomly chosen pre-answer positions and see whether the P4 elevation disappears.
If this is right
- Structured-answer formats that force an early action slot push the wrong-commitment rate to 100%, so instruction design can lock in pre-commitment rather than prevent it.
- Extended chain-of-thought is not a fix: a 4,096-token thinking budget leaves walk rates at 85–100%, and nearly all correct answers occur in thinking mode, meaning thinking rescues only a few percent.
- Because even correct rollouts read as walk-leaning before commitment, a monitor that audits reasoning after the fact may be too late; the detectable signal is earlier.
- Training-free activation oracles can detect answer pre-commitment without task-specific probe training, extending the supervised-probe result to a hint-free, no-training setting.
- Any negative result from an activation oracle on a new task must be accompanied by a positive control using the same wording; wording alone moved the control from 2/16 to 11/16.
Where Pith is reading between the lines
- If pre-commitment generalizes beyond this one question, the practical target for faithfulness interventions shifts from 'improving reasoning' to 'detecting and interrupting commitment before it is written'—for example, a real-time monitor on P4-style read-outs.
- The oracle's brittleness suggests that many earlier negative results with training-free oracles on safety tasks may be wording artifacts; re-testing them with closed, in-distribution questions could revise those conclusions.
- A direct testable extension: vary the strength of the decisive premise (e.g., distance 100 m vs 10 km) and see whether the pre-commit walk read-out rate scales with the surface heuristic, or stays constant—this would separate 'true commitment' from 'generic short-distance default.'
- A second extension: apply the same P0–P5 protocol to other premise-violating trivia questions to see whether the walk-leaning internal default is a general 'first plausible answer' phenomenon rather than a car-wash-specific heuristic.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies answer pre-commitment in Qwen3-8B using a minimal car-wash probe (walk vs. drive, where only drive is correct). It makes three claims. First, behaviorally, the model commits to the wrong answer in 85–100% of sampled rollouts across five system-prompt conditions and in 100% of greedy rollouts, with a 4,096-token thinking budget not repairing the failure. Second, at the activation level, a pretrained, training-free activation oracle reads 'walk' from pre-commit hidden states at rates above a neutral-context baseline (68% vs. 17%), including in rollouts that eventually answer 'drive' (5/6); the authors explicitly frame these results as preliminary. Third, methodologically, the same oracle, activations, and positions succeed or fail depending on question wording, with an open question failing a positive control (2/16) and a closed question passing it (11/16). The paper includes a detailed audit trail for the scoring pipeline, a reproducible code/data package, and a limitations section that candidly discloses small samples, non-random selection, single-layer probing, and judge/target base-model sharing.
Significance. If the behavioral result holds, it is a useful and robust addition to the literature on unfaithful chain-of-thought: it demonstrates a near-deterministic answer-first failure on a minimal, premise-critical task in an open-weight model, with careful scoring validation and a disclosed truncation artifact that strengthens confidence in the 85–100% rates. The methodological finding—that per-wording positive controls are necessary for activation-oracle studies—is a genuinely valuable, quantified caution for the field. The activation-level evidence is appropriately labeled preliminary and, as currently controlled, is not yet conclusive; the central comparison against a non-task-matched neutral baseline is the main obstacle. The paper's strengths are its honesty, reproducibility (scripts and data committed, judge pre-registration, manual gates, cluster-aware analyses where used), and the clear separation of established prior work from the incremental claims.
major comments (3)
- [§3.2–3.4] The neutral-context baseline is not task-matched. The oracle's 17% 'walk' default is measured on 8 unrelated prompts (philosophy, arithmetic, recipes, code), while the target probes are on car-wash deliberation text. Given the paper's own demonstration that the oracle is extremely wording-sensitive (§3.1: 2/16 vs. 11/16 with fixed activations) and tracks single-word lexical fields (§3.4), the 17% baseline is not a valid counterfactual for the car-wash probes. The headline 68% vs. 17% comparison may reflect the oracle's reaction to the task domain or question frame rather than the model's commitment state. This is load-bearing for the activation claim. The authors should add a task-matched control: e.g., activations from the same car-wash prompt under a condition where the model does not commit (a neutral continuation, an 'I don't know' response, or a non-decision control question), probe
- [§3.3 and Limitations] The drive-committing rollout claim rests on n=6, with deterministic but non-random selection and 4 of 10 drive rollouts excluded by the regex localizability rule. The P4 pre-commit anchor itself is defined by the rejected regex scorer's span, used only where it agreed with the judge; however, the rejected scorer's blind spots are inherited in the selection. The 5/6 P4 and 6/6 walk-majority results are prominent in the abstract and Section 3.3, but the selection and anchoring issues are load-bearing because the drive group is the sharpest evidence against a text-recovery artifact. Please provide a sensitivity analysis: hand-localized spans for all 10 drive rollouts, or a robustness check across alternative anchor definitions, and report the drive-group results under that analysis. The current sample is too small and too filtered to support the strength of the claim as stated.
- [§3.3, Table 3] The p-values reported in Table 3 (p=.005, p=.005, p=.0007) are one-sided Fisher exact tests comparing per-rollout P4 read-outs to a per-probe baseline of 23 probes from 8 prompts. The baseline itself is clustered (multiple probes per prompt), and the paper elsewhere correctly moves to cluster-aware majority comparisons (§3.4), but Table 3 does not. The apparent precision of p=.0007 is overstated when baseline probes are clustered and the baseline is not task-matched. Please report a cluster-aware test or a bootstrap over prompts, and present the baseline as a distribution of prompt-level walk shares, not simply 4/23.
minor comments (5)
- [§4] Typo: 'Karvonen et al. (2025)’ report' should read 'Karvonen et al.'s (2025) report'.
- [Appendix A] The sentence '1 false positive, 43/44 field errors were missed commitments' is confusing: the holdout is 32 cases, so '44 field errors' needs clarification (e.g., are these per-lens errors? total error instances across the 4 lenses?). Please rephrase with exact denominators.
- [§2.1] STAR is not expanded at first use. Please define it (e.g., 'Situation, Task, Action, Result') or describe the format, since Condition C is a central result.
- [§3.4] The ±25-token lexical stratification is described as one operationalization of 'local'; the paper should briefly justify why 25 tokens and not, say, the full preceding deliberation context, especially since the oracle is reported to be sensitive to single-word fields.
- [Abstract and §3.3] The phrase 'notably, rollouts that eventually answer drive also read as walk-leaning before commitment (5/6)' appears in the abstract despite the paper's own caveat that the drive-group result rests on n=6 and preliminary selection. Consider moving this specific number to the body or adding a stronger caveat in the abstract.
Circularity Check
No circularity: behavioral rates and oracle read-outs are independent empirical measurements; the closest concerns are external validity, not constructional circularity.
full rationale
I walked the paper's derivation chain. The behavioral claim (85–100% wrong-commitment rates) is an empirical measurement scored by an LLM judge that was validated against human labels (96.9% synthetic agreement, 20/20 manual gate); the judge is not defined in terms of the target outcome. The activation-level claim uses a pretrained, training-free oracle (Karvonen et al., 2025) with no task-specific probe training, and the read-outs are compared against a neutral-context baseline; neither the oracle nor the baseline is fitted to the target rollouts, so the 68%-vs-17% comparison is not forced by construction. The paper explicitly frames Section 3 as preliminary, discloses small samples, clustering, non-random rollout selection, single-layer probing, and the judge/target base-model overlap. The methodological finding (question wording moves a positive control from 2/16 to 11/16) is an empirical observation, not a self-verifying claim. There are no self-citations by the author, no imported uniqueness theorem, and no ansatz smuggled in via citation. The most load-bearing assumption—that the 17% neutral baseline is a valid counterfactual for car-wash pre-commit probes—is a threat to external validity, not circularity: the oracle's outputs are not defined in terms of the target labels. The paper's own stratification (Sections 3.2–3.4) attempts to address lexical leakage, and the remaining limitations are stated honestly. No step reduces to its own inputs by definition.
Axiom & Free-Parameter Ledger
axioms (4)
- domain assumption The car-wash probe has a single correct answer: 'drive' (the car must be at the car wash).
- domain assumption The public Qwen3-8B activation oracle (Karvonen et al., 2025) produces valid natural-language read-outs of the model's internal state for the question used.
- domain assumption Teacher-forced prefill activations over the generated sequence match generation-time hidden states.
- domain assumption The Qwen3-8B LLM judge's labels approximate human judgments of 'committed wrong'.
read the original abstract
Chat models sometimes commit to an answer and then produce reasoning that justifies it rather than deriving it -- even when the answer contradicts a task premise. We study a minimal probe: "I want to wash my car. The car wash is 100 meters away. Should I walk or drive?" Only drive works (the car must be at the car wash), yet models overwhelmingly recommend walking. (1) Behavioral reproduction: on Qwen3-8B across five system-prompt conditions (210 rollouts), the wrong commitment occurs in 85-100% of sampled rollouts per condition and 100% of greedy rollouts, in both thinking and non-thinking modes; a 4,096-token thinking budget does not repair it. (2) Preliminary activation-level evidence: probing hidden states with a pretrained, training-free activation oracle (no task-specific probe training) at positions before the answer text is emitted, "walk" read-outs exceed a neutral-context baseline (68% vs. 17%; walk-committing rollouts p=.005, drive-committing rollouts p=.005, Fisher exact) -- notably, rollouts that eventually answer drive also read as walk-leaning before commitment (5/6). The oracle's default on unrelated content is "drive" (83%), so the read-outs are not lexical bias; stratifying by literal walk/drive occurrence shows they are not text recovery either (spans containing "drive" still read out walk; in balanced lexical fields, per-rollout walk-majorities beat a per-prompt neutral baseline 15/22 vs. 1/8, p=.01; drive-committing rollouts 6/6, p=.002). Samples are small and the within-rollout positional gradient is not significant (p=.34); we frame these results as preliminary. (3) Methodological: with fixed oracle, activations, and positions, question wording alone moves a positive control from 2/16 (open question) to 11/16 (closed); negative oracle results are uninterpretable without per-wording positive controls.
Reference graph
Works this paper leans on
-
[1]
(2025).Circuit Tracing: Revealing Computational Graphs in Language Models.Transformer Circuits Thread
Ameisen, E., Lindsey, J., Pearce, A., et al. (2025).Circuit Tracing: Revealing Computational Graphs in Language Models.Transformer Circuits Thread. https://transformer-circuits.pub/2025/ attribution-graphs/methods.html
2025
-
[2]
Boppana, S., Ma, A., Loeffler, M., Sarfati, R., Bigelow, E., Geiger, A., Lewis, O., Merullo, J. (2026). Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought.arXiv:2603.05488
Pith/arXiv arXiv 2026
-
[3]
(2025).Reasoning Models Don’t Always Say What They Think.arXiv:2505.05410
Chen, Y., Benton, J., Radhakrishnan, A., Uesato, J., Denison, C., et al. (2025).Reasoning Models Don’t Always Say What They Think.arXiv:2505.05410
Pith/arXiv arXiv 2025
-
[4]
Karvonen, A., Chua, J., Dumas, C., Fraser-Taliente, K., Kantamneni, S., Minder, J., Ong, E., Sen Sharma, A., Wen, D., Evans, O., Marks, S. (2025).Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers.arXiv:2512.15674
arXiv 2025
-
[5]
Cox, K., Kianersi, D., Garriga-Alonso, A. (2026).Decoding Answers Before Chain-of-Thought: Evidence from Pre-CoT Probes and Activation Steering.arXiv:2603.01437. 8
Pith/arXiv arXiv 2026
-
[6]
Esakkiraja, E., Rajeswar, S., Akhiyarov, D., Venkatesaramani, R. (2026).Therefore I am. I Think. arXiv:2604.01202
Pith/arXiv arXiv 2026
-
[7]
(2023).Measuring Faithfulness in Chain-of-Thought Reasoning.arXiv:2307.13702
Lanham, T., Chen, A., Radhakrishnan, A., et al. (2023).Measuring Faithfulness in Chain-of-Thought Reasoning.arXiv:2307.13702
Pith/arXiv arXiv 2023
-
[8]
(2025).On the Biology of a Large Language Model.Transformer Cir- cuits Thread.https://transformer-circuits.pub/2025/attribution-graphs/biology.html
Lindsey, J., Gurnee, W., Ameisen, E., et al. (2025).On the Biology of a Large Language Model.Transformer Cir- cuits Thread.https://transformer-circuits.pub/2025/attribution-graphs/biology.html
2025
-
[9]
Mirtaheri, P., Belkin, M. (2026).Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probing.arXiv:2603.17199
arXiv 2026
-
[10]
(2024).LatentQA: Teaching LLMs to Decode Activations Into Natural Language.arXiv:2412.08686
Pan, A., Chen, L., Steinhardt, J. (2024).LatentQA: Teaching LLMs to Decode Activations Into Natural Language.arXiv:2412.08686
arXiv 2024
-
[11]
Scalena, D., Candussio, S., Bortolussi, L., Fersini, E., Nissim, M., Sarti, G. (2026).Beyond the Commitment Boundary: Probing Epiphenomenal Chain-of-Thought in Large Reasoning Models.arXiv:2606.13603
Pith/arXiv arXiv 2026
-
[12]
Turpin, M., Michael, J., Perez, E., Bowman, S. R. (2023).Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting.arXiv:2305.04388
Pith/arXiv arXiv 2023
-
[13]
(2025).Qwen3 Technical Report.arXiv:2505.09388
Yang, A., Li, A., Yang, B., et al. (2025).Qwen3 Technical Report.arXiv:2505.09388. 9
Pith/arXiv arXiv 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.