Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation

Carlo Di Cicco

REVIEW 2 major objections 5 minor 10 references

Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation

T0 review · 2 major / 5 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Code correctness is linearly decodable from LLM hidden states before generation

desk verdict Solid, honestly-reported single-model probe result; the main claim survives its own confound control, but the paper needs to fix the abstract/body number mismatch and test difficulty proxies beyond prompt length. read the letter →

arxiv 2606.14530 v3 pith:MLOUKLCO submitted 2026-06-12 cs.LG

Carlo Di Cicco This is my paper

classification cs.LG

keywords linearprobinghiddenstatescodecorrectnesspre-generationresidualizationconfoundcontrolLiveBenchrepresentationengineering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether the correctness of code a model has not yet written is already legible in its internal representation of the prompt alone. On 444 LiveCodeBench tasks with Qwen3-4B-Instruct-2507, a linear probe trained on the hidden state at the final prompt token predicts whether the first-attempt code will pass unit tests with a held-out AUC of 0.931. The signal survives removing the linear effect of prompt length, dropping only slightly to 0.911 AUC, while a prompt-length-only baseline reaches 0.754. The paper also investigates whether the hidden-state shift from a failed attempt to a repair attempt carries a geometric signature of repair success, and finds that the apparent signal is explained away by repair-context covariates. The contribution is a clean empirical demonstration that correctness-relevant information exists before generation, plus a residualization diagnostic that supports that result while overturning the repair-direction claim.

What carries the argument

The central object is the prompt-final hidden state vector from layer 29 of Qwen3-4B-Instruct-2507, a 2560-dimensional representation captured before generation begins. The key technique is linear probing with a three-step pipeline (standard scaling, PCA retaining 95% variance, L2-regularized logistic regression) evaluated with nested cross-validation, and the key confound-control tool is per-dimension residualization: regressing each hidden dimension on prompt length and subtracting the fit. For the repair analysis, the machinery is a contrastive direction v = mean(success deltas) − mean(failure deltas) in hidden-state difference space, tested by magnitude and split-half cosine against labe

What would settle it

A direct test would re-run the same probing pipeline on a dataset where prompt length is decorrelated from difficulty, or where task difficulty is controlled by matching tasks on all observable features except the pass/fail label. If the residualized AUC drops to near the prompt-only baseline under such matching, the purported correctness signal would be revealed as a difficulty artifact.

Watch

Extended reading notes

Core claim

The central claim is that the hidden state of a large language model at the last prompt token, captured on a prompt-only forward pass before any output token is generated, linearly decodes the eventual pass/fail status of the model's first-attempt code. With nested cross-validation for layer and regularization selection, the probe achieves a leakage-free outer-test AUC of 0.931±0.008 across 50 random splits. After residualizing each of the 2560 hidden dimensions against prompt length, the probe still reaches 0.911±0.010, well above the prompt-length-only baseline of 0.754±0.014, and nonlinear baselines on prompt length do not close the gap. The companion claim about a repair-success directio

Load-bearing premise

The pre-generation probe claims a correctness-specific signal, but it controls only for prompt length; if other properties of task difficulty are correlated with hidden states and with pass/fail, the residualized probe may still be reading difficulty rather than correctness.

Editorial extensions

If this is right

If pre-generation hidden states encode correctness, future decoding-time interventions could steer or detect correct reasoning before sampling begins, without needing to generate or execute code.
The residualization diagnostic provides a reusable template for separating genuine internal signals from confounded covariates in any probe study, and the paper shows it can support one claim while overturning another.
The prompt-length baseline and its nonlinear variants establish a concrete floor: any correctness probe on this data must clear roughly 0.75 AUC in held-out prediction to claim signal beyond input-length artifacts.
The repair-direction result warns that contrastive directions in activation space are not self-interpreting; they must be checked against covariates that differ between groups before being named as features.
The per-attempt pass-rate collapse (15.8% at attempt 0, 36.4% at attempt 1, then single digits) suggests that on this benchmark the model gains nearly all its repair benefit from the first feedback loop, and later attempts provide too few successes for trajectory-level analysis.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper controls for only one confound in the pre-generation probe, prompt length. Other task-difficulty correlates such as problem category, output structure, or token distribution may also be entangled with hidden states and pass/fail, so the residualized AUC of 0.911 should be read as a lower bound on a correctness-specific signal only under the assumption that prompt length is the dominant d
The layer landscape peaking at the shallowest captured layer (29) rather than deeper suggests that the correctness signal may be present even earlier in the network; a lower-layer sweep on models with more accessible layers would clarify where the information first appears.
The method can be extended to a testable hypothesis: if the pre-generation signal is genuinely about correctness rather than difficulty, the residualized probe trained on one benchmark should transfer to other code benchmarks with matched difficulty distributions; if transfer fails, the signal is likely benchmark-specific.
The repair-direction null result may be read as evidence that the model's repair ability is driven by explicit textual error feedback rather than by an internal self-evaluation mechanism, a distinction that could be tested by ablating parts of the repair prompt (error output versus fix instruction) and measuring whether the contrastive direction reappears.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the take: the central claim in the full text—that code correctness is linearly decodable from the prompt-final hidden state before any generation—is well-supported by the experimental design. Nested CV prevents layer-selection leakage, the prompt-length residualization is honestly motivated, and the nonlinear prompt-only baselines rule out a linear-model artifact. The repair-direction negative result is a genuine credit: the author applies the same residualization tool, shows it collapses the raw direction, and reads that as "correlate, not isolated feature." That is the right discipline. The work is upstream of the previous probing papers [2,3], which read hidden states of generated code.

The soft spots are real but not fatal. The abstract and body report different headline AUCs (0.881 vs 0.931 raw, 0.842 vs 0.911 residualized, 0.657 vs 0.754 baseline). That is a serious trust issue for a reader trying to verify claims; it looks like version drift, but it has to be fixed. The title in the arXiv metadata also differs from the full-text title. Second, the main probe controls only prompt length. The paper notes difficulty correlates with length, but difficulty is not reducible to length. The hidden state at the final prompt token can encode problem category, expected output structure, difficulty tier—all of which predict pass/fail. Without a TF-IDF/prompt-content baseline or residualization on difficulty tier, the "correctness" decoding is partially confounded with difficulty. The 0.911 vs 0.754 gap is suggestive, but the claim as stated overreaches. This is the weakest point.

The repair analysis's data-cleaning filters are reasonable (repetition-loop tasks, truncated generations), and the covariate-balance check before residualization is a nice methodological addition. The conclusion that the direction is mediated by repair context is well argued. The limitations section is honest, including the note that the probe layer peaks at the lower edge of captured range.

Who is this for? Probing researchers, interpretability-for-code people, and anyone building early-exit or compute-saving systems. The math and data handling look clean; no circularity; citations are appropriate. A serious referee should see it. Recommendation: accept for peer review, with the requirement that the abstract/body numbers be reconciled and ideally one or two content-based confound baselines be added.

Referee Report

2 major / 5 minor

Summary. The paper investigates whether the hidden state at the final prompt token, captured before any output token is generated, linearly decodes whether Qwen3-4B-Instruct-2507's first-attempt code will pass unit tests. On 444 LiveCodeBench tasks, it reports a nested-CV, leakage-free probe AUC of 0.931±0.008 (raw), 0.911±0.010 after per-dimension OLS residualization against prompt length, and 0.754±0.014 for a prompt-length-only logistic baseline. It also asks whether self-repair leaves a geometric signature: a contrastive direction between failed and repaired attempts is significant in raw deltas but loses significance after residualizing against repair-context covariates. The paper concludes that pre-generation hidden states contain a robust correctness signal and that its residualization diagnostic can either confirm or overturn a geometric interpretation.

Significance. If the pre-generation result survives stronger confound controls, it is a valuable empirical and methodological contribution: it suggests that correctness-relevant information exists before sampling, and it demonstrates a careful nested-CV probing pipeline with explicit leakage controls and a negative result for the repair direction. The code/data release and the honest reporting of null results are strengths. However, the central claim's current support is weakened by an incomplete confound analysis and a numerical mismatch between the abstract and the full text.

major comments (2)

[§3.1, §3.3, Table 3] The residualized configuration removes only the linear effect of prompt_tokens from each hidden-state dimension. The paper uses the gap between the residualized probe (0.911) and the prompt-only baseline (0.754) to conclude that hidden states carry a correctness signal not reducible to prompt length. But the hidden state at the final prompt token can also encode task difficulty (e.g., LiveCodeBench difficulty tier, problem type, expected output structure), and pass/fail is strongly difficulty-dependent. Since the paper itself notes that LiveCodeBench difficulty correlates with prompt length, linear residualization on length is not sufficient to isolate correctness. To support the central claim, the authors should control for difficulty (e.g., residualize against difficulty tier, report within-tier AUCs) or add a prompt-content baseline (e.g., TF-IDF or prompt-embedding features). Without
[Abstract vs. §3.3, Table 3] The abstract reports raw AUC 0.881±0.008, residualized 0.842±0.010, and prompt-length baseline 0.657±0.014. The full text (Table 3) reports 0.931±0.008, 0.911±0.010, and 0.754±0.014. These are materially different and the paper does not explain which set is authoritative. Because these numbers are the core quantitative claim of the paper, the discrepancy must be resolved before publication — either by correcting the abstract or by explicitly reporting the source of each set of numbers.

minor comments (5)

[Abstract] The phrase 'none of the nonlinear models tested improves upon it' is ambiguous. The nonlinear models are prompt-only baselines, not nonlinear probes on hidden states; please clarify to avoid implying a comparison with nonlinear hidden-state probes.
[§1] There are several missing-space typos in the introduction, e.g., 'whethera', 'alsowhat', and 'internallyas'. A copyedit pass would help.
[§4.2, Tables 6–7] The repair-direction residualization includes two nearly collinear continuous covariates (∆prompt_tokens and code_length_attempt_0, r=0.995). The paper gives an in-sample R2 argument that the redundancy is harmless, but a more direct check would be to run the residualization with each covariate set separately and show the qualitative collapse is unchanged. This is not central to the pre-generation claim, but it would strengthen the methodological contribution.
[§5] The limitation about the captured layer range is honest. To support the claim that the repair-direction verdict is stable across layers, consider reporting the layer-30/35 results in a small table or appendix rather than only in prose.
[§2.1/§3.3] The paper does not report the distribution of prompt_tokens. Since the prompt-only baseline is a single scalar feature, a sentence on its range or variance would help calibrate the AUC values.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the probe results are held-out empirical measurements, not derivations from fitted targets.

full rationale

I walked the paper's derivation chain systematically. The central claim is that a linear probe on the hidden state at the final prompt token, captured before generation, decodes first-attempt pass/fail on LiveCodeBench. The labels come from unit-test outcomes after generation; the features are captured before generation; the probe is evaluated on untouched outer-test folds with nested-CV layer and regularization selection. No parameter is fitted to the test labels, and no predictive quantity is constructed from the target it is said to predict. The residualized configuration regresses each hidden-state dimension on prompt_tokens using only the training fold and subtracts that fit from both train and test; this is a legitimate confound adjustment, not a circular step. The prompt-only baseline is a comparison, not the source of the probe's signal. The repair-direction analysis is also self-contained: the raw contrastive direction is tested against label-shuffled nulls, and the conditional residualization uses three observable covariates that are checked to differ between groups. The paper even reports a symmetric negative result when the same residualization tool overturns the repair-direction interpretation. There is no load-bearing self-citation: references [1], [3], and [4] are methodological or prior-work citations, none by the present author, and none is invoked as a uniqueness theorem that forces the paper's choices. The main weakness—that only prompt length is controlled and that other difficulty-related prompt properties could drive the residualized probe's AUC—is a validity/confounding concern, not a circularity. Likewise, the abstract/body AUC discrepancy is an inconsistency, not a circular reduction. Under the stated rules, no quoted equation or fitted parameter reduces to its own input, so the honest finding is no significant circularity, score 0.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new entities are postulated. The main free parameters are probe hyperparameters, but they are selected via nested CV so they do not inflate the reported AUC. The residualization regressions are auxiliary. The axioms are standard for a probing study, with the main fragility being the single-confound control.

free parameters (4)

Probe regularization strength C = 0.01 (modal across 50 outer splits)
Regularization strength selected by 5-fold CV inside training folds. Reported AUC depends on selection but is leakage-free due to nested CV.
Probe layer index = 29 (modal selection; 25 of 50 splits)
Layer hyperparameter selected by nested CV; AUC landscape peaks at layer 29-30.
PCA variance threshold = 0.95 (fixed)
PCA retaining 95% cumulative variance, fit on training fold; standard choice, not tuned.
Per-dimension OLS intercept and slope for residualization = 2560 pairs (one per hidden-state dimension)
Each hidden-state dimension is regressed on prompt_tokens; linear effect subtracted. These are fit on each train fold, but they are auxiliary to the confound removal and cannot create signal.

assumptions (5)

domain assumption LiveCodeBench unit tests provide ground truth for code correctness.
Used to label pass/fail for all probes and repairs.
domain assumption The hidden state at the final prompt token is a fixed-dimension vector and a valid substrate for probing.
The analysis captures layer-29 hidden state at the final prompt token; the paper assumes this represents the model's prompt comprehension.
standard math Linear separability (via logistic regression) is a meaningful measure of decodability.
Standard assumption in probing literature; the paper cites [1].
domain assumption Residualizing each hidden state dimension against prompt length removes the linear confound.
The residualization procedure (OLS on each dimension) is assumed to adequately model the linear effect of prompt length; no test of nonlinear residuals is reported.
domain assumption The model's generation is reproducible via the provided code (sampling parameters are fixed).
Sampling temperature/seed not reported in text; reproducibility relies on the code.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation." pith.science (2026). https://pith.science/paper/MLOUKLCO

@misc{pith2026260614530,
  author       = {Pith},
  title        = {Pith review of: Code Correctness Is Linearly Decodable from LLM Hidden States Before Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MLOUKLCO}},
  note         = {Machine review of arXiv:2606.14530}
}

read the original abstract

Large language models encode rich information in their hidden states. This work asks whether the correctness of code that Qwen3-4B-Instruct-2507 has not yet generated is already legible in its hidden states, evaluated on a set of 444 tasks from LiveCodeBench. The correctness of the model's first-attempt code is linearly decodable from the hidden state at the final prompt token, captured before any output token is generated, with a leakage-free held-out AUC of 0.881 +/- 0.008 across 50 outer splits. To assess whether this signal is explained by prompt length, each hidden state dimension is residualized with respect to its linear effect. The probe still achieves an AUC of 0.842 +/- 0.010, substantially above a logistic prompt-length baseline of 0.657 +/- 0.014, and none of the nonlinear models tested improves upon it. A companion question about whether self-repair leaves a geometric signature in the model's hidden states could not be answered, because successful repairs following a failed first attempt are too rare in this setting to support the analysis. The contribution is both empirical and methodological, providing evidence that pre-generation hidden states contain a robust signal of eventual code correctness, together with a confound-control diagnostic that quantifies how much of that signal survives adjustment for prompt length.

Review history (2 revisions) →

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

10 extracted references · 6 linked inside Pith

[1]

Understanding intermediate layers using linear classifier probes, 2016

Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2016. arXiv:1610.01644

arXiv 2016
[2]

Cor- rectness assessment of code generated by large language models using internal representations,

Tuan-Dung Bui, Thanh Trong Vu, Thu-Trang Nguyen, Son Nguyen, and Hieu Dinh Vo. Cor- rectness assessment of code generated by large language models using internal representations,
[3]

On LLMs’ internal repre- sentation of code correctness, 2025

Francisco Ribeiro, Claudio Spiess, Prem Devanbu, and Sarah Nadi. On LLMs’ internal repre- sentation of code correctness, 2025. arXiv:2512.07404

arXiv 2025
[4]

Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down ap- proach to...

arXiv 2023
[5]

How many tries does it take? iterative self-repair in LLM code generation across model scales and benchmarks, 2026

Johin Johny Arimbur. How many tries does it take? iterative self-repair in LLM code generation across model scales and benchmarks, 2026. arXiv:2604.10508

arXiv 2026
[6]

Qwen3 technical report, 2025

Qwen Team. Qwen3 technical report, 2025. arXiv:2505.09388

arXiv 2025
[7]

LiveCodeBench: Holistic and contamination free evaluation of large language models for code, 2024

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code, 2024. arXiv:2403.07974

arXiv 2024
[8]

Pedregosa, G

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pret- tenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Per- rot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learning Research, 12:2825–2830, 2011

2011

Show all 10 references

[9]

Cawley and Nicola L

Gavin C. Cawley and Nicola L. C. Talbot. On over-fitting in model selection and subsequent selection bias in performance evaluation.Journal of Machine Learning Research, 11:2079–2107, 2010. 12
[2025]

arXiv:2501.12934. 11

arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Understanding intermediate layers using linear classifier probes, 2016

Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2016. arXiv:1610.01644

arXiv 2016

[2] [2]

Cor- rectness assessment of code generated by large language models using internal representations,

Tuan-Dung Bui, Thanh Trong Vu, Thu-Trang Nguyen, Son Nguyen, and Hieu Dinh Vo. Cor- rectness assessment of code generated by large language models using internal representations,

[3] [3]

On LLMs’ internal repre- sentation of code correctness, 2025

Francisco Ribeiro, Claudio Spiess, Prem Devanbu, and Sarah Nadi. On LLMs’ internal repre- sentation of code correctness, 2025. arXiv:2512.07404

arXiv 2025

[4] [4]

Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J

Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down ap- proach to...

arXiv 2023

[5] [5]

How many tries does it take? iterative self-repair in LLM code generation across model scales and benchmarks, 2026

Johin Johny Arimbur. How many tries does it take? iterative self-repair in LLM code generation across model scales and benchmarks, 2026. arXiv:2604.10508

arXiv 2026

[6] [6]

Qwen3 technical report, 2025

Qwen Team. Qwen3 technical report, 2025. arXiv:2505.09388

arXiv 2025

[7] [7]

LiveCodeBench: Holistic and contamination free evaluation of large language models for code, 2024

Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code, 2024. arXiv:2403.07974

arXiv 2024

[8] [8]

Pedregosa, G

F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pret- tenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Per- rot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learning Research, 12:2825–2830, 2011

2011