Pith. sign in

REVIEW 1 major objections 5 minor 25 references

Latent Programming Horizons in Coding Agents

T0 review · 1 major / 5 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Coding agents internally track program correctness up to 25 steps ahead

desk verdict The current-state probing is solid and new; the 25-step lookahead claim is confounded by label persistence and needs an easy fix. read the letter →

arxiv 2607.05188 v1 pith:MTSRJSUC submitted 2026-07-06 cs.LG cs.SE

André Silva , Han Tu , Martin Monperrus This is my paper
classification cs.LGcs.SE
keywords codingagentagentsacrossbenchmarkscodeeditslanguage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper shows that when a language model acts as a coding agent — iteratively reading, editing, and testing code across dozens of steps — its internal hidden states carry a linearly decodable representation of the program it is working on. A simple logistic regression probe on those hidden states can predict whether the current code compiles, passes its test suite, reduces failing tests, or introduces regressions, reaching AUC up to 0.83. The authors then show something they find more surprising: probes trained to predict the outcome of edits that have not yet been written still perform above chance up to roughly 25 agent steps in advance. They call this the agent's latent programming horizon — the idea that the model's internal state already contains information about the shape of future programs well before those changes are committed to disk. The probes transfer across two different software-engineering benchmarks without retraining, suggesting the encoded signal is a general property of how these models represent code, not an artifact of one task distribution. The experiments cover two open-weight models and over 22,000 trajectories on real-world codebases.

What carries the argument

The method is straightforward probing: collect the residual stream vector at each agent step, train a logistic regression classifier to predict a binary program property (e.g., does the code pass its test suite), and measure AUC. For the horizon experiments, the probe at step t is trained to predict the property label at step t+k, where k ranges from 0 to 50. Labels are computed by actually checking out each program version at each edit event and running the test suite. The shuffled-label control confirms the signal lives in the representation, not in the probe's capacity to fit arbitrary labels.

What would settle it

If one conditioned the horizon analysis only on (t, t+k) pairs where at least one edit occurs between t and t+k, and the above-chance performance collapsed, the latent programming horizon claim would be weakened to a current-state encoding claim.

Watch

Extended reading notes

Core claim

The central object the paper introduces is the latent program representation: the internal encoding, in a transformer's residual stream, of the semantic state of the program being edited. The authors show this representation exists, is linearly decodable, is strongest in intermediate layers, and extends forward in time — the latent programming horizon — meaning the hidden state at step t carries information about program properties at step t+k that the agent has not yet materialized through edits. The paper demonstrates this by training separate linear probes for each of four program properties (well-formedness, full correctness, partial correctness, regression) at each of several lookahead,

Load-bearing premise

The lookahead probes predict the label of the program k steps ahead, but if the agent makes no edit in those k steps, the label is carried forward unchanged from the last edit. This means the probe may be predicting the current state rather than a genuinely future state, and the paper does not report how often this label-carry-forward occurs, so the 25-step horizon could be partly inflated by periods of no change.

Editorial extensions

If this is right

  • If the latent programming horizon is real, one could build lightweight monitors that read hidden states to predict whether an agent's upcoming edits will succeed or introduce regressions, before those edits are made — enabling early intervention or course correction.
  • The finding that program properties are most decodable in intermediate layers suggests that mechanistic interpretability work on coding agents should focus probing and steering efforts on mid-depth representations rather than final-layer outputs.
  • Cross-benchmark transfer of probes implies a shared internal vocabulary for program state across different codebases and task types, which could support model-agnostic monitoring tools.
  • If the 25-step horizon reflects genuine internal planning rather than label persistence, it would suggest that coding agents develop something analogous to the look-ahead representations found in game-playing networks, opening questions about how training objectives (next-token prediction, RL from test feedback) cultivate this forward representation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. This paper studies whether the residual streams of language models acting as coding agents linearly encode properties of the evolving program (parsability, correctness, partial correctness, regressions). The authors collect 22,714 agentic trajectories from two models (Laguna-XS.2, Qwen3.6-35B-A3B) on two benchmarks (SWE-Bench-Verified, SWE-Bench-Pro), extract hidden states, and train logistic regression probes. They find that current program properties are decodable with AUC up to 0.83, that the signal peaks in intermediate layers, and that probes transfer across benchmarks without retraining. The paper's most novel claim is the 'latent programming horizon': probes trained to predict program properties k steps ahead remain above chance up to k=25 steps.

Significance. The current-state probing results (Section 4.1) are well-supported and represent a solid contribution to the interpretability of coding agents, extending single-function correctness probing to multi-step agentic editing on real codebases. The experimental design is rigorous: the shuffled-label controls collapse to 0.50, the task-split protocol prevents leakage, and the cross-dataset transfer experiment provides genuine external validity. The inverted-U layer pattern is consistent across models and properties. However, the paper's most novel finding—the 25-step lookahead horizon—rests on a label persistence confound that is not addressed in the current text, weakening the central novelty claim.

major comments (1)
  1. §3.2 and §4.2: The lookahead claim is confounded by label persistence. Section 3.2 defines labels as carried forward from the most recent edit: 'every generated token position t with e_i ≤ t < e_{i+1} receives the label of edit e_i.' For the lookahead probe at horizon k, the target is y^φ_{σ(t)+k}. If no edit occurs in the window [σ(t), σ(t)+k], then y^φ_{σ(t)+k} = y^φ_{σ(t)}, and the probe is effectively doing current-state prediction—which Section 4.1 already shows works at AUC ~0.77. Table 1 shows a median of only 2 edits per trajectory across trajectories with a median of 52–56 steps, meaning the average inter-edit gap is roughly 17–26 steps. For k up to 25, a substantial fraction of (t, t+k) pairs will have zero intervening edits. The observed smooth decay from AUC ~0.77 at k=0 to ~0.55 at k=25 is consistent with a mixture where the no-edit fraction (performing at k=0 levels) is逐渐d,
minor comments (5)
  1. §3.4: The section title reads 'Probing Program Poperties' — typo for 'Properties'.
  2. Figure 6: The y-axis ranges differ across panels (e.g., 0.50–0.75 for Verified vs. 0.50–0.85 for Pro), making visual comparison of decay rates across benchmarks slightly misleading. Consider using a consistent range.
  3. Table 1: The 'Med. edits' column shows a median of 2 for Laguna-Verified, but the text in §3.6 says 'a median number of two and a mean of 3.5 edits per trajectory.' It would help to include the mean in the table for completeness.
  4. Figure 16 caption: 'shown at layer 20 (best or near-best by probe validation AUC for both models)' — but probed layers are at indices 1, 11, 21, 31, 40 (§3.4). Layer 20 was not probed. Please clarify or correct.
  5. §4.1: The text notes that WELL-FORMEDNESS on SWE-Bench-Verified collapses to near chance 'always below 0.60,' but Table 5 shows layer 31 at 0.571 and layer 21 at 0.566. The phrasing 'always below 0.60' is technically correct but could be more precise about the range.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and the constructive engagement with our work. The referee correctly identifies a genuine confound in our lookahead analysis that we must address. Below we respond point by point.

read point-by-point responses

  1. Referee: §3.2 and §4.2: The lookahead claim is confounded by label persistence. Section 3.2 defines labels as carried forward from the most recent edit: 'every generated token position t with e_i ≤ t < e_{i+1} receives the label of edit e_i.' For the lookahead probe at horizon k, the target is y^φ_{σ(t)+k}. If no edit occurs in the window [σ(t), σ(t)+k], then y^φ_{σ(t)+k} = y^φ_{σ(t)}, and the probe is effectively doing current-state prediction—which Section 4.1 already shows works at AUC ~0.77. Table 1 shows a median of only 2 edits per trajectory across trajectories with a median of 52–56 steps, meaning the average inter-edit gap is roughly 17–26 steps. For k up to 25, a substantial fraction of (t, t+k) pairs will have zero intervening edits. The observed smooth decay from AUC ~0.77 at k=0 to ~0.55 at k=25 is consistent with a mixture where the no-edit fraction (performing at k=0 levels) is逐渐d

    Authors: The referee is correct. This is a genuine confound in our current analysis, and we appreciate the precise way it has been articulated. Because labels are carried forward between edits, a substantial fraction of (t, t+k) pairs at large k will have identical labels at t and t+k, meaning the lookahead probe is effectively doing current-state prediction on those pairs. Given the edit frequency statistics in Table 1 (median 2 edits per trajectory, median 52–56 steps), this fraction is non-trivial for k in the 10–25 range, and the smooth decay we report is indeed consistent with a mixture of no-edit pairs (performing near k=0 levels) and edit pairs (potentially performing near chance). We cannot rule out that the entire above-chance signal at large k is driven by this persistence artifact rather than by genuine lookahead in the hidden state. This is a real weakness in the current manuscript. We will address it in the revision as follows. First, we will add an explicit analysis of the no-edit fraction at each horizon k, reporting the proportion of (t, t+k) pairs where the label is unchanged due to zero intervening edits. Second, and more importantly, we will re-run the lookahead analysis stratified by whether at least one edit occurs in the window [σ(t), σ(t)+k]. If the above-chance signal persists in the edit-occurred stratum at large k, the horizon claim is supported; if it collapses, we will retract the 25-step horizon claim and reframe the contribution accordingly. Third, we will add a baseline that predicts y^φ_{σ(t)+k} using only y^φ_{σ(t)} (i.e., a label-persistence oracle), so that the reader can see exactly how much of the lookahead AUC is attributable to persistence versus genuine future-state encoding. We acknowledge that, as the manuscript currently stands, the 25- revision: no

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: probes are trained on externally computed labels with task-disjoint splits and cross-dataset transfer provides independent grounding

full rationale

The paper trains linear probes on hidden states to predict program properties (parsability, test passage, regression) whose labels are computed externally via parsing and test execution (Section 3.3), not from the hidden states themselves. The train/test split is by task identifier, preventing same-task leakage (Section 3.4). The cross-dataset transfer experiment (Section 4.1, Figure 5) trains probes on one benchmark and evaluates on another without retraining, providing independent validation that the signal is in the representation rather than an artifact of fitting. The shuffled-label control (Section 3.4, Figure 3) confirms probes are not exploiting capacity. The lookahead probes (Section 3.5) predict future labels y^φ_{σ(t)+k} that are not in the current hidden state by construction — they are computed from future edit events — so the prediction target is genuinely external to the input. One author (Silva) appears in a related-work citation (Silva et al., 2025) on gradient-based program repair, but this citation is contextual and not load-bearing for any derivation in the present paper. The label-persistence concern raised by the reader (that labels are carried forward between edits, potentially inflating the lookahead horizon) is a correctness risk about confounding, not a circularity issue — the labels are still computed externally and the probe does not define them. No step in the paper's derivation chain reduces to its inputs by construction.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The free parameters are standard probe hyperparameters selected by search, not ad hoc constants. The axioms are standard probing assumptions plus the label-persistence rule, which is the most fragile. The invented entities are conceptual framings backed by operational measurements.

free parameters (6)
  • Probe learning rate = varies, e.g. 5.19e-4 to 2.95e-2
    Selected by 20-trial random search maximizing validation AUC per probe/layer/model/dataset (Table 3, Table 4).
  • Probe weight decay = varies, e.g. 3.81e-3 to 5.19e-5
    Selected by random search (Table 3, Table 4).
  • Probe batch size = 256, 512, or 1024
    Selected by random search (Table 3, Table 4).
  • Probe patience = 10, 25, or 50
    Selected by random search (Table 3, Table 4).
  • Probed layers = {1, 11, 21, 31, 40}
    Chosen subset of transformer layers to probe, not exhaustive.
  • Hidden state sampling stride = every 5 tokens
    Chosen for storage efficiency (Section 3.6).
assumptions (4)
  • standard math Linear probes are sufficient to detect whether a property is encoded in hidden states.
    Standard probing assumption from Alain & Bengio 2017; the paper uses logistic regression and validates with shuffled-label controls (Section 3.4).
  • domain assumption Program properties (well-formedness, correctness, regression) are binary and well-defined at each edit event.
    Section 3.3 defines four binary labels computed by parsing/compiling and running test suites. This is reasonable but simplifies continuous notions of correctness.
  • standard math The residual stream is the correct locus for probing program representations.
    Following Elhage et al. 2021; the residual stream is the standard probing target (Section 2).
  • domain assumption Labels persist from the last edit until the next edit, so every token position has a label.
    Section 3.2: 'every generated token position t with e_i ≤ t < e_{i+1} receives the label of edit e_i.' This is load-bearing for the horizon claim.
invented entities (2)
  • Latent program representation independent evidence
    purpose: Conceptual framing for what the residual stream encodes about the program.
    The paper provides operational evidence via probe AUC above chance and cross-dataset transfer. The entity is a framing, not a new physical object.
  • Latent programming horizon independent evidence
    purpose: The extent to which future edit outcomes are decodable from current hidden states.
    Measured by lookahead probe AUC above chance up to k=25 (Figure 6). The horizon is a measured quantity, not a postulated mechanism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Latent Programming Horizons in Coding Agents." pith.science (2026). https://pith.science/paper/MTSRJSUC

@misc{pith2026260705188,
  author       = {Pith},
  title        = {Pith review of: Latent Programming Horizons in Coding Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MTSRJSUC}},
  note         = {Machine review of arXiv:2607.05188}
}
read the original abstract

A coding agent solving a software-engineering task spends dozens of steps reasoning, editing code, and running tests, yet little is known about what the underlying language model internally represents about the program it is working on. We show that the residual streams of language models under coding agents linearly encode properties of the evolving program: a logistic-regression probe on hidden states is able to decode whether the current code parses, passes its test suite, reduces the number of failing tests, and introduces regressions, reaching AUC up to 0.83 for correctness across two models and two benchmarks. Our second finding is more surprising: these representations run ahead of the agent's own edits. Probes trained to predict the outcome of future edits (before they are materialized and written on disk) achieve performance above chance up to roughly 25 steps in advance. We call this the agent's latent programming horizon. As a proof of external validity, we show that the probes transfer across benchmarks without retraining. Our positive results open calls for more research in mechanistic interpretability of coding agents.

Figures

Figures reproduced from arXiv: 2607.05188 by the authors.

Figure 1
Figure 1. A coding agent iteratively edits code and runs tests (top); the hidden states at [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Our method to train probes to predict program properties. A residual-stream [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Best-layer AUC for each probe, model, and dataset. Each bar shows the peak AUC [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: AUC-ROC by transformer layer for all four probes at [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Cross-dataset transfer AUC at the best layer for each probe and model. Gray bars [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Lookahead AUC for the two strongest properties on [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Left: distribution of trajectory lengths (in agent steps) per (model, dataset) combi [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Lookahead AUC for all four probes on Laguna-XS.2 (SWE-Bench-Verified), k ≤ 50. Panels show • FULL CORRECTNESS (top left), ▲ WELL-FORMEDNESS (top right), ■ PARTIAL CORRECTNESS (bottom left), and ♦ REGRESSION (bottom right). A probe at token position t predicts the progr…
Figure 9
Figure 9. Figure 9: Lookahead AUC for all four probes on Laguna-XS.2 (SWE-Bench-Verified), k ≤ 15, using the full trajectory sample without the length filter required for k ≤ 50. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p021_9.png]
Figure 10
Figure 10. Figure 10: Lookahead AUC for all four probes on Laguna-XS.2 (SWE-Bench-Pro), k ≤ 50. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 11
Figure 11. Figure 11: Lookahead AUC for all four probes on Laguna-XS.2 (SWE-Bench-Pro), k ≤ 15, using the full trajectory sample without the length filter. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Lookahead AUC for all four probes on Qwen3.6-35B-A3B (SWE-Bench-Verified), k ≤ 50. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Lookahead AUC for all four probes on Qwen3.6-35B-A3B (SWE-Bench-Verified), k ≤ 15, using the full trajectory sample without the length filter. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Lookahead AUC for all four probes on Qwen3.6-35B-A3B (SWE-Bench-Pro), k ≤ 50. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: Lookahead AUC for all four probes on Qwen3.6-35B-A3B (SWE-Bench-Pro), k ≤ 15, using the full trajectory sample without the length filter. Panel layout matches [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: Two example resolved trajectories, hand-selected from a small set of long [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    Laguna M.1/XS.2 Technical Report

    Julien Abadji, Marah Abdin, Connor Adams, Eric Alcaide, Mustafa Altun, Michele Artoni, Junze Bao, Uday Barar, Vassilis Bekiaris, Arkadii Bessonov, et al. Laguna m. 1/xs. 2 technical report.arXiv preprint arXiv:2605.27605,

    work page Pith review arXiv

  2. [2]

    Knowing before saying: Llm representations encode information about chain-of-thought success before completion

    Anum Afzal, Florian Matthes, Gal Chechik, and Yftah Ziser. Knowing before saying: Llm representations encode information about chain-of-thought success before completion. In Findings of the Association for Computational Linguistics: ACL 2025, pp. 12791–12806,

    work page 2025

  3. [3]

    Understanding intermediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In5th International Conference on Learning Representations, ICLR 2017, Workshop Track Proceedings,

    work page 2017

  4. [4]

    Neural Program Synthesis with a Differentiable Fixer

    URL https://openreview.net/forum?id=HJ4-rAVtl. Matej Balog, Rishabh Singh, Petros Maniatis, and Charles Sutton. Neural program synthesis with a differentiable fixer.arXiv preprint arXiv:2006.10924,

    work page Pith review arXiv 2006

  5. [5]

    Eliciting Latent Predictions from Transformers with the Tuned Lens

    Nora Belrose, Igor Ostrovsky, Lev McKinney, Zach Furman, Logan Smith, Danny Halawi, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens.arXiv preprint arXiv:2303.08112,

    work page Pith review arXiv

  6. [6]

    No answer needed: Predicting LLM answer accuracy from question-only linear probes, 2025

    Iv´an Vicente Moreno Cencerrado, Arnau Padr´es Masdemont, Anton Gonzalvez Hawthorne, David Demitri Africa, and Lorenzo Pacchiardi. No answer needed: Predicting llm answer accuracy from question-only linear probes.arXiv preprint arXiv:2509.10625,

    work page arXiv

  7. [7]

    Cwm: An open-weights llm for research on code generation with world models.arXiv preprint arXiv:2510.02387,

    12 Jade Copet, Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, Emily McMilin, Michel Meyer, Yuxiang Wei, et al. Cwm: An open-weights llm for research on code generation with world models.arXiv preprint arXiv:2510.02387,

    work page arXiv

  8. [9]

    arXiv:2511.14773

    URL https://arxiv.org/abs/2511.14773. Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, et al. Swe-bench pro: Can ai agents solve long-horizon software engineering tasks?arXiv preprint arXiv:2509.16941,

    work page arXiv

Show all 25 references
  1. [10]

    Code correctness signals in llm hidden states: Pre-generation probing and repair geometry.arXiv preprint arXiv:2606.14530,

    Carlo Di Cicco. Code correctness signals in llm hidden states: Pre-generation probing and repair geometry.arXiv preprint arXiv:2606.14530,

  2. [11]

    Toy models of superposition.arXiv preprint arXiv:2209.10652,

    Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, et al. Toy models of superposition.arXiv preprint arXiv:2209.10652,

    arXiv

  3. [12]

    Localized calibrated uncertainty in code language models

    David Gros and Prem Devanbu. Localized calibrated uncertainty in code language models. arXiv preprint arXiv:2512.24560,

  4. [13]

    Codecircuit: Toward inferring llm-generated code correctness via attribution graphs.arXiv preprint arXiv:2602.07080,

    Yicheng He, Zheng Zhao, Zhou Kaiyu, Bryan Dai, Jie Fu, and Yonghui Yang. Codecircuit: Toward inferring llm-generated code correctness via attribution graphs.arXiv preprint arXiv:2602.07080,

  5. [14]

    Designing and interpreting probes with control tasks

    John Hewitt and Percy Liang. Designing and interpreting probes with control tasks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijcnlp), pp. 2733–2743,

    2019

  6. [15]

    Swe-bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pp

    Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? InInternational Conference on Learning Representations, volume 2024, pp. 54107–54157,

    2024

  7. [16]

    Wei Ma, Shangqing Liu, Mengjie Zhao, Xiaofei Xie, Wenhang Wang, Qiang Hu, Jie Zhang, and Yang Liu

    URL https://transformer-circuits.pub/2025/attribution-graphs/biology.html. Wei Ma, Shangqing Liu, Mengjie Zhao, Xiaofei Xie, Wenhang Wang, Qiang Hu, Jie Zhang, and Yang Liu. Unveiling code pre-trained models: Investigating syntax and semantics capacities.ACM Transactions on So...

    2025

  8. [17]

    Neural programmer: Inducing latent programs with gradient descent.arXiv preprint arXiv:1511.04834,

    Arvind Neelakantan, Quoc V Le, and Ilya Sutskever. Neural programmer: Inducing latent programs with gradient descent.arXiv preprint arXiv:1511.04834,

    arXiv

  9. [18]

    Scott Reed and Nando De Freitas

    URL https://qwen.ai/ blog?id=qwen3.6. Scott Reed and Nando De Freitas. Neural programmer-interpreters.arXiv preprint arXiv:1511.06279,

    arXiv

  10. [19]

    On llms’ internal representation of code correctness.arXiv preprint arXiv:2512.07404,

    Francisco Ribeiro, Claudio Spiess, Prem Devanbu, and Sarah Nadi. On llms’ internal representation of code correctness.arXiv preprint arXiv:2512.07404,

  11. [20]

    Gradient-based program repair: Fixing bugs in continuous program spaces.arXiv preprint arXiv:2505.17703,

    14 Andr´e Silva, Gustav Thor´en, and Martin Monperrus. Gradient-based program repair: Fixing bugs in continuous program spaces.arXiv preprint arXiv:2505.17703,

  12. [21]

    Tact: Mitigating overthinking and overacting in coding agents via activation steering.arXiv preprint arXiv:2605.05980,

    Yuan Sui, Yulin Chen, Yibo Li, Xue Jiang, Yufei He, Yihong Dong, Xiaoxin He, Tianyu Gao, and Bryan Hooi. Tact: Mitigating overthinking and overacting in coding agents via activation steering.arXiv preprint arXiv:2605.05980,

    arXiv

  13. [22]

    Mechanistic interpretability of code correctness in llms via sparse autoencoders.arXiv preprint arXiv:2510.02917,

    Kriz Tahimic and Charibeth Cheng. Mechanistic interpretability of code correctness in llms via sparse autoencoders.arXiv preprint arXiv:2510.02917,

  14. [23]

    Planning in a recurrent neural network that plays sokoban.arXiv preprint arXiv:2407.15421,

    Mohammad Taufeeque, Philip Quirke, Maximilian Li, Chris Cundy, Aaron David Tucker, Adam Gleave, and Adri`a Garriga-Alonso. Planning in a recurrent neural network that plays sokoban.arXiv preprint arXiv:2407.15421,

    arXiv

  15. [24]

    Model-agnostic correctness assessment for llm-generated code via dynamic internal representation selection.arXiv preprint arXiv:2510.02934,

    Thanh Trong Vu, Tuan-Dung Bui, Thu-Trang Nguyen, Son Nguyen, and Hieu Dinh Vo. Model-agnostic correctness assessment for llm-generated code via dynamic internal representation selection.arXiv preprint arXiv:2510.02934,

  16. [25]

    Openhands: An open platform for ai soft- ware developers as generalist agents

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: An open platform for ai soft- ware developers as generalist agents. InInternational Conference on Learning Representations, volume ...

    2025

  17. [26]

    per probe and (model, dataset) combination. ▲ WELL-FORMEDNESSis nearly always positive, which suppresses its AUC despite a real signal being present (see Limitations); • FULLCOR- RECTNESSand ■ PARTIALCORRECTNESSare more balanced, consistent with their higher AUC. Table 2: Agen...

    2020

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.