Pith.

Computation and Language · arXiv:2609.38812 v1 · submitted 30 Sep 2026 · 33 references

Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification

Yingfeng Luo, Shaowei Wei, Daixin Wang, Dingyang Lin, Kaiyan Chang, Weiqiao Shan, Tong Zheng, Zhiqiang Zhang

show 2 more authors

Jingbo Zhu, Tong Xiao

The paper says

“Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. Yet how trustworthy such self-verification is remains poorly understood.”

more of the abstract

To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent's subsequent verification and recovery behavior. Applying it to ten terminal agents on TerminalBench2.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61.43% of incorrect candidates are detected and only 49.36% of detected errors are successfully repaired. These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors. Motivated by these findings, we propose Student-Conditioned Verification Distillation (SCVD), which lets the student first produce a candidate solution and distills a stronger teacher's subsequent verification and recovery from the same interaction context. Across three Qwen3.5 backbones, SCVD improves \textscPass@1 on TerminalBench2.1 by 9.74–16.85 percentage points over the corresponding base models and by 4.49–8.61 points over the standard full-trajectory distillation, while avoiding the pronounced out-of-distribution degradation of full-trajectory distillation on SWE-bench Verified.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 2 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. Two-part paper: the SCVD distillation result is well-controlled and credible, while the diagnostic measurement is genuinely new but rests on annotation validation too thin to treat the headline rates as settled. This line is wrong

3 major objections · 4 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 33 references say.

The paper says, at Section B.1, annotation prompt Step 5

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The coding of the verification outcome conflates 'the check produced negative evidence' with 'the agent detected the error'. The prompt instructs judges to label z=error whenever negative evidence is visible 'even when the agent initially fails to notice it', while Table 1(b) defines EDR as 'Does verification detect an incorrect candidate?' and the abstract reports EDR 61.43% as a detection rate.

more of objection 1

Because the same unnoticed-evidence cases are also added to the RSR denominator (Pr(yf=1 | yc=0, z=error)), this coding simultaneously inflates EDR and deflates RSR, the two headline numbers of Finding 2 and Finding 3. The paper should either recode to separate 'evidence exposed' from 'agent noticed and acted on the evidence' or report both variants explicitly as a sensitivity analysis, since the current numbers are not cleanly interpretable as detection and repair success.

The paper says, at Section B.1, annotation validation

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The annotation pipeline is validated against only 20 manually reviewed trajectories total, with per-judge joint accuracies of 76.67%-86.67% and a three-way exact agreement of 74.29% on the candidate-boundary tuple (roughly a quarter of trajectories lack unanimous boundaries, resolved by majority vote or the DeepSeek tie-breaker).

more of objection 2

Since tc determines the replayed yc and every diagnostic rate, judge errors that are correlated with model strength or task type — rather than random noise — could shift the headline EDR, VPR, and RSR values. A larger, stratified human re-annotation (covering all ten agents) with per-model agreement statistics, plus a report of the headline rates under adjudicated labels, is needed before these rates are treated as settled.

The paper says, at Section B.1, Step 3 (verification trigger)

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The verification trigger counts any deliberate cat/head/ls/grep/stat inspection of a produced artifact as verification. Section B.3 defends near-universal VTR against the scaffold-instruction concern (Terminus-2 does not mandate verification), but it does not address the breadth of the coding itself. Under such a broad definition, VTR 99.53% is partly definitional, and the paper's conclusion that 'initiation is not the weakness' would …

more of objection 3

… be more convincing if backed by a stricter coding that requires a check to yield task-relevant evidence about at least one final requirement, with the resulting VTR reported as a sensitivity value.

4 minor objections

The paper says, at Table 3

Pith says: minor objection 1 of 4

The three non-teacher prefix conditions overlap within their standard deviations (31.46±4.21, 31.83±5.05, 33.33±1.40), so the '4.12-point' advantage of the self-prefix over the teacher-prefix is not established with three runs; either add runs or frame this contrast as suggestive rather than decisive.

The paper says, at Section 4.3

Pith says: minor objection 2 of 4

SCVD improving initial candidate accuracy (ICA +5.50 points over Base) despite the loss being applied only to the teacher continuation is a surprising result worth flagging as needing direct verification, since it could partly reflect properties of the diagnostic replay applied to post-training trajectories rather than a behavioral transfer.

show 2 more

The paper says, at Section 2.2

Pith says: minor objection 3 of 4

The two boundary types ('clean' versus 'includes first verification') should be explained in the main text rather than only in the appendix prompt, in particular the guarantee that in the 'includes first verification' case the verification call does not modify the candidate state, since otherwise the replayed yc would not be the candidate's objective correctness.

The paper says, at Figure 2(a)

Pith says: minor objection 4 of 4

The VTR tick in Figure 2(a) is labeled 99.5 while the text reports 99.53; align the rounding for consistency.

What would settle it

Independently re-annotate a held-out sample of 150 to 200 trajectories stratified across the ten models with two human coders who are not the authors, using a pre-registered protocol that separately records (a) the candidate boundary tc, (b) the verification outcome z following the paper's published definition, and (c) a stricter marker for whether the agent itself acknowledged the negative evidence. Recompute EDR, VPR, and RSR under the human labels and report bootstrapped confidence intervals. If any headline rate shifts by more than about five percentage points, or if the stricter 'agent noticed' marker materially lowers EDR, then the diagnostic findings should be reported with corresponding uncertainty and the detection-versus-evidence distinction should be made explicit.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 2 Oct 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic agrees 1 min 6 s · 24,806 tokens read, 7,487 written
  3. Editor wrote the verdict above 22 s · 25,867 tokens read, 3,640 written
  4. Referee major revision: 3 major objections, 4 minor 55 s · 25,224 tokens read, 6,911 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The part of this paper that will actually move the field is SCVD, and it's well supported. Task- and size-matched FTD comparison, three Qwen3.5 backbones, three runs, official evaluator for filtering and scoring, and a real OOD benchmark on SWE-bench Verified. The 9.74–16.85 point gains over base and 4.49–8.61 over FTD are credible, and the OOD preservation is the most striking result: FTD drops 6.4–25.9 points out-of-distribution while SCVD preserves or improves. The prefix-source ablation in Table 3 is a nice control — matching the prefix to the target student beats teacher prefixes by 4.12 points, which directly supports the "condition on student states" thesis. The continuation-only loss with masked context is a clean, simple variant, and the comparison is better controlled than most distillation papers I've seen recently. The decomposition in Figure 3 (W2R vs. R2W) is also a useful piece of analysis.

The diagnostic half is a real contribution in framing. Jointly measuring candidate correctness by replaying through the candidate boundary into the official evaluator, alongside the agent's own verification outcome and the final outcome, is a measurement I haven't seen done this way. The central claim — agents almost always initiate verification (99.53%) but only detect 61.43% of bad candidates and repair only 49.36% of detected errors — is a clean statement about where self-verification breaks, and the r=0.98 correlation between repair success and final accuracy is striking.

The soft spot is real, and it's the one the stress-test flags. All four headline rates rest on LLM-judge annotation of the candidate boundary and verification outcome, and the validation is thin: 20 manually reviewed trajectories, joint accuracies of 76.67–86.67%, and only 74.29% three-way agreement on the boundary tuple. A quarter of trajectories lack unanimous boundary annotation, and errors there shift the replayed candidate correctness and directly move EDR, VPR, and RSR. The replay pipeline is competent — 98% valid labels — but it's only as good as the boundary it replays to, and annotation uncertainty is not propagated into the metrics. The authors do acknowledge the scaffold dependence in B.3, but they don't address the definitional blur: the prompt counts cat/head/ls/grep as verification and labels a check "exposes error" when negative evidence is visible even if the agent never notices it. That conflates "the check surfaced error evidence" with "the agent detected an error." Note this direction inflates EDR, so it actually reinforces the "detection is the bottleneck" headline rather than breaking it. VTR is the more vulnerable number. The authors are honest elsewhere: they explicitly acknowledge sharing the student-conditioned principle with Lyu et al. and Zhao et al., and the limitations section is fair about scope.

Who's it for: anyone working on agent distillation or self-verification in coding agents. The SCVD claim is independent of the diagnostic rates, so the core contribution survives even if the annotation concerns are upheld. My recommendation: send it to peer review, with a larger human-validated annotation set and uncertainty propagation as the main revision request.

The strongest objection

The four diagnostic rates that carry the headline findings — EDR 61.43%, VPR 51.52%, RSR 49.36%, and also VTR 99.53% — all depend on the annotated candidate boundary tc and the verification outcome z. The validation backing these annotations is thin: Section B.1 reports joint accuracies of 76.67% to 86.67% against only 20 manually reviewed trajectories, and the three-way exact agreement on the candidate-boundary tuple is 74.29%. That means roughly a quarter of trajectories have no unanimous boundary annotation even among the three LLM judges, and the remaining disagreements are resolved by voting or by the DeepSeek tie-breaker. If judge errors on tc are correlated with task type or model weakness rather than random, the replayed candidate correctness yc could be wrong for a nontrivial fraction of trajectories, directly shifting EDR, CPR, VPR, and RSR. The replay pipeline is competent — 98.00% of submitted replays produce valid labels — but it can only be as good as the annotated boundary it replays to. A second, related concern is definitional: the annotation prompt counts any deliberate cat/head/ls/grep inspection as verification, and it instructs judges to label a check as exposes error whenever negative evidence is visible, even if the agent never notices it. This can push VTR upward and blurs the distinction between 'the check produced error evidence' and 'the agent detected an error', which is exactly what EDR and RSR are supposed to measure. The scaffold-dependence discussion in Section B.3 partially defuses the VTR concern, but it does not address the semantic gap between evidence availability and agent recognition. Importantly, this weakness does not appear to undermine the SCVD contribution: the task- and size-matched FTD comparison, three backbones, three runs, official evaluator, and OOD SWE-bench results provide independent support for the distillation claims.

The reader's weakest assumption points to the LLM-judge annotation of tc and z with only 20 human-validated trajectories and limited three-way agreement. My independent read lands on the same load-bearing concern: every diagnostic metric in the headline — EDR, VPR, RSR, and to a lesser extent VTR — is downstream of those annotations, and the candidate-boundary tuple is the least reliable part of the pipeline because z could be systematically mislabeled by prompt-induced judgments about what counts as verification or as an exposed error. The mismatch between the number of submitted replays stated in Table 4 (2,549) and the sum of per-model replay denominators (2,545) is a minor arithmetic discrepancy that does not affect this concern. I also see the annotation prompt's instruction to mark a check as exposes error when negative evidence is visible even if the agent never notices it as a real conceptual risk: EDR and RSR are described in detection and repair terms, but the measurement captures evidence availability rather than agent recognition. That said, the SCVD half of the paper is independently credible: the FTD comparison is task- and size-matched, the evaluation uses three runs and the official evaluator, the gains are consistent across three backbones, and the OOD result on SWE-bench Verified moves in the expected direction. I therefore do not see grounds to strengthen the verdict beyond the reader's CONDITIONAL; the condition should be satisfied by validating the annotation layer on a larger human-labeled set and by clarifying what z measures. My recommendation is to keep the verdict unchanged.

The referee's report

If the diagnostic numbers hold, the paper makes a valuable contribution: it is the first, to my knowledge, to jointly measure, in the terminal-agent setting, whether a candidate being checked is objectively correct and what the agent's own check concludes, with a separation of initiation, detection, and recovery. The SCVD contribution is well controlled: the FTD comparison is task- and size-matched by intersecting verifier-passing trajectories, three backbones with three runs each are reported, final filtering uses the official evaluator, and OOD behavior is checked on an external benchmark (SWE-bench Verified). The replay pipeline is competent (98.00% of submitted replays produce valid labels; per-model coverage reported in Table 4). The main risk to the diagnostic half is that the headline rates rest on LLM-judge annotations with a thin human validation and with an operationalization that blurs 'evidence exposed' with 'agent detected'; these are fixable within scope, so the correct outcome is revision rather than rejection.

Pith's reading of the paper (a machine's summary, not the authors')

The paper asks whether terminal agents can trust the verification checks they run on their own completed solutions in command-line environments. It builds a diagnostic framework that locates the first complete candidate solution in each trajectory, replays it in a fresh environment to determine objective correctness, and compares that against the agent's own verification outcome. Across ten agents on TerminalBench2.1, verification is attempted after 99.53% of candidates, yet only 61.43% of incorrect candidates are detected and only 49.36% of detected errors are repaired, making a passed check weak evidence of correctness (51.52%). The paper concludes that the bottleneck is not initiating checks but detecting and repairing errors, and proposes Student-Conditioned Verification Distillation (SCVD), which raises PASS@1 by 9.74–16.85 points over the base models while avoiding the out-of-distribution degradation of full-trajectory distillation.

What carries the argument. The diagnostic framework rests on the candidate boundary t_c, the earliest interaction point at which a materially complete solution exists, obtained by tool-call-level annotation with three LLM judges, and on candidate-state replay, which re-executes the trajectory up to t_c in a fresh environment and runs the official evaluator to obtain the objective correctness y_c of the candidate. From the tuple (y_c, z, y_f) of candidate correctness, verification outcome, and final outcome, the paper defines six metrics—VTR, VOR, EDR, ESP, CPR, VPR, and RSR—that separate verification initiation from reliability and recovery. SCVD's mechanism is continuation-only distillation: the student-generated prefix is kept as context but masked from the loss, and supervision is applied only to the teacher's verification-and-repair continuation, conditioned on the student's own candidate states rather than teacher-generated ones.

Core claim. The paper's central claim is that terminal agents' self-verification is initiated almost universally but is unreliable where it matters: error detection and error recovery. Using candidate-state replay with the official evaluator to label the first complete candidate as objectively correct or not, the authors show that error signals are usually reliable (92.65% of error signals correspond to genuinely incorrect candidates) but many incorrect candidates escape detection (EDR 61.43%), and of those detected, only about half are successfully repaired (RSR 49.36%). Because repair success correlates with final task accuracy at Pearson r = 0.98, the recovery stage is the lever that most affects end-to-end performance. The companion contribution is SCVD: the student produces the candidate, a stronger teacher continues from that same interaction context and environment state to verify and repair it, and fine-tuning supervises only the teacher continuation. On task- and size-matched training sets, SCVD beats both the base models and full-trajectory distillation on TerminalBench2.1 while largely preserving out-of-distribution performance on SWE-bench Verified, where FTD degrades sharply.

The premise it rests on. The headline diagnostic rates rest on LLM-judge annotations of the candidate boundary and verification outcome that were validated against only 20 manually reviewed trajectories, with joint accuracies of 76.67–86.67% and three-way exact agreement of 74.29% on the boundary tuple, so if annotation error is not random across models and task types the reported EDR, VPR, and RSR could shift; separately, counting any deliberate cat/head/ls/grep inspection as verification may inflate the near-universal VTR.

If the paper is right (Pith's inferences, not the authors')

Post-candidate interaction is not a final consistency check but a stage where agents can gain on average 25.4 accuracy points, so improving verification and recovery is a direct route to higher task success.

A passed verification signal should be treated as weak evidence: at a VPR of 51.52%, roughly half of candidates that pass the agent's own checks are still incorrect, so stopping on a clean check is risky.

Conditioning teacher demonstration on learner-induced states, not teacher states, transfers verification and recovery behavior better, as shown by the 4.12-point advantage of student-generated prefixes over teacher prefixes.

Distilling verification and recovery can improve the student's initial candidate accuracy even though the loss excludes the prefix, suggesting the supervision reshapes earlier problem-solving decisions.

SCVD achieves its accuracy gains with fewer agent turns (12.0–35.6% fewer) and often lower total token cost than base, trading longer within-turn generation for fewer interaction rounds.

The near-universal VTR of 99.53% may be partly definitional: the annotation protocol counts any deliberate cat/head/ls/grep inspection as verification, and the paper's own scaffold discussion (Section B.3) only partially defuses this, so the 'initiation is solved' conclusion deserves a test with a stricter definition that requires genuinely diagnostic checks such as executing the produced program.

The three LLM judges were calibrated on only 20 manually reviewed trajectories with joint accuracies of 76.67–86.67% and three-way exact agreement of 74.29% on the candidate-boundary tuple, so a larger human-annotation study is the natural next check on whether EDR, VPR, and RSR are systematically off in one direction.

The decomposition showing SCVD improves initial candidate accuracy by 5.50 points suggests verification-conditioned training may act as a form of implicit plan shaping; testing whether SCVD's gains persist when the teacher is withheld at inference time would isolate the policy-change effect from any scaffold behavior.

Because SCVD avoids FTD's OOD collapse, the method's robustness may come from not overfitting the student to teacher-typical action patterns; a stress test on farther-distribution environments (beyond SWE-bench) would probe how far that robustness extends.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. Yet how trustworthy such self-verification is remains poorly understood. To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent's subsequent verification and recovery behavior. Applying it to ten terminal agents on TerminalBench2.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61.43% of incorrect candidates are detected and only 49.36% of detected errors are successfully repaired. These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors. Motivated by these findings, we propose Student-Conditioned Verification Distillation (SCVD), which lets the student first produce a candidate solution and distills a stronger teacher's subsequent verification and recovery from the same interaction context. Across three Qwen3.5 backbones, SCVD improves \textscPass@1 on TerminalBench2.1 by 9.74–16.85 percentage points over the corresponding base models and by 4.49–8.61 points over the standard full-trajectory distillation, while avoiding the pronounced out-of-distribution degradation of full-trajectory distillation on SWE-bench Verified.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-01

The authors' main claim, as the classifier picked it out 0.99

“These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.26 · human participants 0.04 · controlled comparison 0.94 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.95 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Terminal agents rely on self-verification to assess and correct their solutions as they solve tasks through interaction with command-line environments. background 1.00 0.05 0.02 asserted 1.00
Yet how trustworthy such self-verification is remains poorly understood. gap 0.99 0.05 0.02 asserted 0.93
To investigate this question systematically, we introduce a diagnostic framework that identifies the first complete solution in each trajectory, determines whether it is objectively correct, and uses this ground truth to quantify the agent's subsequent verification and recovery behavior. method 0.90 0.93 0.04 asserted 0.96
Applying it to ten terminal agents on TerminalBench2.1, we find that verification is nearly universal after a complete candidate is formed, yet only 61.43\% of incorrect candidates are detected and only 49.36\% of detected errors are successfully repaired. result 1.00 0.97 0.20 asserted 1.00
These results show that the main weakness in self-verification lies not in initiating verification, but in detecting and repairing errors. interpretation 1.00 0.94 0.76 asserted 1.00
Motivated by these findings, we propose Student-Conditioned Verification Distillation (SCVD), which lets the student first produce a candidate solution and distills a stronger teacher's subsequent verification and recovery from the same interaction context. method 0.78 0.78 0.06 asserted 0.97
Across three Qwen3.5 backbones, SCVD improves \textsc{Pass@1} on TerminalBench2.1 by 9.74--16.85 percentage points over the corresponding base models and by 4.49--8.61 points over the standard full-trajectory distillation, while avoiding the pronounced out-of-distribution degradation of full-trajectory distillation on SWE-bench Verified. result 0.99 0.95 0.24 asserted 1.00

What kind of paper the classifier reports this is: method or tool 0.72 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-01: 1 request, 5,884 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. Overview of solution-level self-verification and SCVD.
Figure 2 of the paper
Figure 2. Self-verification diagnostics across ten terminal agents on TerminalBench2.1. Metric definitions
Figure 3 of the paper
Figure 3. Decomposition of final accuracy. Blue denotes initially correct candidates that remain correct, green
Figure 4 of the paper
Figure 4. Performance versus inference cost on TerminalBench2.1. Costs are averaged over common-success

Figures are the authors', reproduced from the paper.

References (33)
  1. [1] Qwen Team , month =. Qwen3.5: Towards Native Multimodal Agents , url = on Pith
  2. [2] Glm-5: from vibe coding to agentic engineering , author=. arXiv preprint arXiv:2602.15763 , year= on PithPith's reviewarXiv
  3. [3] Aman Madaan and Niket Tandon and Prakhar Gupta and Skyler Hallinan and Luyu Gao and Sarah Wiegreffe and Uri Alon and Nouha Dziri and Shrimai Prabhumoye and Yiming Yang and Shashank Gupta and Bodhisattwa Prasad Majumder and Katherine Hermann and Sean Welleck and Amir Yazdanbakh... on Pith
  4. [4] Noah Shinn and Federico Cassano and Ashwin Gopinath and Karthik Narasimhan and Shunyu Yao , editor =. Reflexion: language agents with verbal reinforcement learning , booktitle =. 2023 , url = on Pith
  5. [5] Zhibin Gou and Zhihong Shao and Yeyun Gong and Yelong Shen and Yujiu Yang and Nan Duan and Weizhu Chen , title =. The Twelfth International Conference on Learning Representations,. 2024 , url = on Pith
  6. [6] Xinyun Chen and Maxwell Lin and Nathanael Sch. Teaching Large Language Models to Self-Debug , booktitle =. 2024 , url = on Pith
  7. [7] Jie Huang and Xinyun Chen and Swaroop Mishra and Huaixiu Steven Zheng and Adams Wei Yu and Xinying Song and Denny Zhou , title =. The Twelfth International Conference on Learning Representations,. 2024 , url = on Pith
  8. [8] Ryo Kamoi and Yusen Zhang and Nan Zhang and Jiawei Han and Rui Zhang , title =. Trans. Assoc. Comput. Linguistics , volume =. 2024 , url =. doi:10.1162/TACL\_A\_00713 , timestamp = on PithPith's reviewdoi:10.1162/tacl
  9. [9] Aviral Kumar and Vincent Zhuang and Rishabh Agarwal and Yi Su and John D. Co. Training Language Models to Self-Correct via Reinforcement Learning , booktitle =. 2025 , url = on Pith
  10. [10] Ruotian Ma and Peisong Wang and Cheng Liu and Xingyan Liu and Jiaqi Chen and Bang Zhang and Xin Zhou and Nan Du and Jia Li , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2502.12853 , eprinttype =. 2502.12853 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2502.12853
  11. [11] Zhi Chen and Zhensu Sun and Yuling Shi and Chao Peng and Xiaodong Gu and David Lo and Lingxiao Jiang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.07900 , eprinttype =. 2602.07900 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2602.07900
  12. [12] Yuxuan Sun and Yuze Zhao and Yufeng Wang and Yao Du and Zhiyuan Ma and Jinbo Wang and Mengdi Zhang and Kai Zhang and Zhenya Huang , editor =. SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering? , booktitle =. 2026 , url =. doi:10.18653/V1/2026.FINDING... on Pithdoi:10.18653/v1/2026.findings-acl.1976
  13. [13] Zeyao Ma and Jing Zhang and Xiaokang Zhang and Jiaxi Yang and Zongmeng Zhang and Jiajun Zhang and Yuheng Jing and Lei Zhang and Hao Zheng and Wenting Zhao and Junyang Lin and Binyuan Hui , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2602.04254 , eprinttype =. 26... on Pithdoi:10.48550/arxiv.2602.04254
  14. [14] Ruoyu Wang and Jierun Chen and Shaowei Wang and Chaofan Tao and Sidi Yang and Yuxin Jiang and Kim. SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review , journal =. 2026 , url =. doi:10.48550/ARXIV.2607.06065 , eprinttype =. 2607.06065 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.06065
  15. [15] Rui Shu and Chun Yong Chong and Xin Zhou and Yun Peng and Zihan Wu and Xu Han and Zeyang Zhuang and Guowen Yuan and Yuan Wang , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.06184 , eprinttype =. 2607.06184 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.06184
  16. [16] Xiangxin Zhao and Han Li and Shuaiting Li and Tianyi Zhao and Earl T. Barr and Federica Sarro and He Ye , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2607.09510 , eprinttype =. 2607.09510 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2607.09510
  17. [17] Baian Chen and Chang Shu and Ehsan Shareghi and Nigel Collier and Karthik Narasimhan and Shunyu Yao , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2310.05915 , eprinttype =. 2310.05915 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2310.05915
  18. [18] Aohan Zeng and Mingdao Liu and Rui Lu and Bowen Wang and Xiao Liu and Yuxiao Dong and Jie Tang , title =. CoRR , volume =. 2023 , url =. doi:10.48550/ARXIV.2310.12823 , eprinttype =. 2310.12823 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2310.12823
  19. [19] Jun Liu and Zhenglun Kong and Peiyan Dong and Changdi Yang and Tianqi Li and Hao Tang and Geng Yuan and Wei Niu and Wenbin Zhang and Pu Zhao and Xue Lin and Dong Huang and Yanzhi Wang , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2505.13820 , eprinttype =. 2505.... on PithPith's reviewarXivdoi:10.48550/arxiv.2505.13820
  20. [20] St. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , booktitle =. 2011 , url = on Pith
show 13 more
  1. [21] Sidi Yang and Chaofan Tao and Jierun Chen and Tiezheng Yu and Ruoyu Wang and Yuxin Jiang and Yiming Du and Wendong Xu and Jing Xiong and Taiqiang Wu and Lifeng Shang and Xiaohui Li and Ngai Wong and Haoli Bai , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2606.03... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.03461
  2. [22] Yuanjie Lyu and Chengyu Wang and Jun Huang and Tong Xu , title =. CoRR , volume =. 2025 , url =. doi:10.48550/ARXIV.2509.14257 , eprinttype =. 2509.14257 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2509.14257
  3. [23] Siyan Zhao and Zhihui Xie and Mengchen Liu and Jing Huang and Guan Pang and Feiyu Chen and Aditya Grover , title =. CoRR , volume =. 2026 , url =. doi:10.48550/ARXIV.2601.18734 , eprinttype =. 2601.18734 , timestamp = on PithPith's reviewarXivdoi:10.48550/arxiv.2601.18734
  4. [24] Large-scale terminal agentic trajectory generation from dockerized environments , author=. arXiv preprint arXiv:2602.01244 , year= on PithPith's reviewarXiv
  5. [25] SETA: Scaling Environments for Terminal Agents , author=. arXiv preprint arXiv:2607.10891 , year= on PithPith's reviewarXiv
  6. [26] Termigen: High-fidelity environment and robust trajectory synthesis for terminal agents , author=. arXiv preprint arXiv:2602.07274 , year= on PitharXiv
  7. [27] Shunyu Yao and Jeffrey Zhao and Dian Yu and Nan Du and Izhak Shafran and Karthik R. Narasimhan and Yuan Cao , title =. The Eleventh International Conference on Learning Representations,. 2023 , url = on Pith
  8. [28] Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces , author =. 2026 , eprint = on Pith
  9. [29] Swe-bench: Can language models resolve real-world github issues? , author=. International Conference on Learning Representations , volume= on Pith
  10. [30] Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework , author=. arXiv preprint arXiv:2608.08950 , year= on PithPith's reviewarXiv
  11. [31] Beyond Fail-to-Pass: Iterative Hardening of Co-Generated Bug Reproduction Tests and Fixes , author=. arXiv preprint arXiv:2607.19843 , year= on PithPith's reviewarXiv
  12. [32] Learning to self-verify makes language models better reasoners , author=. arXiv preprint arXiv:2602.07594 , year= on PitharXiv
  13. [33] Dream-RSI: Recursive Self-Improvement through Evolving Worlds , author =. arXiv preprint arXiv:2609.14858 , year = on PithPith's reviewarXiv

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment