Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework

Chenglin Li; Shin Hwei Tan; Tse-Hsun (Peter) Chen; Yisen Xu; Zehao Wang

REVIEW 3 major objections 5 minor 12 references

Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A coding agent can independently verify its own patch by reconstructing the problem the patch solves—with the original issue withheld—and comparing that reconstruction with the reported problem; on SWE-bench Verified this signal raises…

desk verdict A genuinely new backward-reconstruction verification idea with a load-bearing 'independence' claim the experiments don't yet support; deserves serious review, not a desk reject. read the letter →

arxiv 2608.08950 v1 pith:6DIPLW5U submitted 2026-08-09 cs.SE

Chenglin Li , Yisen Xu , Zehao Wang , Shin Hwei Tan , Tse-Hsun (Peter) Chen This is my paper · ORCID

classification cs.SE

keywords RETRACEcodingagentspatchverificationbidirectionalreconstructionSWE-benchVerifiedself-refinementinference-timeprogramrepair

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RETRACE is a training-free verification framework that gives a coding agent an independent check on its own patch: after the agent produces a patch, the framework reconstructs the repair rationale from the issue and trajectory, then reconstructs the problem the patch appears to address from the patch and trajectory alone (without the original issue), and reconciles the two views into a submit-or-revise decision. The central claim is that this bidirectional reconstruction produces a verification signal that predicts patch correctness without test execution or ground-truth patches. On SWE-bench Verified, the framework raises Pass@1 from 56.2% to 63.2% with GPT-5-mini and from 75.8% to 79.4% with MiniMax-2.5 on the mini-SWE-agent scaffold and delivers comparable gains on OpenHands. Ablations show both directions contribute, and the Self-Refine comparison is offered as evidence that the improvement comes from the verification signal, not from extra inference alone.

What carries the argument

The mechanism that carries the argument is bidirectional reconstruction with reconciliation. Forward reconstruction takes the original issue $I$ and the forward trajectory view $\tau_f$ and produces a repair rationale $\phi=F(I,\tau_f)=(E,r,s)$ consisting of recovered repository evidence, a logical reasoning chain, and an intended repair; backward reconstruction takes the candidate patch $p$ and the backward trajectory view $\tau_b$ (which excludes the original issue) and produces $\hat{I}=B(p,\tau_b)$, a description of the problem the patch actually implements; reconciliation then computes the alignment verdict $v\in\{\text{Same},\text{Partial},\text{Different}\}$ between $\hat{I}$ and $I$, checks the patch against $\phi$, and selects a reconciliation action $a\in\{\text{Submit},\text{Revise-Patch},\text{Revisit-Reasoning},\text{Revise-Both}\}$ along with an ordered revision plan. What this machinery does is convert the unanswerable question 'is this patch correct?' into a comparative question about whether the problem the patch solves matches the problem that was reported.

What would settle it

Run RETRACE on a held-out set of SWE-bench issues and compare the fail-to-pass test pass rate of patches the framework labels 'Same' against those it labels 'Different'; if the 'Same' patches are not noticeably more likely to pass, the alignment verdict is not a correctness signal.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that a patch's correctness can be read off the correspondence between two reconstructions of the same repair: a forward reconstruction $\phi=F(I,\tau_f)$ that recovers, from the original issue and the trajectory view, the evidence, reasoning chain, and intended repair, and a backward reconstruction $\hat{I}=B(p,\tau_b)$ that recovers, from the candidate patch and the backward trajectory view while withholding the original issue, the problem the patch appears to address. The reconciliation stage compares these and the original issue to emit a verdict (Same, Partial, Different) and a decision (Submit, Revise-Patch, Revisit-Reasoning, Revise-Both) with an ordered revision plan. The paper reports that on SWE-bench Verified this mechanism raises Pass@1 from 56.2% to 63.2% with GPT-5-mini and from 75.8% to 79.4% with MiniMax-2.5 on mini-SWE-agent, and from 37.5% to 56.7% and 62.5% to 70.0% on OpenHands with the two backbones.

Load-bearing premise

The load-bearing premise is that the backward reconstruction is genuinely independent: the same model that wrote the patch, when the original issue is hidden, can describe the problem the patch solves from the patch itself, and that description's match to the original issue predicts whether the patch is correct.

Editorial extensions

If this is right

Coding agents can validate candidate patches without test execution or ground-truth patches, so verification works in settings where no reliable test oracle exists.
The framework is scaffold-agnostic: RETRACE improves Pass@1 on both mini-SWE-agent and OpenHands while leaving the underlying agent's tools and edit loop untouched.
Forward and backward reconstruction address complementary failure modes, and reconciliation rescues issues that neither direction resolves alone; removing either stage on the 120-issue subset lowers Pass@1 from 60.8% to 56.7%.
The gain is not explained by extra computation alone: with a comparable inference budget, Self-Refine fails to improve (and slightly lowers) the mini-SWE-agent baselines.
Because the added stages share cached inputs and reduce the agent's exploratory output, per-issue cost stays at or below the baseline despite the additional inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural stress test is to decouple the verifier from the generator: run the backward reconstruction with a different backbone model than the one that produced the patch. If the gains persist, independence is real; if they vanish, the framework is largely restating the generator's own interpretation.
The same reconstruct-and-compare pattern could transfer to other generative tasks where the output must satisfy a hidden specification, such as summarization faithfulness, code review, or mathematical proof checking: ask the output to re-derive the question it answers, then compare. The paper does not claim this; it is an implication of the mechanism.
The alignment verdict could be calibrated against SWE-bench's fail-to-pass tests on a development subset to learn a confidence threshold for automatic submission, converting RETRACE from a revision loop into a selective verification oracle. This extends the paper's design rather than a claim it makes.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's my take on RETRACE. The core idea is genuinely new: after a coding agent produces a patch, RETRACE does a backward pass that reconstructs the problem statement from the patch and trajectory while withholding the original issue, then compares that reconstruction to the original issue to decide whether to submit or revise. That mechanism is not in the cited self-refinement or test-time scaling work, and it is a clever way to get a verification signal without tests or ground truth. The paper is also well-put-together: clean three-stage design, ablations showing both forward and backward stages contribute, a cost analysis showing no overhead, and transfer to a second scaffold without modification. Credit where due.

The soft spot is the load-bearing word “independent.” The backward reconstruction is done by the same backbone model that generated the patch, and the trajectory it uses was produced while the agent had the issue in context. Stripping “explicit restatements” from τ_b does not strip the model's own interpretation, because p and τ_b are causal outputs of a process that started from I. So the alignment score between bI and I can be a self-consistency check, not an independent verification. The paper's Limitations section acknowledges the same-backbone point, but no experiment tests the independence directly. The forward-only ablation already lifts Pass@1 by 6.7 points on the 120-issue subset, so the gains do not require the backward mechanism to be independent. And the Self-Refine baseline is not tuned on mini-SWE-agent — it degrades both backbones — so the claim that gains come from the verification signal rather than extra inference is weaker than it looks.

There are also experimental robustness issues: single run with greedy decoding, no confidence intervals, and a 120-issue ablation subset with no reported seed. These are fixable.

Overall, the paper deserves a serious referee. The idea is useful and the direction is promising, but the current evidence does not establish the “independent verification signal” claim. I would condition acceptance on multiple seeds, code release, a better-tuned Self-Refine baseline, and either a heterogeneous verifier or an ablation that varies the model used for backward reconstruction. If the mechanism is reframed as structured self-verification, the empirical gains may hold, but the independence claim has to be earned.

For your reading group: worth a maybe. I would cite it if writing on coding agent verification.

Referee Report

3 major / 5 minor

Summary. This manuscript introduces RETRACE, a training-free post-generation verification framework for coding-agent patches. After a patch p is generated from an issue I and trajectory τ, RETRACE (i) forward-reconstructs a repair rationale φ=(E,r,s) from I and the trajectory; (ii) backward-reconstructs the addressed problem bI = B(p,τ_b) with the original issue withheld; and (iii) computes an alignment verdict A(I,bI) and a reconciliation decision R that selects Submit or a targeted revision plan. Experiments on SWE-bench Verified report Pass@1 gains of 7.0 and 3.6 percentage points for GPT-5-mini and MiniMax M2.5 on mini-SWE-agent, additional gains on OpenHands, ablations on a random 120-issue subset, and a cost analysis. The paper claims that the improvement comes from the independent verification signal rather than from additional inference alone.

Significance. If the main claim holds, RETRACE would be a practical, scaffold-agnostic verification signal that does not require test execution or ground-truth patches; the cost analysis and ablations suggest the mechanism is inexpensive and that both stages matter. The design is mostly clean and uses an external standard benchmark, and the ablations on the 120-issue subset are a useful first decomposition of the gains. However, the load-bearing premise—that the backward reconstruction is truly independent of the original issue—is asserted rather than demonstrated, and the Self-Refine comparison does not isolate the claimed mechanism. No code or artifact is provided, so prompt-level reproducibility cannot be checked from the manuscript alone.

major comments (3)

[Backward Reconstruction of the Addressed Problem; Limitations] The central claim of an 'independent verification signal' is not supported by the experiments. The backward view τ_b is defined to exclude I and its explicit restatements, but τ_b still contains observations and decisions made while the agent had I available, and p is causally downstream of I. With the same backbone model performing B, bI can be a rationalization of the generator's own patch rather than an external check. The Limitations section acknowledges that both stages rely on the same backbone model and that the signal depends on issue quality, but no experiment varies whether I is withheld or uses a different verifier model. A control that compares RETRACE with and without access to I in the backward stage (or that uses a second model for bI and the alignment verdict) is needed before the Pass@1 gains can be attributed to independence rather than to extra structured inference.
[Experiments, Table 1 and Table 3; 'Performance of RETRACE'] The claim that the improvement stems from the verification signal rather than from additional computation alone is not established by the Self-Refine comparison. Self-Refine differs from RETRACE in prompt structure, evidence recovery, and revision guidance, and it is not tuned on mini-SWE-agent, where it degrades both baselines, whereas it does improve OpenHands. This confound prevents attributing the difference to the presence of a verification signal. A matched control using the same RETRACE prompts and revision loop but with access to I in the backward stage would isolate the withheld-issue mechanism.
[Experimental Settings; Tables 2 and 3] The quantitative claims rest on a single run per configuration with no confidence intervals, and the 120-issue ablation subset is described only as 'random' with no seed or selection procedure reported. With n=500, the headline 7.0-point gain has an approximate standard error of 2.2 points, but the OpenHands comparison and the ablations use n=120 with substantially larger uncertainty, and a single run cannot rule out seed- or scaffold-specific artifacts. Reporting multiple seeds or at least bootstrap confidence intervals, and fixing the subset seed in the experimental protocol, is needed to support the reported Pass@1 differences.

minor comments (5)

[Related Work] The sentence 'evolutionary refinement (Zeng et al. 2025a), trajectory replay (Ding and Zhang 2025), distillation-conditioned rollouts (Gandhi et al. 2025a), or .' ends with an incomplete relative clause and an orphaned period.
[Forward Reconstruction of Repair Reasoning] In the 'Evidence recovery' paragraph, the phrase 'retains the retains the code and program relationships' contains a duplicated fragment that should be corrected.
[Performance of RETRACE] The sentence 'despite using a comparable inference despite a comparable inference budget' is grammatically broken and should be rewritten.
[References] Several references are duplicated with different labels (e.g., Gandhi et al. 2025a/2025b, CodeMonkeys 2025a/2025b, Trae Agent 2025a/2025b, Satori-SWE 2025a/2025b), and the author list for Li et al. 2026b contains the malformed entry 'Tse-Hsun; and Chen'; these should be consolidated and corrected.
[Ablation Studies] The text refers to 'Figure 2' for the overlap analysis, but no Figure 2 image appears in the manuscript; the figure should be included or the reference removed.

Circularity Check

1 steps flagged · score 2.0 of 10

The 'independent' backward signal is asserted by prompt-level exclusion while its inputs are causally downstream of the withheld issue; the empirical SWE-bench results remain external and non-circular.

self definitional [Backward Reconstruction of the Addressed Problem (definition of τ_b and bI = B(p, τ_b))]
"Let τ_b denote the backward view of the agent trajectory, containing the repository observations and implementation context used to produce the patch, but excluding the original issue I and explicit restatements of it. ... By withholding I, the backward path must reconstruct the addressed problem from the patch rather than repeating the task originally given to the agent."

The inputs to B(p, τ_b) are p and τ_b, both generated by an agent that had I in context: τ_b records observations and decisions made under I, and p is the agent's output conditioned on I. Excluding I from the backward prompt removes the literal issue text but not the information about I encoded in the trajectory and patch. The reconstructed problem bI is therefore a function of the same generator's interpretation, and the alignment verdict A(I, bI) is a self-comparison by the same backbone. The claimed 'independent verification signal' thus reduces by construction to structured self-review; the paper's Limitations acknowledge the same-backbone reliance but do not test whether withholding I changes the verdict.

full rationale

The empirical core of the paper is self-contained against SWE-bench Verified: no parameters are fitted to the benchmark, Pass@1 is measured by external fail-to-pass and pass-to-pass tests, and the comparison with Self-Refine is a real controlled contrast rather than a renamed version of the result. No equation-level reduction of the reported gains to an input occurs, and the author self-citations (Li et al. 2026a, 2026b) are background references, not load-bearing. The only near-circular element is the labeling of the backward reconstruction as 'independent': the paper defines independence by excluding the issue text from the prompt, but the trajectory and patch are causally downstream of the issue and produced by the same backbone model, so the backward signal is at best structured self-review. This is a conceptual circularity in how the verification signal is characterized, and the Limitations section candidly acknowledges the same-backbone reliance without testing whether withholding the issue changes outcomes. Because the benchmark measurements themselves remain external and non-circular, the overall circularity score is low.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central claim rests on no fitted constants, but it relies on several domain assumptions about what the LLM can reconstruct and what the trajectory records. These assumptions are explicit in the design and are acknowledged only partially in Limitations.

free parameters (2)

Revision budget = not reported (configured but value absent)
The reconciliation loop repeats until Submit or until the budget is exhausted; the budget value directly controls how many revision attempts are allowed and therefore affects Pass@1.
Seed for the 120-issue ablation subset = unreported
The ablations and OpenHands comparison use a random 120-issue subset with no seed, so the selection cannot be audited or independently reproduced.

assumptions (4)

domain assumption A natural-language alignment verdict between the reconstructed problem and the original issue is a reliable proxy for patch correctness without executing tests.
Central to the verification signal; the paper does not validate this proxy against test outcomes, and Limitations admits that vague reports weaken the alignment verdict.
domain assumption Backward reconstruction is independent when the original issue is withheld from the prompt, even though the same backbone model generated the patch and trajectory.
Independence is claimed in the Backward Reconstruction section; Limitations notes that both reconstructions rely on the same backbone model, which weakens the independence claim.
domain assumption The recorded agent trajectory is a faithful and sufficient record of exploration, reasoning, and patch-generation context.
Forward and backward stages both consume the trajectory; if the trajectory is lossy, the reconstructions may omit decisive evidence.
domain assumption SWE-bench Verified, a Python-focused benchmark of 500 issues, is representative enough to support general claims about coding-agent verification.
Limitations acknowledges that the evaluation targets Python projects and may not generalize to other languages or domains.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework." pith.science (2026). https://pith.science/paper/6DIPLW5U

@misc{pith2026260808950,
  author       = {Pith},
  title        = {Pith review of: Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6DIPLW5U}},
  note         = {Machine review of arXiv:2608.08950}
}

read the original abstract

Autonomous coding agents powered by large language models can now generate code patches directly from bug reports, but a fundamental gap remains: once a patch is produced, no mechanism independently verifies whether it truly resolves the reported problem. Prior work has sought to address this through iterative self-refinement and inference-time scaling, but these approaches either review the patch under the same interpretation that produced it or broaden candidate generation without verifying individual patches, and neither provides an explicit verification signal for assessing patch correctness. We propose RETRACE, a training-free post-generation verification framework that derives such a signal through bidirectional reconstruction and reconciliation. When a coding agent generates a candidate patch for an issue, RETRACE performs forward reconstruction to build an explicit repair rationale from the issue and the agent's trajectory; backward reconstruction then independently infers, from the patch and its trajectory alone and without access to the original issue, a description of the problem the patch appears to address, and compares this reconstruction against the original issue to produce an alignment verdict; a reconciliation stage then checks the consistency between the forward rationale and the patch, diagnoses the source of any misalignment, and either submits the patch or produces targeted revision guidance. Evaluated on SWE-bench Verified with two backbones (GPT-5-mini and MiniMax-2.5), RETRACE lifts Pass@1 by 7.0% and 3.6% respectively on the mini-SWE-agent scaffold, and delivers comparable gains on OpenHands without modification. Ablation experiments show that both the forward and backward stages contribute to the overall improvement and that adding reconciliation yields further gains.

Figures

Figures reproduced from arXiv: 2608.08950 by the authors.

**Figure 2.** Overlap of resolved issues among three configura [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages

[4]

Ma, Y.; et al

Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Auto- mated Software Improvement.CoRR, abs/2411.00622. Ma, Y.; et al

arXiv
[5]

Self-Refine: Iterative Refinement with Self- Feedback. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds.,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16,

work page 2023
[6]

https://www.minimax.io/models/text

MiniMax M2.5: SOTA in Coding and Agent, De- signed for Agent Universe. https://www.minimax.io/models/text. Accessed: 2026-07-23. Ni,Z.;Wang,H.;Zhang,S.;Lu,S.;He,Z.;Tang,Z.;Hu,S.;Li,B.; Hu, C.; Jiao, B.; et al

work page 2026
[7]

https://openai.com/index/ introducing-gpt-5/

Introducing GPT-5. https://openai.com/index/ introducing-gpt-5/. Accessed: 2026-07-01. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S

work page 2026
[8]

Reflexion: language agents with verbal reinforcement learning. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds.,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16,

work page 2023
[9]

F.; Tang, X.; Zhuge, M.; Pan, J.; Song, Y.; Li, B.; Singh, J.; Tran, H

Wang, X.; Li, B.; Song, Y.; Xu, F. F.; Tang, X.; Zhuge, M.; Pan, J.; Song, Y.; Li, B.; Singh, J.; Tran, H. H.; Li, F.; Ma, R.; Zheng, M.; Qian, B.; Shao, Y.; Muennighoff, N.; Zhang, Y.; Hui, B.; Lin, J.;andetal.2025. OpenHands:AnOpenPlatformforAISoftware Developers as Generalist Agents. InThe Thirteenth International Conference on Learning Representations...

work page 2025
[10]

Yang,J.;Jimenez,C.E.;Wettig,A.;Lieret,K.;Yao,S.;Narasimhan, K.;andPress,O.2024.SWE-agent:Agent-ComputerInterfacesEn- ableAutomatedSoftwareEngineering

Demystifying llm-basedsoftwareengineeringagents.ProceedingsoftheACMon Software Engineering, 2(FSE): 801–824. Yang,J.;Jimenez,C.E.;Wettig,A.;Lieret,K.;Yao,S.;Narasimhan, K.;andPress,O.2024.SWE-agent:Agent-ComputerInterfacesEn- ableAutomatedSoftwareEngineering. InGlobersons,A.;Mackey, L.;Belgrave,D.;Fan,A.;Paquet,U.;Tomczak,J.M.;andZhang,C., eds.,Advances i...

work page 2024
[11]

Zeng, G.; Shen, M.; Chen, D.; Qi, Z.; Das, S.; Gutfreund, D.; Cox, D.; Wornell, G.; Lu, W.; Hong, Z.-W.; and Gan, C. 2025a. Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering.arXiv preprint arXiv:2505.23604. Zeng, G.; Shen, M.; Chen, D.; Qi, Z.; Das, S.; Gutfreund, D.; Cox, D.; Wornell, G. W.; Lu, W.; Hong, Z.; and Gan, C...

arXiv

Show all 12 references

[12]

Au- toCodeRover: Autonomous Program Improvement.arXiv preprint arXiv:2404.05427

arXiv
[2023]

Dihan,M.L.;andKhan,M.A.R.2026.SWE-Shepherd:Advancing PRMs for Reinforcing Code Agents.CoRR, abs/2604.10493

TeachingLarge LanguageModelstoSelf-Debug.arXivpreprintarXiv:2304.05128. Dihan,M.L.;andKhan,M.A.R.2026.SWE-Shepherd:Advancing PRMs for Reinforcing Code Agents.CoRR, abs/2604.10493. Ding, Y.; and Zhang, L

2026 arXiv
[2024]

Li, C.; Xu, Y.; Wang, Z.; Tan, S

Swe-bench: Can language models resolve real-worldgithubissues? InInternationalConferenceonLearning Representations, volume 2024, 54107–54157. Li, C.; Xu, Y.; Wang, Z.; Tan, S. H.; and Chen, T. 2026a. Are BenchmarkTestsStrongEnough?Mutation-GuidedDiagnosisand Augmentation of Re...

2024 arXiv
[2025]

Ehrlich, R.; Brown, B.; Juravsky, J.; Clark, R.; Ré, C.; and Mirho- seini, A

SWE-Replay: Efficient Test- Time Scaling for Software Engineering Agents.arXiv preprint arXiv:2601.22129. Ehrlich, R.; Brown, B.; Juravsky, J.; Clark, R.; Ré, C.; and Mirho- seini, A. 2025a. CodeMonkeys: Scaling Test-Time Compute for Software Engineering.arXiv preprint arXiv:2...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [4]

Ma, Y.; et al

Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Auto- mated Software Improvement.CoRR, abs/2411.00622. Ma, Y.; et al

arXiv

[2] [5]

Self-Refine: Iterative Refinement with Self- Feedback. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds.,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16,

work page 2023

[3] [6]

https://www.minimax.io/models/text

MiniMax M2.5: SOTA in Coding and Agent, De- signed for Agent Universe. https://www.minimax.io/models/text. Accessed: 2026-07-23. Ni,Z.;Wang,H.;Zhang,S.;Lu,S.;He,Z.;Tang,Z.;Hu,S.;Li,B.; Hu, C.; Jiao, B.; et al

work page 2026

[4] [7]

https://openai.com/index/ introducing-gpt-5/

Introducing GPT-5. https://openai.com/index/ introducing-gpt-5/. Accessed: 2026-07-01. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S

work page 2026

[5] [8]

Reflexion: language agents with verbal reinforcement learning. In Oh, A.; Naumann, T.; Globerson, A.; Saenko, K.; Hardt, M.; and Levine, S., eds.,Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16,

work page 2023

[6] [9]

F.; Tang, X.; Zhuge, M.; Pan, J.; Song, Y.; Li, B.; Singh, J.; Tran, H

Wang, X.; Li, B.; Song, Y.; Xu, F. F.; Tang, X.; Zhuge, M.; Pan, J.; Song, Y.; Li, B.; Singh, J.; Tran, H. H.; Li, F.; Ma, R.; Zheng, M.; Qian, B.; Shao, Y.; Muennighoff, N.; Zhang, Y.; Hui, B.; Lin, J.;andetal.2025. OpenHands:AnOpenPlatformforAISoftware Developers as Generalist Agents. InThe Thirteenth International Conference on Learning Representations...

work page 2025

[7] [10]

Yang,J.;Jimenez,C.E.;Wettig,A.;Lieret,K.;Yao,S.;Narasimhan, K.;andPress,O.2024.SWE-agent:Agent-ComputerInterfacesEn- ableAutomatedSoftwareEngineering

Demystifying llm-basedsoftwareengineeringagents.ProceedingsoftheACMon Software Engineering, 2(FSE): 801–824. Yang,J.;Jimenez,C.E.;Wettig,A.;Lieret,K.;Yao,S.;Narasimhan, K.;andPress,O.2024.SWE-agent:Agent-ComputerInterfacesEn- ableAutomatedSoftwareEngineering. InGlobersons,A.;Mackey, L.;Belgrave,D.;Fan,A.;Paquet,U.;Tomczak,J.M.;andZhang,C., eds.,Advances i...

work page 2024

[8] [11]

Zeng, G.; Shen, M.; Chen, D.; Qi, Z.; Das, S.; Gutfreund, D.; Cox, D.; Wornell, G.; Lu, W.; Hong, Z.-W.; and Gan, C. 2025a. Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering.arXiv preprint arXiv:2505.23604. Zeng, G.; Shen, M.; Chen, D.; Qi, Z.; Das, S.; Gutfreund, D.; Cox, D.; Wornell, G. W.; Lu, W.; Hong, Z.; and Gan, C...

arXiv