Pith.

Artificial Intelligence · arXiv:2611.15276 v1 · submitted 11 Nov 2026 · 118 references

Verify by Doing: Co-Evolving Agent Execution and Evaluation without External Feedback

Yitong Lin, Jianda Chen, Yihao Hu, Pan Wang, Hang Wang, Zhihao Wen, Xiaoxue Ren, Wei Wu

The paper says

“Agent self-improvement requires feedback about task success, yet external feedback may be unavailable at test time. Agents must then infer success from observable evidence, and incomplete verification can misguide subsequent updates.”

more of the abstract

We introduce Verify by Doing (VbD), a method that co-evolves Actor and Evaluator harnesses within a single task while keeping model weights fixed. The Evaluator actively inspects artifacts and environment state with tools; its findings ground an internal signal for output selection and guide updates to both execution and verification procedures. Proposed Evaluators undergo comparative review on common candidates, while Actor updates require support from their own fresh outputs. Across AutomationBench, τ3 Banking, and selected GDPval tasks, VbD achieves the highest overall scores among the compared methods operating without external feedback, with gains of 3.3, 4.1, and 2.3 percentage points over the strongest baselines on the main backbone. Ablations support the contributions of active verification and Evaluator evolution, and selection from fixed candidate pools shows that improved verification can recover successful outputs without generating new ones.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. Solid, well-executed paper on within-task agent adaptation that deserves a serious referee. This line is wrong

5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 118 references say.

5 minor objections

The paper says, at Section 5.2 / Appendix B.3

Pith says: minor objection 1 of 5

The adaptation of RQGM uses a Beta(1+S, 1+F) quantile for utility estimation (Equation 7 in the appendix). It would be helpful to state whether this Beta is a prior-posterior or simply a heuristic, and whether the quantile parameter ε=0.05 was chosen post-hoc or justified by pilot experiments.

The paper says, at Section 5.1 / Tables 1–3

Pith says: minor objection 2 of 5

All main results report mean ± sample SD across three runs. For a claim about 'highest overall scores', a direct statistical comparison (e.g., paired bootstrap intervals or a randomization test) between VbD and the next-best baseline on each benchmark would strengthen the claim. The overlapping error bars make it unclear whether the observed differences are reliable.

show 3 more

The paper says, at Figure 3

Pith says: minor objection 3 of 5

The x-axis scale (cumulative tokens per task) is compressed; consider adding tick marks or switching to a logarithmic scale so that the cost progression is easier to read. The arrow labelling '5.6 pp' is not defined until the main text; the caption should be self-contained.

The paper says, at Section 4.3, Eq. (3)

Pith says: minor objection 4 of 5

The Evaluator retention rule requires agreement in both presentation orders, but the text does not specify how ties or undecided votes are handled. Clarify whether 'undecided' from the reviewer counts as a vote for neither, and whether the absence of a tested proposal (V_E false) always retains the incumbent.

The paper says, at Appendix C.9

Pith says: minor objection 5 of 5

The Finance024 failure case is instructive but its description refers to 'file timestamp as business date' without showing the exact tool output that caused the misreading. Adding a short excerpt of the retrieval result would help readers understand the boundary of reliable evidence interpretation.

What would settle it

Replicate the fixed-candidate Evaluator selection experiment (Figure 4b) using an alternative reviewer LLM (e.g., Qwen3.8-Flash-Next) instead of the same backbone. If the retained VbD Evaluator still outperforms the BoN and RQGM baselines by a similar margin, the robustness of the comparative review is further confirmed. If the margin erodes, the reviewer's model choice would be a load-bearing factor.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 13 Nov 2026

  1. Reader holds up confidence high
  2. Skeptic agrees 28 s · 39,604 tokens read, 3,416 written
  3. Editor wrote the verdict above 14 s · 40,926 tokens read, 597 written
  4. Referee minor revision: 5 minor 18 s · 39,849 tokens read, 2,146 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

This is a clean piece of work. The central idea—co-evolving the actor and the evaluator within a single task by using tool-based verification findings as the internal signal—is sensibly designed and honestly tested. The gains are modest (3–4 percentage points on AutomationBench and Banking, 2.3 on GDPval) but consistent across three benchmarks and two backbones, and the ablations show both active verification and evaluator evolution contribute. The paper’s structural decisions (comparative review under both presentation orders, separate retention rules for evaluator adoption, output selection, and actor adoption) address a real design fragility that prior methods like RQGM gloss over. The case studies are well chosen; they show both successes and a clear failure (Finance024, where a timestamp is misinterpreted). The limitations section is upfront about shared model errors, inspectability constraints, and the limited three-round scope—no overclaiming.

The soft spots are proportionate. The gains are real but not large; the method adds inference cost for verification, editing, and review. The three-run evaluation with SD is standard for this area, not a weakness. The biggest assumption—that the LLM-based comparative review reliably picks better evaluators—is acknowledged, and the fixed-candidate selection experiment provides reasonable supporting evidence. The paper does not claim this works for every task, and it does not. The within-task setting means harnesses reset per task, so there is no cross-task transfer; the authors flag this as untested. That is a scope note, not a flaw.

Who is this for? Researchers working on test-time adaptation for LLM agents, particularly those interested in methods that do not depend on external reward signals or human labels. The paper makes a credible case that improving execution and verification together is more effective than improving either alone. It deserves a serious referee. I would support acceptance with minor comments about clarifying the budget for the fixed-candidate evaluator selection experiment and noting that the three-round horizon limits the conclusions about longer-term co-evolution dynamics.

The strongest objection

The paper's central claim—that VbD achieves the highest scores among compared methods without external feedback on three benchmarks—is credible and well-supported by the presented experiments, ablations, case studies, and repeatability on a second backbone. The weakest assumption, as the reader identifies, is the reliability of the LLM-based comparative review (Section 4.3) and the separate retention rules. However, the paper acknowledges this limitation explicitly and provides evidence that the process works on average: the fixed-candidate evaluation shows improved selection by the evolved Evaluator, and the Actor gating analysis (Table 11) shows that accepted updates are usually improvements (precision 77.6%). The paper also documents failures (case C.9), which are expected and do not undermine the overall trend. The ablations and the cross-backbone replication indicate that the improvements are robust. No internal inconsistency or critical flaw that would overturn the central claim is evident from a careful reading. The paper honestly reports its limitations and does not overclaim.

The reader's weakest assumption is the reliability of the LLM-based comparative review and the retention rules. The paper acknowledges the risk and provides supporting evidence: the fixed-candidate experiment shows that the evolved Evaluator selects better outputs, and the Actor gating analysis shows high TNR. The case studies document both successes and failures, lending credibility. No unacknowledged or fatal weakness is present. The central claim is therefore supported, and the verdict should remain ACCEPT (UNCHANGED).

The referee's report

If the results hold, VbD makes a credible contribution to test-time agent adaptation: it couples the improvement of execution procedures with the improvement of the verification procedure that supplies the training signal, all within a single task and without external labels. The paper's strengths include thorough evaluation across three benchmarks and two LLM backbones, controlled ablations, a fixed-candidate analysis that isolates selection quality, detailed case studies, and explicit discussion of limitations (shared model errors, need for inspectable environments, three-round horizon). These design choices give the reader confidence that the gains are not artifacts of a single backbone or a brittle evaluation setup.

Pith's reading of the paper (a machine's summary, not the authors')

The paper introduces Verify by Doing (VbD), a method that allows an agent to improve its task performance at test time without any external feedback, such as benchmark scores or expert labels. Instead, VbD co-evolves two harnesses: an Actor that executes the task and an Evaluator that actively inspects artifacts and environment state using tools. The Evaluator's findings serve as an internal signal to select better outputs and to guide updates to both harnesses. Comparative review and separate retention rules ensure that changes to verification are tested before they affect output selection, and that Actor updates are only adopted when their own fresh output is strictly favored. Across AutomationBench, τ³ Banking, and selected GDPval tasks, VbD achieves the highest overall scores among methods that operate without external feedback, with gains of 3.3, 4.1, and 2.3 percentage points over the strongest baselines.

What carries the argument. The central machinery is the co-evolution of two editable harnesses (Actor and Evaluator) driven by active verification. The Actor harness contains instructions, skills, and working rubrics for task execution; the Evaluator harness contains instructions, rubrics, and skills for inspecting evidence. In each round, both the current and a proposed Actor execute the task, producing fresh candidates. Both the retained and a proposed Evaluator independently inspect the same candidate pool using tool calls to gather evidence about task completion. A fixed reviewer compares the two Evaluators' reports in both presentation orders; the Evaluator is adopted only if both orders favor it. The retained Evaluator then selects an output from eligible candidates. For Actor adoption, the proposed Actor's own fresh output must be the selected one, and both the initial comparison and the final comparison must show strict support. An editor uses the findings and history to propose updates for the next round. All tasks use frozen model weights; only the harness text changes.

Core claim. VbD is built on the premise that the verification procedure itself should be a target of test-time improvement, not a fixed component. It achieves this by making the Evaluator actively gather evidence through tool calls—inspecting cells, reading messages, recomputing values—rather than passively judging the Agent's trace. Those findings are then used to propose changes to both the Actor's instruction, skills, and checks, and to the Evaluator's own criteria and skills. The key innovation is that Evaluator updates are reviewed by comparing their reports on the same candidate pool in both presentation orders; an update is adopted only if both orders favor it. Actor updates require the additional condition that the proposed Actor's own fresh output be selected over the incumbent by the retained Evaluator. This separation prevents verification drift from enabling unwarranted Actor updates and allows improved verification to recover successful outputs that were already present in the candidate pool. In controlled experiments, VbD outperforms strong baselines including Red Queen Gödel Machine (which also co-evolves agent and evaluator) and other harness optimization methods, while rejecting 89.3% of observed regressions compared to 71.2% for the next best method. Ablations confirm that both active verification and evaluator evolution contribute positively, and fixed-candidate experiments show that the evolved Evaluator improves pass rate by 4.6 percentage points over the initial Evaluator by selecting better outputs already in the pool.

The premise it rests on. The load-bearing premise is that the LLM-based comparative review process can reliably distinguish better evaluation reports and that the strict retention rules prevent harmful updates, even though the paper acknowledges that the Actor, Evaluator, and reviewer may share errors in evidence interpretation, and the Finance024 case study shows a concrete failure where misinterpretation led to selecting a worse output.

If the paper is right (Pith's inferences, not the authors')

VbD's approach suggests that test-time agent improvement does not require external labels; the verification procedure itself can be adapted to provide a reliable internal signal.

By validating that improved verification can recover successful outputs from a fixed candidate pool, VbD implies that many failures in agent systems may stem from incomplete evaluation rather than poor generation.

The strict actor-adoption rule, which requires the new actor's output to be selected over the incumbent, provides a principled guard against regression that other methods lack.

If VbD's design scales to longer horizons and cross-task transfer, it could reduce the need for human-annotated validation sets in deployment.

The ablations show that both active verification and evaluator evolution are necessary; leaving either out hurts performance on every task set, indicating that the two components are complementary.

VbD's reliance on LLM-generated reports and comparative review could be vulnerable to systematic biases in the reviewer model, especially when both Evaluators share a blind spot. Future work might introduce adversarial evaluator generation or external ground-truth probes to test this.

The method's need for resettable environments and observable artifacts limits its applicability to stateless or irreversible tasks. An interesting extension would be to adapt VbD to partially observable or non-resettable settings by maintaining belief states or using reversible actions.

While the paper evaluates only three rounds per task, the trajectory of partial credit improvement suggests that continued co-evolution might yield further gains, though with diminishing returns and increasing computational cost. Longer-horizon experiments could test the sustainability of improvements.

The Finance024 failure shows that misinterpretation of evidence (a file timestamp as a business date) can lead to incorrect selection; a more robust Evaluator could incorporate uncertainty quantification or cross-check with multiple independent sources before reporting findings.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Agent self-improvement requires feedback about task success, yet external feedback may be unavailable at test time. Agents must then infer success from observable evidence, and incomplete verification can misguide subsequent updates. We introduce Verify by Doing (VbD), a method that co-evolves Actor and Evaluator harnesses within a single task while keeping model weights fixed. The Evaluator actively inspects artifacts and environment state with tools; its findings ground an internal signal for output selection and guide updates to both execution and verification procedures. Proposed Evaluators undergo comparative review on common candidates, while Actor updates require support from their own fresh outputs. Across AutomationBench, τ3 Banking, and selected GDPval tasks, VbD achieves the highest overall scores among the compared methods operating without external feedback, with gains of 3.3, 4.1, and 2.3 percentage points over the strongest baselines on the main backbone. Ablations support the contributions of active verification and Evaluator evolution, and selection from fixed candidate pools shows that improved verification can recover successful outputs without generating new ones.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-13

The authors' main claim, as the classifier picked it out 1.00

“Across AutomationBench, $\tau^3$ Banking, and selected GDPval tasks, VbD achieves the highest overall scores among the compared methods operating without external feedback, with gains of 3.3, 4.1, and 2.3 percentage points over the strongest baselines on the main backbone.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.32 · human participants 0.03 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.97 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Agent self-improvement requires feedback about task success, yet external feedback may be unavailable at test time. background 0.89 0.05 0.02 asserted 0.62
Agents must then infer success from observable evidence, and incomplete verification can misguide subsequent updates. background 0.83 0.05 0.02 asserted 0.79
We introduce Verify by Doing (VbD), a method that co-evolves Actor and Evaluator harnesses within a single task while keeping model weights fixed. contribution 0.86 0.87 0.10 asserted 1.00
The Evaluator actively inspects artifacts and environment state with tools; its findings ground an internal signal for output selection and guide updates to both execution and verification procedures. method 0.98 0.81 0.09 asserted 1.00
Proposed Evaluators undergo comparative review on common candidates, while Actor updates require support from their own fresh outputs. method 0.98 0.73 0.04 asserted 0.97
Across AutomationBench, $\tau^3$ Banking, and selected GDPval tasks, VbD achieves the highest overall scores among the compared methods operating without external feedback, with gains of 3.3, 4.1, and 2.3 percentage points over the strongest baselines on the main backbone. result 1.00 0.96 0.70 asserted 1.00
Ablations support the contributions of active verification and Evaluator evolution, and selection from fixed candidate pools shows that improved verification can recover successful outputs without generating new ones. result 0.91 0.96 0.15 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.91

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 5,640 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (14)
Figure 1 of the paper
Figure 1. Active verification of a budget workbook. The task requires correct payer assignments, consistent expense totals, and after-tax revenue. Initial checks inspect labels and headers; an updated Evaluator skill retrieves expense and revenue cells while the workbook remains selected. Offline recomputation checks expense totals and after-tax revenue from the retrieved cells. update the verification procedure itself. Incomplete verifi￾cation can leave blind spots unresolved and misguide later adaptation [Wang et al., 2026a]. Red Queen Gödel Machine jointly evolves agents and evaluators,…
Figure 2 of the paper
Figure 2. Verify by Doing overview. Verification findings provide evidence for the internal signal, guiding output selection and updates to both harnesses. Comparative review and fresh trials support separate adoption decisions for Evaluator and Actor changes. The contact-sheet example connects one finding to changes in both roles. procedures before they guide output selection and Actor retention decisions separately. 3 Problem Setting We study agent improvement within a task 𝑥, given its request, materials, and permitted tools. Each Actor candidate links an execution record, resulting state,…
Figure 4 of the paper
Figure 4. AutomationBench. (a) Actor update gating within each method’s own evolution runs: TNR and precision. (b) Evaluator selection from shared, fixed BoN pools: pass rate and partial credit. TP and FP count accepted improvements and regressions; TN counts rejected regressions. We report TNR = TN TN + FP , Precision = TP TP + FP . VbD rejects 89.3% of observed regressions compared with 71.2% for RQGM, while maintaining similar precision (77.6% versus 77.0%;
Figure 5 of the paper
Figure 5. Verification and harness updates across benchmarks. A shared evolution loop uses different evidence: application state, policy requirements, or deliverable contents. The examples connect execution requirements to checks of their observable effects
Figure 6 of the paper
Figure 6. shows VbD increasing from 66.35% to 70.26%, a gain of 3.91 percentage points. RHO increases from 62.83% to 66.83%. VbD exceeds BoN from Round 2 onward and Round 1 Round 2 Round 3 62 64 66 68 70 Partial credit (%) +1.63 pp 66.35 69.00 70.26 VbD BoN RQGM HGM RHI RHO
Figure 7 of the paper
Figure 7. Sales063: execution and verification across rounds. Columns trace HGM, RQGM, and VbD using DeepSeek-V4-Flash-0731 on the same AutomationBench task across three rounds. The comparison highlights the missed reply restriction
Figure 8 of the paper
Figure 8. Supplier strategy: verification and retention across methods and rounds. For DeepSeek-V4-Flash-0731 on GDPval, previews record page counts during execution; final DOCX lengths use a common renderer. VbD and RQGM meet the requested range. C.8 Success without Actor Adoption In the customer-health scoring task Sales047, a successful output is selected without adopting its Actor update (Fig￾ure 10). In Round 2, both nomination orders strictly favor the proposal for retrieving the scoring policy and verifying persisted updates. Both final comparisons declare equiva￾lence: the reports…
Figure 9 of the paper
Figure 9. Marketing088: adapting verification changes the feedback used for selection. Using DeepSeek-V4-Flash-0731 on AutomationBench with candidate availability fixed, the retained Evaluator distinguishes similar recorded business actions by checking final-response scope. It selects B in both presentation orders; self-preference and the initial Evaluator disagree and fall back to A. Online selection reasons and independent offline outcomes are shown separately. D Behavioral Dynamics of Evolution Figures 11–12 and
Figure 10 of the paper
Figure 10. Sales047: a passing output selected without Actor adoption. For DeepSeek-V4-Flash-0731 on AutomationBench, the final comparison lacks strict support for retention. Bottom-row outcomes come from offline checks unavailable during selection. RQGM shows a smaller change in Verify, while RHO remains relatively stable across all three categories. VbD combines reduced Search with increased Verify, a direc￾tion also seen in the external feedback reference. This pattern is consistent with VbD’s design: verification find￾ings guide both output assessment and checks performed during…
Figure 11 of the paper
Figure 11. Selected outcomes and Actor behavior in Rounds 1–3. For DeepSeek-V4-Flash-0731 on AutomationBench, the top row estimates passing selected outputs by tool-call count, normalized to round totals. The middle row shows actions per attempt in five-call bins; the bottom row magnifies Verify frequency. Dark gray curves show the Round 1 reference. Appendix D.3 defines the measurements. −20 −15 −10 −5 0 Change in calls / attempt Finance (100) HR (100) Marketing (100) Operations (100) Sales (100) Support (100) (a) Search calls −5 0 5 10 15 Change in attempts with Verify (pp) Finance (100) HR…
Figure 12 of the paper
Figure 12. Behavioral changes by domain. For DeepSeek-V4-Flash-0731 on AutomationBench, panels show changes from Round 1 to Round 3 in Search calls per attempt (left) and attempts with verification (right, pp). Whiskers show 95% bootstrap intervals over tasks. D.2 Task Outcome Visualization The task projection displays Qwen3.8-Flash-Next outcomes on individual AutomationBench tasks
Figure 13 of the paper
Figure 13. Actor behavior across rounds on AutomationBench. Curves show Search, Read, and Verify calls per attempt with Qwen3.8-Flash-Next for VbD, four evolution baselines, and the external feedback reference (dashed). Shading shows 95% bootstrap intervals over tasks. Appendix D.3 details the measurements. TF-IDF task descriptions using 50-dimensional Truncat￾edSVD, normalization, and t-SNE with PCA initialization and perplexity 35. Both panels share 200 tasks and coor￾dinates; outcome labels determine display inclusion and annotations, while the projection uses task text. Orange highlights…
Figure 14 of the paper
Figure 14. Task outcomes in t-SNE space. Using Qwen3.8- Flash-Next on 200 AutomationBench tasks, panels compare (a) initialization with final selection and (b) VbD with self-preference. Orange (Hard) marks VbD successes missed by the reference with external feedback in this comparison of individual tasks; gray (Normal) provides context. Green/red rings indicate label gains/losses across the same tasks. D.3 Behavioral Measurement Protocol For each method and backbone, we analyze both Actor executions per round on all 600 AutomationBench tasks across three runs, including failed attempts and…

Showing 13 of the paper's 14 figures. Figures are the authors', reproduced from the paper.

References (118)
  1. [1] Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. GEPA : Refle... on PithPith's reviewarXiv
  2. [2] Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self- RAG : Learning to retrieve, generate, and critique through self-reflection. arXiv preprint arXiv:2310.11511, 2023. URL https://arxiv.org/abs/2310.11511 on PithPith's reviewarXiv
  3. [4] Chaoqun Cui, Jing Huang, Shijing Wang, Liming Zheng, Qingchao Kong, and Zhixiong Zeng. Agentic reward modeling: Verifying GUI agent via progressive trajectory-grounded interaction. arXiv preprint arXiv:2602.00575, 2026. URL https://arxiv.org/abs/2602.00575 on PithPith's reviewarXiv
  4. [5] DeepSeek-AI . DeepSeek-V4-Flash-0731 : Model card. Hugging Face model repository, 2026 a . URL https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731 on Pith
  5. [6] DeepSeek-AI . DeepSeek-V4-Flash : Model card. Hugging Face model repository, 2026 b . URL https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash on Pith
  6. [7] Hongxin Ding, Baixiang Huang, Yue Fang, Weibin Liao, Zheng Li, Jinyang Zhang, Zhijing Wu, Junfeng Zhao, and Yasha Wang. EvoRubrics : Dynamic rubrics as rewards via adversarial co-evolution for LLM reinforcement learning. arXiv preprint arXiv:2606.23038, 2026. URL https://arxiv... on PithPith's reviewarXiv
  7. [8] Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 10835--10866. PMLR, 2023. URL https://proceedings.ml... on Pith
  8. [9] Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC : Large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations , 2024 on Pith
  9. [10] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2310.01798 on PithPith's reviewarXiv
  10. [11] Alex Iacob, Andrej Jovanovic, William F. Shen, Daniel Burkhardt, Meghdad Kurmanji, Nurbek Tastan, Lorenzo Sani, Niccolo Alberto Elia Venanzi, Ambroise Odonnat, Zeyu Cao, Bill Marino, Xinchi Qiu, and Nicholas D. Lane. The red queen G \"o del machine: Co-evolving agents and thei... on PithPith's reviewarXiv
  11. [12] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy : Compiling declarative language model calls into self-impr... on PithPith's reviewarXiv
  12. [13] Jan Hendrik Kirchner, Yining Chen, Harri Edwards, Jan Leike, Nat McAleese, and Yuri Burda. Prover-verifier games improve legibility of LLM outputs. arXiv preprint arXiv:2407.13692, 2024. URL https://arxiv.org/abs/2407.13692 on PithPith's reviewarXiv
  13. [15] Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In International Conference on Learning Representations, 2024. URL https://arxiv.org/abs/2305.20050 on PithPith's reviewarXiv
  14. [16] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-Refine : Iterati... on PithPith's reviewarXiv
  15. [17] Jun Nie, Yonggang Zhang, Jun Song, Qianshu Cai, Dahai Yu, Yike Guo, Xinmei Tian, and Bo Han. TTHE : Test-time harness evolution. arXiv preprint arXiv:2607.08124, 2026 on PithPith's reviewarXiv
  16. [20] Qwen Team . Qwen3.8-Flash-Next : Model card and inference configuration. Hugging Face model repository, 2026. URL https://huggingface.co/Qwen/Qwen3.8-Flash-Next on Pith
  17. [22] Quan Shi, Alexandra Zytek, Pedram Razavi, Karthik Narasimhan, and Victor Barres. -Knowledge : Evaluating conversational agents over unstructured knowledge. arXiv preprint arXiv:2603.04370, 2026. URL https://arxiv.org/abs/2603.04370 on PitharXiv
  18. [24] Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. In Advances in Neural Information Processing Systems, volume 35, 2022. URL https://papers.neurips.cc/paper_files/paper/2022/hash/3d719fee332caa23d5038b8a90e81796-A... on Pith
  19. [25] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In International Conference on Learning Representations , 2025. URL https://arxiv.org/abs/2408.03314 on PithPith's reviewarXiv
  20. [26] Binghai Wang, Chenlong Zhang, Dayiheng Liu, Jiajun Zhang, Jiawei Chen, Mingze Li, Mouxiang Chen, Rongyao Fang, Siyuan Zhang, Xuwu Wang, Yuheng Jing, Zeyao Ma, and Zeyu Cui. The verification horizon: No silver bullet for coding agent rewards. arXiv preprint arXiv:2606.26300, 20... on PithPith's reviewarXivdoi:10.48550/arxiv.2606.26300
show 80 more
  1. [28] Wenyi Wang, Piotr Piekos, Li Nanbo , Firas Laakom, Yimeng Chen, Mateusz Ostaszewski, Mingchen Zhuge, and J \"u rgen Schmidhuber. Huxley-- G \"o del machine: Human-level coding agent development by an approximation of the optimal self-improving machine. In International Confere... on PitharXiv
  2. [29] Siwei Wu, Jincheng Ren, Yizhi Li, Haau-Sing Li, Chengran Yang, Yuxuan Zhang, Weicheng Gu, Jian Yang, Riza Batista-Navarro, Chuanyi Zhang, Xianglong Liu, Ming Zhou, Bryan Dai, and Chenghua Lin. ModularRSI : Modular and generalizable recursive harness self-improvement. arXiv pre... on PithPith's reviewarXiv
  3. [30] Tianhao Wu, Weizhe Yuan, Olga Golovneva, Jing Xu, Yuandong Tian, Jiantao Jiao, Jason Weston, and Sainbayar Sukhbaatar. Meta-rewarding language models: Self-improving alignment with LLM -as-a-meta-judge. arXiv preprint arXiv:2407.19594, 2024 a . URL https://arxiv.org/abs/2407.19594 on PithPith's reviewarXiv
  4. [31] Zhenyu Wu, Qingkai Zeng, Zhihan Zhang, Zhaoxuan Tan, Chao Shen, and Meng Jiang. Large language models can self-correct with key condition verification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 12846--12867. Association fo... on Pithdoi:10.18653/v1/2024.emnlp-main.714
  5. [32] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct : Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2210.03629 on PithPith's reviewarXiv
  6. [33] Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Xian Li, Sainbayar Sukhbaatar, Jing Xu, and Jason E. Weston. Self-rewarding language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages... on Pith
  7. [35] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin G \"odel machine: Open-ended evolution of self-improving agents. In International Conference on Learning Representations (ICLR) , 2026 b . URL https://openreview.net/forum?id=pUpzQZTvGY on Pith
  8. [37] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow : Automating agentic workflow generation. In International Conference on Learning Repre... on Pith
  9. [38] Xing Zhang, Guanghui Wang, Yanwei Cui, Ziyuan Li, Wei Qiu, Bing Zhu, and Peiyang He. Who grades the grader? co-evolving evaluation metrics and skills for self-improving LLM agents. arXiv preprint arXiv:2607.12790, 2026 d on PithPith's reviewarXiv
  10. [39] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LLM -as-a-judge with MT -bench and chatbot arena. In Advances in Neural Information Proce... on PithPith's reviewarXiv
  11. [40] Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning, acting, and planning in language models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine ... on Pith
  12. [41] Yajie Zhou, Ao Li, Ashwin Silla, Zaoxing Liu, and Vyas Sekar. AIChilles : Automatically uncovering hidden weaknesses in ai-evolved systems. arXiv preprint arXiv:2606.15834, 2026 on PithPith's reviewarXiv
  13. [42] Mingchen Zhuge, Changsheng Zhao, Dylan R. Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, and J \"u rgen Schmidhuber. Agent-as-a-judge: Evaluate agents with agents. In Procee... on Pith
  14. [43] JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolution , author =. arXiv preprint arXiv:2608.25593 , year = on PithPith's reviewarXiv
  15. [44] AutomationBench , author =. arXiv preprint arXiv:2604.18934 , year = on PithPith's reviewarXiv
  16. [45] Aspire: Can Models Self-Evolve from Vague Goals? , author =. arXiv preprint arXiv:2608.31111 , year = on PithPith's reviewarXiv
  17. [46] Starace, Giulio and Jaffe, Oliver and Sherburn, Dane and Aung, James and Chan, Jun Shern and Maksin, Leon and Dias, Rachel and Mays, Evan and Kinsella, Benjamin and Thompson, Wyatt and Heidecke, Johannes and Glaese, Amelia and Patwardhan, Tejal , journal =. PaperBench: Evaluating on Pith
  18. [47] Zhang, Jenny and Hu, Shengran and Lu, Cong and Lange, Robert and Clune, Jeff , booktitle =. Darwin. 2026 , url = on Pith
  19. [48] Agrawal, Lakshya A. and Tan, Shangyin and Soylu, Dilara and Ziems, Noah and Khare, Rishi and Opsahl-Ong, Krista and Singhvi, Arnav and Shandilya, Herumb and Ryan, Michael J. and Jiang, Meng and Potts, Christopher and Sen, Koushik and Dimakis, Alexandros G. and Stoica, Ion and ... on Pith
  20. [49] Inspect. 2024 , howpublished = on Pith
  21. [50] Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations , author =. arXiv preprint arXiv:2411.00640 , year = on PithPith's reviewarXiv
  22. [51] Enrichment Strategies for Clinical Trials to Support Approval of Human Drugs and Biological Products: Guidance for Industry , author =. 2019 , note = on Pith
  23. [52] Principal Stratification in Causal Inference , author =. Biometrics , volume =. 2002 , note = on Pith
  24. [53] Adaptive Increase in Sample Size when Interim Results are Promising: A Practical Guide with Examples , author =. Statistics in Medicine , volume =. 2011 , note = on Pith
  25. [54] Increasing the Sample Size when the Unblinded Interim Result is Promising , author =. Statistics in Medicine , volume = on Pith
  26. [55] Prediction-Powered Inference , author =. Science , volume =. 2023 , note = on Pith
  27. [56] Iacob, Alex and Jovanovic, Andrej and Shen, William F. and Burkhardt, Daniel and Kurmanji, Meghdad and Tastan, Nurbek and Sani, Lorenzo and Venanzi, Niccolo Alberto Elia and Odonnat, Ambroise and Cao, Zeyu and Marino, Bill and Qiu, Xinchi and Lane, Nicholas D. , journal =. The... on Pith
  28. [57] The Verification Horizon: No Silver Bullet for Coding Agent Rewards , author =. arXiv preprint arXiv:2606.26300 , year = on PithPith's reviewarXiv
  29. [58] Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference , author =. arXiv preprint arXiv:2606.05922 , year = on PithPith's reviewarXiv
  30. [59] On the Role of Preference Variance in Preference Optimization , author =. arXiv preprint arXiv:2510.13022 , year = on PitharXiv
  31. [60] Zhou, Yajie and Li, Ao and Silla, Ashwin and Liu, Zaoxing and Sekar, Vyas , journal = on Pith
  32. [61] Recursive Harness Self-Improvement , author =. arXiv preprint arXiv:2607.15524 , year = on PithPith's reviewarXiv
  33. [62] Nie, Jun and Zhang, Yonggang and Song, Jun and Cai, Qianshu and Yu, Dahai and Guo, Yike and Tian, Xinmei and Han, Bo , journal = on Pith
  34. [63] Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification , author =. arXiv preprint arXiv:2608.27311 , year = on PithPith's reviewarXiv
  35. [64] Self-Harness: Harnesses That Improve Themselves , author =. arXiv preprint arXiv:2606.09498 , year = on PithPith's reviewarXiv
  36. [65] Chen, Mengzhuo and Wang, Junjie and Liu, Zhe and Wang, Yawen and Zheng, Haiming and Wang, Qing , journal =. From Failed Trajectories to Reliable on Pith
  37. [66] Zhang, Xing and Wang, Guanghui and Cui, Yanwei and Li, Ziyuan and Qiu, Wei and Zhu, Bing and He, Peiyang , journal =. Who Grades the Grader? Co-Evolving Evaluation Metrics and Skills for Self-Improving on Pith
  38. [67] Liu, Boyang and Jin, Senjie and Wang, Peixin and Yin, Zhangyue and Wang, Yibo and Zhou, Yuhao and Zhang, Zhihao and Liang, Xinbing and Zhu, Shizheng and Wang, Yuhui and Tong, Jingqi and Zhu, Dingwei and Xi, Zhiheng and Zhang, Jiazheng and Bai, Clive and. arXiv preprint arXiv:2... on PithPith's reviewarXiv
  39. [68] Hu, Yihao and Wen, Zhihao and Liu, Xiujin and Wang, Pan and Zhang, Xin and Wu, Wei , journal =. 2026 , url = on Pith
  40. [69] Metamorphic Testing: A New Approach for Generating Next Test Cases , author = on Pith
  41. [70] Metamorphic Testing: A Review of Challenges and Opportunities , author =. ACM Computing Surveys , volume =. 2018 , doi = on Pith
  42. [71] Liang, Linfeng and Tan, Chenkai and Deng, Yao and Cai, Yingfeng and Chen, T. Y. and Zheng, Xi , journal = on Pith
  43. [72] Khan, Shehroz and Mughees, Abdullah and Sudheerbabu, Gaadha and Ahmad, Tanwir and Truscan, Dragos , journal =. Multi-Agent on Pith
  44. [73] Bidirectional Empowerment of Metamorphic Testing and Large Language Models: A Systematic Survey , author =. arXiv preprint arXiv:2605.13898 , year = on PithPith's reviewarXiv
  45. [74] Zhang, Jiayi and Xiang, Jinyu and Yu, Zhaoyang and Teng, Fengwei and Chen, Xionghui and Chen, Jiaqi and Zhuge, Mingchen and Cheng, Xin and Hong, Sirui and Wang, Jinlin and Zheng, Bingnan and Liu, Bang and Luo, Yuyu and Wu, Chenglin , booktitle = on Pith
  46. [75] Zhao, Bingchen and Zhang, Jenny and Whitehouse, Chenxi and Jiang, Minqi and Shvartsman, Michael and Charnalia, Abhishek and Magka, Despoina and Shavrina, Tatiana and Dunfield, Derek and Mac Aodha, Oisin and Bachrach, Yoram , journal = on Pith
  47. [76] Yan, Sikuan and Bahloul, Ahmed and Nie, Ercong and Schwarzmann, Susanna and Trivisonno, Riccardo and Tresp, Volker and Ma, Yunpu , journal = on Pith
  48. [77] Jiang, Wen and Chu, Mingmin and Tian, Yimeng and Zhang, Qianxin and Yang, Haofei and Yang, Rui and Liu, Yang and Lv, Tao and Li, Fangming , journal = on Pith
  49. [78] Chen, Yifan and Li, Haitao and Ai, Qingyao and Zhu, Fengbin and Chua, Tat-Seng and Zhang, Min and Liu, Yiqun , journal = on Pith
  50. [79] Zhu, Jiayin and Mao, Kelong and Guo, Yudong and He, Dengbo and Xu, Sulong and Gu, Simiu and Yue, Yutao , journal = on Pith
  51. [80] Guan, Xin and Hu, Xiaomeng and Huang, Shen and Wang, Zhenyi and Zhang, Bo and Li, Zijian and Xie, Pengjun and Liu, Bo and Cao, Jiuxin , journal = on Pith
  52. [81] Recursive Experiential--Working Memory Evolution for Long-Horizon Agent Harnesses , author =. arXiv preprint arXiv:2608.24876 , year = on PithPith's reviewarXiv
  53. [82] Mi, Qirui and Ma, Zhijian and Yang, Mengyue and Li, Haoxuan and Wang, Yisen and Zhang, Haifeng and Wang, Jun , booktitle = on Pith
  54. [83] Policy-Conditioned Counterfactual Credit for Verifiable Reinforcement Learning of Long-Horizon Language Agents , author =. arXiv preprint arXiv:2606.05263 , year = on PithPith's reviewarXiv
  55. [84] Rethinking the Evaluation of Harness Evolution for Agents , author =. arXiv preprint arXiv:2607.12227 , year = on PithPith's reviewarXiv
  56. [85] Cui, Chaoqun and Huang, Jing and Wang, Shijing and Zheng, Liming and Kong, Qingchao and Zeng, Zhixiong , journal =. Agentic Reward Modeling: Verifying. 2026 , url = on Pith
  57. [86] Zhang, Jiazheng and Fu, Ziche and Xi, Zhiheng and Jing, Wenqing and Chai, Mingxu and He, Wei and Zhang, Guoqiang and Fan, Chenghao and An, Chenxin and Chen, Wenxiang and Liu, Zhicheng and Pan, Haojie and Zhu, Dingwei and Gui, Tao and Zhang, Qi and Huang, Xuanjing , journal = on Pith
  58. [87] Verifiable Process Rewards for Agentic Reasoning , author =. arXiv preprint arXiv:2605.10325 , year = on PithPith's reviewarXiv
  59. [88] DeepSeek Harness , author =. 2026 , howpublished = on Pith
  60. [89] Dream-RSI: Recursive Self-Improvement through Evolving Worlds , author =. arXiv preprint arXiv:2609.14858 , year = on PithPith's reviewarXiv
  61. [90] Madaan, Aman and Tandon, Niket and Gupta, Prakhar and Hallinan, Skyler and Gao, Luyu and Wiegreffe, Sarah and Alon, Uri and Dziri, Nouha and Prabhumoye, Shrimai and Yang, Yiming and Gupta, Shashank and Majumder, Bodhisattwa Prasad and Hermann, Katherine and Welleck, Sean and Y... on Pith
  62. [91] Reflexion: Language Agents with Verbal Reinforcement Learning , author =. arXiv preprint arXiv:2303.11366 , year = on PithPith's reviewarXiv
  63. [92] Gou, Zhibin and Shao, Zhihong and Gong, Yeyun and Shen, Yelong and Yang, Yujiu and Duan, Nan and Chen, Weizhu , booktitle = on Pith
  64. [93] Wang, Wenyi and Piekos, Piotr and. Huxley--. International Conference on. 2026 , url = on Pith
  65. [94] Shi, Quan and Zytek, Alexandra and Razavi, Pedram and Narasimhan, Karthik and Barres, Victor , journal =. 2026 , url = on Pith
  66. [95] Xia, Peng and Han, Rujun and Wang, Zifeng and Chen, Yanfei and Zhuang, Yufan and Lee, Yoonho and Huang, Chengsong and Yu, Han and CuiZhu, Zhongying and Ming, Yifei and Yao, Huaxiu and Gokturk, Burak and Pfister, Tomas and Lee, Chen-Yu , journal=. 2026 , url= on Pith
  67. [96] Srikanth, Dhruv and Zhao, Bingchen and Xu, Dixing and Wu, Yuxiang and Jiang, Zhengyao , journal=. Recursive Self-Improvement of. 2026 , url= on Pith
  68. [97] Wang, Zefeng and Yan, Minxi and Bi, Jinhe and Yan, Sikuan and Tresp, Volker and Ma, Yunpu , journal=. 2026 , url= on Pith
  69. [98] Fan, Tianyu and Huang, Chao , journal=. 2026 , url= on Pith
  70. [99] Measuring the Performance of Our Models on Real-World Tasks , author=. 2025 , url= on Pith
  71. [100] Asai, Akari and Wu, Zeqiu and Wang, Yizhong and Sil, Avirup and Hajishirzi, Hannaneh , journal =. Self-. 2023 , url = on Pith
  72. [101] Self-Rewarding Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , url = on Pith
  73. [102] Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models , author =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , url = on Pith
  74. [103] Zeng, Aohan and Liu, Mingdao and Lu, Rui and Wang, Bowen and Liu, Xiao and Dong, Yuxiao and Tang, Jie , booktitle =. 2024 , url = on Pith
  75. [104] Chen, Zehui and Liu, Kuikun and Wang, Qiuchen and Zhang, Wenwei and Liu, Jiangning and Lin, Dahua and Chen, Kai and Zhao, Feng , booktitle =. 2024 , url = on Pith
  76. [105] Patwardhan, Tejal and Dias, Rachel and Proehl, Elizabeth and Kim, Grace and Wang, Michele and Watkins, Olivia and. arXiv preprint arXiv:2510.04374 , year = on PithPith's reviewarXiv
  77. [106] Snell, Charlie and Lee, Jaehoon and Xu, Kelvin and Kumar, Aviral , booktitle =. Scaling. 2025 , url = on Pith
  78. [107] Large Language Monkeys: Scaling Inference Compute with Repeated Sampling , author =. arXiv preprint arXiv:2407.21787 , year = on PithPith's reviewarXiv
  79. [108] Dror, Rotem and Baumer, Gili and Shlomov, Segev and Reichart, Roi. The Hitchhiker ' s Guide to Testing Statistical Significance in Natural Language Processing. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018... on Pithdoi:10.18653/v1/p18-1128
  80. [109] Scaling Laws for Reward Model Overoptimization , author =. Proceedings of the 40th International Conference on Machine Learning , series =. 2023 , url = on Pith

The first 100 of 118 references are listed here. All of them are in this paper's citation record.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment