Pith.

Machine Learning · arXiv:2610.16491 v2 · submitted 12 Oct 2026

Drafting Agents, Not Just Tokens: Speculative Recursive Self-Improvement for Agentic Systems

Xin Xu, Ke Cheng, Hao Wang, Yangru Huang, Yanghao Xiao, Zhichao Chen, Zhisheng Zhong, Xun Liu

show 6 more authors

Yi Li, Qian Li, Haibin Lv, Jun Zhang, Jie Jiang, Zhouchen Lin

The paper says

“Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget.”

more of the abstract

A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version often exhausts the budget before those sequences yield a verified improvement. Inspired by the draft–verify principle of speculative decoding, we propose Speculative Recursive Self-Improvement (SpecRSI), a two-level framework coupling inner-loop agent search with outer-loop improver learning. SpecRSI drafts successors before their predecessors are fully evaluated, while adopting only fully evaluated, admissible versions. In the inner loop, Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. In the outer loop, Group Verification Optimization (GVO) updates the improver from completed searches while keeping the real evaluator fixed. We evaluate SpecRSI on scientific optimization (including active queue management (AQM) for network infrastructure), coding, and general agent-system benchmarks. Under matched evaluation-call budgets, SpecRSI improves scientific terminal gain over Janus by 60.0%–96.3% and area under the budget curve by 32.1%–91.6%, while achieving higher means across all 50 scientific backbone–task pairs and outperforming MetaRSI-v1 by 1.30–2.70 percentage points (pp) in system score across twenty backbones. On instance-level comparisons, it further improves PyBaMM terminal gain by 142.6% and SWE-Pro success by 6.0 pp at 7.0% and 9.2% lower token cost, respectively. These results demonstrate the efficacy of the proposed approach.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 14 Oct 2026 · of version 1 of the paper; version 2 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

The passes disagreed. Reader: holds up. Skeptic: agrees with the reader. Referee: major revision.

SpecRSI is a novel and well-tested framework for agent self-improvement with speculative drafting, but the generalization of the continuation scorer to unseen backbones is not directly evaluated, and the reported lifecycle cost advantage is modest. This line is wrong

3 major objections · 5 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at §3.3.2, §4.2, Appendix C

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The continuation scorer bQ (Section 3.3.2) is central to VATS budget allocation, yet its prediction accuracy on states from backbones not used during training is not directly evaluated. The paper reports RMSE of 0.85 (Table 2(b)) on training data and ablations on the reference backbone (Figure 4), but these do not isolate scorer calibration on held-out backbones (e.g., GPT-6-Astra).

more of objection 1

Table S11 reports diagnostics for an 'evolved evaluator', not the final continuation scorer. Since the cross-backbone claim (expansion from §4.2: outperforms MetaRSI-v1 on each of twenty backbones) is a key contribution, the authors should either (a) provide a direct calibration study of bQ's predictions against realized returns on backbones excluded from training, or (b) explicitly discuss the expected robustness of the method to miscalibration and why the observed across-backbone advantage is unlikely to be an artifact of poor scorer generalization.

The paper says, at §4.5, Table 3(b)

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The lifecycle cost advantage over MetaRSI-v1 is 5.7% (total-cost index 106.3 vs. 112.7), which is modest given the increased algorithmic complexity of SpecRSI. The paper should discuss the practical significance of this margin: whether it is statistically significant across seeds, whether it is consistent across backbones, and whether the complexity overhead of VATS+GVO is justified by this improvement.

more of objection 2

Table S2 shows a 7.2% cost premium over Janus; the lifecycle ledger uses a different cost endpoint, making the net benefit harder to evaluate. Clarify the relationship between these cost figures.

The paper says, at §G.4.2, Proposition 7

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

The three-check reversal result is theoretically clean but relies on a highly specific finite setting with checks, dispatch costs, and known conditional laws. The paper should discuss how this result connects to the full VATS method: e.g., whether similar scope reversals occur in the real agent-search experiments, or whether the result primarily motivates the design of the scorer and bundle construction rather than providing a testable prediction.

more of objection 3

The current connection is implicit; a more explicit bridging paragraph would strengthen the theoretical narrative.

5 minor objections

The paper says, at Abstract

Pith says: minor objection 1 of 5

The abstract states 'In the main scientific comparison, terminal gains exceed Janus by 33.3%–40.0%' but does not define what 'main scientific comparison' refers to until Section 4.2. Consider adding brief context.

The paper says, at §4.5, Table 3(a)

Pith says: minor objection 2 of 5

The terms 'Overhead', 'Missing (%)', and 'Net' are defined in the table caption but would benefit from a sentence explaining each. In particular, 'Net is not recomputed as Gain minus Overhead' is confusing—what is Net then?

show 3 more

The paper says, at Figure 2

Pith says: minor objection 3 of 5

'Macro-average score' is used but defined only implicitly (text says 'four-task macro-average score across twenty backbones'). Add a footnote or figure caption explaining which four tasks and how averaging is performed.

The paper says, at General

Pith says: minor objection 4 of 5

The paper alternates between 'SpecRSI (ours)' and 'Ours' in figures and tables. Unify the label for consistency (e.g., 'SpecRSI (ours)' everywhere).

The paper says, at §F.5

Pith says: minor objection 5 of 5

Equation (21) states 'The input manuscript does not identify these settings for the reported deployment'. While honest, this leaves the reader unsure whether KL regularization was actually used. Please specify the β_KL value used (even if zero) for the main results.

What would settle it

On a held-out search from the Qwen3.5-35B-A3B backbon with the final trained improver, collect all paid states S_t and realized returns G_t. Then, on the same held-out state distribution, run searches using the same frozen improver but with a different backbon (GPT-6-Astra). For each state, record bQ predictions and realized G_t. Compute Spearman rank correlation across all states. Also compute pairwise accuracy: for states with ≥2 legal plan alternatives, what fraction of the time does the plan with highest bQ also have highest realized G_t? Report these metrics. If Spearman correlation <0.3 or pairwise accuracy ≤0.55 (binomial test aginst chance, n≥50), the scorer is insufficiently calibrated to support the claim that V ATS's budget allocation drives cross-backbone gains.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 14 Oct 2026

  1. Reader holds up confidence moderate
  2. Skeptic agrees 39 s · 47,124 tokens read, 4,159 written
  3. Editor wrote the verdict above 22 s · 47,992 tokens read, 2,013 written
  4. Referee major revision: 3 major objections, 5 minor 25 s · 47,481 tokens read, 2,926 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The paper's core idea—drafting successor agents before their parents are fully verified—is a genuine extension of speculative decoding to agent revision, and the two-level loop with VATS and GVO is new. The empirical coverage is impressive: 50 backbone-task pairs, 20 models, extensive ablations, and released code. The results are consistent and the method outperforms Janus and MetaRSI-v1 across multiple domains.

The main soft spot is the continuation scorer bQ. The stress-test concern about its calibration on held-out backbones is fair. Table S11 reports diagnostics for an 'evolved evaluator,' but the paper does not directly measure bQ's prediction accuracy on states from a backbone not used during training (e.g., GPT-6-Astra). The ablations in Figure 4 remove scorer refinement or replace it with greedy selection, which degrades performance, but those are on the reference backbone. If the scorer generalizes poorly, VATS budget allocation could become suboptimal on other backbones. The paper acknowledges this indirectly by noting that predictions never replace real evaluation, and the overall gain still holds with real verification, so it is not a load-bearing flaw, but it deserves scrutiny.

Other soft spots are minor: the 5.7% lifecycle cost advantage over MetaRSI-v1 is modest, and some planning ablation gains are small (e.g., 4.73 to 5.99 pp at 70% budget). The paper is honest about these—it reports standard deviations and does not overclaim.

Overall, this is a serious contribution to automated agent improvement. The method is well-motivated, the experiments are thorough, and the writing is clear. The open code and detailed appendices support reproducibility. I would bring this to a reading group focused on agentic systems or automated RL, and I think a serious editor should send it to peer review. I would not cite it in my own work in the next 12 months because it is not directly in my area, but I would recommend it to colleagues working on RSI.

The strongest objection

The learned continuation scorer bQ (Section 3.3.2) is central to V ATS budget allocation. The paper's ablation studies (Section 4.3, Figure 4) show that removing scorer refinement or replacing it with greed selection reduces gain, but these ablations are performed with the reference backbone (Qwen3.5-35B-A3B) and do not isolate the scorer's prediction accuracy. A direct hold-out evaluation of bQ's predictions against realized net returns G_t on states from a backbone not used during training (e.g., GPT-6-Astra) is missing. Table S11 reports predictive diagnostics for an older 'evolved evaluator', not for the final continuation scorer. The scorer's claimed RMSE of 0.85 (from Table 2(b)) is on training data and may not reflect out-of-distribution performance. If the scorer is poorly calibrated on unseen backbones, the verification-aware search could allocate budget suboptimally, undermining the claim that SpecRSI's advantage persists across twenty system backbones. This is load-bearing because the method's entire planning signal depends on accurate continuation valuation.

The reader identifies the learned scorer's accuracy as the weakest assumption, and I concur that this is the single most load-bearing cncern for the central claim. However, the paper provides substantial indirect evidence that the scorer contributes meaningfully: ablation studies (Section 4.3, Figure 4) show that components depending on the scorer outperform alternatives, and the cross-backbon evaluation (Section 4.2, Figure 2) demonstrates consistent gains across twenty backbones, which would be unlikely if the scorer were catastrophcally miscalibrated on new backbones. The paper acknowledges that the scorer is trained on data from the reference backbone and that actor updates require recalibration (Section 3.4). The proposed concrete test would provide a direct check of the scorer's generalization, but the current evidence is sufficiently strong to maintain the ACCEPT verdict. Therefore I recommend no change.

The referee's report

If the results hold, SpecRSI represents a meaningful advance in recursive self-improvement for agents by decoupling drafting from verification and jointly selecting edits and check bundles under a budget. The method is evaluated across a wide range of backbones and tasks, with controlled ablations separating the contributions of work scope, planning, and learning. The theoretical analysis in Appendix G.4.2 provides an elegant demonstration that compressed success summaries are insufficient for optimal verification scheduling, motivating the verification-aware design. However, the lifecycle cost advantage over the strongest baseline (MetaRSI-v1) is modest (5.7%), and the central reliance on a learned continuation scorer whose cross-backbone calibration is not directly evaluated tempers the significance until that gap is addressed.

Pith's reading of the paper (a machine's summary, not the authors')

The paper introduces SpecRSI, a two-level framework for agent improvement that drafts successor agents before their predecessors are fully evaluated, but requires complete verification before adoption. The inner loop uses Verification-Aware Tree Search (VATS) to allocate budget between further edits and outstanding checks based on predicted net verified return. The outer loop uses Group Verification Optimization (GVO) to learn the improver from completed searches. In scientific tasks, terminal gains exceed Janus by 33.3\u201340.0%, and across twenty backbones SpecRSI outperforms MetaRSI-v1 by 1.85\u20132.38 percentage points in macro-average system score. The key insight is that separating drafting from verification, while jointly planning edits and checks, turns exploration into verified progress more efficiently.

What carries the argument. The core mechanism is Verification-Aware Tree Search (VATS), which at each step constructs feasible completion bundles (sets of verification checks) and draft plans, scores each continuation with a learned function $\hat{Q}_\phi(S, \omega)$ predicting net verified gain net of future expense and time, and then selects the next action via a learned policy $\pi_\psi$. The scoring function $\hat{Q}$ evaluates state\u2013work pairs, not just candidate quality, and is trained by Group Verification Optimization (GVO) using realized suffix returns and pairwise comparisons from independent complete searches. The framework also freezes the verification contract $\Gamma$ and the improver during each inner search, updating the improver only after a group of searches is complete.

Core claim. SpecRSI's central claim is that an agent improvement system can achieve better verified gains under a fixed budget by permitting executable successor drafts to be generated before their parents are fully evaluated, while still insisting on complete verification before any version is adopted. The verification-aware tree searcher (VATS) predicts the net verified return of continuing to edit and check from a given state, and uses that prediction to allocate the remaining budget. This allows the system to explore promising lines of edits without committing to full evaluation of every intermediate step, and to bundle together complementary checks that individually look poor but together unlock a gain. The group verification optimizer (GVO) then updates the improver\u2014drafter, scorer, and policy\u2014using the difference between independent complete searches as a learning signal. Empirical evidence across scientific discovery, system tasks, and code/prompt/workflow optimization shows that SpecRSI consistently produces better successors than baselines that either maintain a single fully-evaluated lineage or evaluate all branches, while often operating at a lower or comparable total cost.

The premise it rests on. The learned continuation scorer $\hat{Q}_\phi$ must accurately predict the net verified gain of further edits and verification checks from any state, because if the scorer is poorly calibrated, VATS may waste budget on unproductive branches or miss valuable ones.

If the paper is right (Pith's inferences, not the authors')

If SpecRSI is correct, agent self-improvement under budget constraints can be formulated as a search over edit trajectories with a learned continuation-value model, analogous to speculative decoding for tokens.

The principle of drafting successors before full evaluation could be applied to other domains with expensive verification, such as automated theorem proving or experimental design.

The group verification optimization (GVO) provides a scalable way to train improvers from multiple complete searches, suggesting that agent improvement can be learned rather than hand-tuned.

The empirical finding that updated improvers outperform even the same released artifact supports the idea that improvement is transferable across tasks and backbones.

The verification-aware budget allocation might generalize to other resource-constrained sequential decision problems, such as hyperparameter optimization or curriculum learning.

The speculative drafting idea could be adapted to settings where verification costs are heterogeneous and interdependent, not just agent code edits; for example, in scientific simulations where some probes are cheap and others expensive.

The leave-one-out advantage baseline used in GVO is a natural fit for any multi-trajectory reinforcement learning setting where independent rollouts from the same start are available, potentially extending meta-learning for robot manipulation.

The paper's ablation showing that scorer-only updates give substantial gains suggests that in many practical systems, improving the value estimator may be more impactful than improving the generator or policy, which could simplify implementation.

Because the method does not assume model-parameter updates, it may be applied to closed-source language models through prompt and tool editing alone, broadening its reach.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version often exhausts the budget before those sequences yield a verified improvement. Inspired by the draft–verify principle of speculative decoding, we propose Speculative Recursive Self-Improvement (SpecRSI), a two-level framework coupling inner-loop agent search with outer-loop improver learning. SpecRSI drafts successors before their predecessors are fully evaluated, while adopting only fully evaluated, admissible versions. In the inner loop, Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. In the outer loop, Group Verification Optimization (GVO) updates the improver from completed searches while keeping the real evaluator fixed. We evaluate SpecRSI on scientific optimization (including active queue management (AQM) for network infrastructure), coding, and general agent-system benchmarks. Under matched evaluation-call budgets, SpecRSI improves scientific terminal gain over Janus by 60.0%–96.3% and area under the budget curve by 32.1%–91.6%, while achieving higher means across all 50 scientific backbone–task pairs and outperforming MetaRSI-v1 by 1.30–2.70 percentage points (pp) in system score across twenty backbones. On instance-level comparisons, it further improves PyBaMM terminal gain by 142.6% and SWE-Pro success by 6.0 pp at 7.0% and 9.2% lower token cost, respectively. These results demonstrate the efficacy of the proposed approach.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-14

The authors' main claim, as the classifier picked it out 0.54

“Additional evaluations show that SpecRSI outperforms Janus in all 50 scientific backbone--task pairs and exceeds MetaRSI-v1 by 1.85%--2.38% in macro-average system score across twenty backbones.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.06 · states limitation 0.32 · human participants 0.04 · controlled comparison 0.91 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Improving an agent through successive revisions requires balancing exploration and evaluation under a limited budget. background 1.00 0.08 0.04 asserted 1.00
A narrow search can miss useful sequences of edits, while fully evaluating every intermediate version can exhaust the budget before those sequences yield a verified improvement. background 0.82 0.10 0.03 asserted 0.72
We introduce Speculative Recursive Self-Improvement (SpecRSI), which drafts executable successor agents before their predecessors are fully evaluated, while requiring complete verification before adoption. contribution 0.82 0.83 0.17 asserted 1.00
Verification-Aware Tree Search (VATS) allocates the budget between further edits and outstanding checks using the predicted verified return of a continuation. method 0.98 0.78 0.05 asserted 1.00
Group Verification Optimization (GVO) learns the improver from completed searches while keeping the real evaluator fixed. method 0.88 0.83 0.04 asserted 0.96
In the main scientific comparison, terminal gains exceed Janus by 33.3%--40.0%. result 1.00 0.92 0.61 asserted 1.00
Additional evaluations show that SpecRSI outperforms Janus in all 50 scientific backbone--task pairs and exceeds MetaRSI-v1 by 1.85%--2.38% in macro-average system score across twenty backbones. result 1.00 0.94 0.21 asserted 1.00
Coding-validation success reaches 69.4% versus HGM's 64.3% at a 20.8% lower target-cost index. result 1.00 0.92 0.09 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.68

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-14: 1 request, 6,249 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (4)
Figure 1 of the paper
Figure 1. Explore ahead, verify before adoption. Agent patterns (a–c) and their token-level counterparts (d–f). Agent checks are external and paid; only the adopted version needs complete, admissible evidence. However, the drafting–confirmation idea cannot be transferred unchanged from tokens to agent revisions. (1) Revisions are revisable programs, not accepted prefixes. A successor may repair or build on a parent that is not yet admissible. (2) Verification consists of paid, complementary checks. External evaluations are not supplied by one target-model pass: checks have heterogeneous costs…
Figure 2 of the paper
Figure 2. Performance across ten backbones. Target-cost indices and macro-average scores on Terminal-Bench 2.1 (TB2.1), SWE-Pro, GPQA-hard100, and AIME (mean ± 1.0 × sample SD across five training seeds). Panels follow Initial Agent means; lines connect model categories. On all ten backbones, SpecRSI scores higher than MetaRSI-v1 at lower target cost but 0.83–0.90% higher total-cost indices. Accurate benefit evaluation is central to guiding RSI toward better agents. It informs which revisions and checks deserve further work, guiding the direction of improvement. Inaccurate evaluation can…
Figure 3 of the paper
Figure 3. Two-level Speculative RSI Loop. VATS allocates drafts and checks under a frozen improver; Accept returns the best complete admissible version. GVO updates the improver between searches; Γ stays fixed. SpecRSI tracks individual evaluation checks. A complete admissible root r anchors tree T = (C, A) of immutable agent versions and executed edit edges. Versions freeze prompts, tools, workflows, and permitted persistent state, including memory; only D/H/M permits model-parameter edits. The uneditable verification contract Γ specifies finite required-item set I, execution/admission…
Figure 4 of the paper
Figure 4. Verification scope and planning ablations. (a) Gain versus total-cost index for one check or bundles at fixed Summary or Full-state information; bars show reported SD. (b) Verified gain and (c) total-cost index across eight budget caps. In (b), means average five reported seed means; SD averages within-seed SDs. Lines join means; bands/bars show ±1 SD. 8

Figures are the authors', reproduced from the paper.

Cited by 1 Pith paper

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment