{"id":"2e930e1a-ff5b-42a0-bdf8-74026060a22b","arxiv_id":"2607.03333","paper_version":1,"verdict":"ACCEPT","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"A training-free self-fork of an agentic LLM predicts its own next tool call early enough to hide 16–37% tool-wait latency, cutting Qwen3-32B GAIA P95 by 18% with accuracy preserved.","lead":"SPORK speeds up LLM agents by forking a cheap probe from the model itself to guess the next tool call early, then running that tool while the model finishes thinking. It cuts tail latency on real web-search agents without retraining or extra predictor models.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No objection that overturns ACCEPT; softest spot is unisolated causal weight of tool-overlap vs D3 for the exact 18% GAIA P95 figure under single-run real-API noise.","rationale":"The reader’s ACCEPT and low correctness risk are right: training-free controller, strict gate, real tools, EQ1-validated envelope, honest failure modes (no-think, format divergence, read-only), open code. The early-intent / thinking-CoT assumption (§3.1–3.2, §6.6) is necessary and already bounded by the paper; I agree it is a primary assumption. I only partially agree that it is the single softest support for the strongest claim as stated: that claim is a measured systems result (18% P95 + ≤1 pp EM), so the load-bearing gap is less “does early intent exist?” (shown) than “is the headline P95 magnitude and its accept-path tool-overlap attribution isolated under single-run real-API noise?” That does not imply a correctness break or require REJECT/CONDITIONAL—the mechanism, cost model, multi-benchmark direction, and quality gate hold. Verdict stays ACCEPT; a multi-seed D3-off ablation would tighten confidence in the exact number and wording, not the paper’s core contribution.","tokens_in":22029,"tokens_out":787,"duration_ms":57041,"concrete_test":"Re-run GAIA Qwen3-32B (same harness, greedy, seed sweep {42,43,44}): ngram baseline vs engine D1+D2 (D3 off) vs D1+D2+D3. Report P95 mean±range and the fraction of the 18% retained without D3; also log per-accepted-turn remaining decode time at commit. If D1+D2 alone keeps ≥~12% P95 cut across seeds, tool-overlap drives the claim; if the cut collapses without D3 or is seed-unstable, narrow the strongest claim’s causal wording and treat 18% as a single-run point estimate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim attributes the GAIA P95 cut (131.9→108.1 s) to overlapping tool execution on exact name+args match. That requires α·t_overlap to dominate on the tail. The paper supports the ingredients well: fork name accuracy (§3.1), logprob gate (§3.2/§4.2), EQ1 residual ≤1.84% on tau2/GAIA (§6.6), and D1 helping mean latency (§6.4). The soft spot is attribution and precision of the headline number, not mechanism existence. Ablation text credits D1 for mean/P50 and D1+D2+D3 for best P95, with “D2+D3 synergy” on rejected retries, but does not publish GAIA P95 for D1+D2 with D3 off, nor the distribution of remaining CoT (realized t_overlap) at gate commit on the N=165 run. Appendix D’s BrowseComp mean t_overlap is only 1.03 s despite large mean T_tool, showing overlap can be modest. P95 at N=165 is ~8 queries and is measured once under real search APIs and vLLM batching—the same nondeterminism the paper cites for EM noise (§6.3–6.4). So the direction and envelope are solid; the exact 18% and the abstract’s emphasis on accept-path tool overlap (vs reject-path D3 drafts plus variance) are the least secured parts of the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"SPORK is a training-free controller that accelerates agentic LLM inference by forking a forced tool-call probe from the running model’s own prefix-cached state, dispatching the predicted tool early, and overlapping tool latency with remaining chain-of-thought. Three mechanisms map to a simple cost model (EQ1): D1 prefix-cache fork (lowers T_oh), D2 logprob confidence gate (raises α·t_overlap), and D3 partial-token accept of rejected probes as speculative-decoding drafts (lowers T*_base on misses). On real tools, the full system cuts Qwen3-32B GAIA P95 from 131.9 s to 108.1 s (−18%) with EM within 1 pp of baseline; similar directional gains appear on HotpotQA and a tau2 latency sweep, and across 4B–32B dense and a MoE model. Correctness is preserved by a strict name+arguments match gate and serial fallback.","tokens_in":22483,"tokens_out":1330,"duration_ms":19972,"significance":"If the result holds, SPORK fills a practical gap between token-level speculative decoding (which does not issue tools earlier) and prior action-level speculation systems that need auxiliary predictors, verifiers, or historical traces. The day-one, API-compatible design, open-source controller, falsifiable EQ1 break-even condition (validated within ~2% residual on tau2/GAIA), and explicit operating envelope (no-think mode, format divergence, short tools) are genuine systems contributions. Real-tool evaluation (GAIA web search, Wikipedia API) and multi-architecture generalization strengthen the claim beyond synthetic stalls. Orthogonality to ngram speculative decoding is demonstrated by stacking, which is useful for production stacks.","major_comments":[{"comment":"§6.4 and Figure 12 attribute best P95 to D1+D2+D3 and mean/P50 to D1, but do not report GAIA P95 (or the accepted-turn t_overlap distribution) for D1+D2 with D3 off. The abstract and §1 frame the 18% primarily as accept-path tool overlap (“on acceptance, the tool result is ready when reasoning ends”), while Appendix D’s BrowseComp decomposition shows mean realized t_overlap of only 1.03 s despite large mean T_tool. Without the missing D1+D2 P95 arm on the same GAIA N=165 run, the causal weight of tool-overlap versus D3 draft recovery (and single-run variance) for the headline 131.9→108.1 s figure remains under-isolated. Please add that ablation (or an equivalent per-turn breakdown of saved tool wait vs. saved tool-call decode) so the central latency claim is attributed cleanly.","section":"§6.4, Figure 12; Abstract; Appendix D"},{"comment":"§6.2 reports a single-seed P95 on N=165 real-API queries (~8 queries at the 95th percentile) under the same search-API and vLLM-batching nondeterminism the paper cites for EM noise (§6.3–6.4). EQ1 is well calibrated on mean speedup, but the abstract’s precise 18% P95 number is a tail statistic on a thin sample. Either multi-seed P95 (or bootstrap CIs) on GAIA, or a clearer statement that the robust claim is directional tail reduction under the EQ1 envelope rather than a single-run 18% point estimate, would make the load-bearing latency claim proportionate to the evidence.","section":"§6.2; Abstract"}],"minor_comments":[{"comment":"§2.2 / Appendix A: EQ1 is presented in two slightly different forms (uniform T_oh vs. accept/reject-specific overhead). A single canonical equation in the main text with the Appendix A expansion would reduce reader friction when checking the tau2 residual claims.","section":"§2.2, Appendix A, EQ1"},{"comment":"§4.3 and §5: D3 is correctly described as an engine-side prototype, but the abstract’s “thin controller over standard completion APIs” phrasing can be read as covering full D1+D2+D3. A one-sentence clarification that the open HTTP path is D1+D2 and D3 needs the SporkProposer integration would avoid overclaim.","section":"Abstract; §4.3; §5"},{"comment":"Figure 10 uses per-panel y-scales and mixed HTTP/engine baselines (Table 3). A short caption note that each model is compared only to its own baseline under the same serving mode would prevent cross-panel misreading of absolute seconds.","section":"Figure 10; Table 3"},{"comment":"§6.5 cross-model case study is valuable; stating GPU count and whether the 4B drafter was co-located or on a second GPU more prominently in the figure caption would make the “avoids a second served model” claim easier to audit.","section":"§6.5, Figure 13"},{"comment":"Typos/style: “Spork” vs “SPORK” capitalization is inconsistent between title/abstract and body; “think-end” / “no-think” could be defined once in §3.2 for readers outside the Qwen3 ecosystem.","section":"Throughout; §3.2"}],"recommendation":"minor_revision","confidential_remarks":"I agree with the reader’s ACCEPT direction on mechanism soundness; I am recommending minor_revision only to force isolation of the headline P95 attribution and more honest tail statistics—both fixable without new algorithms. Novelty relative to Speculative Actions, DualSpec, PASTE, and Nichols et al. is adequately differentiated (self-fork, no traces/aux models). Scope fits a systems/DC venue well. No integrity concerns; open-source link is a plus if the artifact is complete."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful bit is simple: instruction-tuned models in thinking mode often already know the next tool name after the first token, so you can fork a forced tool-call probe off the shared prefix cache, gate on the probe’s own logprobs, and hide real tool latency behind the remaining CoT—without a draft model, traces, or retraining. That fills a real gap left by Speculative Actions, DualSpec, PASTE, and related work.\n\nWhat they did well is systems hygiene. EQ1 is an accounting identity over measurable α, t_overlap, T_oh, and T*_base, and the tau2 floor sweep matches it within ~2%. Fork-at-start name accuracy is high across five benchmarks (74.6–99.6% on Qwen3-32B). The strict exact name+args gate keeps correctness by construction (EM within 1 pp, sometimes better). They ablate D1/D2/D3, show self-probe beats a 4B cross-drafter on acceptance, open-source the controller, and write an honest operating envelope: no-think mode kills the window, format divergence kills α, and they only speculate read-only tools. Generalization across 4B–32B dense and a MoE is more than most agent-serving papers bother with.\n\nSoft spots, in proportion. The headline 18% GAIA P95 (131.9→108.1 s) is a single N=165 real-API run; P95 is ~8 queries, and search/vLLM nondeterminism is the same noise they cite for EM. Ablations credit D1 for mean/P50 and full D1+D2+D3 for best P95, but they do not publish GAIA P95 for D1+D2 with D3 off, nor the realized t_overlap distribution at gate commit on that run. BrowseComp’s mean t_overlap of 1.03 s shows accept-path overlap can be modest, so some of the tail win may be reject-path draft recycling plus variance. That weakens the abstract’s emphasis on accept-path tool overlap more than it weakens the mechanism. Free knobs (θ, retry budget, step size) are ordinary; θ is swept on F1. D3 is a vLLM-specific proposer integration, not pure HTTP.\n\nThis is for people building agent runtimes and LLM serving stacks who care about wall-clock under real tools. The central argument holds; the weakest assumption is checked and bounded. I would send it to referees and I would cite the self-fork + cost-model framing. Engage.","headline":"Training-free self-fork for agent tool wait is real, useful, and bounded; the 18% GAIA P95 is directionally solid but not cleanly isolated from D3 and single-run API noise.","tokens_in":23104,"tokens_out":655,"would_cite":true,"duration_ms":5840,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A forked probe from the running model can start tool calls early, cutting agent tail latency without extra predictors or training.","keywords":["LLM agents","tool use","speculative execution","inference acceleration","self-speculation","prefix cache","chain-of-thought"],"falsifier":"Run the same real-tool agent suite in no-think mode or with a native tool format that diverges from the forced probe: if name accuracy collapses or speedup falls below 1× as the paper’s own tau2 no-think and XML boundary cases predict, the self-speculation claim fails.","tokens_in":22932,"feed_emoji":"⚡","tokens_out":677,"duration_ms":5439,"temperature":0.7,"pith_summary":"Agent systems often wait on external tools after the model finishes reasoning, and that wait eats a large share of end-to-end time. SPORK claims the same model already knows its next tool early: a short forced probe forked after the first token predicts the tool name with high accuracy, so the controller can dispatch the tool while the remaining chain-of-thought still decodes. A cost model says speculation pays when accepted overlap beats probe overhead; three mechanisms push that balance—prefix-cache sharing cuts probe cost, a logprob confidence gate filters bad guesses, and rejected probes recycle as draft tokens for the main stream. On real web-tool GAIA with Qwen3-32B, P95 falls 18% while exact-match accuracy stays within one point of the serial baseline, with the same idea holding across model sizes and dense versus mixture-of-experts architectures when thinking-mode CoT is long enough.","feed_headline":"Self-forked probes cut agent P95 latency 18%","feed_subtitle":"No extra model or traces: the running LLM predicts its own tool call and hides the wait behind remaining reasoning.","key_machinery":"SPORK’s self-speculative fork: after the main stream’s first token, a prefix-cache-sharing probe with a forced tool-call opener emits a candidate call; a min-span logprob gate decides whether to dispatch the tool early; on exact match the result is ready when reasoning ends, otherwise serial fallback runs and the probe’s verified prefix is reused as speculative-decoding draft tokens (D1–D3 against the EQ1 break-even).","core_discovery":"Instruction-tuned models in thinking mode already expose their next tool call early enough that a training-free self-fork can hide tool latency behind remaining decode: a start-of-generation probe predicts Qwen3-32B tool names at 74.6–99.6% across five benchmarks, and a strict name-and-arguments gate plus fallback keeps the agent lossless while cutting GAIA P95 from 131.9 s to 108.1 s.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["SPORK self-fork hides tool wait, cuts agent P95 18%","Training-free probe predicts tools early for 18% latency cut","Self-speculative fork overlaps tool calls with LLM reasoning","Own-model fork trims Qwen3-32B GAIA P95 from 131.9s to 108.1s","Forked self-prediction hides tool latency behind remaining decode"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The model must already reveal its next tool early enough, with separable confidence and a long enough thinking-mode reasoning window, so that accepted overlap beats the cost of running the probe.","fun_headline_variants_meta":{"raw":{"variants":["SPORK self-fork hides tool wait, cuts agent P95 18%","Training-free probe predicts tools early for 18% latency cut","Self-speculative fork overlaps tool calls with LLM reasoning","Own-model fork trims Qwen3-32B GAIA P95 from 131.9s to 108.1s","Forked self-prediction hides tool latency behind remaining decode"]},"model":"grok-4.5","effort":"low","cost_usd":0.006016,"raw_usage":{"total_tokens":1668,"prompt_tokens":950,"num_sources_used":0,"completion_tokens":88,"cost_in_usd_ticks":60160000,"prompt_tokens_details":{"text_tokens":950,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":630,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":950,"tokens_out":88,"duration_ms":5152,"temperature":1.0,"reasoning_tokens":630,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-12T03:12:24.595660+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same real-tool agent suite in no-think mode or with a native tool format that diverges from the forced probe: if name accuracy collapses or speedup falls below 1× as the paper’s own tau2 no-think and XML boundary cases predict, the self-speculation claim fails.","supporting_citations":[],"review_version":1}