{"id":"be3890ad-0bf2-4c09-8039-dd1d25358743","arxiv_id":"2607.26253","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SARA uses a Bayesian stopping rule on a prompt's first few responses to abandon saturated response groups early and reallocate rollouts, matching accuracy with up to 67% fewer rollouts than dynamic sampling.","lead":"Training reasoning AI models currently wastes most compute on practice problems where the model already gets all answers right or all answers wrong. SARA watches the first few answers, stops spending on problems whose outcome is already decided, and uses the freed effort on fresh problems, matching accuracy with up to 67% less compute.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's expected-savings proof uses the wrong prompt count: Eq. 6 divides by Phi, but SARA's per-prompt effectiveness probability is Psi < Phi; the advertised rollout-savings guarantee is not established.","rationale":"The reader's weakest_assumption focused on the i.i.d. Bernoulli model, which is a real limitation for tree/multi-turn rollouts but is satisfied by construction in the paper's single-turn MATH/Countdown experiments, where each rollout is an independent sample from pi_theta. The more direct threat to the paper's stated contributions is the algebraic error in Theorem 2's proof: the expected number of prompts needed to fill B effective groups is not B/Phi under SARA's early stopping. This error does not by itself falsify the empirical headline numbers, which are measured rather than derived, but it removes the theoretical backing for the 22% savings claim and leaves the magnitude of the savings dependent on a corrected calculation. The reader's rationale already asks for the Theorem 2 derivation to be fixed, so this is a partial agreement rather than a new disagreement. The verdict stays CONDITIONAL: the central empirical claim is plausible and the method is coherent, but the theoretical guarantee and the exact savings figure need verification before full acceptance.","tokens_in":17831,"tokens_out":14501,"duration_ms":131511,"concrete_test":"Using logged per-prompt stopping times and rewards from a SARA run, estimate the empirical distribution of gamma (or directly of Tmix). Compute Psi = mean(1 - gamma^{n_a} - (1-gamma)^{n_a}), E[N] = mean(min(Tmix, n_a)), Phi = mean(1 - gamma^k - (1-gamma)^k), then form corrected_SARA_cost = (B/Psi) * E[N] and DS_cost = (B/Phi) * k. If corrected_SARA_cost >= DS_cost, Theorem 2's savings claim fails; if corrected_SARA_cost < DS_cost, the theorem's proof still needs to be re-derived with the B/Psi denominator before the theoretical savings claim can be accepted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In App. C.4, Theorem 2 derives cost_SARA = (B/Phi) * E[N(gamma)] in Eq. 6, claiming that 'both schemes must evaluate, in expectation, B/Phi prompts to obtain B effective groups.' That is correct for dynamic sampling, which always runs a full group and is effective with probability Phi = E[1 - gamma^k - (1-gamma)^k]. But SARA stops each prompt at n_a = k - floor(tau_low*(k+1)) if no mix has occurred, so under SARA a prompt is effective only if Tmix <= n_a, with probability Psi = E[1 - gamma^{n_a} - (1-gamma)^{n_a}] < Phi. The expected number of prompts SARA must evaluate is therefore B/Psi, not B/Phi, and the correct expected cost is (B/Psi) * E[min(Tmix, n_a)]. Because Psi < Phi, the theorem's formula understates SARA's cost by a factor of Phi/Psi. The claim that SARA is strictly cheaper than dynamic sampling is thus not proven by Theorem 2 as written; the empirical 22% savings may still hold, but it is not backed by the advertised theoretical guarantee. This is load-bearing because the abstract explicitly lists 'expected rollout savings' among the paper's proven contributions.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SARA, a sequential adaptive rollout allocator for RLVR/GRPO training. It treats per-step rollout collection as an optimal-stopping problem over a stream of prompts: a Beta posterior over each prompt's success rate is updated from the group's own rollouts, and a closed-form posterior-predictive probability peff of eventual group effectiveness drives a two-threshold rule that commits mixed groups, abandons likely-saturated groups after a short probe, and reallocates the freed budget to fresh prompts. The authors claim proofs of abandonment reliability, expected rollout savings relative to dynamic sampling, fixed-budget yield dominance over uniform allocation, and a lower bound connecting effective-group yield to the GRPO gradient norm. Experiments on 1.5B/3B models for math and planning report that SARA matches dynamic-predictive sampling (DPS) while using 22% fewer rollouts than dynamic sampling (DS), and that SARA composed with DPS exceeds DS accuracy at 67% fewer rollouts.","tokens_in":18178,"tokens_out":12565,"duration_ms":115906,"significance":"If the theoretical claims hold, the core idea is useful and well motivated: a group's effectiveness is often decided before the full k rollouts are spent, and the closed-form Beta-Binomial predictor is elegant and parameter-free. The composition of an in-sample sequential allocator with predictive prompt selection is a sensible and practical contribution, and the reported empirical gains are substantial. The paper is clearly written and the empirical protocol is detailed. However, the advertised expected-savings theorem is not proven as written; this is a load-bearing gap because the abstract and Section 3.4 explicitly list 'expected rollout savings' among the proven contributions.","major_comments":[{"comment":"The derivation of cost_SARA uses B/Phi for both schemes. Under SARA, a prompt is evaluated only up to N = min(Tmix, n_a), and it yields an effective group only when Tmix <= n_a. The probability of this is Psi = E[1 - gamma^{n_a} - (1-gamma)^{n_a}], which is strictly less than Phi = E[1 - gamma^k - (1-gamma)^k] whenever tau_low > 0. Hence SARA must evaluate B/Psi prompts in expectation, not B/Phi, and Eq. (6) should be corrected (e.g., cost_SARA = (B/Psi) E[N], or a per-effective-group comparison). The claimed strict inequality versus DS = (B/Phi) k is therefore not established by the current proof. Because 'expected rollout savings' is advertised as a proven result, this needs a corrected proof or a weakened statement.","section":"App. C.4, Eq. (6); Theorem 2 in Section 3.4"},{"comment":"The theorem states n*(gamma) = O(log(1/tau_low) / log(1/max(gamma,1-gamma))) for saturated prompts. Under the default uniform prior used in all experiments, the abandonment boundary is n_a = k - floor(tau_low (k+1)), which is linear in k, and Eq. (5) gives E[N(gamma)] <= n_a. The logarithmic bound appears only in a parenthetical about an informative prior, which is not the default. The theorem as stated overclaims the strength of the guarantee for the configuration actually evaluated.","section":"Section 3.4 Theorem 2 / App. C.4"},{"comment":"The exchange argument says reallocated budget starts fresh prompts that are 'effective with probability Phi.' But under SARA a fresh prompt is effective only if it mixes before n_a, i.e., with probability Psi < Phi. The proof does not account for this reduction in per-prompt effectiveness and therefore does not establish the stated strict fixed-budget yield dominance. The conclusion may still be true, but a quantitative per-rollout yield comparison is needed.","section":"App. C.5, Theorem 3"}],"minor_comments":[{"comment":"The main-text statement of Theorem 4 omits assumptions (i) and (ii) that appear in App. C.6. As written, 'Consequently' is not a logical consequence; the assumptions should be stated in the theorem.","section":"Section 3.4 Theorem 4 / App. C.6"},{"comment":"There are small inconsistencies between Table 1 and Table 3: e.g., DPS average 53.2 vs 53.1, SARA 53.1 vs 53.3, SARA+DPS 54.8 vs 54.9. Please reconcile or state that the numbers come from different runs/roundings.","section":"Tables 1 and 3"},{"comment":"The notation peff vs p_eff is used inconsistently. Please unify the symbol and the typesetting of the closed-form predictor.","section":"Eq. (2) and surrounding text"},{"comment":"The labels in Figure 6c appear clipped/overlapping, especially the y-axis 'effective wrongly abandoned (%)'. A cleaner layout would help readability.","section":"Figure 6c"}],"recommendation":"major_revision","confidential_remarks":"The main obstacle is the proof gap in Theorem 2/App. C.4: the expected-savings claim is not established with the current B/Phi accounting. This is fixable in principle by redoing the comparison with B/Psi, but it is a central advertised contribution. The paper also needs to align Theorem 2's logarithmic claim with the uniform-prior analysis and to tighten Theorem 3's exchange argument. I recommend major revision rather than rejection: the empirical results and the core sequential-allocation idea are valuable, and the theoretical issues appear local rather than fatal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the core idea is fresh: treat per-group rollout collection as a sequential stopping problem with a closed-form Beta-Binomial predictor, and reallocate budget from abandoned saturated groups. That is a real departure from prompt-level selection or full-group filtering. Second, the theory's headline result—expected rollout savings—has a bug in the proof. In App. C.4, Eq. 6 divides by Phi, the probability that a full k-group is effective, when SARA's per-prompt effectiveness probability is Psi = E[1 - gamma^{n_a} - (1-gamma)^{n_a}] with n_a < k. Since Psi < Phi, the expected number of prompts SARA must evaluate is B/Psi, not B/Phi, so the claimed cost is understated by a factor of Phi/Psi. The strict savings over dynamic sampling is therefore not proven as written. The empirical 22% savings may still hold, but it is not backed by the advertised theorem.\n\nWhat is genuinely good: the problem framing is right, the closed-form peff is clean, the composition with DPS makes sense, and the ablations (tau_low, n0, group size) are thoughtful. The paper is well-written and the experiments cover several models and benchmarks. The early-decidability figure is convincing.\n\nSoft spots: (1) the Theorem 2 bug is load-bearing because the abstract lists expected savings as a proven contribution. (2) The default commit_min=k means groups that mix early are run to full size anyway, so the theoretical early-commit model does not match the main experimental configuration; early-commit is only an ablation. (3) No error bars or seed variance are reported; numbers are averages over 16 samples but with no variance. (4) The closest baseline, selective rollouts, is mentioned but not compared in Table 1. (5) No code or artifacts are released yet. The i.i.d. Bernoulli assumption is acknowledged but not stress-tested for tree or multi-turn rollouts—for the current single-turn math tasks it is likely fine.\n\nWho this is for: RLVR practitioners and researchers working on training efficiency. It is a solid method-level contribution that should be submitted to a serious venue, but it needs a revised proof and more experimental rigor before acceptance.\n\nMy recommendation: send to peer review. The idea is good enough that referees should see it, but with the expectation of a major revision to fix the theorem and add reproducibility.","headline":"Genuinely new sequential-allocation idea with plausible empirical gains, but Theorem 2's savings proof has a real denominator bug that undercuts the advertised guarantee.","tokens_in":18634,"tokens_out":4673,"would_cite":true,"duration_ms":40666,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62L10","68T05"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims most rollout compute in RLVR is wasted on already-decided saturated groups, and that SARA — a sequential allocator using each prompt's own early rollouts — recovers dynamic-sampling accuracy at 22–67% fewer rollouts.","keywords":["reinforcement learning with verifiable rewards","rollout allocation","sequential analysis","optimal stopping","GRPO","dynamic sampling","Beta-Bernoulli model","early stopping"],"falsifier":"Run the same protocol on a task where within-group rollouts share a long prefix (e.g. multi-turn planning) and measure the empirical probability that an abandoned group would have been effective. If it exceeds τlow by a wide margin, the i.i.d. assumption is violated and the claimed savings are not reliable. Alternatively, on a dataset with non-monotone rewards where effectiveness is not absorbing, check whether the fraction of groups decided after n rollouts still reaches 1 as predicted.","tokens_in":17703,"feed_emoji":"🎯","tokens_out":4726,"duration_ms":39083,"temperature":0.7,"pith_summary":"The paper argues that most of the rollout budget in RLVR training is wasted on prompts whose groups are already saturated — all correct or all incorrect — and that the verdict is usually reached within the first few rollouts. It proposes SARA, an allocator that probes each prompt with a few rollouts, updates a Beta belief over its success rate, and stops early: commit a group once it is mixed, abandon it once it is almost surely saturated, and reallocate the freed budget to fresh prompts. SARA needs no extra prediction rollouts, and the authors prove bounds on abandonment reliability, expected savings, and fixed-budget yield. Empirically it matches a dynamic-sampling oracle's accuracy at 22% fewer rollouts, and composing it with predictive prompt selection beats the oracle by a small margin at 67% fewer rollouts. The contribution is a budget axis orthogonal to prompt selection: deciding in-sample, from a group's own rollouts, rather than predicting before or paying after.","feed_headline":"Sequential stopping saves 22–67% of RLVR rollouts at equal accuracy","feed_subtitle":"SARA decides each prompt's value from its own first few rollouts, matching dynamic-sampling accuracy at much lower cost.","key_machinery":"The central object is the closed-form posterior-predictive effectiveness probability peff(n,s) = 1 − B(α0, β0+n+r)/B(α0, β0+n) (with symmetric all-pass form), computed from a Beta(α0,β0) prior and observed successes s in n rollouts with r=k−n remaining. It is exact for the event 'completed group of size k is mixed', and it defines the stopping rule: commit when mixed, abandon when peff < τlow, otherwise continue. The two-threshold rule is a one-sided sequential probability ratio test of 'saturated' against 'mixed', with zero type-I error (a committed group is always effective) and type-II error bounded by τlow. The same predictor doubles as a priority index for reallocating abandoned budget","core_discovery":"The central claim is that a prompt's training value — whether its rollout group will be effective (mixed successes) or saturated — is statistically decided long before the full group is generated, so collecting remaining rollouts is pure waste. SARA operationalizes this: for each prompt it keeps a Beta posterior over the success rate, evaluates a closed-form posterior-predictive probability peff that the completed group will be effective, and applies a two-threshold sequential probability-ratio-style rule. Effectiveness is absorbing (once mixed, always mixed) and along an all-same prefix peff decreases monotonically, so every prompt has a well-defined stopping time. The authors prove that ab","pith_inferences":["The same early-decidability argument should transfer to agentic and tree-based rollouts if the predictor is conditioned on prefix history rather than a single success rate — the authors flag this as future work, and it is the most direct path to testing the method beyond i.i.d. settings.","If the conditional-i.i.d. assumption fails in practice, the closed-form peff will be miscalibrated; a concrete fix is to learn a per-prefix effectiveness model and re-derive the stopping boundary, which would determine how much of the claimed savings survive correlated rollouts.","SARA's Beta posterior could be reused as a difficulty score for test-time compute allocation: an early-abandoned prompt is likely easy or hard, and the posterior gives a per-prompt uncertainty that a test-time sampler could query."],"forward_implications":["SARA matches the accuracy of dynamic sampling at 22% fewer rollouts on math reasoning and planning with 1.5B and 3B models on a single GPU.","Composing SARA with predictive prompt selection yields the best pass@1, slightly above the dynamic-sampling oracle, at 67% fewer rollouts — near-uniform cost.","SARA replaces oversample-and-filter: it assembles B effective groups at roughly B/Φ expected rollouts instead of Bk/Φ, and the savings grow with group size k.","Because abandoned all-fail traces are the longest, token savings (23%) exceed rollout savings, cutting GPU-hours roughly in line.","SARA is algorithm-agnostic: it improves PPO, RLOO, and Reinforce++ as well as GRPO when it replaces uniform rollout collection at matched effective-batch size."],"fun_headline_variants":["Stop wasting rollouts: SARA sees prompt value in first few tries","SARA cuts RLVR rollout costs by up to 67% with early stopping","SARA: know a prompt is a dud before wasting compute","SARA knows when a prompt's gold is gone—stops rollouts early","SARA frees budget from dud prompts, boosts efficiency"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The entire closed-form predictor and stopping boundary assume that rollouts within a group are conditionally independent and identically distributed Bernoulli draws given a per-prompt success rate; if real rollouts are correlated (shared prefixes, multi-turn state) or rewards are continuous, the early-abandonment decisions that produce the savings can be miscalibrated — a limitation the paper states explicitly in App. G.","fun_headline_variants_meta":{"raw":{"variants":["Stop wasting rollouts: SARA sees prompt value in first few tries","SARA cuts RLVR rollout costs by up to 67% with early stopping","SARA: know a prompt is a dud before wasting compute","SARA knows when a prompt's gold is gone—stops rollouts early","SARA frees budget from dud prompts, boosts efficiency"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000739,"raw_usage":{"total_tokens":3173,"prompt_tokens":819,"completion_tokens":2354,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":2257}},"tokens_in":563,"tokens_out":2354,"duration_ms":17676,"temperature":1.0,"reasoning_tokens":2257,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T00:21:38.455710+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same protocol on a task where within-group rollouts share a long prefix (e.g. multi-turn planning) and measure the empirical probability that an abandoned group would have been effective. If it exceeds τlow by a wide margin, the i.i.d. assumption is violated and the claimed savings are not reliable. Alternatively, on a dataset with non-monotone rewards where effectiveness is not absorbing, check whether the fraction of groups decided after n rollouts still reaches 1 as predicted.","supporting_citations":[],"review_version":1}