{"id":"e1b77a3b-d9a8-4ef0-ae48-ea27d28319e7","arxiv_id":"2607.28077","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"LEEPS adaptively splits RLVR prompts into exploit and explore portfolios and ranks uncertain prompts by latent-neighbor success estimates, raising average math scores 2.6–3.7% over the best prior pre-rollout selector at fixed rollout budget.","lead":"LEEPS picks which math prompts to train on before expensive LLM rollouts, mixing proven useful prompts with uncertain ones scored by latent neighbors. It modestly beats prior prompt selectors on Qwen math models at matched rollout cost, with only seconds of extra sampling time.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Headline gains rest on single-run ~1–1.7 point overall-score gaps with best-checkpoint selection and no seed variance.","rationale":"The reader’s CONDITIONAL verdict already flags missing multi-seed error bars and small absolute lifts; I treat that statistical fragility as the single most load-bearing issue for the strongest_claim as worded (an empirical win and stated % gains), more directly than the static K-NN transfer assumption. The K-NN concern is real for mechanism and external validity (Fig. 3 is offline AUROC on a held-out split of the same pool; representations/neighbors never refresh), and ablations show latent guidance matters—but the claim that LEEPS “achieves the highest average score” with those relative gains stands or falls first on whether Table 1 differences replicate. Code release helps but does not replace seed-averaged numbers. No internal inconsistency found in the algorithms or budget matching versus DPS/MoPPS; contribution remains ACCEPT-shaped for the niche if the gaps hold under the concrete multi-seed check. Hence verdict stays CONDITIONAL (UNCHANGED).","tokens_in":21501,"tokens_out":606,"duration_ms":49983,"concrete_test":"Re-train LEEPS and DPS for ≥3 independent seeds on Qwen2.5-Math-1.5B with identical hyperparameters, fixed Bcand ratio, and a pre-registered evaluation rule (final step or mean of last 5 evals—no peak pick). If the seed-mean overall-score gap is ≤0.5 or a 95% CI on the paired difference includes 0, the headline relative-gain claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is an empirical ranking win: under matched rollout budgets, LEEPS posts the highest six-benchmark average, with 2.6% and 3.7% relative gains over DPS (Table 1: 39.16 vs 38.17 on 1.5B; 47.53 vs 45.83 on 7B). Those absolute gaps come from single training trajectories. The manuscript reports neither multi-seed means/SEs nor significance tests, despite high-variance ingredients (temperature-1 group rollouts, random Bcand draws, GRPO). Reporting further favors peaks: Llama results explicitly take “the checkpoint with the highest average score within the first 160 steps” (Table 5); ablations use “best overall scores within the model-specific selection ranges” (Table 2); main curves (Fig. 5) fluctuate by more than the claimed margins. OOD lifts are 0.14 and 0.10 points (Table 3)—inside ordinary run noise. If across-seed SD is on the order of ~1 overall point, the stated ranking and relative-gain percentages are not established, independent of whether the static K-NN design is sound.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes LEEPS, a pre-rollout prompt sampler for RLVR/GRPO that reduces waste from zero-variance rollout groups. Candidates are split into an exploit portfolio (historically non-trivial prompts, weighted by Bernoulli variance â(1−â)) and an explore portfolio (unseen or latest zero-variance prompts). Rollout budget is allocated by an adaptive explore fraction ρ chosen so that a linear mixture of recent portfolio non-trivial ratios tracks a target τ. Exploration is further ranked by a count-decayed latent uncertainty score that estimates success rate from cosine K-NN neighbors in a fixed final-token hidden-state cache (no extra selection rollouts). On DAPO-Math-17K with Qwen2.5-Math-1.5B/7B, under matched rollout budgets, LEEPS reports the highest six-benchmark average versus GRPO, DS, MoPPS, and DPS (relative gains 2.6% and 3.7% over DPS), plus the highest average on three OOD suites, ablations of random explore / uniform exploit / no portfolios, Llama-3.2-3B results, sensitivity to τ/K, and ~2s per-step sampling overhead. Code is linked.","tokens_in":21802,"tokens_out":1512,"duration_ms":34950,"significance":"If the empirical ranking holds under matched budgets, LEEPS is a useful systems contribution to efficient RLVR: it targets the explore–exploit tension that the paper documents between DPS (high non-trivial ratio, narrow coverage) and MoPPS (broader coverage, lower early non-trivial ratio), without the extra generation cost of post-rollout filters like DS. Strengths include a clear problem formulation, matched-budget and rollout-normalized comparisons, training diagnostics (non-trivial and medium ratios, coverage CCDF), component ablations, a second backbone (Llama), OOD checks, overhead profiling, and public code. The method is heuristic rather than theoretically guaranteed, but that is appropriate for this empirical systems setting. The main significance risk is whether the reported 1–1.7 point overall-score margins and OOD tenths-of-a-point lifts survive multi-seed evaluation and fixed checkpoint protocols.","major_comments":[{"comment":"Table 1 and the abstract’s 2.6%/3.7% relative gains over DPS rest on single training trajectories per method/scale, with no multi-seed means, standard errors, or significance tests. GRPO with temperature-1 group rollouts and random Bcand draws is high-variance; Fig. 5 curves fluctuate by amounts comparable to or larger than the 0.99–1.70 absolute overall-score gaps. Without seed replication (or at least final-checkpoint multi-run reporting), the central ranking claim and the stated relative percentages are not established at the precision claimed.","section":"Table 1; Fig. 5; Abstract"},{"comment":"Several tables select favorable checkpoints rather than a pre-specified final or average-over-window metric: Table 5 explicitly takes “the checkpoint with the highest average score within the first 160 steps”; Table 2 uses “best overall scores within the model-specific selection ranges.” Combined with single runs, peak-picking can reorder methods when curves cross (Fig. 5). Please report a fixed protocol (e.g., last checkpoint, or mean of last K evals) for all methods, and apply the same protocol to ablations and Llama.","section":"Table 2; Table 5; Fig. 5"},{"comment":"Table 3 OOD averages favor LEEPS by only +0.14 (1.5B) and +0.10 (7B) over the strongest baselines, with mixed per-benchmark wins/losses. The abstract and conclusion state that LEEPS “achieves the highest average score across the three evaluated OOD … benchmarks at both model scales” as a parallel headline to the in-domain gains. Those margins are well inside plausible run noise given the issues above; either multi-seed OOD results or substantially softer wording is needed so OOD is not presented as equally established.","section":"Table 3; Abstract; Conclusion"},{"comment":"Latent-Guided Exploration (Eqs. 6–9; Fig. 3) uses a static pre-RL K-NN graph on fixed layers (22/26) and never refreshes representations as the policy changes. Fig. 3 is an offline AUROC on a held-out split of the same training pool, not a demonstration that neighbor non-triviality remains predictive late in training. The random-exploration ablation (Table 2, Fig. 6) supports some value of the score, but does not test representation staleness. A load-bearing check would be mid/late-training AUROC of âlat_p vs. realized non-triviality, or a cheap refresh ablation; without it, the claim that latent locality guides exploration “as the policy evolves” is only weakly supported.","section":"Methods, Latent-Guided Exploration; Fig. 3; Eq. (9)"}],"minor_comments":[{"comment":"Figure 1 right panel y-axis is log-scaled with uneven tick labels (0,1,10,20,50,100); a clearer log scale or explicit note would help read the coverage CCDF.","section":"Figure 1"},{"comment":"Notation mixes a(p), âp, and âlat_p; a short symbol table in Preliminaries or Methods would reduce scanning cost.","section":"Preliminaries; Methods"},{"comment":"Algorithm 1 is helpful; briefly state how q_explore and q_exploit are estimated from the “adaptive window” of 3 (Table 4)—e.g., simple moving average of batch non-trivial indicators—so the ρ update is fully reproducible from the main text.","section":"Algorithm 1; Table 4"},{"comment":"Related Work cites concurrent/arXiv pre-rollout selectors (MoPPS, DPS) appropriately; ensure camera-ready citation keys and any venue updates are consistent.","section":"Related Work"},{"comment":"Minor prose spacing artifacts appear throughout (“improvesthereasoning”, “beforerolloutgeneration”); clean for camera-ready.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"Fit for a solid empirical ML/NLP venue after revision. The method is plausible and the experimental suite is broader than many RLVR systems papers, but the headline percentages are currently over-precise relative to single-run, peak-selected evidence. I would not accept on the present statistics; multi-seed or a hardened fixed-checkpoint protocol would likely move this to minor revision/accept. No integrity concerns; code link is a plus."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: LEEPS is a practical pre-rollout prompt selector for GRPO-style RLVR that actually tries to solve the explore/exploit tension the prior selectors leave open. The absolute lifts are modest and the main tables are single-run, but the method is clear, the diagnostics are useful, and the paper is worth referee time.\n\nWhat is new is not “prompt selection for RLVR” — MoPPS, DPS, and online filters already own that shelf — but the specific pairing: (1) split candidates into exploit vs explore portfolios and set the explore fraction ρ so the batch’s expected non-trivial ratio tracks a target τ, and (2) rank uncertain prompts by a count-decayed latent-neighbor success estimate from a static final-token K-NN cache. That combination is coherent, cheap online (~2s/step), and better motivated than pure posterior sampling or pure dynamics scoring.\n\nWhat it does well: problem framing is tight (zero-variance groups waste generation); Algorithm 1 and the weight formulas are reproducible; experiments cover two Qwen-Math scales under matched rollouts, ablations that isolate random explore / uniform exploit / no portfolios, Llama-3.2-3B transfer, OOD averages, τ/K/calibration sensitivity, and overhead. Figure 4–5 and the non-trivial/medium-ratio curves are more informative than the headline percentages. Code is linked. Citation pattern looks fair to the immediate baselines.\n\nSoft spots, in proportion: the stress-test is right on statistics. Table 1’s 0.99 and 1.70 overall-point edges over DPS, and the 2.6%/3.7% relative claims, come from single trajectories with best-checkpoint flavor in places (explicitly on Llama and ablations). OOD margins are noise-sized. No multi-seed SEs. That does not sink the paper — full RLVR multi-seed is expensive — but it means the ranking is not locked. Second, the static pre-RL K-NN (fixed layers, no representation refresh) is a real modeling bet; Fig. 3’s offline AUROC helps, and ablations show latent guidance beats random explore, but transfer as the policy moves is assumed more than stress-tested. Free parameters (τ, ρ0, K, layer, cold-start) are documented and somewhat swept; not hidden.\n\nWho it is for: people building RLVR data pipelines who already care about non-trivial ratio vs coverage. Not a theory paper. I would bring it to a methods reading group if we are in that lane; otherwise skim the algorithm and Fig. 4–5. I would cite it when discussing pre-rollout selection. Send it to peer review — ask for multi-seed or at least variance discussion and tighter checkpoint protocol, not a desk reject.","headline":"Solid pre-rollout RLVR sampler with a coherent explore/exploit + latent-neighbor design; gains are real-looking but single-run and small, so treat the ranking claims as provisional.","tokens_in":22475,"tokens_out":691,"would_cite":true,"duration_ms":24896,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A pre-rollout sampler that mixes known useful prompts with latent-neighbor exploration raises math RL scores without extra rollouts.","keywords":["RLVR","prompt sampling","explore-exploit","latent neighbors","GRPO","mathematical reasoning","pre-rollout selection","non-trivial ratio"],"falsifier":"Replace latent-neighbor ranking with uniform sampling from the explore portfolio (or refresh/remove the static K-NN graph) and check whether the reported average gains on the six math benchmarks and the high non-trivial ratio disappear under the same rollout budget.","tokens_in":22288,"feed_emoji":"🎯","tokens_out":933,"duration_ms":19275,"temperature":0.7,"pith_summary":"When language models learn from checkable answers, many prompts waste generation because every sample in a group is right or every sample is wrong, so the update gets no signal. Earlier pre-rollout selectors either keep recycling a small set of historically useful prompts or explore so widely that the batch fills with uninformative ones. This paper argues that you can keep a high share of informative prompts and still cover more of the training pool by splitting candidates into an exploit portfolio and an explore portfolio, then allocating the fixed rollout budget from their recent non-trivial rates, and by ranking uncertain prompts with a static nearest-neighbor estimate built from model hidden states and past neighbor outcomes. On two math-specialized model sizes the method posts the best average across six math benchmarks under matched rollout budgets, improves faster during training, transfers to three out-of-domain reasoning suites, and adds only about two seconds of online sampling cost per step.","feed_headline":"Prompt sampler lifts math RL scores without extra rollouts","feed_subtitle":"Explore–exploit portfolios plus latent neighbors beat prior selectors under matched budgets","key_machinery":"LEEPS: Adaptive Explore–Exploit Portfolio Allocation plus Latent-Guided Exploration. Candidates are split into an exploit set of previously non-trivial prompts (weighted by success-rate variance) and an explore set of unseen or zero-variance prompts; the explore fraction is chosen so the expected batch non-trivial ratio stays near a target; uncertain prompts are ranked by a count-decayed score from cosine neighbors in a fixed final-token hidden-state cache whose observed success rates estimate intermediate difficulty.","core_discovery":"Under matched rollout budgets and without any extra selection rollouts, LEEPS—the latent-guided explore–exploit prompt sampler—attains the highest average score on six mathematical reasoning benchmarks for both Qwen2.5-Math-1.5B and 7B, with relative gains of 2.6% and 3.7% over the strongest prior pre-rollout baseline, while also leading on three OOD general-reasoning benchmarks and adding roughly two seconds of online sampling overhead per training step.","pith_inferences":["If latent locality of non-triviality holds more generally, the same portfolio-plus-neighbor pattern could apply to other verifiable domains such as code or formal proof without redesigning the reward.","Periodic light refresh of the neighbor graph might be needed when policy shift breaks the offline AUROC signal; that is a natural stress test the paper leaves open.","Curriculum and adaptive rollout-count methods could plug into the same explore–exploit quota rather than competing with it."],"forward_implications":["Pre-rollout RLVR can raise final math scores and training speed without paying the extra generation cost of post-rollout filtering.","Keeping separate exploit and explore portfolios with a target non-trivial ratio sustains both informative batches and broader prompt coverage than pure exploitation or pure exploration.","Static representation-space neighbors plus historical neighbor outcomes can steer exploration without online re-encoding or extra rollouts.","Gains under this sampler also show up on OOD general-reasoning suites, not only in-domain math.","Online selection overhead stays on the order of two seconds per step once the neighbor cache is built."],"fun_headline_variants":["LEEPS latent sampler lifts math RLVR scores under matched budgets","Explore-exploit portfolios beat prior selectors on six math benchmarks","Latent neighbors guide prompt reuse for 2.6-3.7% RL gains","Highest average math and OOD scores with 2s sampling overhead","Adaptive exploit-explore allocation tops RLVR without extra rollouts"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"A fixed nearest-neighbor map of prompt hidden states, built once before training, keeps predicting which currently uncertain prompts will produce mixed right-and-wrong rollouts as the model changes.","fun_headline_variants_meta":{"raw":{"variants":["LEEPS latent sampler lifts math RLVR scores under matched budgets","Explore-exploit portfolios beat prior selectors on six math benchmarks","Latent neighbors guide prompt reuse for 2.6-3.7% RL gains","Highest average math and OOD scores with 2s sampling overhead","Adaptive exploit-explore allocation tops RLVR without extra rollouts"]},"model":"grok-4.5","effort":"low","cost_usd":0.003771,"raw_usage":{"total_tokens":1271,"prompt_tokens":855,"num_sources_used":0,"completion_tokens":75,"cost_in_usd_ticks":37708000,"prompt_tokens_details":{"text_tokens":855,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":341,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":855,"tokens_out":75,"duration_ms":6259,"temperature":1.0,"reasoning_tokens":341,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T18:31:48.845079+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace latent-neighbor ranking with uniform sampling from the explore portfolio (or refresh/remove the static K-NN graph) and check whether the reported average gains on the six math benchmarks and the high non-trivial ratio disappear under the same rollout budget.","supporting_citations":[],"review_version":1}