{"id":"94e75e71-ac32-4543-a950-06333f7e7cd4","arxiv_id":"2608.09217","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Task 'learnability' (reward response to continued RL training) is reproducible and predictive, and the lightweight probe estimator TRAJVAL improves LLM RL task sampling, though probe-compute accounting weakens the efficiency claim.","lead":"This paper proposes measuring how much a reasoning task improves with continued reinforcement learning training, not just how hard it is now, and uses that measure to sample tasks more efficiently. The authors show the signal is stable across training runs and that a cheap 'probe' version gives modest accuracy gains, though the probe itself costs almost as much compute as the training it is meant to speed up.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never reports the reliability of the probe-estimated delta, the quantity that defines the TRAJVAL score; a 20-epoch probe ranking may not transfer to the 1-epoch (Math) or 3-epoch (Logic) main run.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the probe-trained model trained for 20 epochs on a 512-task subset must faithfully represent per-task response under the main run's 1-epoch (Math) or 3-epoch (Logic) budget. I agree, and I sharpen it: the paper validates the probe using correlations for r_early and r_late but never for delta, which is the component that distinguishes learnability from solvability in Equation 2. Difference-score reliability is a real concern, because two highly correlated endpoint estimates can produce a delta estimate with much lower rank correlation. The within-pre-bin experiment in Section 5.3 is good evidence for the oracle notion of learnability, but it does not test the actual estimator. This is why the conditional verdict is appropriate rather than a rejection: the scientific finding of a reproducible, predictive learnability axis is independently supported by oracle-trajectory analyses and cross-context stability results, while the practical method TRAJVAL still needs the missing probe-delta reliability evidence. The compute-accounting issue noted by the reader is also relevant, but the probe-fidelity question is more fundamental because it concerns whether the method measures what it claims; I therefore focus this critique on that point.","tokens_in":24070,"tokens_out":9037,"duration_ms":87726,"concrete_test":"Take the actual TRAJVAL probe (|P|=512, T=20) on Qwen3-1.7B Math, split the pool within the three pre-bins into high- and low-probe-delta subpools exactly as in Section 5.3 but using the probe estimator rather than oracle trajectories, and train each subpool for the main 1-epoch budget. If the high-probe-delta subpool does not beat the low-probe-delta subpool in Best Acc and AUC (as the oracle split does in Table 2), the probe fails as a proxy for the actual training schedule and the method's gains cannot be attributed to learnability.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central methodological step is Section 4.1 Reduction II: the 20-epoch probe on |P|=512 is used as a proxy for a later training state, and the per-task score s(x) = (1 - r_early(x)) * max(0, delta(x)) is used to weight the main run, which lasts only 1 epoch (Math) or 3 epochs (Logic) per Table 4. The paper validates the probe by reporting Spearman correlations of 0.940 for r_early and 0.876 for r_late between probe-derived and oracle values, but it never reports the corresponding correlation for delta or for s(x). Since delta = r_late - r_early is a difference score, its reliability can be far lower than that of either endpoint. Figure 9 only compares probe-based scores at intermediate probe epochs with the same probe's final score; that is self-consistency, not agreement with the oracle or with main-run utility. Section 5.3's within-pre-bin analysis uses oracle 20-epoch trajectories to define high- and low-delta subpools, so it does not validate the actual TRAJVAL estimator. Consequently, the claim that TRAJVAL captures learnability rather than solvability rests on an unmeasured quantity. If probe-delta rank correlation is materially below 0.876, the gains in Table 1 and the small gap between pre-only and full TRAJVAL in Table 3 may be mostly due to the headroom term, which is a solvability signal.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes task learnability as a regime-conditional property of how positively a task's reward responds to continued RL post-training, distinct from snapshot solvability. It reports empirical evidence from Qwen3-1.7B/4B and Llama-3.2-3B on math and logic that per-task reward trajectories are heterogeneous, reproducible across independently sampled probe contexts (Cohen's kappa = 0.776 for the six-profile taxonomy and 0.879 for the binary learnable grouping), and predictive of downstream accuracy. It then introduces TRAJVAL, a static prior estimated from a short probe RL run on a subset of the pool, using early/late endpoint rewards to score tasks as s(x) = (1 - r_early(x)) * max(0, Delta(x)). Experiments show TRAJVAL improves over uniform sampling and online schedulers, and a within-pre-bin analysis indicates that Delta carries information beyond initial solvability.","tokens_in":24455,"tokens_out":6736,"duration_ms":57847,"significance":"If the central claims hold, the paper adds a useful dimension to LLM RL data allocation: rather than filtering only by current pass rate, one can prioritize tasks with high expected positive response to training. The oracle-based within-pre-bin analysis and the cross-context reproducibility measurements are genuinely informative, and the code release is a plus. However, the practical and conceptual case rests on an unvalidated link: the probe-based Delta and score are never checked against oracle values, and the probe cost is not compared with the main-run compute budget. These gaps need to be closed before the headline efficiency claims are fully supported.","major_comments":[{"comment":"The paper reports Spearman correlations of 0.940 for r_early and 0.876 for r_late between probe-derived and oracle endpoint estimates, but never reports the analogous correlation for Delta(x) or the TRAJVAL score s(x). Because Delta is a difference of two noisy estimates, its reliability can be far lower than that of either endpoint, and s(x) is a nonlinear function of both. Figure 9 only shows agreement of intermediate probe scores with the same probe's final score, which is self-consistency and does not validate the estimator against the oracle. Please report the probe-versus-oracle rank correlation for Delta and for s, and ideally show how sampling weights change when they are recomputed from probe-estimated rather than oracle Delta.","section":"§4.1, Eq. (1)–(2); Fig. 9"},{"comment":"The substitution of a 20-epoch probe on a 512-task subset for the main run is not validated against the main run's actual training budget. The main runs use 1 epoch over 17,398 tasks for Math and 3 epochs over 4,998 tasks for Logic, so the 20-epoch probe response may rank tasks differently from the 1-3 epoch response that the main run actually exploits. The reported endpoint correlations are computed against a 20-epoch oracle, not against epoch-1 or epoch-3 responses. Please provide evidence that the probe-derived ordering transfers to the short main-run horizon, or state more cautiously what the probe is a proxy for.","section":"§4.1, Reduction II; Table 4"},{"comment":"The 'lightweight' characterization of TRAJVAL is not supported by the cost accounting. The probe runs T=20 epochs on |P|=512, while the main runs consume 1 epoch over 17,398 tasks (Math) or 3 epochs over 4,998 tasks (Logic). In task-epoch units the probe is roughly 10,240/17,398 ~= 59% of the Math main-run compute and 10,240/14,994 ~= 68% of the Logic main-run compute. Appendix B.5 measures only the per-step selector overhead (0.01% of pipeline time) and omits this dominant probe training cost. Please report end-to-end compute including the probe and show that total-compute efficiency, not just steps-to-baseline, improves.","section":"§4.1; Appendix B.5; Table 4"},{"comment":"The within-pre-bin analysis validates oracle learnability, not the TRAJVAL estimator. The high-Delta and low-Delta subpools in Table 2 are formed from full 20-epoch oracle trajectories, so the result that Delta predicts utility beyond matched pre is about the oracle quantity. It does not show that the probe-derived TRAJVAL score preserves the same signal. Given that Table 3 shows pre-only reaches 0.4084 Best Acc on Math versus 0.4105 for full TRAJVAL, the marginal contribution of the probe-derived Delta term to the headline gains remains unclear. Please repeat the within-pre-bin comparison using probe-estimated Delta, and add an analysis of how much of the Table 1 gain survives when the headroom term is removed.","section":"§5.3, Table 2; Table 3"}],"minor_comments":[{"comment":"The pie chart in Figure 6(c) omits the Forgetting category in its legend, although this category appears in the profile taxonomy and in the other panels; please make the legends consistent.","section":"Figure 6"},{"comment":"The symbol T is overloaded: in the S2B definition it denotes the baseline's Best Acc, while in Section 4 it denotes the probe horizon. Please rename one of them.","section":"Appendix A.2.2 and §4.1"},{"comment":"Equation (1) uses acc(x, theta_t), but the text frequently refers to mean reward; please clarify whether the endpoint statistics are per-prompt binary accuracy, group-mean reward, or a normalized variant.","section":"Eq. (1) and surrounding text"},{"comment":"The statement that 'the score ranking stabilizes early' relies on Figure 9, which measures agreement between intermediate probe scores and the same probe's final score; this is self-consistency and should be phrased as such rather than as validation against the oracle.","section":"§5.4, 'Diagnostic analyses'"},{"comment":"The abstract and Section 3.2 state that learnability is 'predictive of downstream utility' before the probe estimator is introduced; consider separating the oracle-level claim from the claim that TRAJVAL approximates oracle learnability, to avoid conflating the two.","section":"Abstract and §3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the core idea is interesting. The main gaps are the missing validation of probe-derived Delta and the incomplete compute accounting; both are fixable with additional experiments and reporting. I would not reject on the current evidence, but I would require those points to be addressed before publication, and I would also ask the editor to weigh whether the 2-seed means without error bars are sufficient for the small differences in Table 1."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here is my read on 2608.09217. The core idea is worth taking seriously: current solvability is not the whole story for task value in LLM RL post-training, and per-task response to a fixed training regime is a measurable, reasonably stable property. The within-pre-bin analysis (Table 2) is the strongest part—matching average pre and showing high-delta subpools beat low-delta subpools on both domains is real evidence that learnability is distinct from solvability. The cross-context kappa values are also decent. Credit where due: they cite Dataset Cartography and ELN2, frame learnability as regime-conditional, and ship code. That is honest work.\n\nThe soft spots are real and mostly about the estimator, not the concept. First, the probe cost accounting is off. T=20 epochs on 512 tasks is roughly 59–68% of the reported main-run step budget (Math 1 epoch, Logic 3 epochs). Calling that \"lightweight\" and claiming the overhead is \"far offset\" by S2B is misleading unless the probe is amortized across models. The cross-regime reuse appendix helps, but the main results appear to use in-regime probes. Second, and more load-bearing: the paper validates probe r_early (0.940) and r_late (0.876) against oracle values but never reports the Spearman correlation for delta or for the actual score s(x). Delta is a difference score; its reliability can be much lower than the endpoints. Figure 9 only shows self-consistency of the probe at different epochs, not agreement with oracle or with main-run training dynamics. Since the main run is only 1–3 epochs and the probe runs 20 epochs, the ordering may not transfer. The ablation in Table 3 shows pre-only gets most of the Best Acc gain, with full TrajVal adding mainly S2B and small AUC gains—so the delta term is carrying less weight than the headline suggests. I would like to see delta reliability and per-seed variance reported, and a corrected compute comparison that includes probe cost.\n\nWho is this for: people working on data selection and task scheduling for LLM RL post-training. It is a solid incremental contribution, not a breakthrough. The scientific claim (learnability distinct from solvability) is supported by the oracle analysis; the practical claim (TrajVal estimates it cheaply) needs another round of evidence. Send it to peer review—it deserves a serious referee—but the referee should push on probe fidelity and honest cost accounting.","headline":"Learnability is distinct from solvability and the within-bin analysis supports that, but the probe estimator's reliability for delta and the true compute cost are both under-reported.","tokens_in":24938,"tokens_out":2262,"would_cite":true,"duration_ms":20933,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A task's response to continued RL training is a stable, measurable property — learnability — distinct from current solvability, and a short probe run can estimate it well enough to improve data selection.","keywords":["task learnability","reinforcement learning post-training","task scheduling","data efficiency","LLM reasoning","probe-based estimation","reward trajectories","training dynamics"],"falsifier":"Train the same model and schedule using TrajVal weights derived from a probe run of the same length as the actual main run (1 epoch on math, 3 on logic) and compare against weights from the full 20-epoch probe; if the short-horizon weights no longer beat uniform sampling, the claimed predictive ordering is an artifact of probe horizon. Alternatively, measure the actual per-task reward gain during the main run's budget and check whether its rank correlation with probe-predicted scores is high; a low correlation would refute the proxy assumption.","tokens_in":23895,"feed_emoji":"🎯","tokens_out":5496,"duration_ms":44022,"temperature":0.7,"pith_summary":"Reinforcement-learning post-training for large language models usually samples tasks uniformly or weights them by how well the current policy already solves them. This paper argues that those signals miss a separate axis: how positively a task responds to further optimization under a fixed training regime, which it names learnability. It reports that per-task reward trajectories under GRPO, a group-relative policy-gradient RL method, are heterogeneous even among tasks with matched initial pass rates, that this heterogeneity reproduces across independently sampled training contexts, and that selecting or weighting tasks by it improves data efficiency on math and logic benchmarks. To make the signal usable before training, the paper introduces TrajVal, which learns a static per-task score from a short probe run on about 3% of the pool plus two inference passes. The practical claim is that a cheap pre-training estimate of learnability yields faster convergence and higher accuracy than uniform sampling, and complements online schedulers.","feed_headline":"Short probe run predicts which tasks will improve under RL training","feed_subtitle":"Two reward measurements per task estimate learnability, accelerating math and logic reasoning training.","key_machinery":"The load-bearing object is the TrajVal score s(x) = (1 − \\bar{r}_early(x)) · max(0, \\bar{r}_late(x) − \\bar{r}_early(x)), built from two endpoint reward estimates per task. The first factor encodes learning headroom; the second encodes directional alignment of the reward trajectory. The estimator rests on two reductions: dense per-task reward trajectories are summarized by mean early-window and late-window accuracies, and full-pool training is replaced by a 20-epoch GRPO probe run on a uniform 512-task subset whose resulting model serves as a stand-in for a later training state. The resulting static weights can be used alone or composed multiplicatively with an online scheduler's dynamic weights as w_final = w_TrajVal · w_online, which the paper argues supplies the stable offline prior that online methods lack early in training.","core_discovery":"The paper's central claim is that learnability is a real and usable dimension of task value in LLM RL post-training: tasks with the same current solvability can differ sharply in how their rewards evolve under continued training, and this difference predicts downstream utility. In a controlled within-pre-bin analysis, after partitioning tasks into terciles by initial accuracy and matching average initial accuracy, the high-Δ subpools outperform low-Δ subpools on both Best Accuracy and AUC across all six domain-bin combinations. The paper further claims the signal is context-robust: agreement across independently sampled training sets reaches Cohen's κ = 0.879 for the binary learnable/non-learnable split, and a probe-trained model obtained from 20 epochs of RL on a 512-task subset preserves per-task endpoint ordering with Spearman ρ = 0.876 for late rewards. TrajVal operationalizes this by scoring each task as s(x) = (1 − \\bar{r}_early(x)) · max(0, \\bar{r}_late(x) − \\bar{r}_early(x)), then converting scores to sampling weights with power compression and a probability floor. If correct, the result reframes task valuation for RL post-training: instead of asking only what the policy can already solve, practitioners should ask which tasks are poised to improve.","pith_inferences":["A natural testable extension is to shorten the probe to roughly 70% of the nominal horizon and verify that the convergence gains persist, since the reported rank correlation reaches 0.85 by epoch 14.","The cross-regime reuse experiments suggest the probe cost can be amortized: a prior estimated on a smaller same-family model transferred to a larger one retained most of the in-regime gain, which if robust would make learnability priors a shareable asset across a model family.","Because learnability is defined as regime-conditional, recomputing the same score under different reward functions or base models would test how much of the signal is task-intrinsic versus regimen-dependent.","If the endpoint-pair estimator transfers to non-binary or model-graded rewards, the same lightweight recipe could apply to open-ended generation, although the paper leaves that case unverified."],"forward_implications":["As a standalone replacement for uniform sampling, TrajVal improves Best Accuracy and AUC in all four domain–scale configurations tested, with steps-to-baseline as low as 40% on Logic at 4B scale.","Multiplying TrajVal weights into the BOTS and GRESO online schedulers improves both accuracy metrics and steps-to-baseline in every reported configuration, indicating the learnability prior is complementary to online solvability estimates.","Training only on tasks labeled learnable, about 41% of the pool, matches the full pool's converged accuracy in 3.6× fewer GRPO steps.","Within each initial-solvability tercile, high-Δ tasks outperform low-Δ tasks with matched average initial accuracy, so the predictive signal is not explained by current pass rate alone.","The learnability score's ranking stabilizes by probe epoch 14 of 20, so the probe horizon can be shortened without materially changing the task ordering."],"supporting_citations":[{"why":"Supplies the GRPO algorithm used in all probe and main training runs.","marker":"Shao et al., 2024"},{"why":"Provides the DAPO-Math training pool for the math domain.","marker":"Yu et al., 2025"},{"why":"Provides the GURU Logic subset used for the logic domain.","marker":"Cheng et al., 2025"},{"why":"Defines the BOTS online scheduler that TrajVal is composed with as a multiplicative prior.","marker":"Shen et al., 2025"},{"why":"Defines the GRESO selective-rollout baseline that TrajVal is composed with.","marker":"Zheng et al., 2025"},{"why":"Supplies the Qwen3 model family used across the main-scale experiments.","marker":"Yang et al., 2025"},{"why":"Provides the training framework used to run the RL experiments.","marker":"Pan et al., 2025"},{"why":"Contributes the trajectory-based sample-characterization perspective that motivates learnability as distinct from snapshot solvability.","marker":"Swayamdipta et al., 2020"}],"fun_headline_variants":["Probe-run signal predicts task improvement in LLM RL","Task learnability prior boosts RL post-training efficiency","Two reward checks gauge which tasks will learn fastest","Static prior on learnability cuts RL training waste"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 20-epoch probe run on a 512-task subset is assumed to reproduce the training dynamics of the main run, which uses the full math or logic pool for only 1 or 3 epochs; if the probe's per-task ordering does not match the main budget's actual response, TrajVal's weights mis-rank tasks under the real training schedule.","fun_headline_variants_meta":{"raw":{"variants":["Probe-run signal predicts task improvement in LLM RL","Task learnability prior boosts RL post-training efficiency","Two reward checks gauge which tasks will learn fastest","Static prior on learnability cuts RL training waste"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000553,"raw_usage":{"total_tokens":2678,"prompt_tokens":1027,"completion_tokens":1651,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":1590}},"tokens_in":643,"tokens_out":1651,"duration_ms":11241,"temperature":1.0,"reasoning_tokens":1590,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:25:54.156859+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same model and schedule using TrajVal weights derived from a probe run of the same length as the actual main run (1 epoch on math, 3 on logic) and compare against weights from the full 20-epoch probe; if the short-horizon weights no longer beat uniform sampling, the claimed predictive ordering is an artifact of probe horizon. Alternatively, measure the actual per-task reward gain during the main run's budget and check whether its rank correlation with probe-predicted scores is high; a low correlation would refute the proxy assumption.","supporting_citations":[],"review_version":2}