{"id":"65411f92-c642-4efe-ae2d-d2e9622c2d9d","arxiv_id":"2412.02331","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"MUSEL selects training samples by multiplying model uncertainty, distance to known inputs, and learning progress, improving sample efficiency for simulated robot effect prediction.","lead":"Robots that learn from their own actions need many trials. This paper presents a rule, MUSEL, for choosing which action to try next, combining model uncertainty, input diversity, and learning progress, and reports better sample efficiency in two simulated tabletop tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an unvalidated uncertainty decomposition, and the reported gains over the strongest baseline, MD-only, are not shown to be statistically reliable.","rationale":"I read the paper in good faith. The method is plausible and the experiments are internally coherent: selecting by Eq. 8 plausibly beats random sampling in these simulated tasks, and the ablation and sampling histograms support a qualitative story about boundary-focused exploration. The reader's weakest_assumption correctly identifies the factorization as heuristic and unverified. I partially agree, but I would shift the load-bearing emphasis to the empirical side: even if the decomposition is heuristic, the central claim 'MUSEL improves learning accuracy and sample efficiency' could survive if the empirical advantage over the strongest baseline (MD-only) is statistically robust. The paper does not provide such evidence: no significance tests, no effect sizes, no confidence intervals, and only graphical RMSE curves with SEM. Since σ and ℓ are correlated and LP is region-constant, the unique contribution of the new decomposition may be small; the one-sphere results suggest this, with MD-only nearly matching MUSEL. The two-sphere results are described as showing a larger gap, but no numeric backing is given. A paired significance test on the existing per-seed data would settle whether the central claim is supported beyond noise. Because the requested analysis is missing and the results cannot be independently checked, the reader's CONDITIONAL verdict remains appropriate; I do not see grounds for rejection, but the paper needs the statistical support and, ideally, release of code and per-seed logs.","tokens_in":9405,"tokens_out":6769,"duration_ms":89358,"concrete_test":"Run paired two-sided Wilcoxon signed-rank tests (or paired bootstrap with 10^4 resamples) on the per-seed RMSE values at each reported checkpoint (500, 1000, 1500, 2000, 2500, 3000) comparing MUSEL vs MD-only in both the one-sphere and two-sphere tasks, and report 95% confidence intervals for the final RMSE difference. If MUSEL is not significantly lower than MD at a majority of checkpoints, or the confidence interval includes zero, the claimed benefit of the model-uncertainty decomposition over a standard diversity heuristic is not established. This test requires only the authors' existing per-seed logs or a rerun of the published experimental protocol.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is the factorization σ(x) = U_model(x)·U_data(x) (Eq. 3) with U_data(x) = 1/(ℓ(x)·LP(x)) (Eq. 4), which is asserted by analogy to [14] and never checked against a ground-truth uncertainty decomposition. In the deterministic tabletop simulator, aleatoric data uncertainty should be near zero, so interpreting 1/(ℓ·LP) as data uncertainty is conceptually suspect. The more decisive problem for the central empirical claim is that the experiments do not separate the proposed model-uncertainty mechanism from the MD component. In the one-sphere task, MD-only 'closely approached' MUSEL, and the only quantitative comparison provided is a boundary-count table (Table 1) rather than RMSE values; in the two-sphere task, the claimed larger gap is described verbally with no effect sizes or significance tests. Because σ and ℓ are strongly correlated (both are large far from training data) and LP is constant within each grid region, the ranking produced by Eq. 8 is largely driven by ℓ. Without paired significance tests, the 10-seed means with SEM error bars do not establish that the proposed decomposition adds a reliable advantage over standard minimum-distance sampling.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces MUSEL, an active-learning score U_model(x) = sigma(x) * ell(x) * LP(x) (Eq. 8) for selecting state-action pairs in continuous supervised effect-prediction regression. The score is motivated by a factorization of SVDKL total uncertainty sigma into model and data parts (Eqs. 3-4), with data uncertainty taken to be the inverse product of minimum distance to training inputs ell and region-level learning progress LP. The method is evaluated in two simulated tabletop pushing tasks against random sampling, single-component sampling, and component-ablated versions, reporting RMSE over 3000 iterations with 10 seeds. The paper claims lower RMSE and higher sample efficiency, particularly in the two-sphere task.","tokens_in":9568,"tokens_out":8256,"duration_ms":88228,"significance":"If substantiated, MUSEL would be a practical and relatively simple active-learning criterion for continuous robot regression, and the use of SVDKL to obtain both total uncertainty and a learned feature space is sensible. The experimental design has strengths: ten seeds with SEM error bars, component ablations, two tasks, and diagnostic analyses of sampling distributions and LP profiles. However, the central decomposition is asserted rather than verified, and the empirical advantage over the strongest baseline, MD-only sampling, is not established statistically. The contribution would be more convincing with paired significance tests, numeric RMSE tables, and an analysis of how much of the ranking is driven by ell alone. No code or supplementary material is referenced, which limits repeatability.","major_comments":[{"comment":"The factorization U(x) = sigma(x) = U_model(x) * U_data(x) with U_data(x) = 1/(ell(x) * LP(x)) is load-bearing for the paper, but it is adopted from [14] and is not tested on the deterministic tabletop tasks. In those tasks the environment transition is deterministic, so the aleatoric component of data uncertainty is arguably negligible; interpreting 1/(ell*LP) as data uncertainty is therefore not self-evident. A concrete check would be to compare the estimated U_model against the actual reduction in predictive error achieved by adding a candidate sample across multiple regions; without such a check, Eq. (8) is one of several possible weightings of sigma, ell, and LP rather than a grounded extraction of model uncertainty. I do not see a circularity problem, since Eq. (8) follows algebraically from the assumed definitions, but the assumption itself needs empirical justification.","section":"Section 3.4, Eqs. (3)-(4)"},{"comment":"The central claim that MUSEL improves both learning accuracy and sample efficiency is not supported by statistical evidence. In the one-sphere task, MD-based selection closely approached MUSEL, and Table 1 reports only boundary-sample counts, not RMSE comparisons; in the two-sphere task the advantage is described as more pronounced without numeric RMSE values, effect sizes, or test statistics. With ten seeds per condition, paired significance tests (e.g., Wilcoxon signed-rank at each iteration) and a table of mean RMSE with confidence intervals should be reported. This is load-bearing because MD-only is the strongest baseline and the abstract's claim depends on a reliable advantage over it.","section":"Section 4.2-4.3, Figs. 4 and 6, Table 1"},{"comment":"The proposed score may be dominated by the ell factor. ell(x) is the minimum distance to training inputs and sigma(x) tends to be large in the same low-density regions, while LP(x) is constant within each grid region by construction (Section 3.4). The paper does not quantify the rank correlation between U_model and ell, nor does it provide an ablation that removes the sigma or LP contribution without also changing ell. Reporting the marginal contribution of each factor, or an experiment in which one factor is held fixed in the ranking, would address the concern that the decomposition adds little over standard minimum-distance sampling.","section":"Section 3.4, Eq. (8)"},{"comment":"Several hyperparameters that directly affect the LP term are missing or under-specified. The window p in 'last p average RMSE errors' is never given; the region-grid resolution is reported as 7x7x7, but the handling of LP for regions with no executed samples is not stated, and the exact mapping implied by Eq. (7) is not fully consistent with the claimed [10^-4, 1] range. The 'prioritized training scheme' with mtrain=2000 is also described only verbally. These omissions prevent reproduction of the experiments and leave open the possibility that the reported rankings depend sensitively on unstated choices.","section":"Section 4.1, Eqs. (6)-(7)"}],"minor_comments":[{"comment":"The notation |r_i|, used as the number of executed inputs in region i, is not defined before use; please add a definition.","section":"Section 3.4, Eq. (6)"},{"comment":"The test grid dimensions are given as 25x20x20 and 20x25x25, but it is not stated which axis corresponds to the push angle and which to posx/posy; clarify the ordering.","section":"Section 4.1"},{"comment":"The right panel lacks a caption describing the line styles and the meaning of 'full MUSEL' in the ablation plot; the current caption only refers to the left panel.","section":"Figure 4"},{"comment":"The model uncertainty estimator Eq. (8) depends on LP computed from executed samples, but the algorithm does not state how LP is initialized for candidate regions that have not been visited yet; please clarify the default LP value.","section":"Algorithm 1, line 7"},{"comment":"The sentence 'Note that the maximum of this value is often used in active learning algorithms for sample selection [34]' is ambiguous because Eq. (4) is an inverse product; rephrase to refer to ell(x) directly.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the applied ML/robotics scope of the venue, but in its current form the evidence for the core claim is not sufficient for acceptance. I would support reconsideration after a revision that adds paired statistical tests, numeric RMSE tables, and a validation or at least a sensitivity analysis of the uncertainty decomposition. The lack of code is a further drawback for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read of Eren & Oztop, arXiv:2412.02331. The genuinely new bit is the acquisition function U_model = σ · ℓ · LP (Eq. 8): SVDKL predictive uncertainty times minimum distance to training inputs times region-level learning progress. That exact product isn't in the cited literature; existing work uses those signals separately. The paper evaluates it on two simulated pushing tasks with ten seeds, ablations, and single-component baselines.\n\nWhat it does well: the empirical setup is honest and reasonably thorough. MUSEL beats random and σ-only sampling, roughly matches MD-only in the one-sphere task, and shows a larger gap over MD-only in the two-sphere task. The ablations are informative: removing MD hurts most, removing LP hurts least, which tells you where the leverage is. The SVDKL backbone is standard but appropriate. The authors also note the high variability of LP, which is a fair self-criticism.\n\nThe soft spots are real but not fatal. The conceptual framing is the weakest part: Eq. 3–4 assert that total uncertainty factorizes into model and data uncertainty, with data uncertainty set to 1/(ℓ·LP). That's a heuristic guess, not a derived result, and in a deterministic simulator the notion of irreducible data uncertainty is dubious. The method would stand better if presented as a product of three complementary exploration signals. The empirical case against MD-only is also under-powered: no significance tests, no effect sizes, and the two-sphere claim is described verbally. The one-sphere gap is modest, and the boundary-count table doesn't substitute for RMSE comparisons. No code or data are released, and one key hyperparameter, the LP window p, is left unspecified.\n\nThat said, the central methodological claim is plausible: the product heuristic works about as well as the best component and often a bit better, without needing the questionable decomposition to be true. It's a legitimate method paper for the robot active-learning niche, not a field-reorganizing result. A serious referee should see it; I'd send it to review with requests for paired significance tests, a clearer statement of the heuristic rather than the data-uncertainty story, and ideally code or a full hyperparameter table.\n\nBottom line: engage with it, but don't let the conceptual gloss outrun the evidence. A conditional acceptance with revisions is the right outcome.","headline":"A useful heuristic combination of three known sampling signals for robot regression, presented with an overclaimed conceptual decomposition and thin statistical evidence for its edge over the strongest baseline.","tokens_in":10180,"tokens_out":2095,"would_cite":false,"duration_ms":26952,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One acquisition score, multiplying predictive variance, learning progress, and input distance, improves sample efficiency in robot effect-prediction tasks.","keywords":["active learning","sample efficiency","model uncertainty","learning progress","deep kernel learning","robot effect prediction","uncertainty decomposition"],"falsifier":"On a deterministic task where the effect function is noiseless, true data uncertainty is zero, so $U_{data}$ should collapse to a constant; if $U_{model}$ computed with the paper's formula still changes sample selection and improves RMSE relative to using $\\sigma$ alone, then the gain comes from heuristics rather than from the claimed decomposition. A direct check would compare the ranking produced by $\\sigma(x)\\ell(x)\\mathrm{LP}(x)$ against a ground-truth decomposition obtained by repeatedly training models on growing datasets to isolate the reducible error for each input.","tokens_in":9084,"feed_emoji":"🤖","tokens_out":3567,"duration_ms":39411,"temperature":0.7,"pith_summary":"This paper claims that an active-learning score called MUSEL lets a robot learning action–effect mappings from its own pushes need fewer executed actions. The score multiplies three terms: the model's predictive variance, a learning-progress signal, and the minimum distance of a candidate state–action pair from already collected inputs. In two simulated tabletop push tasks, MUSEL reaches lower prediction error than random sampling and than any of the three terms used alone, with the largest gap in the more complex two-sphere task. This matters because self-supervised robot learning pays per executed action, so choosing which actions to try directly controls data-collection cost.","feed_headline":"Score picks robot's next push to cut data needs","feed_subtitle":"MUSEL weighs uncertainty, learning progress, and input distance to learn effect prediction from fewer executed actions.","key_machinery":"The load-bearing object is the multiplicative uncertainty decomposition $U(x)=\\sigma(x)=U_{model}(x)U_{data}(x)$, adopted from the literature, together with the assumption that learning-progress rate and distance to training data jointly capture irreducible data uncertainty. Learning progress is computed on a grid of regions in input space: per-region average errors over the last $p$ iterations are linearly fitted against step counts, and the fitted slope is mapped into $[10^{-4},1]$ via an arctan transform. The product in equation (8) ranks candidate inputs, and the top-$k$ are executed and incorporated into the training set, with the Stochastic Variational Deep Kernel Learning model providing $\\sigma(x)$ and the predictive mean.","core_discovery":"On the paper's own terms, the central discovery is that model uncertainty for a continuous state–action input $x$ can be extracted from the total predictive uncertainty $\\sigma(x)$ by dividing out an estimate of data uncertainty $U_{data}(x)=1/(\\ell(x)\\,\\mathrm{LP}(x))$, where $\\ell(x)$ is the minimum Euclidean distance to training inputs and $\\mathrm{LP}(x)$ is a region-level learning-progress estimate. The resulting acquisition score $U_{model}(x)=\\sigma(x)\\ell(x)\\mathrm{LP}(x)$ is used to rank candidates from a population sampled over the continuous input space, and the top-$k$ are executed and added to the training set. The paper reports that this score outperforms random selection and single-criterion selection in a one-sphere push task, and exceeds even the strongest single component (minimum-distance sampling) in a two-sphere collision task, where convergence is noticeably earlier.","pith_inferences":["The paper does not compare MUSEL against a simple additive combination of the same three normalized components; if a weighted sum matched its performance, the multiplicative factorization would not be the essential ingredient.","In the paper's deterministic simulator, true aleatoric data uncertainty is near zero, so the theory predicts $U_{data}\\approx 1$; a direct comparison of $\\sigma$-only sampling with full MUSEL in that regime would separate the factorization's contribution from the coverage effect of minimum-distance sampling.","The $7\\times7\\times7$ LP grid is a fixed free parameter, and its adequacy will need re-examination as the state–action dimensionality grows in more complex robot tasks.","The claimed generality to any effect-prediction setting presumes a metric, locally smooth input space; discrete or symbolic state representations would break the minimum-distance and region-local learning-progress estimates."],"forward_implications":["If the score is right, self-supervised effect prediction can run with fewer executed robot actions, lowering energy, oversight, and experimental time in real-robot deployment.","The two-sphere result suggests the advantage over random and MD-only sampling grows with task complexity, and earlier convergence may make the method usable in online learning settings.","Because the score uses only quantities available during training (predictive variance, prediction errors, and input distances), it can be added to an existing SVDKL-based learner without an external oracle.","The ablation experiments imply the three terms are partly redundant: dropping minimum distance hurts most, while dropping learning progress or predictive variance hurts least, so a leaner variant may retain most of the benefit."],"supporting_citations":[{"why":"Supplies the statistical independence assumption and the multiplicative data/model uncertainty decomposition used in equation (3).","marker":"[14]"},{"why":"Provides the Stochastic Variational Deep Kernel Learning backbone that outputs calibrated predictive variance and supports scalable training.","marker":"[31]"},{"why":"Defines the max-min distance sampling used both as the MD component of MUSEL and as a strong baseline in the experiments.","marker":"[34]"},{"why":"Establishes learning progress as an intrinsic motivation signal that MUSEL borrows to estimate data uncertainty.","marker":"[21]"},{"why":"Introduces the sparse variational GP approximation that makes the SVDKL backbone scalable to streaming datasets.","marker":"[12]"}],"fun_headline_variants":["MUSEL: fewer robot demos, smarter effect learning","Uncertainty score cuts robot learning data needs","Robot picks informative pushes via model uncertainty","Sample-efficient robot effect prediction with MUSEL","New metric guides robot sampling for faster learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme assumes that the product of learning progress and minimum distance to training inputs correctly measures the part of prediction error that no amount of extra data can remove, and that this factorization holds as a plain multiplication of three scalar signals.","fun_headline_variants_meta":{"raw":{"variants":["MUSEL: fewer robot demos, smarter effect learning","Uncertainty score cuts robot learning data needs","Robot picks informative pushes via model uncertainty","Sample-efficient robot effect prediction with MUSEL","New metric guides robot sampling for faster learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000209,"raw_usage":{"total_tokens":1384,"prompt_tokens":897,"completion_tokens":487,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":513,"completion_tokens_details":{"reasoning_tokens":417}},"tokens_in":513,"tokens_out":487,"duration_ms":6218,"temperature":1.0,"reasoning_tokens":417,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T23:35:09.658477+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a deterministic task where the effect function is noiseless, true data uncertainty is zero, so $U_{data}$ should collapse to a constant; if $U_{model}$ computed with the paper's formula still changes sample selection and improves RMSE relative to using $\\sigma$ alone, then the gain comes from heuristics rather than from the claimed decomposition. A direct check would compare the ranking produced by $\\sigma(x)\\ell(x)\\mathrm{LP}(x)$ against a ground-truth decomposition obtained by repeatedly training models on growing datasets to isolate the reducible error for each input.","supporting_citations":[{"cited_title":"IEEE Access (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the statistical independence assumption and the multiplicative data/model uncertainty decomposition used in equation (3)."},{"cited_title":"Advances in neural information processing systems29 (2016)","cited_arxiv_id":null,"evidence_quote":"Provides the Stochastic Variational Deep Kernel Learning backbone that outputs calibrated predictive variance and supports scalable training."},{"cited_title":"Information Sciences 474, 90–105 (2019)","cited_arxiv_id":null,"evidence_quote":"Defines the max-min distance sampling used both as the MD component of MUSEL and as a strong baseline in the experiments."},{"cited_title":"IEEE transactions on evolutionary computation 11(2), 265–286 (2007)","cited_arxiv_id":null,"evidence_quote":"Establishes learning progress as an intrinsic motivation signal that MUSEL borrows to estimate data uncertainty."}],"review_version":1}