{"id":"9baeee39-5bc2-4a33-b042-a35c9e155032","arxiv_id":"2607.23108","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"For high-precision manipulation, required demonstration count grows as log(N) ∝ 1/(P−c), where the fitted c varies with sensors, expert, and task complexity.","lead":"This paper reports an empirical scaling law for high-precision robotic manipulation: the number of demonstrations needed grows super-exponentially as the target precision approaches a fitted limit c. The authors propose c as a measurable capability ceiling of the whole system, offering a new way to debug and plan precision assembly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-c fit in §IV-A enforces SR-independence rather than testing it; separate c estimates per target SR could contradict the central claim.","rationale":"The reader's weakest assumption—that c independence is enforced rather than tested—is exactly the most load-bearing soft spot. The paper's headline claim rests on c being a system-intrinsic, SR-independent parameter; the fitting protocol in Section IV-A imposes that property by grid-searching a single c across all target-SR curves. No independent estimate of c per SR is reported, so the evidence is circular. A separate-c fit with bootstrap confidence intervals is a concrete, feasible check that would settle the issue. If the check fails, the central claim is unsupported. If it passes, the law is more credible. The reader's CONDITIONAL verdict appropriately reflects this uncertainty, and my stress-test does not change that verdict. I agree with the reader's identification, though I would emphasize that the lack of seed replication and the extrapolation of N values amplify the concern: even a separate-c analysis based on the current single-run data could be inconclusive, so a fuller test would ideally include multiple training seeds per (N,P) condition.","tokens_in":11488,"tokens_out":5729,"duration_ms":60464,"concrete_test":"Re-fit the precision scaling law separately for each target success rate (SR=0.5, 0.7, 0.9), allowing a distinct c per curve rather than a shared c. Use bootstrap resampling over the 300 evaluation trials per (N,P) condition to propagate uncertainty into N and obtain confidence intervals for each c. If the three c confidence intervals do not overlap (or exclude each other's point estimates), the SR-independence hypothesis is falsified and the central claim fails. As a secondary check, hold out one precision level (e.g., P=8 mm) from fitting and compare its predicted N at each SR to the empirically derived N; large deviations would indicate the law is not predictive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central novelty is that the limit precision c is an intrinsic, target-SR-independent property of the agent system. Section IV-A states that a single c is grid-searched to maximize the sum of R² across all three target-SR curves, 'enforcing' the hypothesis that c is independent of success rate. This is circular: the data are never allowed to show that c varies with target SR. The reported high R² for each SR under a shared c is a result of the imposed constraint, not evidence for it. If separate c values were estimated for each target SR and found to differ meaningfully, then (i) c is not a system-intrinsic metric, (ii) the precision law's functional form may be an artifact of forcing a common asymptote, and (iii) the 'emergent property' interpretation collapses. Moreover, the N values used in these fits are themselves derived from interpolating or extrapolating the success-rate power-law fits, which are based on single training runs (no seeds) and only 4–6 precision levels; uncertainties in N are never propagated into the c estimate or the R². Thus the empirical support for the load-bearing claim is weaker than the paper implies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a 'precision scaling law' for imitation learning in closed-world manipulation: for a fixed target success rate, the number of demonstrations N required grows super-exponentially as the task precision P approaches a limit c, modeled as log(N) = m/(P − c) + n. The authors further claim that c is not a fixed physical constant but an intrinsic, target-SR-independent property of the whole agent system (sensors, expert, task complexity). The law is validated in ManiSkill3 on Peg Insertion, Stack Cuboid, and Roll Ball, with power-law SR-vs-N fits used to interpolate N at target SRs, and a shared-c grid search used to fit the precision law. Ablations on Peg Insertion show c varying with observation modality, expert strategy, and task complexity.","tokens_in":11849,"tokens_out":3122,"duration_ms":37784,"significance":"If the central claim holds, c would be a practically useful, quantitative system-capability metric and the scaling law would give a predictive tool for data requirements in high-precision tasks. The paper also makes a useful conceptual shift from open-world scaling-for-breadth to closed-world scaling-for-precision. Positive features include high in-sample R² values across three tasks and several ablations, a clear demonstration that the law fails for an under-capacity model (R² = 0.22 vs 0.99), and a falsifiable functional form. However, the load-bearing claim that c is independent of target success rate is not actually tested—it is imposed by the fitting protocol—and the supporting statistics rest on single training runs and on N values obtained by interpolation/extrapolation from prior fits. These issues are fixable, but they currently weaken the empirical support for the paper's headline interpretation.","major_comments":[{"comment":"Each (N, P) data point corresponds to a single training run, yet the precision-law fits use N values obtained by interpolating or extrapolating the SR power-law fits from those same single runs. No seed variance is reported, and uncertainties from the first-stage fits are never propagated into the second-stage estimates of N or c. With only 4–6 precision levels per task, a single outlier run could materially change a, b, and hence the inferred N and c. The authors should either run multiple seeds (at least 3) or use a bootstrap/joint-fitting procedure to report confidence intervals on c and on the predicted N. They should also state explicitly which target-SR N values are interpolated versus extrapolated; extrapolation beyond the observed SR range (e.g., SR = 0.9 when the largest N only reaches lower SRs) would be a serious concern.","section":"IV-A, 'Policy Training Sweep' and 'Curve Fitting'"},{"comment":"The reported R² values are in-sample and are computed on the same data used to select c by grid search. This is not a predictive validation of the law, and it does not distinguish the proposed 1/(P−c) form from other flexible two- or three-parameter curves. Since the stated practical value of the law is prediction (Section V), the authors should provide a held-out test, e.g., fit on a subset of precision levels and predict N at the remaining levels, or report leave-one-precision-out cross-validated error. Such a test would also strengthen the ablations in Table III, which otherwise share the same circular fitting protocol.","section":"IV-B and IV-C, Tables II and III"}],"minor_comments":[{"comment":"Typo: 'disabling the wrist-mounted camer' should be 'camera'.","section":"IV-C, 'Impact of Observation Modality'"},{"comment":"The phrase 'these two axes of scaling, Breadth and Depth, are largely orthogonal' is clear, but 'Depth' is used informally. Consider defining it once in the introduction to avoid confusion with the U-Net model capacity discussion in IV-D.","section":"II-A"},{"comment":"For Roll Ball, P is described as the radius of the target region, but Table I uses mm. That is fine, but the range 35–200 mm is very different from the peg/stack ranges; the text should make explicit that the absolute scale of c is task-dependent and not comparable across tasks.","section":"IV-A, Table I"},{"comment":"The simulation-only scope is appropriately acknowledged. The paper would be stronger if the authors also noted that Diffusion Policy is a single algorithm choice and that c may depend on the learning algorithm, not only on sensors and expert.","section":"V, Limitations"},{"comment":"Panel (d)–(f) would benefit from showing the raw (P, N) points with error bars, rather than only the linearized fit lines, so readers can assess scatter and extrapolation regions.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The central empirical claim is interesting and the paper is well within scope for ICRA. The main issue is not the existence of a scaling phenomenon—the high in-sample R² and the model-capacity failure mode suggest something real—but the fact that the SR-independence of c is assumed by construction rather than measured. This is fixable in revision with separate per-SR c fits, uncertainty quantification, and a held-out evaluation. If those analyses contradict SR-independence, the paper's headline interpretation would need substantial revision. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is the first paper I know to propose a concrete precision scaling law for closed-world manipulation: log(N) ∝ 1/(P−c), with c as a system capability limit. Second, the paper's headline interpretation — that c is an intrinsic property independent of the target success rate — is partly an artifact of the fitting procedure, not a discovery. The authors grid-search a single c that maximizes the summed R² across three target-SR curves, which imposes the constancy they then claim to validate.\n\nWhat it does well: the experimental effort is real — over 100 training runs in ManiSkill3, three tasks, tens of thousands of evaluations. The failure-rate power law in N replicates an earlier result, but the 1/(P−c) form is new and fits the data remarkably well (R² > 0.97 in most cases). The ablations are the most convincing part: removing the wrist camera raises c from 2.35 to 3.85 mm, a cleaner expert strategy lowers it to 1.27 mm, and reduced task complexity lowers it to 1.00 mm. Those shifts make sense and suggest c is capturing something real. The model-capacity failure (R² = 0.22 versus 0.99) is an honest and useful caveat. The authors also clearly state the simulation-only scope and BC-only setting.\n\nThe soft spots are exactly where the reader's stress-test lands. Section IV-A states that the grid search 'enforces the hypothesis that c is a constant property.' So the high R² for each target SR under a shared c is not evidence for that hypothesis. You need to fit c separately for each target SR and show the differences are within uncertainty. Second, the N values at target success rates are interpolated or extrapolated from power-law fits that themselves have error, and each (N,P) data point is a single run. No seed variance is reported, and no held-out precision point is predicted. The in-sample R² is a fit, not a validation. That said, the functional form could have failed and did fail for the small-capacity model, so there is genuine signal here. The paper overstates the strength of the evidence, but not to the point of empty rhetoric.\n\nWho this is for: anyone working on high-precision imitation learning, assembly, or scaling laws in robotics. The c metric and the ablations are thought-provoking. The paper deserves a serious referee, but with required revisions: separate c fits, uncertainty quantification, multi-seed runs, and an out-of-sample prediction, e.g., hold out one precision level and predict the N needed for a target SR. I'd recommend peer review, not desk rejection, with the expectation of substantial revision.","headline":"An interesting empirical scaling law for precision in manipulation, but the central claim that c is target-SR-independent is enforced by the fitting protocol rather than tested.","tokens_in":12297,"tokens_out":2079,"would_cite":true,"duration_ms":22984,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"To hold success rate fixed, required demonstrations grow super-exponentially as target precision nears a system-defined limit c.","keywords":["precision scaling law","imitation learning","high-precision manipulation","behavior cloning","limit precision","system capability metric","super-exponential scaling","data scaling"],"falsifier":"Estimate c independently for two different target success rates, e.g., 0.5 and 0.9, using fresh data at several precision levels; if the two fitted c values differ beyond statistical uncertainty, the claim that c is a single system-level constant fails.","tokens_in":11409,"feed_emoji":"🤖","tokens_out":7567,"duration_ms":66348,"temperature":0.7,"pith_summary":"This paper tries to establish a quantitative law connecting the amount of demonstration data needed by an imitation-learning robot to the precision demanded by a closed-world manipulation task. The claim is that to hold a fixed success rate, the number of demonstrations N grows super-exponentially as the required precision P approaches a system-specific ceiling c, captured by log N ∝ 1/(P−c). The paper further claims that c is not a fixed physical limit but an emergent property of the whole agent—sensors, expert demonstrations, and task complexity—so it can serve as a measurable capability metric. If true, engineers could predict data costs before scaling up, and use c as a diagnostic for debugging high-precision systems. The evidence comes from simulated peg insertion, cuboid stacking, and ball rolling tasks using a standard behavior-cloning policy.","feed_headline":"Scaling law says robot precision data needs grow super-exponentially","feed_subtitle":"A scaling law connects demonstrations to tolerance, and the fitted limit c benchmarks the whole robot system.","key_machinery":"The paper's central object is the precision scaling identity log(N) = m/(P−c)+n, in which P is the task precision (a tolerance in millimeters), N is the number of demonstrations, and c is the 'limit precision'—the best precision the system can ever reach, in the same units as P. The identity does two kinds of work: it converts the intuitive 'curse of precision' (each extra decimal of accuracy costs vastly more data) into a quantitative prediction, and it turns c into a scalar that summarizes the entire agent's capability. The empirical method couples two fits: a per-P power law for failure rate versus data, and then a single global c chosen to maximize the coefficient of determination across","core_discovery":"Across three simulated manipulation tasks, the paper reports the precision scaling law log(N) = m/(P−c)+n: at a fixed target success rate, the number of demonstrations N needed to reach precision P grows super-exponentially as P approaches the system's limit precision c. The authors find that a single shared c fits data for three different target success rates in each task, leading them to interpret c not as a fixed property of the task geometry but as an emergent capability of the whole agent—sensors, expert data, and task complexity. Ablations on a peg-insertion task support this: removing the wrist camera raises c from 2.35 mm to 3.85 mm; switching to a less ambiguous expert demonstration","pith_inferences":["If c is truly independent of the target success rate, a short data sweep at one moderate precision could be used to extrapolate the full data/precision frontier, letting teams A/B test hardware or data changes without exhaustive full-scale training.","The form log N ∝ 1/(P−c) suggests a 'precision frontier' analogous to compute-efficient frontiers: for any budget of demonstrations, there is an optimal precision to target, and pushing past it wastes data.","A direct testable extension: on physical hardware, c should track measurable system noise—sensor resolution, control latency, and calibration error—so it could become a standard benchmark number reported alongside robot setups.","Demonstration clarity lowering c more than expert success rate hints that automatic data cleaning or demonstration editing could be a cheaper route to precision than collecting more or better teleoperation data."],"forward_implications":["With c fitted from a few experiments, engineers can predict the demonstration budget needed for any target precision before running a large data collection.","Because c changes with system components, it provides a quantitative, directly comparable capability metric for sensor suites, expert policies, and task formulations.","The super-exponential growth near c implies that brute-force data scaling is an inefficient path to sub-millimeter reliability; improving system components is the lever that moves the limit.","The law holds across image-based assembly and a state-based dynamic task, indicating it may generalize beyond a single task family."],"fun_headline_variants":["Precision scaling law: demos explode as tolerance nears its limit","Robots need super-exponential demos to hit precision targets","Precision wall is not fixed—it's set by the robot's whole system","Wrist camera lowers robot precision limit, says new scaling law","Data-to-precision law: super-exponential cost as you approach the edge"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper assumes that the limit precision c is a single constant for the system regardless of the target success rate, because it fits one shared c across all target success-rate curves; if c actually shifts with the target success rate, the interpretation of c as an intrinsic capability metric collapses.","fun_headline_variants_meta":{"raw":{"variants":["Precision scaling law: demos explode as tolerance nears its limit","Robots need super-exponential demos to hit precision targets","Precision wall is not fixed—it's set by the robot's whole system","Wrist camera lowers robot precision limit, says new scaling law","Data-to-precision law: super-exponential cost as you approach the edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000585,"raw_usage":{"total_tokens":2585,"prompt_tokens":740,"completion_tokens":1845,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":1750}},"tokens_in":484,"tokens_out":1845,"duration_ms":13099,"temperature":1.0,"reasoning_tokens":1750,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T03:33:51.350127+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Estimate c independently for two different target success rates, e.g., 0.5 and 0.9, using fresh data at several precision levels; if the two fitted c values differ beyond statistical uncertainty, the claim that c is a single system-level constant fails.","supporting_citations":[],"review_version":1}