{"id":"5197580d-3fde-4a1b-ba45-43a558d9e8c7","arxiv_id":"2504.16183","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Accounting for shape-completion uncertainty in the grasp quality score improves the success rate of the top five ranked grasps in table-top robot experiments.","lead":"Robots that fill in hidden parts of objects can also estimate how unsure they are about the filled-in points. This paper uses that uncertainty to re-rank grasp candidates, and reports higher success rates for the top five grasps in table-top tests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The +7 p.p. rank-5 gain is not attributable to the uncertainty signal: the proposed arm generates 15 candidates and re-ranks them, while baselines keep the top 5 by the original score, and no control replaces uncertainty with a placebo re-ranking.","rationale":"The central empirical claim is that MC-dropout uncertainty re-ranking raises rank-5 grasp success (58% vs 51% over GPD+3DSGrasp). The 3000-attempt physical experiment is a genuine strength, and the paper is honest about the 15-candidate choice in Section IV. The load-bearing gap is causal isolation. The proposed pipeline and the baseline differ in the number of candidates offered to the selector and in the score used to select; no factorial ablation or placebo arm separates these. The reader's REJECT is the right verdict; I would only sharpen the needed control: re-ranking the top 15 by the original GPD score is likely to reproduce the baseline top 5, so the decisive arm is a non-uncertainty re-ranking of the same 15 candidates. If a random or visibility-based perturbation replicates the gain, the uncertainty measurement is not the operative ingredient; if it does not, the claim gains support. The Monte Carlo dropout premise is also asserted rather than verified, but the experimental-control gap is the more immediate obstacle to the headline claim. Verdict unchanged.","tokens_in":10700,"tokens_out":12325,"duration_ms":123168,"concrete_test":"Run a single control experiment on the logged top-15 candidates: (A) re-rank the same 15 by the original GPD score alone and execute the top 5; (B) re-rank the same 15 by Equation (2) with the uncertainty term replaced by an equal-magnitude random perturbation (or by a binary observed-vs-completed penalty) and execute the top 5. If arm B reproduces the 58% success of GPD+ours, the claim is not specific to MC-dropout uncertainty; if arm A differs from the reported GPD+3DSGrasp 51%, candidate-count selection is also a confound. Report bootstrap confidence intervals over the 10 objects for each arm.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table I supports the headline claim only by comparing pipelines that differ in two ways at once. The GPD+3DSGrasp baseline executes the top 5 grasps of GPD's original ranking, while GPD+ours generates 15 GPD candidates, re-scores all 15 with Equation (2), and executes the new top 5. Even if GPD's top 5 from a 15-candidate request coincide with its top 5 from a 5-candidate request, the uncertainty-modified score can reorder the top 5 or promote candidates originally ranked 6-15; the experiment therefore separates neither the effect of considering a larger candidate set nor the effect of the uncertainty penalty itself. Section IV reports only that generating 5, 10, or 15 grasps has similar runtime, not that success is insensitive to candidate count. No arm re-ranks the same 15 candidates using the unmodified GPD score, a random perturbation, or a non-uncertainty heuristic such as distance from the camera. Without that control, the +7 p.p. over GPD+3DSGrasp could be produced by any re-ranking signal that pulls grasps toward the visible, low-standard-deviation side; Section VI explicitly notes this side is the robot-near side, so the Monte Carlo dropout uncertainty may not be the operative ingredient.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to quantify the uncertainty of a point-cloud shape-completion network (3DSGrasp) at inference using Monte Carlo dropout with T=60 forward passes, producing both an averaged completed point cloud and a per-point standard deviation. This uncertainty is incorporated into the GPD grasp quality score by subtracting a weighted sum of standard deviations over the points inside each grasp candidate's bounding box (Eq. (2)), after which the top 15 grasp candidates are re-ranked and the new top 5 are executed. Real-robot experiments on 10 YCB objects with a Kinova Gen3 and Robotiq 2F-85 compare top-5 grasp success against GPD, GPD+3DSGrasp, PointNetGPD, and PointNetGPD+3DSGrasp. The paper reports average success rates of 58% for GPD+ours versus 51% for GPD+3DSGrasp and 35% for GPD, and concludes that accounting for completion uncertainty improves grasp ranking.","tokens_in":10880,"tokens_out":7554,"duration_ms":67597,"significance":"If the causal claim were established, the contribution would be practically useful: a plug-in uncertainty penalty that can be added to any point-cloud grasp scorer without retraining, validated on a real robot with a substantial number of trials (3000 grasp attempts across all methods). The paper also evaluates two grasp generators (GPD and PointNetGPD), which gives useful breadth, and it builds on a peer-reviewed backbone (3DSGrasp) rather than introducing a new architecture. However, as detailed below, the headline comparison is not yet controlled for candidate-pool size or for the effect of re-ranking per se, and the uncertainty signal rests on an unverified property of the pretrained network. The manuscript contains the ingredients of a useful empirical study, but the current evidence does not isolate the uncertainty term as the cause of the reported improvement.","major_comments":[{"comment":"The headline comparison is confounded by asymmetric candidate generation. The proposed pipeline explicitly generates 15 GPD candidates and then re-ranks them ('we made the decision of generating the rank 15 grasp candidates'), while the GPD and GPD+3DSGrasp baselines execute the top 5 of the original GPD ranking. The experiment therefore varies two factors at once: the size of the candidate pool (15 versus 5) and the uncertainty-based re-scoring. No condition applies the same 15-to-5 selection with the original GPD score, with a random perturbation, or with a non-uncertainty heuristic such as penalizing distance from the camera. Without such a control, the +7 p.p. improvement over GPD+3DSGrasp cannot be attributed to the uncertainty signal. This concern is reinforced by Section VI, which states that the proposed score 'favours the visible regions of the object that is also the side that is closest to the robot'; any heuristic that prefers visible, near-side points could reproduce the same effect.","section":"Section IV / Table I"},{"comment":"The reported success rates are point estimates without confidence intervals, significance tests, or per-trial variance. With 50 grasp attempts per object per method, the difference between 58% and 51% on the average could easily arise from sampling noise, especially because object-level success rates vary widely (e.g., 2% versus 28% for 'Pitcher with lid'). The paper should report binomial confidence intervals or a paired comparison across trials, and should identify which pairwise differences are statistically reliable.","section":"Table I / Section V"},{"comment":"The uncertainty weight Wu is hand-tuned (10^5 for GPD and 10^-1 for PointNetGPD) and no sensitivity analysis is provided. Because the penalty term scales linearly with Wu and the GPD scores are in hundreds or thousands, the final ranking can be dominated by the uncertainty term at the chosen scale. The paper notes that two objects (mustard and jell-o) did not improve and attributes this to a 'balancing issue,' but it does not investigate how the result depends on Wu. A sweep over Wu, or an automatic principled calibration, is necessary to show that the reported ranking is not an artifact of a single favorable weight.","section":"Equation (2) / Section IV"},{"comment":"The Monte Carlo dropout uncertainty estimate is only valid if the deployed 3DSGrasp network actually contains dropout layers that were active during training and can be re-enabled at inference. The paper does not report the network architecture's dropout placement, the dropout rate, or any check that the T=60 forward passes produce nonzero variance. If the released pretrained model has no dropout in the relevant layers, the reported standard deviations would be zero or constant, and Equation (2) would reduce to the original score. The authors should verify and document this property, for example by showing the distribution of predicted standard deviations over passes and comparing with a network with dropout disabled.","section":"Section III-B / III-C"}],"minor_comments":[{"comment":"There are typographical errors: 'It is important to not that' should read 'note that', and 'BoudingBox' should be 'BoundingBox'.","section":"Section IV / Algorithm 1"},{"comment":"The heading sequence is inconsistent: both Sections V and VI are titled 'Robot Experiments'; the second one should probably be 'Discussion' or 'Additional Experiments'.","section":"Section V / VI"},{"comment":"Table II is described as an 'ablation study', but it actually compares against results reported in other papers under different experimental protocols. This should be relabeled as a comparison to published methods, and the assumptions (rank-1 execution, different object sets, and different robot setups) should be stated in the table caption.","section":"Table II"},{"comment":"Please clarify whether the robot re-grasps the same object placement for every method's top-5 candidates (a paired design) or whether each method is evaluated in separate placements; the sentence 'in each trial we execute the rank 5 grasp candidates for all methods' is ambiguous.","section":"Section V"}],"recommendation":"major_revision","confidential_remarks":"The paper is not circular: the uncertainty scoring is tested against real-robot outcomes, and the 3DSGrasp backbone is independently published. The main risk is the experimental-control gap, which is substantial but fixable by adding the missing control conditions and sensitivity analyses. I therefore recommend major revision rather than rejection. The editor may also wish to ask the authors to release or document the exact dropout configuration of the pretrained 3DSGrasp model, since the entire uncertainty signal depends on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The idea here is clean and worth taking seriously: run Monte Carlo dropout on a point-cloud shape completion network during inference, get per-point standard deviations, and subtract the sum of those uncertainties inside each grasp candidate from that candidate's quality score. That is a genuinely new coupling, and the paper is honest about building on 3DSGrasp and on Lundell et al.'s earlier MC dropout work. The real-robot experiment is large: 3000 grasp attempts, ten YCB objects, identical hardware and planner across conditions. The paper does earn credit for shipping a practical, plug-in procedure that any point-based grasp generator could adopt.\n\nBut the soft spots are load-bearing. The central comparison is confounded. As Section IV states, the proposed pipeline requests 15 grasps from GPD, applies Equation (2), and executes the new top 5. The baselines request only 5 and execute their original top 5. So the comparison varies both the candidate pool size and the re-ranking signal at once. There is no arm that, say, generates 15 candidates and re-ranks them with the unmodified GPD score, or with a random perturbation, or with a simple visibility penalty. Without that control, the reported +7 p.p. over GPD+3DSGrasp cannot be attributed to the uncertainty term. The paper even notes in Section VI that the uncertainty penalty systematically favors the robot-near side of the object, which is also where the camera sees ground-truth points; that makes me suspect the operative ingredient is distance from the observed view, not the dropout variance per se.\n\nOther issues are smaller but real. Success rates are reported without confidence intervals or significance tests, and with 10 trials per object, the per-object differences are noisy. The weight Wu is hand-tuned (10^5 for GPD, 10^-1 for PointNetGPD) with no sensitivity analysis, so one cannot tell whether the result is robust to that choice. And the paper does not verify that the pretrained 3DSGrasp model actually uses dropout in a way that makes the MC dropout approximation meaningful; that is an assumption, not a demonstrated fact.\n\nWho is this for? People working on shape-completion-based grasping will want to know about the uncertainty-penalized scoring idea and the large dataset. As a submission, though, the evidence does not support the stated claim. I would not desk-reject it, because the concept and the experiment scope are worth referee time, but I would send it back for major revision with a demand for the missing control and for statistical support. The path to a defensible paper is clear: add a 15-to-5 control without uncertainty, run a sensitivity sweep over Wu, and report intervals.","headline":"The plug-in uncertainty penalty is a sensible idea, but the headline +7 p.p. claim rests on a comparison that changes two variables at once and the evidence as presented does not isolate the uncertainty signal.","tokens_in":11493,"tokens_out":1442,"would_cite":false,"duration_ms":16019,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding shape-completion uncertainty to grasp scores raises real top-5 grasp success to 58 percent.","keywords":["uncertainty estimation","Monte Carlo dropout","shape completion","grasp quality","point cloud","robot grasping","single-view perception","grasp re-ranking"],"falsifier":"Have each baseline rank its top five from the same fifteen candidates the proposed method uses; if a baseline then matches the 58 percent success, the uncertainty term contributed nothing. Also run the proposed pipeline with dropout switched off at inference, and if the re-ranking still helps, the reported standard deviations are not the cause.","tokens_in":10408,"feed_emoji":"🤖","tokens_out":6657,"duration_ms":59440,"temperature":0.7,"pith_summary":"The paper tries to establish that a robot grasps partially seen objects more reliably when the grasp planner is told how unsure the shape-completion network is about the region it will touch. It estimates that uncertainty by leaving dropout on during inference, running the completion network 60 times on a single-view point cloud, and taking the per-point standard deviation of the completed shape. Each grasp candidate's quality score is then reduced by a weighted sum of the standard deviation inside the gripper's contact volume, and the candidates are re-ranked. On a real seven-degree-of-freedom arm with a two-finger gripper and ten household objects, the top five re-ranked grasps succeeded 58 percent of the time, against 51 percent for the same pipeline without uncertainty and 35 percent for the grasp generator alone. The paper's message is that uncertainty in reconstructed geometry is usable signal for grasp selection.","feed_headline":"Uncertainty boosts robot grasp success to 58 percent","feed_subtitle":"Re-ranking grasps by shape-completion uncertainty outperforms the same pipeline without it.","key_machinery":"The machinery is a score update of the form $S' = S - W_u \\sum_{p\\in CP'} \\sigma_u(p)$, applied to each of the fifteen grasp candidates: crop the completed point cloud to the bounding box defined by the gripper's contact points, sum the per-point standard deviations inside that crop, and subtract the scaled sum from the original grasp score. The standard deviations come from Monte Carlo dropout, with dropout left active at inference and $T=60$ forward passes averaged into the completed cloud and a variance map. The weight $W_u$ is calibrated to the score scale of the grasp detector ($10^5$ for the GPD-style scores, $10^{-1}$ for the PointNetGPD-style scores) so the uncertainty term has comparable influence across metrics. Re-ordering by $S'$ is what moves low-uncertainty, visible-side grasps into the top five.","core_discovery":"The central claim is that a penalisation term built from the local standard deviation of a completed point cloud re-ranks grasp candidates better than the original quality score alone. The method does not modify the grasp generator or the shape-completion network; it keeps dropout active at inference, averages 60 forward passes into a completed cloud, and computes the per-point standard deviation of those passes. For each candidate, the completed points inside the gripper's contact bounding box are cropped, their standard deviations summed, and that sum, scaled by a weight, is subtracted from the grasp score. Because the highest uncertainty lies on the side of the object farthest from the observed view, the re-ranking systematically favours grasps on the visible, reachable side. In the real-robot trials the re-ranked top five averaged 58 percent success, seven percentage points above the same pipeline without the uncertainty term and twenty-three points above the grasp generator on the partial view alone.","pith_inferences":["If the mechanism is the penalty on high-variance reconstructed regions, then any calibrated per-point uncertainty source, such as ensemble disagreement or learned variance, should produce a similar re-ranking; this is testable without changing the rest of the pipeline.","The reported comparison is not fully controlled on candidate-pool size, since the proposed method draws its top five from fifteen candidates while the baselines take five directly from their original scores; a baseline that also selects five from fifteen using the original score would isolate the uncertainty contribution.","The same score-update idea could be applied to other grippers by replacing the contact bounding box with the corresponding contact region, and to suction grasps by penalising the uncertainty under the suction cup."],"forward_implications":["Any off-the-shelf point-cloud grasp detector can be re-ranked with this uncertainty term without retraining or changing the set of generated candidates.","Grasps on the observed, near side of the object are systematically preferred, because points visible to the camera carry near-zero standard deviation, which also tends to avoid unreachable grasps.","The added cost is modest: roughly four seconds for the completed cloud plus uncertainty and two seconds for re-ranking, keeping the pipeline usable online.","The same uncertainty penalty transfers to a second grasp generator and improves its top-five success from 32 to 49 percent on average, suggesting the gain is not tied to one detector.","Precision at the top of the ranking becomes a better predictor of execution success, which is the quantity a robot actually acts on."],"supporting_citations":[{"why":"Supplies the pre-trained point-cloud shape-completion network whose inference is repeated for uncertainty estimation.","marker":"[4]"},{"why":"Generates the grasp candidates and the original quality scores that the uncertainty term re-ranks.","marker":"[5]"},{"why":"Provides the Monte Carlo dropout procedure used to turn repeated inferences into a mean completed cloud and per-point variance.","marker":"[31]"},{"why":"Supplies the alternative grasp detector used to show the re-ranking transfers beyond one generator.","marker":"[32]"},{"why":"Is the rank-1 uncertainty-aware shape-completion baseline compared in the ablation table.","marker":"[26]"}],"fun_headline_variants":["Uncertainty-aware grasp ranking lifts success to 58%","Re-rank grasps by shape uncertainty, hit 58% success","Shape-completion uncertainty picks reachable grasps, lifting success","Uncertainty term in grasp score boosts top-5 success to 58%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the experimental comparison being fair: the proposed pipeline re-ranks fifteen generated candidates and keeps five, while the baselines take five directly from their original scores, and the uncertainty estimate only means something if the pre-trained completion model actually used dropout during training.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty-aware grasp ranking lifts success to 58%","Re-rank grasps by shape uncertainty, hit 58% success","Shape-completion uncertainty picks reachable grasps, lifting success","Uncertainty term in grasp score boosts top-5 success to 58%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000569,"raw_usage":{"total_tokens":2686,"prompt_tokens":931,"completion_tokens":1755,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":547,"completion_tokens_details":{"reasoning_tokens":1679}},"tokens_in":547,"tokens_out":1755,"duration_ms":11790,"temperature":1.0,"reasoning_tokens":1679,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:10:12.912772+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have each baseline rank its top five from the same fifteen candidates the proposed method uses; if a baseline then matches the 58 percent success, the uncertainty term contributed nothing. Also run the proposed pipeline with dropout switched off at inference, and if the re-ranking still helps, the reported standard deviations are not the cause.","supporting_citations":[{"cited_title":"3DSGrasp: 3D Shape-Completion for Robotic Grasp,","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained point-cloud shape-completion network whose inference is repeated for uncertainty estimation."},{"cited_title":"Grasp Pose Detection in Point Clouds,","cited_arxiv_id":null,"evidence_quote":"Generates the grasp candidates and the original quality scores that the uncertainty term re-ranks."},{"cited_title":"Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning,","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo dropout procedure used to turn repeated inferences into a mean completed cloud and per-point variance."},{"cited_title":"PointNetGPD: Detecting Grasp Configurations from Point Sets,","cited_arxiv_id":null,"evidence_quote":"Supplies the alternative grasp detector used to show the re-ranking transfers beyond one generator."},{"cited_title":"Robust grasp planning over uncertain shape completions,","cited_arxiv_id":null,"evidence_quote":"Is the rank-1 uncertainty-aware shape-completion baseline compared in the ablation table."}],"review_version":1}