{"id":"0cd887e3-3849-40fb-a98a-116354332a21","arxiv_id":"2506.20045","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Consensus disagreement among off-the-shelf 6-DoF pose estimators, encoded as signed 6D differences, lets a lightweight MLP predict simulated grasp success more accurately than an ADD-based uncertainty baseline.","lead":"The paper trains small neural networks to predict whether an open-loop robot grasp, guided by an RGB-based 6-DoF pose estimate, will succeed before the robot moves. The predictor uses disagreement among three off-the-shelf pose estimators, and joint training across objects improves accuracy relative to per-object models and to an uncertainty-based baseline.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains rely on selecting MLP checkpoints by best test-set accuracy, making Table III's comparison to the training-threshold baseline unfair; the MLP-O synergy claim is not established unless re-evaluated with validation-based selection.","rationale":"Good-faith reading: the paper is not proposing a new pose estimator; it claims that a lightweight MLP over principal-supporting 6D pose differences can forecast simulator grasp success, and that joint training across objects improves prediction. This is plausible, and the pipeline is described in unusual detail. Using real images with simulated grasping is a reasonable first step, and the paper explicitly acknowledges its reference-grasp and simulator limitations. However, the central evidence, Table III, is vulnerable to a concrete internal-validity flaw. Because the test set is used for checkpoint selection, the reported MLP accuracies are selected rather than evaluated. This is not a matter of consensus disagreement; it is a methodological error that can inflate all MLP rows and, more importantly, can differentially inflate the configurations that overfit the test split, so the MLP-O improvement is not trustworthy. The baseline's threshold fitting on the training set makes the comparison asymmetric. The proposed concrete test — validation-based checkpoint selection with multiple seeds and significance tests — is inexpensive with the released pipeline and would settle whether the central claim survives. If the margins collapse, the verdict should remain conditional on re-evaluation; if the margins persist, the paper's strongest contribution is supported. I therefore keep the reader's CONDITIONAL verdict unchanged but sharpen the reason: the sim-to-real proxy is a legitimate external-validity concern, yet the test-set selection is a more immediate threat to the internal conclusion. The paper's transparency about choosing the best test checkpoint is a point in its favor, and the GitHub release is good practice, but reproducing a flawed protocol does not validate it.","tokens_in":13306,"tokens_out":5454,"duration_ms":69033,"concrete_test":"Split the 3,152 object-image samples into train/validation/test sets (e.g., 70/15/15), stratified by object and outcome, and train each MLP configuration with 10 random seeds. Select the checkpoint on validation accuracy only; report test accuracy with mean, standard deviation, and paired McNemar or bootstrap tests against the baseline. For the baseline, fit the ADD threshold on the training set (optionally select on validation), never on the test set. If the validation-selected MLP-O minus baseline is not consistently positive, or MLP-O no longer exceeds per-object MLP, then Table III's headline improvement is an artifact of test-set checkpoint selection. This directly tests the central claim without requiring new real-robot data.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing problem is in the evaluation protocol, not the simulator. Section V.A states: \"We report results for the model checkpoint with best test-set accuracy.\" Thus, every MLP entry in Table III is selected from 3,000 checkpoints using the same 20% test split that is later used to report accuracy. This is a test-set selection bias: the reported MLP accuracies are optimistically tuned to that particular split. The baseline in Section V.B, by contrast, learns its threshold on the training set (\"we use the same training set used by the MLPs\"), with no equivalent test-set sweep. The comparison is therefore asymmetric, and the claimed margins — e.g., MLP-O vs. Baseline: 0.898 vs. 0.792 for the parallel gripper and 0.905 vs. 0.863 for the underactuated hand — could arise from differential overfitting to the test split even if the method has no true advantage. The central findings (MLP beats baseline; MLP-O beats per-object MLP) are both drawn from this table. No repeated splits or confidence intervals are provided, so the ordinal relationships in Table III are not shown to be stable. This is an internally correctable flaw, but as it stands the strongest claim is not established by the evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method to predict, before execution, whether an open-loop robotic grasp guided by a 6-DoF pose estimate from a single RGB image will succeed. The method designates one of three pose estimators (EPOS, GDRNPP, ZebraPose) as the Principal Estimator, forms signed 6D differences between the principal and the two supporting estimators, and trains small MLPs on these differences with labels generated by MuJoCo grasping trials. Experiments on YCB-V and LM-O with a parallel gripper and an underactuated hand compare per-object MLPs, joint-object (MLP-O), joint-gripper (MLP-G), and joint-object-gripper (MLP-OG) variants against an ADD-threshold baseline derived from Shi et al. The paper reports that MLP-O achieves the highest average prediction success.","tokens_in":13504,"tokens_out":6027,"duration_ms":64132,"significance":"If the quantitative claims hold, this is a practical contribution: it connects pose uncertainty to a downstream manipulation task and shows that consensus among off-the-shelf estimators can predict grasp success. The paper ships code, data, and a reproducible pipeline, and the idea of training jointly across objects is a useful finding. However, the evaluation protocol contains a test-set selection bias that prevents the central quantitative claims from being accepted as reported.","major_comments":[{"comment":"The sentence \"We report results for the model checkpoint with best test-set accuracy\" means the same 20% test split is used both for model selection and for reporting. The baseline in Section V.B learns its threshold on the training set only. This asymmetry makes the comparisons in Table III unfair: the MLP numbers are optimistically selected, while the baseline is not. The claimed margins, e.g., MLP-O vs. Baseline 0.898 vs. 0.792 for the parallel gripper, could be an artifact of this selection. The evaluation should be redone using a validation set for checkpoint selection (or the final-epoch model), with repeated splits and confidence intervals.","section":"Section V.A, Table III"},{"comment":"The authors drop object-image pairs where any of the three estimators fails to make a detection or prediction. This filters out exactly the high-uncertainty cases in which a consensus-based uncertainty estimate is most needed. The reported accuracies are therefore conditional on all estimators succeeding, which may not hold in practice. Please report the number of dropped pairs and discuss or address how the method would handle missing detections at inference time.","section":"Section V.A"},{"comment":"The grasping trials are entirely simulated with a single handcrafted reference grasp per object, approximated mass and friction coefficients, no scene clutter, and a 5 cm success tolerance. No real-robot validation or validation against a real grasping dataset is provided. The abstract's claim that the method predicts whether \"a grasp guided by an image-based pose estimate will succeed\" is therefore only established in this simplified simulation proxy. This is a load-bearing limitation for the paper's practical significance; the authors should either temper the claim to the simulated setting or add a real-world validation.","section":"Sections III.B and IV.C"},{"comment":"The central claim that joint training across objects improves prediction (MLP-O vs. MLP) is a key contribution, but it rests on the same test-selected checkpoints. Because all MLP variants are selected using the test split, the ordinal relationships among MLP, MLP-O, MLP-G, and MLP-OG are also potentially biased. After fixing the selection protocol, the synergy claim needs to be re-established with a validation-based comparison. No confidence intervals or significance tests are provided, so it is unclear whether the reported differences are stable across data splits.","section":"Table III and Section V.C"}],"minor_comments":[{"comment":"The phrase \"the so-calledsim2real gap\" is missing a space; it should read \"the so-called sim2real gap.\"","section":"Introduction"},{"comment":"The statement that \"translation error is a better indicator of grasp failure than rotation error\" is presented without supporting quantitative analysis; please provide the evidence or soften the claim.","section":"Section V.B"},{"comment":"The error functions e_R and e_t are used without being defined; please define these metrics explicitly (e.g., angular error and Euclidean translation error).","section":"Section III.B, Eqs. (1)-(2)"},{"comment":"The sentence \"Training 90 MLPs (three PEs, two grippers, 15 objects)\" appears to count only the per-object, per-gripper MLPs, but the paper also trains MLP-O, MLP-G, and MLP-OG variants; please clarify the total number of trained networks.","section":"Section V.A"},{"comment":"The input representation uses raw Euler angle differences, which are sensitive to the chosen Euler convention and have singularities. Consider using a more canonical pose difference, such as the logarithmic map of the relative SE(3) transformation, to avoid these issues.","section":"Section III.C"}],"recommendation":"major_revision","confidential_remarks":"The test-set selection bias is the most serious issue and is correctable within the manuscript's scope. The contribution is still of interest if the evaluation is redone with a validation-based protocol. The self-citation in refs [52]-[53] is not a concern. The paper fits the cs.RO scope well."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper proposes a lightweight MLP that predicts grasp success from signed 6D differences between a principal and supporting RGB pose estimators, trained on simulator outcomes. That is a legitimate new application, and it could be genuinely useful for grasp abstention policies. But the headline numbers are inflated by test-set checkpoint selection, so the main comparison is not currently trustworthy.\n\nWhat is actually new: the signed 6D PE-SE difference representation, the promotion of one ensemble member to principal, and the empirical finding that joint training across objects helps. The pipeline is clearly described, the code and data are public, and the paper is honest about the sim-to-real proxy. The central claim is not circular: labels come from simulator trials, and predictions are evaluated on a held-out split. The citation pattern is fine; the self-citations in the underactuated hand model are about the tool, not the result.\n\nThe load-bearing flaw is in the evaluation protocol. Section V-A states that results are reported for the model checkpoint with best test-set accuracy. That is test-set selection bias: the MLPs are tuned to the same split later used for reporting, while the baseline in V-B learns its threshold on the training set only. The margins in Table III—MLP-O vs. baseline, and MLP-O vs. per-object MLP—could therefore be differential overfitting. No confidence intervals or repeated splits are given, so the ordinal claims are not established. This is internally correctable, but as written the paper's strongest findings rest on an unfair comparison.\n\nThe second concern is the sim-to-real proxy: one handcrafted reference grasp per object, approximated masses and friction coefficients, a 5 cm success tolerance, and no scene clutter. The paper is upfront about these choices, but without real-robot validation there is no evidence the learned predictor transfers. That is a real limitation, not a fatal one, if the work is positioned as a simulation-level study.\n\nWho should read it: researchers working on RGB-based manipulation, pose uncertainty, or grasp abstention. It deserves a serious referee. The fix is clear: re-evaluate with validation-set checkpoint selection, report error bars over repeated splits, and ideally add physical robot trials or at least a stronger transfer argument. With that, this could be a solid contribution. Without it, the cross-object synergy claim is not supported.","headline":"A useful but currently over-claimed consensus-uncertainty grasp predictor: the cross-object synergy result will not stand until the test-set checkpoint selection is fixed.","tokens_in":14090,"tokens_out":2463,"would_cite":true,"duration_ms":29539,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Disagreement among pose estimators predicts whether a robotic grasp will succeed, before the robot moves.","keywords":["6-DoF object pose estimation","grasp success prediction","uncertainty quantification","ensemble consensus","RGB-only perception","robotic grasping","MuJoCo simulation","multi-layer perceptron"],"falsifier":"Run the same perception-to-grasp pipeline on a physical robot arm with the same objects and grippers, replacing simulated labels with real grasp outcomes; if the MLP's predicted success probabilities do not reliably rank-order or threshold real successes and failures, the central claim that simulated consensus differences predict real grasp success is refuted.","tokens_in":13055,"feed_emoji":"🤖","tokens_out":3345,"duration_ms":37221,"temperature":0.7,"pith_summary":"The paper tries to establish that an RGB-only grasping agent can predict, before executing a grasp, whether that grasp will succeed, by looking at how much several off-the-shelf 6-DoF pose estimators disagree with each other. It trains a lightweight MLP on the signed six-dimensional differences between a chosen principal pose estimate and supporting estimates, using simulated grasp outcomes as labels. The resulting predictor outperforms a threshold-based uncertainty baseline, and training one predictor jointly across all objects improves accuracy further. If correct, a robot could use this cheap pre-execution signal to abstain from grasps that would likely fail.","feed_headline":"Disagreeing pose estimates predict grasp failure before the grasp","feed_subtitle":"A lightweight network trained on simulated grasps lets RGB-only robots abstain from attempts that would fail.","key_machinery":"The load-bearing object is the signed 6D pose-difference vector in $\\mathbb{R}^6$: for each pair of principal and supporting estimator, the three translation differences and three Euler-angle rotation differences are concatenated. These difference vectors are computed after transforming all poses into a common reference frame, which preserves each estimator's error relative to ground truth while allowing the simulator to use one handcrafted reference grasp per object. The MLP maps these differences, plus one-hot object or gripper identifiers in the wider training configurations, through five fully connected layers with ReLU activations and a sigmoid output to a predicted grasp-success probability.","core_discovery":"The paper's central claim is that signed 6D pose differences between a Principal Estimator and Supporting Estimators carry enough information about downstream task outcome that a small multi-layer perceptron can predict grasp success or failure before the grasp is attempted. This consensus-based signal outperforms the ADD-threshold baseline by 4.5% on average per object, and by 7.27% when networks are trained jointly on all objects. The paper further claims that preserving the six translation and rotation components separately, rather than collapsing them into a single ADD value, is what lets the network learn which geometric errors actually cause task failure.","pith_inferences":["The authors leave implicit that the same consensus signal could serve as a confidence measure for other pose-guided manipulation tasks such as insertion, placement, or tool use, where the cost of failure is also high.","A natural testable extension is to replace the simulated grasp labels with real-robot outcomes on a subset of trials; if the MLP's predictions remain well calibrated on real grasps, the simulator proxy is validated, and if not, the difference would quantify the sim-to-real gap.","The MLP's independence from the absolute pose suggests a possible transfer path to new objects or unseen viewpoints, provided the distribution of pose differences stays similar; this could be checked by training on one object set and testing on another.","The 5 cm success tolerance and handcrafted reference grasps define an implicit task difficulty; varying the tolerance or using optimized grasps would likely shift the learned decision boundary and could be used to probe how conservative the predictor should be."],"forward_implications":["A grasping agent can use the MLP's output as a gating signal, refusing to attempt grasps whose predicted success probability falls below a threshold.","Because the predictor operates on pose differences rather than absolute poses, it can be combined with any number of off-the-shelf detectors, pose estimators, and grippers without retraining them.","Joint training across objects is beneficial, indicating that diverse objects share enough structure in how pose error translates to grasp failure to support a single predictor.","Translation error, especially along the viewing direction, is a stronger driver of grasp failure than rotation error, so uncertainty metrics that preserve this distinction are more useful for downstream tasks.","Training a single network across both grippers is less effective, implying that gripper differences are too large to be captured by a one-hot identifier alone."],"supporting_citations":[{"why":"Supplies the ADD-based ensemble uncertainty baseline that the MLP predictors are compared against.","marker":"[7]"},{"why":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline.","marker":"[25]"},{"why":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline.","marker":"[26]"},{"why":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline.","marker":"[27]"},{"why":"The MuJoCo physics simulator that generates the binary grasp-success labels used to train and evaluate the MLPs.","marker":"[54]"},{"why":"Supplies the BOP benchmark framework and datasets (YCB-V and LM-O) that provide real RGB images and ground-truth poses.","marker":"[3]"},{"why":"Provides the ensembling rationale that disagreement among predictors indicates uncertainty, which the consensus approach adapts.","marker":"[37]"},{"why":"Related work that models grasp success under pose uncertainty through pose perturbation, which the paper discusses and contrasts with its real-estimator-difference approach.","marker":"[39]"},{"why":"Defines the ADD metric whose aggregation the paper argues loses geometric information needed for grasp-success prediction.","marker":"[55]"}],"fun_headline_variants":["Pose disagreement predicts grasp failure before grasp","RGB robot abstains when pose estimates disagree","6D pose consensus stops robotic grasping failures","Lightweight net uses pose differences to veto failed grasps","Sim-to-real pose disagreement forecasts grasp outcome"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The MuJoCo grasping protocol, with one handcrafted reference grasp per object, approximated masses and friction coefficients, a 5 cm success tolerance, and no scene clutter, faithfully represents real grasping from the estimated poses.","fun_headline_variants_meta":{"raw":{"variants":["Pose disagreement predicts grasp failure before grasp","RGB robot abstains when pose estimates disagree","6D pose consensus stops robotic grasping failures","Lightweight net uses pose differences to veto failed grasps","Sim-to-real pose disagreement forecasts grasp outcome"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00047,"raw_usage":{"total_tokens":2265,"prompt_tokens":793,"completion_tokens":1472,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":409,"completion_tokens_details":{"reasoning_tokens":1403}},"tokens_in":409,"tokens_out":1472,"duration_ms":12599,"temperature":1.0,"reasoning_tokens":1403,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:57:18.357622+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same perception-to-grasp pipeline on a physical robot arm with the same objects and grippers, replacing simulated labels with real grasp outcomes; if the MLP's predicted success probabilities do not reliably rank-order or threshold real successes and failures, the central claim that simulated consensus differences predict real grasp success is refuted.","supporting_citations":[{"cited_title":"Fast Uncertainty Quantification for Deep Object Pose Estimation,","cited_arxiv_id":null,"evidence_quote":"Supplies the ADD-based ensemble uncertainty baseline that the MLP predictors are compared against."},{"cited_title":"EPOS: Estimating 6D Pose of Objects with Symmetries,","cited_arxiv_id":null,"evidence_quote":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline."},{"cited_title":"GDR-Net: Geometry- Guided Direct Regression Network for Monocular 6D Object Pose Estimation,","cited_arxiv_id":null,"evidence_quote":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline."},{"cited_title":"ZebraPose: Coarse to Fine Surface Encoding for 6DoF Object Pose Estimation,","cited_arxiv_id":null,"evidence_quote":"One of the three pose estimators used as Principal or Supporting Estimator in the consensus pipeline."},{"cited_title":"MuJoCo: A physics engine for model-based control,","cited_arxiv_id":null,"evidence_quote":"The MuJoCo physics simulator that generates the binary grasp-success labels used to train and evaluate the MLPs."},{"cited_title":"BOP Challenge 2022 on Detection, Segmen- tation and Pose Estimation of Specific Rigid Objects,","cited_arxiv_id":null,"evidence_quote":"Supplies the BOP benchmark framework and datasets (YCB-V and LM-O) that provide real RGB images and ground-truth poses."},{"cited_title":"Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles,","cited_arxiv_id":null,"evidence_quote":"Provides the ensembling rationale that disagreement among predictors indicates uncertainty, which the consensus approach adapts."},{"cited_title":"Robotic Task Success Evaluation Under Multi-modal Non-Parametric Object Pose Uncertainty","cited_arxiv_id":"2403.10874","evidence_quote":"Related work that models grasp success under pose uncertainty through pose perturbation, which the paper discusses and contrasts with its real-estimator-difference approach."},{"cited_title":"BOP Challenge 2020 on 6D Object Localization,","cited_arxiv_id":null,"evidence_quote":"Defines the ADD metric whose aggregation the paper argues loses geometric information needed for grasp-success prediction."}],"review_version":1}