Pith. sign in

REVIEW 3 major objections 9 minor 42 references

Dataset gives robots grasps tested by physics, not just geometry

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 21:46 UTC pith:WEBMYHLY

load-bearing objection GraspIT is a dataset paper that combines photorealistic RGBD, reachability-filtered grasps, staged slip-test scoring, and a sim-to-real registration link — a combination no prior dataset offers. The pipeline is open-source and Docker-containerized, and the dataset statistics are internally consistent. The main weakness is that the physics defaults (uniform friction µ=1.0, uniform density 1000 kg/m the 3 major comments →

arxiv 2607.05869 v1 pith:WEBMYHLY submitted 2026-07-07 cs.RO cs.CV

GraspIT: A Dataset Bridging the Sim-to-Real gap and back for Validated Grasping SE(3) Pose Generation

Paul Koch. Adem Karakurt , Andr\'e Sers This is my paper
classification cs.RO cs.CV
keywords realscenesannotateddatasetsforce-closuregraspinggraspitisaac
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces GraspIT, a dataset and generation pipeline for robotic grasping that combines photorealistic RGBD imagery with grasp quality scores derived from a four-stage physical slip test in simulation. The central claim is that force-closure analysis—the standard method for scoring grasp candidates—is insufficient, and that staged physical perturbation testing (lift, horizontal oscillation, pendulum swing) produces more informative quality labels, including graded hard negatives that look geometrically valid but fail under dynamic loads. The dataset comprises roughly 2.3 million grasp candidates across 1,035 simulated and 100 real tabletop scenes, with 83% passing the slip test at a good-quality threshold. A bidirectional registration pipeline links real captured scenes to simulation, back-projecting validated grasp labels onto real camera frames so that one physical capture session yields both real sensor data and unlimited synthetic views with consistent annotations.

Core claim

The paper's central object is the four-stage slip test: a sequential physical perturbation protocol (trajectory-to-closure, lift, horizontal oscillation, pendulum swing) executed by parallel simulated Franka Panda arms. This test produces a continuous quality score in five discrete base levels (0, 0.25, 0.50, 0.75, 1.0) with additional slip penalties for positional displacement beyond 2 cm or rotational slip beyond 10 degrees at any executed stage. The key finding is that 17% of grasp candidates that passed force-closure analysis failed the slip test, and 4.15% of candidates nominally passed two or more stages but were penalized below the good threshold due to excessive slip—these constitute

What carries the argument

The four-stage slip test scoring function (Equations 1-2), the cuRobo trajectory planning reachability filter, the Real-Sim bidirectional registration via multi-view ICP, and the Docker-containerized Isaac Sim scene generation pipeline.

Load-bearing premise

The paper assumes that Isaac Sim's default physics parameters—uniform friction coefficient of 1.0 and uniform object density of 1000 kg/m³ for all objects—produce grasp quality scores that meaningfully transfer to real-world robotic execution. Real objects vary in material, friction, and mass distribution, and the paper provides no real-robot validation that the slip-test scores correlate with physical execution success.

What would settle it

Train a grasp-quality predictor on GraspIT's slip-test scores and evaluate its correlation with real Franka Panda execution success rates. If the predictor trained on slip-test labels does not significantly outperform one trained on force-closure scores when measured against real-robot grasp success, the central premise that staged physical validation produces more transferable quality labels is unsupported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Grasp-quality regressors trained on GraspIT's graded hard negatives could learn to anticipate physical failure modes that force-closure scores cannot expose, potentially closing the gap between analytic grasp scoring and real-robot execution success.
  • The Real-Sim registration loop could be applied to existing datasets like GraspNet or BOP to upgrade their analytic annotations to slip-test-validated labels without recollecting real RGBD observations.
  • If the slip-test scores transfer to real execution, vision-language-action models pretrained on GraspIT's physically grounded labels could reduce the depth-estimation deficit that current semantic encoders exhibit when processing robotic manipulation scenes.
  • The open containerized pipeline allows other labs to extend the dataset to new robot kinematic chains, sensor modalities, and object corpora, potentially creating a shared physical-grounding foundation across manipulation research.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 9 minor

Summary. GraspIT introduces a dataset and generation pipeline for robotic grasping that combines photorealistic RGBD observations, robot-reachability-filtered grasp candidates, a four-stage physical slip-test producing continuous quality scores, and a bidirectional Real↔Sim registration link. The initial release comprises ~316k annotated frames across 1,035 simulated and 100 real scenes, with ~2.3M slip-test-validated grasp candidates (82.94% annotated good, s≥0.50). The slip-test protocol evaluates grasps through trajectory planning, vertical lift, horizontal oscillation, and pendulum swing, producing a graded score with slip penalties (Eq. 2). The pipeline is Docker-containerized and open-source.

Significance. The dataset addresses a genuine gap: no prior dataset simultaneously provides photorealistic RGBD, robot-reachability-filtered grasps, staged physics-validated quality scores, complete 3D shape, and a Real↔Sim registration link (Tables 1, 3). The four-stage slip-test with continuous scoring and explicit slip penalties (Eq. 2) is a thoughtful design that produces graded hard negatives — the 94,475 penalized grasps that pass stages but score below 0.50 are a concrete asset for training grasp-quality critics. The open-source, containerized generation system and the ability to import existing datasets (BOP, GraspNet) into the pipeline enhance community value. The Real↔Sim loop with TRELLIS-based 3D reconstruction and ICP registration is a practical contribution.

major comments (3)
  1. §3.5, stage 4 (pendulum swing) and Table 2: The pendulum test is described as testing 'resistance to moment arms created by off-center mass' and is the most discriminating filter (883,733 candidates, 38.84% fail here). However, §3.5 also states all objects use uniform density (1000 kg/m³), which forces center-of-mass to equal the geometric centroid. This means the stage that does the most work in separating good from bad grasps is evaluating a property most distorted by the uniform density assumption. The paper should either (a) provide real-robot validation that pendulum-test outcomes correlate with physical execution for objects with non-uniform mass distribution, or (b) explicitly qualify the 'physically validated' label in the title and abstract to note that validation is under uniform-density assumptions, and discuss the implications for the pendulum stage specifically.
  2. §3.5 and title/abstract: The central differentiating claim is that slip-test scores are 'physically validated' and more meaningful than force-closure metrics. However, µ=1.0 (comparable to rubber-on-rubber) is used for all object-fingertip interactions regardless of material. The lift stage (1.45% failure) and horizontal oscillation stage (7.36% failure) are likely too lenient for low-friction materials (metal, plastic) where real slip occurs at lower thresholds. Without any real-robot validation showing correlation between slip-test scores and execution success — which the paper acknowledges as absent in limitation (i) — the claim that these scores are more physically grounded than force-closure remains untested. At minimum, a small-scale real-robot validation (even 50-100 grasps across a few object types) would substantially strengthen the central claim. If this is not feasible for the
  3. §3.5, Eq. (2): The slip-penalty formula uses hand-set constants (2cm positional threshold, 10° orientational threshold, denominator 12). The paper provides a brief calibration rationale (a single violation on a 0.75-base grasp stays marginally good; two violations on a 0.50-base grasp push to bad). However, no sensitivity analysis is provided for how the good/bad classification changes with different threshold or denominator values. Given that 94,475 grasps (4.15%) fall in the penalized bucket — the paper's highlighted 'informative hard negatives' — the stability of this bucket under reasonable parameter perturbations should be reported.
minor comments (9)
  1. Table 2: The 'Fail pendulum (s=0.75)' row reports 883,733 (38.84%), but the text in §3.6 states '94.26% survive the pendulum test.' These are consistent (100% - 38.84% ≈ 61.16% fail, not 5.74% fail), so the 94.26% figure appears to be an error or refers to a different denominator. Please clarify.
  2. Table 2: The 'w/ trajectory' row (2,014,689, 88.54%) and 'No Trajectory to closure (s=0.00)' row (260,663, 11.46%) sum to 2,275,352, which is correct. However, the subsequent stage rows (fail lift, fail horizontal, fail pendulum, full success) should also sum to 2,014,689 (those with trajectories). Adding 33,029 + 94,475 + 167,549 + 883,733 + 835,903 = 2,014,689. This checks out but would benefit from explicit labeling that these rows are conditional on trajectory success.
  3. §3.4: The UR-5 arm is used for real scene capture, while the Franka Panda is used for grasp validation. This mismatch is not discussed — please clarify whether grasp labels generated for the Franka are intended to transfer to UR-5 setups or whether the real branch is purely for visual data capture.
  4. Figure 2 caption: 'Robot School' is referenced but not defined until §3.5. A forward reference or brief inline definition would help.
  5. §3.2: Objects are 'randomly scaled to a maximum side length of [2,20]% of the maximum table side length.' Please clarify whether this means the scaling range is [2%, 20%] or whether 2-20% is applied to some base dimension.
  6. Table 1: The '≈' symbol for partial provision is used inconsistently across datasets (e.g., GraspNet-1B column D shows '≈' without explanation). A footnote defining the threshold for '≈' vs '✓' would improve clarity.
  7. §3.5: The Franka Hand closure force is stated as 70N. The default Franka Hand specification is typically ~70N maximum, but please confirm this is the commanded force used in simulation, not the hardware maximum.
  8. Abstract: 'which existing datasets lack to provide jointly' is grammatically awkward; consider 'which no existing dataset provides jointly.'
  9. References [2] and [3] (RT-1, RT-2) are cited in the introduction but not included in the reference list with full bibliographic details matching the numbered scheme.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for a thorough and constructive report. The referee correctly identifies that the uniform-density assumption (1000 kg/m³) most distorts the pendulum stage — the stage doing the most filtering work — and that the friction coefficient (μ=1.0) and slip-penalty constants lack sensitivity analysis and real-robot validation. We agree with the substance of all three major comments and will revise accordingly: (1) we will qualify the 'physically validated' language in the title and abstract, add a dedicated discussion of the uniform-density assumption's impact on the pendulum stage, and note real-robot validation as future work; (2) we will add an explicit caveat about the μ=1.0 assumption and its effect on low-friction materials, and scope the 'more physically grounded than force-closure' claim accordingly; (3) we will add a sensitivity analysis for the slip-penalty parameters. A small-scale real-robot validation is a genuinely valuable suggestion that we cannot complete within the revision timeline, and we are transparent about this below.

read point-by-point responses

  1. Referee: §3.5, stage 4 (pendulum swing) and Table 2: The pendulum test is described as testing 'resistance to moment arms created by off-center mass' and is the most discriminating filter (883,733 candidates, 38.84% fail here). However, §3.5 also states all objects use uniform density (1000 kg/m³), which forces center-of-mass to equal the geometric centroid. This means the stage that does the most work in separating good from bad grasps is evaluating a property most distorted by the uniform density assumption. The paper should either (a) provide real-robot validation that pendulum-test outcomes correlate with physical execution for objects with non-uniform mass distribution, or (b) explicitly qualify the 'physically validated' label in the title and abstract to note that validation is under uniform-density assumptions, and discuss the implications for the pendulum stage specifically.

    Authors: The referee is correct on both counts. The uniform-density assumption (1000 kg/m³) forces center-of-mass to coincide with the geometric centroid, which directly undermines the pendulum stage's stated purpose of testing resistance to moment arms from off-center mass. We acknowledge that this is the stage doing the most filtering work (38.84% of candidates fail here), and it is precisely the stage most affected by the assumption. We cannot complete option (a) — real-robot validation with non-uniform-mass objects — within the revision timeline. We will therefore implement option (b) in full. Specifically: (1) We will qualify 'physically validated' in the title and abstract to read 'physically validated under uniform-density assumptions' or equivalent language that makes the assumption visible at first encounter. (2) We will add a dedicated paragraph in §3.5 discussing why the pendulum stage is most affected: with uniform density, the moment arm reduces to the geometric offset between the grasp contact line and the centroid, which is still non-trivial for irregular geometries but does not capture real-world cases where mass distribution is non-uniform (e.g., a hollow container, a tool with a heavy handle). (3) We will note in the limitations that the pendulum-stage failure rates should be interpreted as lower bounds on stringency for non-uniform objects — a grasp that fails under uniform density would likely also fail with non-uniform mass, but some grasps that pass may fail in reality. We agree this qualification is essential for honest use of the dataset. revision: yes

  2. Referee: §3.5 and title/abstract: The central differentiating claim is that slip-test scores are 'physically validated' and more meaningful than force-closure metrics. However, μ=1.0 (comparable to rubber-on-rubber) is used for all object-fingertip interactions regardless of material. The lift stage (1.45% failure) and horizontal oscillation stage (7.36% failure) are likely too lenient for low-friction materials (metal, plastic) where real slip occurs at lower thresholds. Without any real-robot validation showing correlation between slip-test scores and execution success — which the paper acknowledges as absent in limitation (i) — the claim that these scores are more physically grounded than force-closure remains untested. At minimum, a small-scale real-robot validation (even 50-100 grasps across a few object types) would substantially strengthen the central claim. If this is not feasible for the

    Authors: We agree that μ=1.0 for all material interactions is a significant simplification and that the low failure rates on the lift (1.45%) and horizontal oscillation (7.36%) stages are likely partly attributable to this high friction coefficient. The referee's observation that these stages would be more discriminating for low-friction materials (metal, plastic) is well-founded. We will make the following revisions: (1) We will add an explicit caveat in §3.5 stating that μ=1.0 is comparable to rubber-on-rubber contact and that the lift and oscillation stages are therefore likely too lenient for low-friction materials; this means the good-grasp rate (82.94%) should be interpreted as an upper bound for such objects. (2) We will scope the claim 'more physically grounded than force-closure' to acknowledge that this is a relative claim — the slip test captures failure modes (trajectory infeasibility, slip under perturbation, moment-arm instability) absent from force-closure, but the absolute score values are contingent on the friction and density assumptions. (3) We will strengthen the existing limitation (i) to explicitly state that without real-robot correlation data, the claim that slip-test scores predict real execution success remains unvalidated, and that such validation is the most important next step. Regarding the small-scale real-robot validation (50-100 grasps): we genuinely agree this would substantially strengthen the paper, but we are unable to complete it within the revision timeline due to hardware access constraints. We will state this explicitly as a standing limitation rather than overpromising. revision: partial

  3. Referee: §3.5, Eq. (2): The slip-penalty formula uses hand-set constants (2cm positional threshold, 10° orientational threshold, denominator 12). The paper provides a brief calibration rationale (a single violation on a 0.75-base grasp stays marginally good; two violations on a 0.50-base grasp push to bad). However, no sensitivity analysis is provided for how the good/bad classification changes with different threshold or denominator values. Given that 94,475 grasps (4.15%) fall in the penalized bucket — the paper's highlighted 'informative hard negatives' — the stability of this bucket under reasonable parameter perturbations should be reported.

    Authors: This is a fair and actionable request. We will add a sensitivity analysis varying the positional threshold (±0.5 cm around 2 cm), the orientational threshold (±2° around 10°), and the denominator (10, 12, 14) and report how the good/bad classification and specifically the penalized bucket (94,475 grasps, 4.15%) change. We expect the penalized bucket to be moderately sensitive to the denominator (since it directly controls how many violations push a grasp below 0.50) and less sensitive to the thresholds (since these control whether a violation is counted at all). We will report the range of penalized-bucket sizes and good-grasp rates under these perturbations and discuss the implications: if the bucket is stable, this strengthens the claim that these are genuinely ambiguous near-misses; if it is unstable, we will note that the bucket boundary should be treated as a soft threshold. This analysis can be computed from the existing slip-test logs without re-running the physics simulation, so it is feasible within the revision timeline. revision: yes

standing simulated objections not resolved
  • Small-scale real-robot validation (50-100 grasps across object types) cannot be completed within the revision timeline due to hardware access constraints. We agree this is the single most valuable addition the paper could receive and will explicitly flag it as the highest-priority future work item, but we cannot honestly commit to including it in the revised manuscript.

Circularity Check

0 steps flagged

Dataset contribution with no derivation chain; score formula uses hand-set design constants, not fitted parameters

full rationale

GraspIT is a dataset paper, not a derivation or prediction paper. The grasp quality score (Eq. 2) uses hand-set constants (2cm threshold, 10° threshold, denominator 12) that are design choices for a labeling scheme, not parameters fitted to a target output. The slip-test scores are produced by running physics simulations, not by fitting parameters to data and then predicting the same data. The denominator-12 calibration is explicitly motivated by external evidence (Dex-Net 4.0's real-robot success rates of 9-98% at varying force-closure scores), not by circular reference to GraspIT's own outputs. The paper does cite prior work by some of the same authors (ref [20], [21] — robot-assisted annotation and MVIP dataset), but these are tangential to the central contribution and not load-bearing for the slip-test validation pipeline. The uniform density and friction assumptions (µ=1.0, 1000 kg/m³) are acknowledged limitations that affect physical fidelity, but this is a correctness concern about whether the simulation matches reality, not a circularity where the output is defined in terms of the input. No step in the paper's pipeline reduces by construction to its own inputs.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 0 invented entities

The paper introduces no new physical entities, particles, forces, or theoretical constructs. It is a dataset and pipeline contribution built entirely from existing software (Isaac Sim, cuRobo, TRELLIS), hardware (Franka Panda, UR-5, RealSense D405), and datasets (ABC). The free parameters are all hand-set design choices for the labeling scheme, not fitted constants. The axioms are domain assumptions about simulation and reconstruction fidelity that are standard in the sim-to-real literature but unvalidated in this paper.

free parameters (7)
  • Slip positional threshold = 2 cm
    Hand-set threshold in Eq. 2 defining when positional slip triggers a penalty. Not derived from data or optimization.
  • Slip orientational threshold = 10 degrees
    Hand-set threshold in Eq. 2 defining when orientational slip triggers a penalty. Not derived from data or optimization.
  • Penalty denominator = 12
    Hand-set constant in Eq. 2. The paper states it is 'calibrated so that a single maximal-slip violation on a 0.75-base grasp still keeps it marginally good,' which is a design choice, not a fitted parameter.
  • Good grasp threshold = 0.50
    Hand-set threshold defining the good/bad binary split. Justified by analogy to Dex-Net 4.0 success rates but not optimized on GraspIT data.
  • Gripper closure force = 70 N
    Fixed force for the Franka Hand gripper used in all slip tests (Section 3.5, stage 2).
  • Friction coefficient = 1.0
    Isaac Sim default friction coefficient applied uniformly to all objects and fingertips (Section 3.5, stage 2). Not tuned per object.
  • Object density = 1000 kg/m³
    Uniform density for all objects in Isaac Sim, used to compute object weight linearly with volume (Section 3.5, stage 2).
axioms (4)
  • domain assumption Isaac Sim's physics engine accurately models grasp dynamics (friction, gravity, contact) sufficiently to produce quality scores that correlate with real-robot execution.
    Invoked throughout Section 3.5. The entire labeling scheme depends on simulation fidelity. The paper provides no real-robot validation of this assumption.
  • domain assumption TRELLIS image-to-3D reconstruction produces meshes of sufficient fidelity for grasp annotation transfer.
    Invoked in Section 3.4 for real object scanning. The paper acknowledges registration offsets (limitation vi) but does not quantify reconstruction error.
  • domain assumption Multi-view ICP with robot forward kinematics provides sufficiently accurate camera poses for back-projection of annotations onto real frames.
    Invoked in Section 3.4-3.5 for the Real↔Sim registration. Accuracy depends on UR-5 forward kinematics precision and ICP convergence quality.
  • domain assumption Antipodal point sampling on downsampleed point clouds (1000 points) generates a representative set of grasp candidates.
    Invoked in Section 3.5 for candidate generation. The ~10% antipodal pass rate and ~307 candidates per object depend on this sampling density.

pith-pipeline@v1.1.0-glm · 16077 in / 3417 out tokens · 490388 ms · 2026-07-08T21:46:30.788113+00:00 · methodology

0 comments
read the original abstract

Robust robotic grasping of novel objects requires datasets that simultaneously provide photorealistic RGB-D observations, physically validated grasp quality annotations, and a principled bridge between simulation and the real world, which existing datasets lack to provide jointly. \textbf{GraspIT} addresses this gap: tabletop scenes in NVIDIA Isaac Sim are annotated via a four-stage physical slip-test on parallel Franka Panda instances, producing trajectory-reachability checks and continuous quality scores beyond force-closure.Of ${\sim}$2.3M candidates, 83% pass as \emph{good} ($s{\geq}0.50$); the 17% that passed force-closure but failed the slip-test provide graded hard negatives. A Real$\leftrightarrow$Sim loop back-projects these labels onto 100 real-world scenes. The release provides ${\sim}$316k annotated RGBD frame sets across 1035 sim and 100 real scenes, with instance masks, 6-DoF poses, physical object properties, and scored 6-DoF grasps. All tools are open-source and Docker-containerized. The trajectory planning within Isaac Sim further allows streaming of high resolution demonstrations for tabletop manipulation policy learning and behavior cloning.

Figures

Figures reproduced from arXiv: 2607.05869 by Andr\'e Sers, Paul Koch. Adem Karakurt.

Figure 1
Figure 1. Figure 1: Grasp ITerative robot tabletop object manipulation dataset: illustration of the four-stage iterative concept. proposed grasp or object manipulation will succeed [8, 41]. Collecting the training data needed to close this gap is expensive. Accurate 6-DoF grasp annotation requires ei￾ther real-robot execution at scale or a physics simulator; real RGBD observations necessitate a physical setup; link￾ing the tw… view at source ↗
Figure 2
Figure 2. Figure 2: GraspIT Real↔Sim Collection Loop. Objects are 3-D scanned via generative image-to-3D reconstruction and imported into Isaac Sim. Real scenes are captured by a robot-mounted camera, registered into the simulator, and extended with synthetic views. Physics-validated grasp labels from the Robot School are back-projected onto all real and synthetic frames, yielding a consistent annotation coordinate frame acro… view at source ↗
Figure 3
Figure 3. Figure 3: GraspIT Scene Variety. Representative Isaac Sim tabletop scenes with ABC objects, randomised table textures, and HDR lighting. Every scene introduces novel object combinations. (a) RGB (b) Depth (c) Masks (d) Grasp+Shape [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: GraspIT Annotation Modalities. RGB, metric depth with sensor noise, per-instance segmentation, and validated 6-DoF grasp candidates overlaid on the 3D point cloud in camera coordinates [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Real↔Sim. Illustration of real Scenes brought to simulation to become grasp annotated [PITH_FULL_IMAGE:figures/full_fig_p007_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 42 canonical work pages · 8 internal anchors

  1. [1]

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky.π 0: A visi...

    work page 2024

  2. [2]

    RT- 1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT- 1: Robotics transformer for real-world control at scale. In Robotics: Science and Systems (RSS), 2023

    work page 2023

  3. [3]

    RT- 2: Vision-language-action models transfer web knowledge to robotic control, 2023

    Anthony Brohan, Noah Brown, Justice Carbajal, et al. RT- 2: Vision-language-action models transfer web knowledge to robotic control, 2023. 1, 2

    work page 2023

  4. [4]

    ShapeNet: An Information-Rich 3D Model Repository

    Angel X. Chang, Thomas A. Funkhouser, Leonidas J. Guibas, Pat Hanrahan, Qi-Xing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, Jianxiong Xiao, Li Yi, and Fisher Yu. Shapenet: An information-rich 3d model repository.CoRR, abs/1512.03012, 2015. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  5. [5]

    Diffu- sion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffu- sion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems (RSS), 2023. 2, 7

    work page 2023

  6. [6]

    Strobl, Matthias Humt, and Rudolph Triebel

    Maximilian Denninger, Dominik Winkelbauer, Martin Sun- dermeyer, Wout Boerdijk, Markus Knauer, Klaus H. Strobl, Matthias Humt, and Rudolph Triebel. Blenderproc2: A procedural pipeline for photorealistic rendering.Journal of Open Source Software, 8(82):4901, 2023. 3

    work page 2023

  7. [7]

    Jacquard: A large scale dataset for robotic grasp detection,

    Amaury Depierre, Emmanuel Dellandr ´ea, and Liming Chen. Jacquard: A large scale dataset for robotic grasp detection,

    work page

  8. [8]

    Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, and Sergey Levine

    Danny Driess, Jost Tobias Springenberg, Brian Ichter, Lili Yu, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, and Sergey Levine. Knowledge insulating vision-language-action models: Train fast, run fast, generalize better, 2025. 1, 2, 3, 7

    work page 2025

  9. [9]

    ACRONYM: A large-scale grasp dataset based on simula- tion

    Clemens Eppner, Arsalan Mousavian, and Dieter Fox. ACRONYM: A large-scale grasp dataset based on simula- tion. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2021. 2, 4, 7

    work page 2021

  10. [10]

    Physical Intelligence et al.π ∗ 0.6: a vla that learns from expe- rience, 2025. 5, 7

    work page 2025

  11. [11]

    Graspnet-1billion: A large-scale benchmark for general ob- ject grasping

    Hao-Shu Fang, Chenxi Wang, Minghao Gou, and Cewu Lu. Graspnet-1billion: A large-scale benchmark for general ob- ject grasping. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11444– 11453, 2020. 1, 2, 4, 7

    work page 2020

  12. [12]

    ManiSkill2: A unified benchmark for generalizable manipulation skills,

    Jiayuan Gu, Fanbo Xiang, Xuanlin Li, et al. ManiSkill2: A unified benchmark for generalizable manipulation skills,

    work page

  13. [13]

    Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes

    Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Ste- fan Holzer, Gary Bradski, Kurt Konolige, and Nassir Navab. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered scenes. InCom- puter Vision – ACCV 2012, pages 548–562, Berlin, Heidel- berg, 2013. Springer Berlin Heidelberg. 2, 4

    work page 2012

  14. [14]

    Tomas Hodan, Pavel Haluza, Step ´an Obdrz´alek, Jiri Matas, Manolis I. A. Lourakis, and Xenophon Zabulis. T-LESS: an RGB-D dataset for 6d pose estimation of texture-less objects. CoRR, abs/1701.05498, 2017. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  15. [15]

    Bop challenge 2023 on detection, segmentation and pose estimation of seen and unseen rigid objects, 2024

    Tomas Hodan, Martin Sundermeyer, Yann Labbe, Van Nguyen Nguyen, Gu Wang, Eric Brachmann, Bertram Drost, Vincent Lepetit, Carsten Rother, and Jiri Matas. Bop challenge 2023 on detection, segmentation and pose estimation of seen and unseen rigid objects, 2024. 2, 3

    work page 2023

  16. [16]

    Effi- cient grasping from RGBD images: Learning using a new rectangle representation

    Yun Jiang, Stephen Moseson, and Ashutosh Saxena. Effi- cient grasping from RGBD images: Learning using a new rectangle representation. InProceedings of the IEEE In- ternational Conference on Robotics and Automation (ICRA), pages 5130–5137, 2011. 2, 7

    work page 2011

  17. [17]

    HomebrewedDB: RGB-D Dataset for 6D Pose Estimation of 3D Objects

    Roman Kaskman, Sergey Zakharov, Ivan Shugurov, and Slo- bodan Ilic. Homebreweddb: RGB-D dataset for 6d pose es- timation of 3d objects.CoRR, abs/1904.03167, 2019. 2

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  18. [18]

    DROID: A large-scale in-the-wild robot manipulation dataset, 2024

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, et al. DROID: A large-scale in-the-wild robot manipulation dataset, 2024. 2, 4, 8

    work page 2024

  19. [19]

    Openvla: An open- source vision-language-action model, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Fos- ter, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kol- lar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open- source vision-language-action model, 2024. 7

    work page 2024

  20. [20]

    Towards robot-assisted data generation with minimal user in- teraction for autonomously training 6d pose estimation in op- erational environments.Procedia CIRP, 120:249–254, 2023

    Paul Koch, Marian Schl ¨uter, Serge Thill, and J ¨org Kr ¨uger. Towards robot-assisted data generation with minimal user in- teraction for autonomously training 6d pose estimation in op- erational environments.Procedia CIRP, 120:249–254, 2023. 56th CIRP International Conference on Manufacturing Sys- tems 2023. 3

    work page 2023

  21. [21]

    Mvip – a dataset and methods for application oriented multi-view and multi-modal industrial part recognition, 2025

    Paul Koch, Marian Schl ¨uter, and J ¨org Kr ¨uger. Mvip – a dataset and methods for application oriented multi-view and multi-modal industrial part recognition, 2025. 2

    work page 2025

  22. [22]

    Abc: A big cad model dataset for geometric deep learning, 2019

    Sebastian Koch, Albert Matveev, Zhongshi Jiang, Francis Williams, Alexey Artemov, Evgeny Burnaev, Marc Alexa, Denis Zorin, and Daniele Panozzo. Abc: A big cad model dataset for geometric deep learning, 2019. 2, 4, 8

    work page 2019

  23. [23]

    Rasim: A range-aware high- fidelity rgb-d data simulation pipeline for real-world appli- cations, 2024

    Xingyu Liu, Chenyangguang Zhang, Gu Wang, Ruida Zhang, and Xiangyang Ji. Rasim: A range-aware high- fidelity rgb-d data simulation pipeline for real-world appli- cations, 2024. 3, 5

    work page 2024

  24. [24]

    Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics, 2017

    Jeffrey Mahler, Jacky Liang, Sherdil Niyaz, Michael Laskey, Richard Doan, Xinyu Liu, Juan Aparicio Ojea, and Ken Goldberg. Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics, 2017. 2

    work page 2017

  25. [25]

    Learning ambidextrous robot grasping policies.Sci- ence Robotics, 4(26):eaau4984, 2019

    Jeffrey Mahler, Matthew Matl, Vishal Satish, Michael Danielczuk, Bill DeRose, Stephen McKinley, and Ken Gold- berg. Learning ambidextrous robot grasping policies.Sci- ence Robotics, 4(26):eaau4984, 2019. 2, 3

    work page 2019

  26. [26]

    Isaac gym: High performance GPU-based physics simulation for robot learning

    Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, 9 Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance GPU-based physics simulation for robot learning. InAdvances in Neural Infor- mation Processing Systems (NeurIPS), 2021. 4

    work page 2021

  27. [27]

    GR00T N1: An open foundation model for gener- alist humanoid robots, 2025

    NVIDIA. GR00T N1: An open foundation model for gener- alist humanoid robots, 2025. 7

    work page 2025

  28. [28]

    Lula robot description and xrdf editor - isaac sim documentation, 2025

    NVIDIA. Lula robot description and xrdf editor - isaac sim documentation, 2025. Accessed: 2026-04-14. 5

    work page 2025

  29. [29]

    Octo: An open-source generalist robot pol- icy, 2024

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, et al. Octo: An open-source generalist robot pol- icy, 2024. 1

    work page 2024

  30. [30]

    Colla., Abhishek Padalkar, et al

    Open X-Emb. Colla., Abhishek Padalkar, et al. Open X- embodiment: Robotic learning datasets and RT-X models,

    work page

  31. [31]

    Learning Transferable Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable vi- sual models from natural language supervision.CoRR, abs/2103.00020, 2021. 1

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  32. [32]

    Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks

    Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. Faster R-CNN: towards real-time object detection with re- gion proposal networks.CoRR, abs/1506.01497, 2015. 1

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  33. [33]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. pages 746–760, 2012. 4

    work page 2012

  34. [34]

    cuRobo: Parallelized Collision-Free Minimum-Jerk Robot Motion Generation

    Balakumar Sundaralingam, Siva Kumar Sastry Hari, Adam Fishman, Caelan Garrett, Karl Van Wyk, Valts Blukis, Alexander Millane, Helen Oleynikova, Ankur Handa, Fabio Ramos, et al. curobo: Parallelized collision-free minimum-jerk robot motion generation.arXiv preprint arXiv:2310.17274, 2023. 3, 5

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  35. [35]

    Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes

    Martin Sundermeyer, Arsalan Mousavian, Rudolph Triebel, and Dieter Fox. Contact-graspnet: Efficient 6-dof grasp gen- eration in cluttered scenes.CoRR, abs/2103.14127, 2021. 2

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  36. [36]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Woj- ciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. InIEEE/RSJ International Conference on Intel- ligent Robots and Systems (IROS), pages 23–30, 2017. 3

    work page 2017

  37. [37]

    Siglip 2: Multilingual vision- language encoders with improved semantic understanding, localization, and dense features, 2025

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier H ´enaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision- language encoders with improved semantic understanding, localization, and dense feature...

    work page 2025

  38. [38]

    BridgeData V2: A dataset for robot learning at scale, 2023

    Homer Walke, Kevin Black, Abraham Lee, et al. BridgeData V2: A dataset for robot learning at scale, 2023. 2

    work page 2023

  39. [39]

    Native and compact structured latents for 3d generation, 2025

    Jianfeng Xiang, Xiaoxue Chen, Sicheng Xu, Ruicheng Wang, Zelong Lv, Yu Deng, Hongyuan Zhu, Yue Dong, Hao Zhao, Nicholas Jing Yuan, and Jiaolong Yang. Native and compact structured latents for 3d generation, 2025. 5

    work page 2025

  40. [40]

    PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

    Yu Xiang, Tanner Schmidt, Venkatraman Narayanan, and Dieter Fox. Posecnn: A convolutional neural network for 6d object pose estimation in cluttered scenes.CoRR, abs/1711.00199, 2017. 2, 4

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  41. [41]

    Understanding the impact of geometric foundation models on vision-language-action models, 2026

    Yurou Yang, Muyuan Lin, Roberto Martin-Martin, Martin Labrie, Shreekant Gayaka, Cheng-Hao Kuo, and Luca Car- lone. Understanding the impact of geometric foundation models on vision-language-action models, 2026. 1, 3, 7

    work page 2026

  42. [42]

    3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,

    Yanjie Ze, Ge Zhang, Kangning Zhang, Chenyou Hu, Ruizhe Wang, and Huazhe Xu. 3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,

    work page