Pith. sign in

REVIEW 4 major objections 3 minor 21 references

VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception

T0 review · 4 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read VinT-6D is a 2.1-million-sample dataset pairing vision, touch, and proprioception for 6D object-in-hand pose estimation, with a baseline showing touch improves accuracy under occlusion.

desk verdict Strong, potentially benchmark-setting dataset paper held back by unmeasured ground-truth accuracy and an arithmetic inconsistency in the headline sample count. read the letter →

arxiv 2501.00510 v2 pith:34I5JHSO submitted 2024-12-31 cs.RO

Zhaoliang Wan , Yonggen Ling , Senlin Yi , Lu Qi , Wangwei Lee , Minglei Lu , Sicheng Yang , Xiao Teng
show 4 more authors
Peng Lu Xu Yang Ming-Hsuan Yang Hui Cheng
This is my paper · ORCID
classification cs.RO
keywords object-in-handposeestimationmulti-modalrobotperceptiontactilesensingproprioceptionsim-to-realtransfer6Droboticmanipulationdatasetbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents VinT-6D, a dataset built to support 6D object-in-hand pose estimation for multi-fingered robotic hands by combining three modalities that are usually collected separately: vision (RGB-D), whole-hand touch, and proprioception. The authors claim it is the first extensive dataset of its kind, with two million simulated samples generated in MuJoCo and rendered in Blender, plus one hundred thousand real samples collected on a custom robot platform. Real objects and hands carry custom marker fixtures tracked by a motion capture system, which the paper asserts provides sub-millimeter object-pose ground truth while the object is held. On top of the dataset, the paper introduces VinT-Net, a baseline that fuses color, depth, and touch point clouds; its experiments show that adding touch to vision raises pose accuracy and makes it degrade more slowly as hand occlusion increases. A sympathetic reader would care because accurate in-hand pose labels at this scale are the missing ingredient for training and benchmarking perception models for dexterous manipulation.

What carries the argument

The load-bearing mechanism is the aligned multi-modal ground-truth pipeline. On the real side, custom-printed fixtures with motion-capture markers are attached to each object, and a flange plate with markers on the hand lets forward kinematics convert activated piezoresistive taxels into a local touch point cloud in the palm frame; the same transformed poses align with RGB-D from a Kinect Azure and stereo cameras. On the simulation side, each taxel is modeled as a force-sensing cuboid on the finger surface, so contact positions rather than raw readings are the shared representation for pressure and vision-based tactile sensors. VinT-Net then treats touch as a complementary point cloud fused with depth features before a keypoint-voting head predicts the object's center and 3D keypoints.

What would settle it

Mount a high-resolution external camera or a second motion-capture setup and independently measure the same object pose during grasps; if the difference between the fixture-based pose and the independent measurement exceeds a millimeter at typical grasp forces, or if marker dropout occurs during recorded segments, the sub-millimeter ground-truth claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that VinT-6D supplies the scale, modality coverage, and label quality needed to train object-in-hand pose estimators that work in real multi-finger grasps. VinT-Sim contributes two million samples by simulating whole-hand taxel distributions on a three-fingered and a four-fingered hand, generating stable grasps in MuJoCo, and rendering photo-realistic RGB-D in Blender. VinT-Real contributes one hundred thousand samples from a platform that uses a motion capture system with custom object fixtures to record object and hand poses, forward kinematics to build touch point clouds, and SAM guided by touch prompts to segment the object from the hand. The dataset covers 25 simulated and 10 real objects across household materials, with occlusion rates from 10% to 90% in simulation. The accompanying VinT-Net uses a U-Net for color features, PointNet++ for depth and touch, and a keypoint-voting pose head; in their experiments the vision-plus-touch model outperforms vision-only on all seven reported real objects and loses less accuracy as occlusion rises from 20% to 50%.

Load-bearing premise

The claim that VinT-Real's object poses are sub-millimeter accurate rests on custom marker fixtures and motion capture staying rigid and fully tracked while the hand holds and moves each object; the paper reports no calibration error, fixture-flex test, reprojection check, or tracking-loss statistics.

Editorial extensions

If this is right

  • Training on the combined simulation-plus-real split gives higher ADD(S) AUC on the reported objects than training on either split alone, supporting the paper's claim that the pipeline narrows the sim-to-real gap.
  • Adding touch and proprioception to vision raises ADD(S) AUC on every real object in the ablation, with the largest gains on objects that are hard for vision, such as the large shaker and tomato soup can.
  • Under increasing hand occlusion, vision-plus-touch accuracy falls from 94.76% to 88.81% at 50% occlusion, while vision-only falls to 80.43%, so the tactile modality is what keeps the estimate usable in heavy occlusion.
  • The dataset's three-finger and four-finger splits with whole-hand taxel distributions give a benchmark that two-finger gripper datasets cannot provide, for methods that need multi-finger contact reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the ground-truth accuracy claim holds, the same collection platform could be extended to category-level or zero-shot object-in-hand pose estimation, which the paper itself names as open future work; the fixture-based labels would then serve as a supervisory signal independent of object-specific meshes.
  • The dataset's choice to store tactile data as contact positions rather than raw taxel readings, if adopted more widely, could let models trained on one tactile sensor type transfer to another with less recalibration; the paper argues for this representation but does not demonstrate cross-sensor transfer.
  • A natural testable extension is to push occlusion beyond 50% to find where touch alone, without vision, becomes the better estimator; the paper's occlusion experiments stop before that crossover.
  • The transparent and reflective objects added to the simulated split could be used to test whether touch and motion-capture labels enable pose estimation where RGB-D alone fails, a regime the paper does not evaluate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper introduces VinT-6D, a multi-modal dataset for 6D object-in-hand pose estimation with two splits: VinT-Sim (synthesized) and VinT-Real (collected on a custom robotic platform). The dataset combines RGB-D vision, whole-hand tactile sensing, and proprioception, and is built for three- and four-fingered robotic hands. The authors describe a MuJoCo/Blender simulation pipeline, a real platform with Vicon motion capture, and a baseline network VinT-Net that fuses vision, touch, and proprioception. Experiments on the real split report that adding touch and proprioception improves pose accuracy, especially under hand-induced occlusion. The appendices describe the hardware setup, tactile sensor modeling, camera calibration, and segmentation procedure in detail.

Significance. If the dataset is released as described, it is a potentially valuable community resource: it is larger than existing real object-in-hand datasets, covers whole-hand tactile sensing rather than only fingertips, and provides an independent motion-capture ground truth that is not derived from the proposed baseline. The simulation pipeline, particularly the 3D scanning of taxel distributions, is concrete and useful for sim-to-real transfer. The VinT-Net baseline is simple and gives initial evidence that touch helps under occlusion. The main open threats are the unquantified sub-millimeter claim for the real-split ground truth and an inconsistent sample-count arithmetic; both are fixable in revision but are load-bearing for the paper's central claims.

major comments (4)
  1. [§3.3.7, §3.1.2, Abstract, Table 1] The dataset-scale arithmetic is internally inconsistent. Section 3.3.7 states "2,500 interactions for each object" with "four different environments, with three camera positions for each," which yields 2,500 × 25 × 4 × 3 = 750,000 VinT-Sim samples, not the claimed "2 million". If the object count is 26 as suggested by Section 3.3.1 (21 YCB objects plus 5 added objects), the total is 780,000; if one instead uses the 125,000 grasps stated in Section 3.1.2, 125,000 × 4 × 3 = 1,500,000. None of these reproduces the abstract's 2M figure. The same inconsistency propagates to Table 1 and Section 6. Please reconcile the counting definitions (per-object, per-hand, per-view) and correct the stated scale.
  2. [§3.2.1 and Appendix A.2.4] The real-split ground-truth pose accuracy is asserted but not measured. The text claims that the Vicon system plus custom fixtures "ensure sub-millimeter accuracy" (Appendix A.2.4), yet no calibration residual for the fixture-to-object transform, no static or dynamic repeatability test, no fixture-deflection test under grasp forces, and no marker-occlusion or tracking-loss statistics are reported; Figures 4 and 15 provide only visual checks. Because every ADD(S) AUC number in Tables 3–6 is evaluated against this ground truth, any unmodeled calibration error would propagate directly into all quantitative conclusions drawn from VinT-Real. Please add a quantitative accuracy characterization, such as reprojection error of the mocap-derived CAD model onto the RGB-D point cloud, worst-case error across the ten objects, and the tracking-loss fraction during the toddler-like motions of Section 3.3.7.
  3. [Tables 3–6] No error bars or repeated-run statistics are reported for any benchmark result. All tables give single point estimates, so the reported improvements (for example, the large shaker row in Table 4, 87.04 vs. 94.66, or the occlusion-robustness gap in Table 6) are not shown to be significant relative to training noise or random split variability. For a paper whose purpose is to establish a benchmark, please provide mean and standard deviation over at least three seeds or train/test splits, and specify the exact splits used.
  4. [§5.3, Table 5] The comparison against prior methods is not sufficiently specified. The paper states that the Object-Hand-Pose method (Wen et al., 2020) was reproduced "by replacing its two-finger gripper with our three-finger hand, which has reduced degrees of freedom. This and other adjustments were made..." but it does not enumerate the other adjustments, the optimization settings, the training data used for the baseline, or the compute budget. Without these details, or released reproduction code and configurations, the claim that VinT-Net "significantly outperforms" prior work in Table 5 is not independently verifiable.
minor comments (3)
  1. [§3.3.1, Section 3, Figure 18] The object count is inconsistent: Section 3.3.1 says 21 YCB objects plus 5 added objects (26 in total), while Section 3 and Figure 18 say 25 objects. Please reconcile this number.
  2. [Throughout] There are several typos and terminology inconsistencies: "Tencnet" in the affiliations, "Internationl Journal of Robotics Research" in the Calli et al. reference, "motion caption system" in the Figure 4 caption, "Benifit" in Section 5.3, the duplicated phrase "robust object segmentation reliable object segmentation" in Section 3.3.6, and inconsistent capitalization between "VinT-Net" and "Vint-Net" in Section 3.3.4.
  3. [§3.3.6 and §4.1] The text refers to "SAM model" and "SAM modal" in places where the intended meaning is the Segment Anything Model; please use consistent terminology such as "SAM" or "the Segment Anything model."

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: dataset ground truth comes from independent simulation and motion capture, and the baseline network is a consumer of that data.

full rationale

The paper's central claims are a dataset with vision, touch, and proprioception, plus a baseline network evaluated on it. The ground-truth poses come from two independent sources: MuJoCo physics simulation for VinT-Sim and a Vicon motion capture system with custom marker fixtures for VinT-Real. These labels are not produced by VinT-Net or by any fitted parameter of the paper; the network is trained and tested against them as an external consumer. Self-citations (Wan et al. 2023; Lin et al. 2023a, 2023b) appear in related-work discussion and in the choice of grasp-generation ideas, but none of these citations supplies the ground-truth mechanism or defines the dataset labels. No uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation: the depth-noise model cites Tolgyessy et al. (2021) and the keypoint-pose module cites He et al. (2020, 2021), both external. The claimed 'sub-millimeter' object-pose accuracy is asserted without calibration or tracking-loss statistics, and the VinT-Sim sample-count arithmetic (2,500 interactions x 4 environments x 3 cameras x 25 objects = 750k, not 2M) is inconsistent; these are empirical/correctness concerns about dataset quality and reporting, not circular derivation steps. The benchmark comparisons in Tables 3-6 are measurements against the dataset, so they do not feed back into the labels. No equation is defined in terms of a predicted quantity, and no fitted parameter is renamed as a prediction. Therefore the paper shows no significant circularity.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

The dataset claims depend on the fidelity of physics simulation, tactile abstraction, motion capture, and SAM-based labeling. None of these are fitted parameters; the network hyperparameters (loss weights) affect only the benchmark demonstration and are not part of the dataset's construction.

free parameters (1)
  • VinT-Net multi-task loss weights (lambda1, lambda2, lambda3) = 1, 2, 1
    Hand-chosen weights for the baseline's semantic, keypoint, and center losses (Eq. 1). They affect the reported benchmark numbers but not the construction or validity of the dataset itself.
assumptions (5)
  • domain assumption MuJoCo contact simulation with cuboid-and-hemisphere taxel proxies yields touch point clouds that transfer to the real piezoresistive sensors.
    Section 3.1.1 models each taxel as a force sensor with non-linear response and records activated positions as touch. This assumes the simulated contact positions match real sensor activations closely enough to support sim-to-real transfer, which is not quantitatively validated.
  • domain assumption Representing tactile readings as contact positions is sensor-agnostic and time-invariant.
    Section 3.1.1 argues this representation is robust to force and time variation. The paper does not measure how much real sensor drift or taxel calibration error remains after this abstraction.
  • domain assumption Vicon motion capture with custom fixtures on objects and a flange plate on the hand provides sub-millimeter ground truth during in-hand manipulation.
    Section 3.2.1 and Appendix A.2.4 assert sub-millimeter accuracy but report no calibration error, fixture-flex test, or tracking-loss statistics.
  • domain assumption SAM guided by touch-point add/remove prompts produces correct segmentation labels for the dataset.
    Section 3.3.6 validates SAM on 100 images of a single object at 20-50% occlusion, then uses SAM to generate labels for the whole real split.
  • domain assumption The Blender depth post-processing (noise, holes, smoothing) faithfully reproduces Kinect Azure TOF characteristics.
    Section 3.1.3 adapts the method of Tolgyessy et al. to emulate Kinect noise; the fidelity of this emulation to the specific camera on the platform is not evaluated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception." pith.science (2026). https://pith.science/paper/34I5JHSO

@misc{pith2026250100510,
  author       = {Pith},
  title        = {Pith review of: VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/34I5JHSO}},
  note         = {Machine review of arXiv:2501.00510}
}
read the original abstract

This paper addresses the scarcity of large-scale datasets for accurate object-in-hand pose estimation, which is crucial for robotic in-hand manipulation within the ``Perception-Planning-Control" paradigm. Specifically, we introduce VinT-6D, the first extensive multi-modal dataset integrating vision, touch, and proprioception, to enhance robotic manipulation. VinT-6D comprises 2 million VinT-Sim and 0.1 million VinT-Real splits, collected via simulations in MuJoCo and Blender and a custom-designed real-world platform. This dataset is tailored for robotic hands, offering models with whole-hand tactile perception and high-quality, well-aligned data. To the best of our knowledge, the VinT-Real is the largest considering the collection difficulties in the real-world environment so that it can bridge the gap of simulation to real compared to the previous works. Built upon VinT-6D, we present a benchmark method that shows significant improvements in performance by fusing multi-modal information. The project is available at https://VinT-6D.github.io/.

Figures

Figures reproduced from arXiv: 2501.00510 by the authors.

Figure 1
Figure 1. Large-scale object-in-hand dataset VinT-6D compris￾ing synthesized and real-world splits naming VinT-Sim and VinT-Real. VinT-Sim aims to generate realistic data across vision, touch, and proprioception. VinT-Real is gathered through a pre￾cisely calibrated and aligned multi-modal robot platform, where a motion capture system obtains accurate object and hand poses. 2023b; Suresh et al., 2023) and grasping (Liang et a… view at source ↗
Figure 2
Figure 2. VinT-Sim Dataset Generation Pipeline. VinT-Sim re￾quires a robotic hand as input, which can have either three or four fingers along with an object model. There are three components involved in this process: (1) Simulating whole-hand touch. (2) Generating tactile data and proprioception information through object-grasp interactions. (3) Rendering each object-grasp scene with various realistic backgrounds and capturin… view at source ↗
Figure 3
Figure 3. Simulated and Real-world Robotic Hands with Whole-Hand Tactile Perception. In VinT-6D, both the three￾fingered Trx hand and the four-fingered Allegro hand are used to generate or collect datasets. These robotic hands are equipped with array-based tactile sensors covering the entire hand, with the simulated sensors distributed similarly to the real-world setup. relationship to accurately replicate tactile contact in … view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Visualization of full tactile points aligned with vision in VinT-Real. Gray points represent the point cloud from the depth camera, blue points depict the transformed model from the motion capture system, and red points indicate the full touch points of the hand. with …
Figure 6
Figure 6. Figure 6: Distribution of occlusion rate and camera poses in VinT-Sim [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Distribution of and occlusion rate and object poses in VinT-Real. 3.3.2. GENERALIZATION OF ROBOTIC HANDS The use of different multi-fingered hands in robotics presents unique challenges that must be addressed. Robotic vision, touch, and proprioception can vary signific…
Figure 8
Figure 8. Figure 8: Overview of VinT-Net. The Frontend receives well-aligned vision and raw touch and proprioception data. Using SAM (Kirillov et al., 2023) and hand’s forward kinematics, it acquires the segmentation label and touch point cloud separately. Then, the Backend predicts the o…
Figure 9
Figure 9. Figure 9: Simulating touch follows the real-world setup. In the upper row, we have an AllegroHand equipped with whole-hand tactile sensors, while in the lower row, we have a Trx-hand with sensors. The distribution of these sensors closely follows that of the real-world setup. A.…
Figure 10
Figure 10. Figure 10: Simulating object-grasp interaction. Selected stable grasping requires stability and preparedness for manipulation. Our simulation-based grasping method replicates real-world scenarios to ensure effective tactile engagement. surfaces, topped with a blue hemisphere, as…
Figure 11
Figure 11. Figure 11: Photorealistic Rendered RGB Images. We present a set of photo-realistic rendered images that depict objects held in hand. A.1.4. SIMULATING VISION Drawing from our prior experience, we have recognized that images captured in MuJoCo for training frequently exhibit a si…
Figure 12
Figure 12. Figure 12: Well-Aligned Vision and Touch Data Visualization. The image on the left shows a color-rendered image of the object being held, while the right side depicts the point cloud generated by the depth camera. The gray points represent the depth, while the red points indicat…
Figure 13
Figure 13. Figure 13: Robotic Pouring Task. Demonstrates the “Perception-Planning-Control” paradigm in a robotic pouring task, highlighting challenges in unstructured household environments due to potential vision obstruction by the robot’s hand. Essential for success is the integration of…
Figure 14
Figure 14. Figure 14: Motion Capture System. The motion capture system accurately captures the poses of the object, hand, and robot base. frequency. During the simulation phase, touch positions are aggregated to form a local touch point cloud. However, translating this into a real-world ap…
Figure 15
Figure 15. Figure 15: Visualization of full tactile points aligned with vision in VinT-Real. Gray points represent the point cloud from the depth camera, blue points represent the transformed model from the motion capture system, and red points indicate the full touch points of the hand. A…
Figure 16
Figure 16. Figure 16: Object-in-hand segmented by large vision model. We use touch and proprioception as cues for the segmentation of a large model. We assign ‘add’ prompts to the activated touch points on the tip of the thumb fingertip while assigning ‘remove’ prompts to those on the inde…
Figure 17
Figure 17. Figure 17: Real-world objects with markers. wine glass vary in size and material distribution, with some, like the stir, presenting a challenge in fingertip touch during rotational holding to prevent dropping. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_17.png]
Figure 18
Figure 18. Figure 18: Synthetic rendering of 25 object models in VinT-Sim [PITH_FULL_IMAGE:figures/full_fig_p020_18.png]
Figure 19
Figure 19. Figure 19: Selected 10 objects used in the VinT-Real dataset [PITH_FULL_IMAGE:figures/full_fig_p020_19.png]
Figure 20
Figure 20. Figure 20: Data collection mirroring toddler-like exploration. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

21 extracted references · 16 canonical work pages

  1. [1]

    https://polyhaven.com

    Polyhaven. https://polyhaven.com. Accessed: 2024-02-01. Shadowhand. https://ninjatek.com/shop/ edge/. Accessed: 2024-02-01. Calli, B., Singh, A., Bruce, J., Walsman, A., Konolige, K., Srinivasa, S., Abbeel, P., and Dollar, A. M. Yale- cmu-berkeley dataset for robotic manipulation research. Internationl Journal of Robotics Research , 36(3):261– 268,

    work page 2024

  2. [7]

    Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation

    Suresh, S., Qi, H., Wu, T., Fan, T., Pineda, L., Lambeta, M., Malik, J., Kalakrishnan, M., Calandra, R., Kaess, M., et al. Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation. arXiv preprint arXiv:2312.13469,

    arXiv

  3. [9]

    In the upper row, we have an AllegroHand equipped with whole-hand tactile sensors, while in the lower row, we have a Trx-hand with sensors

    Simulating touch follows the real-world setup. In the upper row, we have an AllegroHand equipped with whole-hand tactile sensors, while in the lower row, we have a Trx-hand with sensors. The distribution of these sensors closely follows that of the real-world setup. A. Appendix. A.1. VinT-Sim A.1.1. S IMULATION SETUP Given the influence of finger configur...

    work page 2023

  4. [10]

    Unidexgrasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iterative generalist-specialist learning

    Wan, W., Geng, H., Liu, Y ., Shan, Z., Yang, Y ., Yi, L., and Wang, H. Unidexgrasp++: Improving dexterous grasping policy learning via geometry-aware curriculum and iterative generalist-specialist learning. arXiv preprint arXiv:2304.00464,

    arXiv

  5. [11]

    Wen, B., Mitash, C., Soorian, S., Kimmel, A., Sintov, A., and Bekris, K. E. Robust, occlusion-aware pose estima- tion for objects grasped by adaptive hands. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 6210–6217. IEEE,

    work page 2020

  6. [12]

    Posecnn: A convolutional neural network for 6d object pose estima- tion in cluttered scenes

    11 VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception Xiang, Y ., Schmidt, T., Narayanan, V ., and Fox, D. Posecnn: A convolutional neural network for 6d object pose estima- tion in cluttered scenes. arXiv preprint arXiv:1711.00199,

    arXiv

  7. [14]

    (2) (Dikhale et al., 2022; Li et al., 2023; Rezazadeh et al.,

    to generate RGB images for vision-based tactile sensors. (2) (Dikhale et al., 2022; Li et al., 2023; Rezazadeh et al.,

    work page 2022

  8. [15]

    pressure-based tactile sensors simulated by attaching depth cameras to each joint of an AllegroHand, with the point cloud serving as tactile feedback. However, these approaches may introduce a significant domain gap compared to real-world tactile sensors, primarily because they do not account for the curved surface of fingers and the typically smaller, mo...

    work page 2012

Show all 21 references
  1. [16]

    However, these approaches often do not address the critical aspect of stable holding in hand and readiness for manipulative actions, which are key focuses of our research

    and (Wan et al., 2023), have emerged. However, these approaches often do not address the critical aspect of stable holding in hand and readiness for manipulative actions, which are key focuses of our research. Simulating object-hand setup: To simulate object-grasp interactions...

    2023

  2. [17]

    We present a set of photo-realistic rendered images that depict objects held in hand

    Photorealistic Rendered RGB Images. We present a set of photo-realistic rendered images that depict objects held in hand. A.1.4. S IMULATING VISION Drawing from our prior experience, we have recognized that images captured in MuJoCo for training frequently exhibit a significan...

    2021

  3. [18]

    The motion capture system accurately captures the poses of the object, hand, and robot base

    Motion Capture System. The motion capture system accurately captures the poses of the object, hand, and robot base. frequency. During the simulation phase, touch positions are aggregated to form a local touch point cloud. However, translating this into a real-world application...

    2014

  4. [21]

    All models of these fixtures will be made publicly available on our website. A.2.5. O BJECT CATEGORIES In constructing the VinT-6D, we applied the following criteria for object selection: (1) Held in Hand : Objects of an appropriate scale that can be comfortably held in hand. ...

    2017

  5. [565]

    After distortion correction, we reproject the depth image into the right camera’s frame, resulting in two well-aligned vision images with resolutions of 640 x 400,

    Using a chessboard, we repeatedly recalibrated both cameras’ intrinsic and extrinsic parameters to ensure precise alignment. After distortion correction, we reproject the depth image into the right camera’s frame, resulting in two well-aligned vision images with resolutions of...

    2023

  6. [2011]

    Tactile pose estima- tion and policy learning for unknown object manipulation

    Kelestemur, T., Platt, R., and Padir, T. Tactile pose estima- tion and policy learning for unknown object manipulation. arXiv preprint arXiv:2203.10685,

    arXiv

  7. [2015]

    Learning a state estimator for tactile in-hand manipulation

    R¨ostel, L., Sievers, L., Pitz, J., and B ¨auml, B. Learning a state estimator for tactile in-hand manipulation. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4749–4756,

    2022

  8. [2016]

    Smith, E., Calandra, R., Romero, A., Gkioxari, G., Meger, D., Malik, J., and Drozdzal, M

    [Online; accessed 25-May-2020]. Smith, E., Calandra, R., Romero, A., Gkioxari, G., Meger, D., Malik, J., and Drozdzal, M. 3d shape reconstruction from vision and touch. Advances in Neural Information Processing Systems, 33:14193–14206,

    2020

  9. [2017]

    However, we believe that simply sticking markers on objects may not yield high-quality pose data due to the uncertainty in marker positions relative to the object frame

    where objects are simply placed on a table and their poses estimated via QR codes, we adopt a motion capture system similar to (Dikhale et al., 2022), which attaches markers directly to the objects. However, we believe that simply sticking markers on objects may not yield high...

    2022

  10. [2020]

    Tracking object’s pose via dynamic tactile interaction

    Lin, Q., Yan, C., Li, Q., Ling, Y ., Lee, W., Zheng, Y ., Wan, Z., Huang, B., and Liu, X. Tracking object’s pose via dynamic tactile interaction. International Journal of Humanoid Robotics, pp. 2350021, 2023a. Lin, Q., Yan, C., Li, Q., Ling, Y ., Zheng, Y ., Lee, W., Wan, Z., ...

    2023

  11. [2021]

    Posefusion: Robust object-in-hand pose estimation with selectlstm

    Tu, Y ., Jiang, J., Li, S., Hendrich, N., Li, M., and Zhang, J. Posefusion: Robust object-in-hand pose estimation with selectlstm. arXiv preprint arXiv:2304.04523,

    arXiv

  12. [2022]

    C., Lo, W.-Y ., et al

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y ., et al. Segment anything. arXiv preprint arXiv:2304.02643,

    arXiv

  13. [2023]

    Fast-grasp’d: Dexterous multi-finger grasp gener- ation through differentiable simulation

    Turpin, D., Zhong, T., Zhang, S., Zhu, G., Liu, J., Singh, R., Heiden, E., Macklin, M., Tsogkas, S., Dickinson, S., et al. Fast-grasp’d: Dexterous multi-finger grasp gener- ation through differentiable simulation. arXiv preprint arXiv:2306.08132,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.