REVIEW 4 major objections 6 minor 81 references
From hand videos to 3D object models in one minute
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 03:59 UTC pith:KH3E2GUY
load-bearing objection Plausible feed-forward alternative to optimization for HOI reconstruction, but the SOTA claim rests on a thin, curated real-data evaluation. the 4 major comments →
ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videos
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ForeHOI is a feed-forward generative network—specifically, a diffusion model that learns to denoise random noise into meaningful structure—that takes a handful of RGB frames of a hand holding an object and directly outputs the object's complete 3D geometry. The central discovery is that two complementary completion tasks—inpainting the per-frame 2D object masks and filling the occluded parts of a 3D voxel grid—mutually reinforce each other when coupled through bidirectional cross-attention inside the denoising network. A hand-pose feature tells the model where occlusions come from, and the joint training on a purpose-built synthetic dataset lets it hallucinate unobserved object surfaces. At
What carries the argument
The key mechanism is a two-branch generative network with bidirectional cross-attention. One branch denoises per-view complete 2D object masks; the other denoises the 3D voxel latent. At each block, mask-branch features are fed into the geometry branch (replacing plain image features) and geometry features are fed back into the mask branch, so contour information guides volume completion and vice versa. This joint 2D/3D completion is what the paper argues overcomes severe hand-induced occlusion.
Load-bearing premise
The model is trained exclusively on synthetic renderings of hand grasps and is expected to transfer to real monocular video with no real-data fine-tuning; if that sim-to-real transfer is narrower than the chosen benchmark clips suggest, the claimed general performance would not hold.
What would settle it
Evaluate ForeHOI on a large, uncurated set of real hand-object videos with ground-truth scans, evenly spanning object categories, hand sizes, and viewpoints, and compare chamfer distance and F-scores against the figures reported for the curated benchmarks; a disproportionate drop on objects or viewpoints absent from the synthetic training set would show the sim-to-real claim is limited.
If this is right
- Reconstruction of hand-held objects from casual monocular video becomes an interactive-time operation (about one minute), making 3D asset creation from daily footage practical.
- The joint 2D-mask/3D-shape completion recipe could be applied to other occlusion-heavy reconstruction tasks, such as objects behind tools or foreground clutter.
- The synthetic dataset of 400,000 hand-object interaction clips supports training without real 3D annotations, which could lower the cost of future hand-object models.
- Object pose estimation emerges from the reconstructed mesh, so a single model can provide both geometry and trajectory for downstream tasks.
- If the speed and accuracy hold, applications like robot grasp planning, egocentric AR annotations, and interactive 3D modeling from phone videos become feasible.
Where Pith is reading between the lines
- A likely implication the paper does not develop: the bidirectional 2D/3D completion mechanism is a general template for any reconstruction task with structured occlusion—e.g., objects partially hidden by vegetation or medical tools—and could be tested by retraining on such data.
- If the sim-to-real transfer is as strong as reported, one testable extension is scaling the synthetic data with more hand shapes, lighting, and object categories; the prediction is that real-world accuracy rises without any real-data fine-tuning.
- The paper leaves pose estimation as a separate render-and-match post-process; integrating pose estimation into the feed-forward network itself is a natural next step and would likely remove the remaining overhead.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ForeHOI, a feed-forward diffusion-based model that reconstructs 3D object geometry from monocular hand-object interaction videos. The method encodes input frames with DINOv2 and WiLoR hand features, then runs a dual-branch diffusion transformer that jointly denoises a 3D object voxel latent and per-frame 2D complete object masks, with bidirectional cross-attention between the branches. The model is trained exclusively on a new synthetic dataset of 400K clips built from GraspXL grasps, Objaverse meshes, and textured MANO hands rendered in Blender. At inference, the voxel latent is upsampled with a masked structured-latent flow, and per-frame object poses are recovered by a render-and-match procedure using VGGT and Mast3R. The paper claims state-of-the-art reconstruction quality on HO3D and HOT3D, with roughly a 100x speedup over optimization-based baselines, and reports ablations supporting the hand-feature and 2D-mask-completion components.
Significance. If the claims are robust, this is a significant contribution: it replaces hour-scale per-object optimization with a feed-forward network, introduces a large-scale synthetic HOI video dataset, and provides evidence that joint 2D mask and 3D shape completion improves reconstruction under occlusion. The ablations in Table 3 are informative, and the dataset release would be valuable to the community. However, the headline SOTA claim rests on a very small and hand-curated HOT3D evaluation, an unspecified train/test hand-feature pipeline, and quantitative results without variance or per-sequence breakdowns. These gaps currently prevent the paper from fully supporting its advertised generalization to 'daily HOI videos.'
major comments (4)
- [Sec. 4.2 / Table 2] The HOT3D evaluation is based on 6 short clips that are 'carefully preprocessed,' including manual cropping so that the object 'won't be too small.' This is exactly the regime where a synthetic-trained model is most likely to succeed, and it contradicts the abstract's claim of eliminating pre-processing and reconstructing from 'daily' videos. Moreover, MagicHOI and HOLD, the strongest video-based competitors, are N/A on HOT3D, so the large margin over EasyHOI, a single-image method, does not establish video-based SOTA. Please provide per-sequence results on all evaluated HOT3D clips, report results on uncurated or automatically selected clips, and include any available comparisons with MagicHOI/HOLD on HOT3D.
- [Sec. 3.1 / Sec. 6.2] The training-time hand feature pipeline is unclear. Sec. 3.1 says hand features are encoded with 'a state-of-the-art hand pose estimation model [49]' (WiLoR), but Sec. 6.2 describes decoding input hand features into a hand mesh through WiLoR's optimization process. It is not stated whether WiLoR is run on the synthetic renderings during training, whether its weights are frozen, or whether the model instead uses ground-truth hand parameters from GraspXL. If training uses ground-truth hand features and inference uses WiLoR predictions, the hand prior at test time is domain-shifted and includes estimation error. This directly affects the validity of the '+ hand feats' ablation in Table 3. Please specify the exact hand-feature supervision at train and test time, and, if there is a mismatch, quantify its effect.
- [Table 2, Sec. 3.3] The reported reconstruction metrics lack any indication of variance. The geometry branch is a stochastic conditional flow matching model, so results can depend on sampling seed, yet the table gives only point values. The HO3D improvement over MagicHOI is modest (CD 0.79 vs 0.86 cm; F@5 68.95 vs 64.53). Without per-sequence breakdowns or error bars over multiple sampling runs, it is impossible to assess whether this difference is meaningful. Please report per-sequence scores and means over at least a few seeds.
- [Table 1] The object pose comparison is not apples-to-apples. Dynhor is evaluated using only its first-step pose because its refinement is not open-sourced, and HOLD is initialized from SfM poses, which are known to be unreliable in this setting. The resulting pose numbers (RPE 1.42 cm, 2.64 deg vs 4.25 cm, 5.25 deg for Dynhor) may overstate the advantage. Either obtain the final Dynhor output from the authors, or clearly state this limitation next to Table 1.
minor comments (6)
- [Eq. (2)] The loss term contains a typo: `(ϵ−x 0])` has an unmatched bracket; should be `(ϵ - x_0)`.
- [Table 2 caption] The caption says 'HOI dataset' but the second block reports on HOT3D. Use the dataset name consistently.
- [Abstract vs Sec. 4.2] The claim of 'eliminating the need for any pre-processing steps' is too strong given the manual curation, undistortion, and cropping of HOT3D clips described in Sec. 4.2. Please soften or explicitly separate training/inference preprocessing from evaluation-protocol preprocessing.
- [Sec. 4.4] Wording errors: 'Our method outperform all the baseline method' should be 'Our method outperforms all baseline methods.' Also, 'the resulted textured mesh has a high degree of randomness, thus influences' should be 'thus influencing.'
- [Throughout] The paper does not specify the number of diffusion sampling steps, the GPU used for inference-time timing, or the breakdown of the 1.1 min (e.g., voxel denoising, SLat decoding, pose refinement). Please include these details.
- [References] Reference formatting appears corrupted in places (e.g., [1] ends with '1', [54] with '1', [43] with '1'). Please regenerate the bibliography.
Circularity Check
No significant circularity: the central claims are supported by an external synthetic-to-real evaluation, not by definitional or fitted-input reductions.
full rationale
I examined the claimed derivation chain: ForeHOI trains a feed-forward diffusion model on a 400K-clip synthetic dataset (GraspXL + Objaverse renders) with both 2D mask and 3D voxel targets supervised by ground-truth synthetic annotations (Sec 3.3, Eq. 2). The real-world results on HO3D and HOT3D (Sec 4.2, Tab. 2) are external benchmarks that are not used to fit the model; the Chamfer distance and F-scores are computed against independently captured ground-truth geometry. The object-pose evaluation (Sec 3.4, Tab. 1) uses a render-and-match plus PnP pipeline against ground-truth poses and is not a training-input quantity. The HOT3D evaluation is limited to 6 hand-curated clips with frame cropping, and the conclusion acknowledges that diffusion models have inherent precision limitations; these are benchmark-validity concerns, not circularity. The self-citations to ReconViaGen [5] (overlapping authors) for cross-attention and SLat-flow architecture are reuse of prior building blocks, and the paper's central claims are carried by its own ablations (Tab. 3) and the external TRELLIS backbone, so the self-citation is not load-bearing proof of the result. No equation reduces to its own input by construction, and no fitted parameter is renamed as a prediction.
Axiom & Free-Parameter Ledger
free parameters (5)
- beta (2D/3D loss balance) =
unreported
- LoRA rank / alpha =
64 / 128
- number of input views =
random 2-6 views
- pose refinement weights lambda_proj, lambda_smooth =
10.0 / 3.0
- hand-alignment weights lambda_contact, lambda_kpoints, lambda_vsmooth =
200.0 / 20.0 / 20.0
axioms (5)
- domain assumption Synthetic data from GraspXL grasps + Objaverse objects + textured MANO hands rendered in Blender is representative enough of real monocular HOI videos for zero-finetune transfer.
- domain assumption WiLoR hand features used as conditioning are reliable at inference and consistent between the training-time and test-time input distributions.
- domain assumption VGGT coarse camera poses and MAST3R correspondences are accurate enough for the render-and-match pose refinement to succeed on real videos.
- standard math The conditional flow-matching objective (Eq. 2) with a shared time step t for the 2D and 3D branches is a sound coupling mechanism.
- domain assumption TRELLIS structured-latent priors, LoRA fine-tuned on the new synthetic dataset, preserve geometry fidelity when input images are heavily occluded.
read the original abstract
The ubiquity of monocular videos capturing daily hand-object interactions presents a valuable resource for embodied intelligence. While 3D hand reconstruction from in-the-wild videos has seen significant progress, reconstructing the involved objects remains challenging due to severe occlusions and the complex, coupled motion of the camera, hands, and object. In this paper, we introduce ForeHOI, a novel feed-forward model that directly reconstructs 3D object geometry from monocular hand-object interaction videos within one minute of inference time, eliminating the need for any pre-processing steps. Our key insight is that, the joint prediction of 2D mask inpainting and 3D shape completion in a feed-forward framework can effectively address the problem of severe occlusion in monocular hand-held object videos, thereby achieving results that outperform the performance of optimization-based methods. The information exchanges between the 2D and 3D shape completion boosts the overall reconstruction quality, enabling the framework to effectively handle severe hand-object occlusion. Furthermore, to support the training of our model, we contribute the first large-scale, high-fidelity synthetic dataset of hand-object interactions with comprehensive annotations. Extensive experiments demonstrate that ForeHOI achieves state-of-the-art performance in object reconstruction, significantly outperforming previous methods with around a 100x speedup. Code and data are available at: https://github.com/Tao-11-chen/ForeHOI.
Figures
Reference graph
Works this paper leans on
-
[1]
Follow my hold: Hand-object interaction re- construction through geometric guidance, 2025
Ayce Idil Aytekin, Helge Rhodin, Rishabh Dabral, and Chris- tian Theobalt. Follow my hold: Hand-object interaction re- construction through geometric guidance, 2025. 3, 1
2025
-
[2]
Push- ing the envelope for rgb-based dense 3d hand pose estima- tion via neural rendering
Seungryul Baek, Kwang In Kim, and Tae-Kyun Kim. Push- ing the envelope for rgb-based dense 3d hand pose estima- tion via neural rendering. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 2
2019
-
[3]
HOT3D: Hand and object tracking in 3D from ego- centric multi-view videos.CVPR, 2025
Prithviraj Banerjee, Sindi Shkodrani, Pierre Moulon, Shreyas Hampali, Shangchen Han, Fan Zhang, Linguang Zhang, Jade Fountain, Edward Miller, Selen Basol, Richard Newcombe, Robert Wang, Jakob Julian Engel, and Tomas Hodan. HOT3D: Hand and object tracking in 3D from ego- centric multi-view videos.CVPR, 2025. 2, 6
2025
-
[4]
Reconstructing hand-object interactions in the wild
Zhe Cao, Ilija Radosavovic, Angjoo Kanazawa, and Jitendra Malik. Reconstructing hand-object interactions in the wild. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 12417–12426, 2021. 3
2021
-
[5]
Reconviagen: Towards accurate multi-view 3d object reconstruction via generation, 2025
Jiahao Chang, Chongjie Ye, Yushuang Wu, Yuantao Chen, Yidan Zhang, Zhongjin Luo, Chenghong Li, Yihao Zhi, and Xiaoguang Han. Reconviagen: Towards accurate multi-view 3d object reconstruction via generation, 2025. 2, 4, 1
2025
-
[6]
Hongyi Chen, Yunchao Yao, Yufei Ye, Zhixuan Xu, Homanga Bharadhwaj, Jiashun Wang, Shubham Tulsiani, Zackory Erickson, and Jeffrey Ichnowski. Web2grasp: Learning functional grasps from web images of hand-object interactions.arXiv preprint arXiv:2505.05517, 2025. 1
Pith/arXiv arXiv 2025
-
[7]
Handos: 3d hand reconstruction in one stage
Xingyu Chen, Zhuheng Song, Xiaoke Jiang, Yaoqing Hu, Junzhi Yu, and Lei Zhang. Handos: 3d hand reconstruction in one stage. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 1, 3
2025
-
[8]
AlignSDF: Pose-Aligned signed distance fields for hand-object reconstruction
Zerui Chen, Yana Hasson, Cordelia Schmid, and Ivan Laptev. AlignSDF: Pose-Aligned signed distance fields for hand-object reconstruction. InECCV, 2022. 3
2022
-
[9]
gsdf: Geometry-driven signed distance functions for 3d hand-object reconstruction
Zerui Chen, Shizhe Chen, Cordelia Schmid, and Ivan Laptev. gsdf: Geometry-driven signed distance functions for 3d hand-object reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12890–12900, 2023
2023
-
[10]
Zerui Chen, Rolandos Alexandros Potamias, Shizhe Chen, and Cordelia Schmid. HORT: Monocular hand-held ob- jects reconstruction with transformers.arXiv preprint arXiv:2503.21313, 2025. 3, 6, 7
Pith/arXiv arXiv 2025
-
[11]
D-grasp: Physi- cally plausible dynamic grasp synthesis for hand-object in- teractions
Sammy Christen, Muhammed Kocabas, Emre Aksan, Jemin Hwangbo, Jie Song, and Otmar Hilliges. D-grasp: Physi- cally plausible dynamic grasp synthesis for hand-object in- teractions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 1
2022
-
[12]
Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018
Blender Online Community.Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam, 2018. 5
2018
-
[13]
Ganhand: Predicting human grasp affordances in multi-object scenes
Enric Corona, Albert Pumarola, Guillem Alenya, Francesc Moreno-Noguer, and Gregory Rogez. Ganhand: Predicting human grasp affordances in multi-object scenes. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 3
2020
-
[14]
Obja- verse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022
Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Obja- verse: A universe of annotated 3d objects.arXiv preprint arXiv:2212.08051, 2022. 2, 3, 5
Pith/arXiv arXiv 2022
-
[15]
Bench- marks and challenges in pose estimation for egocentric hand interactions with objects
Zicong Fan, Takehiko Ohkawa, Linlin Yang, Nie Lin, Zhis- han Zhou, Shihao Zhou, Jiajun Liang, Zhong Gao, Xu- anyang Zhang, Xue Zhang, Fei Li, Liu Zheng, Feng Lu, Karim Abou Zeid, Bastian Leibe, Jeongwan On, Seungryul Baek, Aditya Prakash, Saurabh Gupta, Kun He, Yoichi Sato, Otmar Hilliges, Hyung Jin Chang, and Angela Yao. Bench- marks and challenges in po...
2024
-
[16]
HOLD: Category-agnostic 3d reconstruction of in- teracting hands and objects from video
Zicong Fan, Maria Parelli, Maria Eleni Kadoglou, Muhammed Kocabas, Xu Chen, Michael J Black, and Otmar Hilliges. HOLD: Category-agnostic 3d reconstruction of in- teracting hands and objects from video. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 494–504, 2024. 3, 6, 7
2024
-
[17]
Fischler and Robert C
Martin A. Fischler and Robert C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Commun. ACM, 24(6):381–395, 1981. 1
1981
-
[18]
Gigahands: A massive annotated dataset of bimanual hand activities
Rao Fu, Dingxi Zhang, Alex Jiang, Wanjia Fu, Austin Fund, Daniel Ritchie, and Srinath Sridhar. Gigahands: A massive annotated dataset of bimanual hand activities. 2025. 1, 3, 6
2025
-
[19]
Honnotate: A method for 3d annotation of hand and object poses
Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vin- cent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. InCVPR, 2020. 2, 3, 6, 8
2020
-
[20]
Keypoint transformer: Solving joint identifica- tion in challenging hands and object interactions for accurate 3d pose estimation
Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, and Vin- cent Lepetit. Keypoint transformer: Solving joint identifica- tion in challenging hands and object interactions for accurate 3d pose estimation. InIEEE Computer Vision and Pattern Recognition Conference, 2022. 2
2022
-
[21]
Keypoint transformer: Solving joint identifica- tion in challenging hands and object interactions for accurate 3d pose estimation
Shreyas Hampali, Sayan Deb Sarkar, Mahdi Rad, and Vin- cent Lepetit. Keypoint transformer: Solving joint identifica- tion in challenging hands and object interactions for accurate 3d pose estimation. InCVPR, 2022. 3, 6
2022
-
[22]
Black, Ivan Laptev, and Cordelia Schmid
Yana Hasson, Gul Varol, Dimitrios Tzionas, Igor Kale- vatykh, Michael J. Black, Ivan Laptev, and Cordelia Schmid. Learning joint reconstruction of hands and manipulated ob- jects. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2019. 3
2019
-
[23]
Black, Ivan Laptev, and Cordelia Schmid
Yana Hasson, G ¨ul Varol, Dimitrios Tzionas, Igor Kale- vatykh, Michael J. Black, Ivan Laptev, and Cordelia Schmid. Learning joint reconstruction of hands and manipulated ob- jects. InCVPR, 2019. 3, 6, 2
2019
-
[24]
Viewdiff: 3d-consistent image generation with text-to-image models
Lukas H ¨ollein, Aljaˇz Boˇziˇc, Norman M¨uller, David Novotny, Hung-Yu Tseng, Christian Richardt, Michael Zollh ¨ofer, and Matthias Nießner. Viewdiff: 3d-consistent image generation with text-to-image models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5043–5052, 2024. 2
2024
-
[25]
Cupid: Pose-grounded generative 3d reconstruction from a single image, 2025
Binbin Huang, Haobin Duan, Yiqun Zhao, Zibo Zhao, Yi Ma, and Shenghua Gao. Cupid: Pose-grounded generative 3d reconstruction from a single image, 2025. 2
2025
-
[26]
Reconstruct- ing hand-held objects from monocular video
Di Huang, Xiaopeng Ji, Xingyi He, Jiaming Sun, Tong He, Qing Shuai, Wanli Ouyang, and Xiaowei Zhou. Reconstruct- ing hand-held objects from monocular video. InSIGGRAPH Asia Conference Proceedings, 2022. 3
2022
-
[27]
Affordpose: A large-scale dataset of hand-object inter- actions with affordance-driven hand pose
Juntao Jian, Xiuping Liu, Manyi Li, Ruizhen Hu, and Jian Liu. Affordpose: A large-scale dataset of hand-object inter- actions with affordance-driven hand pose. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 14713–14724, 2023. 3, 6, 2
2023
-
[28]
In-hand 3d object recon- struction from a monocular rgb video
Shijian Jiang, Qi Ye, Rengan Xie, Yuchi Huo, Xiang Li, Yang Zhou, and Jiming Chen. In-hand 3d object recon- struction from a monocular rgb video. InProceedings of the AAAI Conference on Artificial Intelligence, pages 2525– 2533, 2024. 3
2024
-
[29]
Hand-held object reconstruction from rgb video with dynamic interaction
Shijian Jiang, Qi Ye, Rengan Xie, Yuchi Huo, and Jim- ing Chen. Hand-held object reconstruction from rgb video with dynamic interaction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12220–12230, 2025. 7
2025
-
[30]
Hand-held object reconstruction from rgb video with dynamic interaction
Shijian Jiang, Qi Ye, Rengan Xie, Yuchi Huo, and Jiming Chen. Hand-held object reconstruction from rgb video with dynamic interaction. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 12220–12230,
-
[31]
Weitai Kang, Haifeng Huang, Yuzhang Shang, Mubarak Shah, and Yan Yan. Robin3d: Improving 3d large lan- guage model via robust instruction tuning.arXiv preprint arXiv:2410.00255, 2024. 2
Pith/arXiv arXiv 2024
-
[32]
Bronstein, and Stefanos Zafeiriou
Dominik Kulon, Riza Alp Guler, Iasonas Kokkinos, Michael M. Bronstein, and Stefanos Zafeiriou. Weakly- supervised mesh-convolutional hand reconstruction in the wild. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), 2020. 2
2020
-
[33]
Ground- ing image matching in 3d with mast3r, 2024
Vincent Leroy, Yohann Cabon, and Jerome Revaud. Ground- ing image matching in 3d with mast3r, 2024. 1
2024
-
[34]
Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model
Jiahao Li, Hao Tan, Kai Zhang, Zexiang Xu, Fujun Luan, Yinghao Xu, Yicong Hong, Kalyan Sunkavalli, Greg Shakhnarovich, and Sai Bi. Instant3d: Fast text-to-3d with sparse-view generation and large reconstruction model. In International Conference on Representation Learning, pages 21896–21920, 2024. 2
2024
-
[35]
Qixiu Li, Yu Deng, Yaobo Liang, Lin Luo, Lei Zhou, Chengtang Yao, Lingqi Zeng, Zhiyuan Feng, Huizhi Liang, Sicheng Xu, Yizhong Zhang, Xi Chen, Hao Chen, Lily Sun, Dong Chen, Jiaolong Yang, and Baining Guo. Scalable vision-language-action model pretraining for robotic manip- ulation with real-life human activity videos.arXiv preprint arXiv:2510.21571, 2025. 3, 6
arXiv 2025
-
[36]
Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape synthesis using large-scale rectified flow models.arXiv preprint arXiv:2502.06608, 2025. 2
Pith/arXiv arXiv 2025
-
[37]
End-to-end hu- man pose and mesh reconstruction with transformers
Kevin Lin, Lijuan Wang, and Zicheng Liu. End-to-end hu- man pose and mesh reconstruction with transformers. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1954–1963, 2021. 2
1954
-
[38]
Zero-1-to-3: Zero-shot one image to 3d object, 2023
Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object, 2023. 2
2023
-
[39]
Semi-supervised 3d hand-object poses es- timation with interactions in time
Shaowei Liu, Hanwen Jiang, Jiarui Xu, Sifei Liu, and Xi- aolong Wang. Semi-supervised 3d hand-object poses es- timation with interactions in time. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14687–14697, 2021. 2
2021
-
[40]
Semi-supervised 3d hand-object poses es- timation with interactions in time
Shaowei Liu, Hanwen Jiang, Jiarui Xu, Sifei Liu, and Xi- aolong Wang. Semi-supervised 3d hand-object poses es- timation with interactions in time. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, 2021. 3
2021
-
[41]
Hoi4d: A 4d egocentric dataset for category-level human- object interaction
Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human- object interaction. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 21013–21022, 2022. 3, 6
2022
-
[42]
Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Gen- erating multiview-consistent images from a single-view im- age.arXiv preprint arXiv:2309.03453, 2023. 2
Pith/arXiv arXiv 2023
-
[43]
Easyhoi: Unleashing the power of large models for reconstructing hand-object interactions in the wild
Yumeng Liu, Xiaoxiao Long, Zemin Yang, Yuan Liu, Marc Habermann, Christian Theobalt, Yuexin Ma, and Wenping Wang. Easyhoi: Unleashing the power of large models for reconstructing hand-object interactions in the wild. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pages 7037–7047, 2025. 2, 3, 7, 1
2025
-
[44]
Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.Advances in neural information processing systems, 32, 2019
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.Advances in neural information processing systems, 32, 2019. 4
2019
-
[45]
Srinivasan, Matthew Tancik, Jonathan T
Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. InECCV, 2020. 3
2020
-
[46]
Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Rus- sell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang- Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nico- las Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patri...
2023
-
[47]
Recon- structing hands in 3d with transformers
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Recon- structing hands in 3d with transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9826–9836, 2024. 2
2024
-
[48]
Barron, and Ben Milden- hall
Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion.arXiv,
-
[49]
Wilor: End-to-end 3d hand localization and reconstruction in-the-wild, 2024
Rolandos Alexandros Potamias, Jinglei Zhang, Jiankang Deng, and Stefanos Zafeiriou. Wilor: End-to-end 3d hand localization and reconstruction in-the-wild, 2024. 1, 2, 3
2024
-
[50]
3d reconstruction of objects in hands without real world 3d supervision
Aditya Prakash, Matthew Chang, Matthew Jin, Ruisen Tu, and Saurabh Gupta. 3d reconstruction of objects in hands without real world 3d supervision. InEuropean Conference on Computer Vision (ECCV), 2024. 3
2024
-
[51]
HTML: A Para- metric Hand Texture Model for 3D Hand Reconstruction and Personalization
Neng Qian, Jiayi Wang, Franziska Mueller, Florian Bernard, Vladislav Golyanik, and Christian Theobalt. HTML: A Para- metric Hand Texture Model for 3D Hand Reconstruction and Personalization. InProceedings of the European Conference on Computer Vision (ECCV). Springer, 2020. 2, 5
2020
-
[52]
Novel- view synthesis and pose estimation for hand-object interac- tion from sparse views
Wentian Qu, Zhaopeng Cui, Yinda Zhang, Chenyu Meng, Cuixia Ma, Xiaoming Deng, and Hongan Wang. Novel- view synthesis and pose estimation for hand-object interac- tion from sparse views. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15100–15111, 2023. 3
2023
-
[53]
High-resolution image syn- thesis with latent diffusion models, 2021
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 2
2021
-
[54]
Javier Romero, Dimitrios Tzionas, and Michael J. Black. Embodied hands: modeling and capturing hands and bodies together.ACM Trans. Graph., 36(6), 2017. 2, 5, 1
2017
-
[55]
Photorealistic text-to-image diffusion models with deep lan- guage understanding
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Sali- mans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep lan- guage understanding. InAdvances in Neural Information Processing Systems, pages 3647...
2022
-
[56]
SuperGlue: Learning feature matching with graph neural networks
Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning feature matching with graph neural networks. InCVPR, 2020. 2
2020
-
[57]
Visual odom- etry [tutorial].IEEE Robotics & Automation Magazine, 18 (4):80–92, 2011
Davide Scaramuzza and Friedrich Fraundorfer. Visual odom- etry [tutorial].IEEE Robotics & Automation Magazine, 18 (4):80–92, 2011. 7
2011
-
[58]
Zero123++: a single image to consistent multi-view dif- fusion base model.CoRR, abs/2310.15110, 2023
Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view dif- fusion base model.CoRR, abs/2310.15110, 2023. 2
Pith/arXiv arXiv 2023
-
[59]
Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material,
Tencent Hunyuan3D Team. Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material,
-
[60]
Hunyuan3d 2.0: Scaling diffu- sion models for high resolution textured 3d assets generation,
Tencent Hunyuan3D Team. Hunyuan3d 2.0: Scaling diffu- sion models for high resolution textured 3d assets generation,
-
[61]
H+o: Uni- fied egocentric recognition of 3d hand-object poses and in- teractions
Bugra Tekin, Federica Bogo, and Marc Pollefeys. H+o: Uni- fied egocentric recognition of 3d hand-object poses and in- teractions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 3
2019
-
[62]
Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151, 2024
Dmitry Tochilkin, David Pankratz, Zexiang Liu, Zixuan Huang, , Adam Letts, Yangguang Li, Ding Liang, Christian Laforte, Varun Jampani, and Yan-Pei Cao. Triposr: Fast 3d object reconstruction from a single image.arXiv preprint arXiv:2403.02151, 2024. 2
Pith/arXiv arXiv 2024
-
[63]
Vggt: Visual geometry grounded transformer
Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 2, 4
2025
-
[64]
Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction
Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689, 2021. 3
Pith/arXiv arXiv 2021
-
[65]
Magichoi: Leveraging 3d priors for accurate hand-object reconstruction from short monoc- ular video clips
Shibo Wang, Haonan He, Maria Parelli, Christoph Gebhardt, Zicong Fan, and Jie Song. Magichoi: Leveraging 3d priors for accurate hand-object reconstruction from short monoc- ular video clips. InProceedings of the IEEE/CVF Interna- tional Conference on Computer Vision (ICCV), pages 5957– 5968, 2025. 3
2025
-
[66]
Shibo Wang, Haonan He, Maria Parelli, Christoph Gebhardt, Zicong Fan, and Jie Song. Magichoi: Leveraging 3d priors for accurate hand-object reconstruction from short monocu- lar video clips.Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025. 2, 6, 7, 1
2025
-
[67]
Novel view synthesis with diffusion models, 2022
Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, and Mohammad Norouzi. Novel view synthesis with diffusion models, 2022. 2
2022
-
[68]
BundleSDF: Neural 6-DoF tracking and 3D reconstruction of unknown objects
Bowen Wen, Jonathan Tremblay, Valts Blukis, Stephen Tyree, Thomas M ¨uller, Alex Evans, Dieter Fox, Jan Kautz, and Stan Birchfield. BundleSDF: Neural 6-DoF tracking and 3D reconstruction of unknown objects. InCVPR, 2023. 3
2023
-
[69]
Reconstructing hand-held objects in 3d.arXiv preprint arXiv:2404.06507, 2024
Jane Wu, Georgios Pavlakos, Georgia Gkioxari, and Jiten- dra Malik. Reconstructing hand-held objects in 3d.arXiv preprint arXiv:2404.06507, 2024. 3
arXiv 2024
-
[70]
Srinivasan, Dor Verbin, Jonathan T
Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan T. Barron, Ben Poole, and Aleksander Ho?y?ski. Reconfusion: 3d reconstruction with diffusion priors. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 21551– 21561, 2024. 2
2024
-
[71]
Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong Yang. Structured 3d latents for scalable and versatile 3d gen- eration.arXiv preprint arXiv:2412.01506, 2024. 2, 5, 7, 1
Pith/arXiv arXiv 2024
-
[72]
Cpf: Learning a contact potential field to model the hand-object interaction
Lixin Yang, Xinyu Zhan, Kailin Li, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Cpf: Learning a contact potential field to model the hand-object interaction. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11097–11106, 2021. 3
2021
-
[73]
Artiboost: Boosting articulated 3d hand-object pose estimation via online exploration and syn- thesis
Lixin Yang, Kailin Li, Xinyu Zhan, Jun Lv, Wenqiang Xu, Jiefeng Li, and Cewu Lu. Artiboost: Boosting articulated 3d hand-object pose estimation via online exploration and syn- thesis. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 2750– 2760, 2022. 3
2022
-
[74]
What’s in your hands? 3d reconstruction of generic objects in hands
Yufei Ye, Abhinav Gupta, and Shubham Tulsiani. What’s in your hands? 3d reconstruction of generic objects in hands. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3895–3905,
-
[75]
Diffusion-guided reconstruction of everyday hand- object interaction clips
Yufei Ye, Poorvi Hebbar, Abhinav Gupta, and Shubham Tul- siani. Diffusion-guided reconstruction of everyday hand- object interaction clips. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 19717–19728, 2023. 3
2023
-
[76]
G-hop: Generative hand-object prior for interac- tion reconstruction and grasp synthesis
Yufei Ye, Abhinav Gupta, Kris Kitani, and Shubham Tul- siani. G-hop: Generative hand-object prior for interac- tion reconstruction and grasp synthesis. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1911–1920, 2024. 3
1911
-
[77]
Predicting 4d hand trajectory from monocular videos.arXiv preprint arXiv:2501.08329,
Yufei Ye, Yao Feng, Omid Taheri, Haiwen Feng, Shubham Tulsiani, and Michael J Black. Predicting 4d hand trajectory from monocular videos.arXiv preprint arXiv:2501.08329,
-
[78]
Ddf-ho: Hand-held object reconstruction via conditional directed distance field
Chenyangguang Zhang, Yan Di, Ruida Zhang, Guangyao Zhai, Fabian Manhardt, Federico Tombari, and Xiangyang Ji. Ddf-ho: Hand-held object reconstruction via conditional directed distance field. InAdvances in Neural Information Processing Systems, pages 56871–56884. Curran Associates, Inc., 2023. 3
2023
-
[79]
GraspXL: Generating grasping motions for di- verse objects at scale
Hui Zhang, Sammy Christen, Zicong Fan, Otmar Hilliges, and Jie Song. GraspXL: Generating grasping motions for di- verse objects at scale. InEuropean Conference on Computer Vision (ECCV), 2024. 2, 3
2024
-
[80]
ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasp- ing and articulation
Hui Zhang, Sammy Christen, Zicong Fan, Luocheng Zheng, Jemin Hwangbo, Jie Song, and Otmar Hilliges. ArtiGrasp: Physically plausible synthesis of bi-manual dexterous grasp- ing and articulation. InInternational Conference on 3D Vi- sion (3DV), 2024. 1
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.