HandOcc: NeRF-based Hand Rendering with Occupancy Networks

Maksym Ivashechkin; Oscar Mendez; Richard Bowden

REVIEW 4 major objections 9 minor 57 references

HandOcc: NeRF-based Hand Rendering with Occupancy Networks

T0 review · 4 major / 9 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hand rendering goes mesh-free with a pose-conditioned NeRF

desk verdict HandOcc is a real new capability—meshless, skeleton-only hand rendering with competitive numbers—but the SOTA claim outruns the tables and the occupancy validation is circular. read the letter →

arxiv 2505.02079 v1 pith:4BZLT7QX submitted 2025-05-04 cs.CV

Maksym Ivashechkin , Oscar Mendez , Richard Bowden This is my paper · ORCID

classification cs.CV

keywords neuralradiancefieldshandrenderingoccupancynetworksskeletonconditioningvariationalautoencodermeshlessinteractinghandsnovelviewsynthesis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a dynamic, photorealistic hand can be rendered from a single image using only an estimated 3D skeleton, with no parametric mesh such as MANO anywhere in the loop. The proposed pipeline trains an occupancy network to encode hand shape from multi-view point clouds, uses a convolutional variational autoencoder to pull appearance and identity from one RGB image into a latent vector, and conditions a NeRF renderer on the resulting occupancy probabilities, features, and appearance embedding. The authors report state-of-the-art or comparable PSNR, SSIM, and LPIPS on the InterHand2.6M benchmark against mesh-based NeRF hand renderers, while also gaining faster ray sampling and a mechanism for handling two interacting hands. The broader point is that meshless neural rendering is not a compromise: the skeleton alone can carry enough shape and appearance information to compete with parametric models.

What carries the argument

The load-bearing object is a skeleton-conditioned occupancy field: a PointNet encoder reads the sparse 3D hand skeleton, and a decoder assigns each query point a surface probability and a feature embedding. That field does four jobs at once: it defines tight ray bounds for NeRF sampling, supplies extra hierarchical surface samples so no separate fine NeRF is needed, provides per-point conditioning features to the radiance MLP, and, for two hands, labels each point by whichever hand's probability is larger so the renderer can separate interacting hands. Around it, a CVAE supplies the appearance vector, a bone-based rigid deformation layer canonicalizes posed points before the NeRF, and a CNN upsamples the low-resolution NeRF output.

What would settle it

Compare the learned occupancy field against the MANO mesh ground truth that InterHand2.6M already provides: if the intersection-over-union against MANO-derived volumes on held-out subjects is far below the reported 80 percent, or if rendering quality collapses when the point-cloud extraction is perturbed, then the pseudo-labeling bias is the cause and the mesh-free claim is weakened.

Watch

Extended reading notes

Core claim

The central claim is that a pose-conditioned NeRF, guided by a learned occupancy field rather than by a MANO mesh, can render animatable hands at state-of-the-art quality. Given only the 3D skeleton, an occupancy network returns, for any 3D point, the probability that the point lies inside the hand plus per-point features that encode spatial structure; these probabilities supply the NeRF's ray bounds and an extra sampling distribution, and they also disambiguate which of two interacting hands a point belongs to. Appearance and identity are injected through a CVAE latent vector extracted from the input image, which the authors show can be swapped to transfer one person's hand appearance onto another person's skeleton. The paper reports that this mesh-free pipeline outperforms the mesh-based baselines on PSNR under the LiveHand, HandNeRF, and HandAvatar protocols, with a small LPIPS gap that it attributes to the ground-truth masks being derived from MANO meshes.

Load-bearing premise

The occupancy network is trained on point clouds mined from multi-view images by color consistency and hand masks, with no ground-truth geometry and only visual checks, so a systematic bias in that pseudo-labeling would degrade the occupancy field, the NeRF conditioning, and the ray bounds on which the whole pipeline rests.

Editorial extensions

If this is right

Mesh-free rendering generalizes beyond hands: any articulated object with a skeleton and multi-view training data can be rendered without a bespoke parametric model.
Because ray bounds come from occupancy probabilities, rendering cost tracks the actual hand surface instead of a fixed volume, which is why the method needs only 16 samples per ray and runs at 7 fps unoptimized.
Interacting hands are handled by occupancy-based assignment, so contact and intersection regions can be rendered by one network rather than separate per-hand models.
Appearance transfer between identities is a byproduct of conditioning the NeRF on CVAE embeddings, as demonstrated by swapping embeddings while keeping the skeleton fixed.
The method's input is a skeleton, so it can consume pose estimates from a standard 3D keypoint detector; the paper shows small Gaussian noise on the skeleton has negligible effect on accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the unvalidated point-cloud pseudo-labelling proves unbiased, the same recipe should transfer to non-hand objects with articulated skeletons, such as animal paws or soft robots, where no MANO-like parametric model exists.
A direct test of the mesh-free claim would be to compare occupancy-derived surface point clouds against the MANO meshes that InterHand2.6M already provides; if the occupancy field reproduces MANO-like volumes without ever seeing them, the 'no mesh needed' claim is strengthened beyond rendering metrics.
The method's dependence on known camera calibration and hand masks during point-cloud extraction means that 'skeleton-only input' applies at inference; a harder, testable extension is training on monocular video where calibration and masks must be estimated jointly.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First thing to know: HandOcc is the first hand-rendering pipeline in the cited literature that feeds NeRF with a skeleton-conditioned occupancy field plus a CVAE appearance embedding, with no MANO or other parametric mesh. That is a genuine new capability, and the paper shows it produces images that are competitive with mesh-based methods on InterHand2.6M and HanCo across three different evaluation protocols. The ablation tables are useful: removing occupancy probabilities or appearance embeddings measurably hurts, especially on interacting hands. The pipeline is described in enough detail to re-implement (point cloud extraction, occupancy network, per-bone deformation, hierarchical sampling, CNN upsampling).

Where it goes soft. First, the abstract says 'state-of-the-art results,' but the tables tell a mixed story: on the LiveHand protocol LPIPS is worse than LiveHand; on the HandNeRF single-hand protocol SSIM trails HandNeRF (0.967 vs 0.974); on interacting hands SSIM is clearly worse (0.939 vs 0.956). 'State-of-the-art' only holds for PSNR and sometimes LPIPS. That needs to be fixed.

Second, the occupancy network is validated with an '80% IoU' but the paper also says ground-truth point clouds are unavailable and the extracted clouds are only visually verified. You cannot compute IoU without a ground truth; the 80% must be measured against the same color-consistency pseudo-labels that trained the network. The stress-test note is right that this is circular, and it matters because the occupancy field sets ray bounds, drives hierarchical sampling, and separates interacting hands. The end-to-end rendering is still evaluated against real benchmark images, so the central rendering claim is not circular—but the paper's internal validation of the occupancy model is. They should compare against MANO-derived geometry from InterHand (which is available) to give an honest outside reference.

Third, no code or data release, and no error bars. For a paper whose headline is a new pipeline, that limits reproducibility; two runs would at least bound the noise.

Overall: this is a solid systems contribution with one inflated claim and one genuinely circular validation. The idea is worth taking seriously, and the benchmark numbers are close enough that a careful referee should engage. Recommend accepting for peer review with the expectation that the authors add an independent occupancy check, temper the SOTA claim, and release code.

Referee Report

4 major / 9 minor

Summary. The paper presents HandOcc, a meshless hand-rendering framework. Given a 3D hand skeleton and a single RGB image, a pretrained skeleton-conditioned occupancy network predicts per-point occupancy probabilities and features; a CVAE extracts an appearance embedding; a NeRF conditioned on occupancy features, probabilities, and appearance renders a low-resolution image; and an SRResNet CNN upsamples and refines the result. The occupancy field is also used to set ray bounds, guide hierarchical sampling, and separate interacting hands. Experiments are reported on InterHand2.6M under the LiveHand, HandNeRF, and HandAvatar protocols, plus HanCo, with ablations on occupancy-probability and appearance-embedding conditioning. The authors claim to be the first to render a dynamic hand with a pose-conditioned NeRF without an underlying MANO mesh, and they claim state-of-the-art results on InterHand2.6M.

Significance. If the pipeline is sound, the contribution is useful: it replaces the MANO mesh with a skeleton-conditioned occupancy field, which could generalize to objects without parametric models; it introduces an occupancy-guided sampling scheme that accelerates NeRF; the CVAE embedding enables identity and appearance transfer; and the low-resolution NeRF plus CNN upsampling offers a practical speed/quality trade-off. Strengths include evaluation on external benchmark images rather than self-generated renderings, ablations showing that occupancy probabilities and appearance embeddings each improve metrics, and a clearly described modular pipeline. The main reservation is that the occupancy network, which is load-bearing for the whole pipeline, is validated only against self-generated pseudo-labels, so the strength of the central geometric claim is unclear. The paper also reports no error bars, and its 'state-of-the-art' claim extends beyond what the tables support. Code and data are not released, which further limits reproducibility of the pseudo-labelling stage.

major comments (4)

[Section III.A, with II.B, II.D, II.E, II.F] The reported 80% IoU for the occupancy network is unverifiable as stated. The paper says 'ground truth point clouds are unavailable' and that the extracted clouds are validated only by 'visual verification on a random subset,' yet the same paragraph reports an 'intersection over union accuracy of our occupancy network is 80% on the validation set.' Because the occupancy network is trained on these pseudo-labels, an IoU computed against the same or similarly derived pseudo-labels cannot validate the geometry; it only measures self-consistency with the extraction pipeline. This matters because the occupancy field sets the ray bounds (Section II.E), drives hierarchical sampling (Section II.F), separates interacting hands (Section II.B), and conditions the NeRF (Section II.D). Any systematic bias in the color-consistency/mask extraction (e.g., self-occluded surfaces missing or background points surviving) propagates into the occupancy network and then into ray sampling and interaction handling. I request an independent validation: compare the predicted occupancy surface against MANO meshes available in InterHand2.6M (e.g., Chamfer distance or voxelized IoU), or against a held-out multi-view reconstruction procedure that does not rely on the same pseudo-labelling, and report the resulting numbers.
[Abstract, Conclusions, Tables I-III] The blanket claim of state-of-the-art results is not supported by the tables. In Table I, our LPIPS (27.92) is worse than LiveHand (25.73) and SMPLpix (26.57); in Table II, for interacting hands our SSIM is consistently lower than HandNeRF (0.931 vs 0.955 on 4 views and 0.939 vs 0.956 on 10 views), and on 10-view single hands our LPIPS (0.038) is marginally worse than HandNeRF (0.037); in Table III, on all three HandAvatar splits our LPIPS is worse than HandAvatar (0.107 vs 0.103, 0.117 vs 0.107, 0.112 vs 0.106). The abstract and conclusion should be revised to state the specific metrics and protocols on which the method is superior, competitive, or worse, rather than claiming state-of-the-art results without qualification.
[All experiments, Tables I-VI] No error bars, confidence intervals, or repeated-run statistics are reported. Given that several reported margins are very small (e.g., Table I PSNR 32.38 vs 32.37 for SMPLpix; Table III val/Capture0 SSIM 0.895 vs 0.890), the superiority claims cannot be assessed. I ask the authors to report the mean and standard deviation over at least three training runs, or to state explicitly that only a single run was performed, so the reader can calibrate the comparison.
[Section III.C and Table I] The evaluation uses ground-truth images masked with the MANO mesh, which may disadvantage meshless methods. The authors note this for LPIPS, but the same masking can affect PSNR and SSIM because the masked ground truth excludes image content outside the mesh. Please report the fraction of masked pixels, and provide an additional comparison on unmasked ground-truth images, or explain why the masking is protocol-mandated and quantify how it affects each metric.

minor comments (9)

[Abstract] The dataset name is written as 'INTER HAND 2.6M' and should be 'InterHand2.6M'.
[Section I.B] There is a typo: 'oportunties' should be 'opportunities'.
[Figure 3 caption] The caption contains 'LiveHnad' and should read 'LiveHand'.
[Table II and references] The method labeled 'Pose-NeRF' cites reference [3], which is Mip-NeRF by Barron et al.; this label/citation mismatch should be corrected.
[References [53,54]] The text cites HanCo as [53,54], but those references are 'Contrastive Representation Learning for Hand Shape Estimation' and FreiHand; the correct HanCo citation should be provided.
[Section II.C, Eq. (2)] The notation in Eq. (2) is confusing: H^X_i and I^X_Hj need explicit definitions distinguishing the skeleton index, the hand image, and the person identity; the equality should be stated as an objective the CVAE is trained to approximate rather than an identity that holds exactly.
[Section II.E and Implementation Details] The occupancy threshold p_min and the maximum-saturation/minimum-distance parameters are not given exact values in the implementation details; please specify them.
[Table IV] For the HanCo experiments, please clarify what 'persons 26 and 29' refers to, why only those subjects were used, and whether HandAvatar was retrained on the same data and protocol.
[Reproducibility] Code and data are not released; given the complexity of the point-cloud pseudo-labelling and the difficulty of reproducing it, please include a reproducibility statement or release the occupancy-network training data and extraction scripts.

Circularity Check

1 steps flagged · score 4.0 of 10

The occupancy IoU is validated against the same pseudo-label pipeline that generated the labels; the final rendering benchmark remains external.

fitted input called prediction [Section III.A (Implementation Details and Reproducibility); cf. Section II.A (Point Cloud Extraction)]
"Since ground truth point clouds are unavailable, visual verification on a random subset of hands confirms their high realism. ... The intersection over union accuracy of our occupancy network is 80% on the validation set. Such performance demonstrates high overlap with the ground truth hand occupancy, enabling reliable shape reconstruction."

The occupancy labels are produced by the paper's own color-consistency and hand-mask extraction (Section II.A), and the paper explicitly states that ground truth point clouds are unavailable. The reported 80% IoU can therefore only be measured against the same pseudo-labels (or an unspecified surrogate), so it measures how well the network reproduces its own training-label source rather than agreement with an independent ground-truth surface. The text nevertheless presents it as 'high overlap with the ground truth hand occupancy'. This is circular because the validation target is defined by the same pipeline that generated the training data.

full rationale

One genuinely circular validation step was found, and no load-bearing self-citation chain. The dense point clouds used to train the occupancy network come from the paper's own color-consistency and mask-filtering pipeline (Section II.A), and the paper explicitly says ground-truth point clouds are unavailable. The 80% IoU in Section III.A can only be an agreement with those same pseudo-labels (or an unspecified surrogate), yet it is presented as demonstrating 'high overlap with the ground truth hand occupancy'. This is a fitted-input-called-prediction: the occupancy network's accuracy is measured against the label source it was trained to reproduce. The bias is not isolated, because the occupancy field sets ray bounds (Section II.E), guides hierarchical sampling (Section II.F), and separates interacting hands (Section II.B). However, the paper's final rendering claims are not forced by this circularity: the NeRF renders images compared against ground-truth photographs from InterHand2.6M and HanCo using PSNR/LPIPS/SSIM, and ablations independently show gains from occupancy conditioning. The state-of-the-art comparisons are externally grounded. No self-citation is used as a load-bearing proof, and no uniqueness theorem or imported ansatz is invoked. The CVAE appearance-transfer behaviour follows from its training objective (Eqs. 2-3) rather than from an independent prediction, but the paper presents it as a designed mechanism, not as a derived law, so it is not counted as circular. Overall: one localized but load-bearing circular validation, with independent content in the central benchmark.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The pipeline relies on two learned priors trained without independent ground truth (occupancy from multi-view color consistency, appearance from single-image CVAE), plus a hand-tuned sampling bound and loss weighting. No new physical entities are proposed.

free parameters (5)

Occupancy probability threshold p_min = 0.1
Sets the lower ray bound and defines the estimated hand surface in Section II.E. Chosen by hand; rendering quality and sampling efficiency depend on it.
Maximum occupancy saturation bound / min distance = 1-2 cm
Defines the upper ray bound in Section II.E. Chosen by hand; affects how much of the hand volume is sampled.
LPIPS loss weight = 0.4
Weight for perceptual loss in end-to-end training (Section III.A). Chosen by hand; balances with L1 (0.6).
L1 loss weight = 0.6
Weight for pixel-wise L1 loss in end-to-end training (Section III.A). Chosen by hand.
Upsampling scale factor = 2
Scale factor for SRResNet upsampling (Section III.A). Chosen by hand; trades NeRF resolution against render speed.

assumptions (6)

domain assumption Multi-view color consistency after normalization identifies true hand points (Section II.A).
The point cloud extraction filters 3D points by low standard deviation of color across views and by hand masks; if illumination or skin-tone variance breaks color consistency, the pseudo-ground-truth occupancy is corrupted.
domain assumption Hand segmentation masks are accurate for filtering point clouds (Section II.A).
Points are re-projected and checked against 2D binary hand masks; mask errors propagate into the occupancy training data.
domain assumption The occupancy network trained on these pseudo-point-clouds generalizes to real hand shapes (Section II.B).
The only reported validation is 80% IoU on a validation set whose ground truth is generated by the same pipeline, so this generalization is assumed rather than independently confirmed.
domain assumption Skeleton-based per-bone rigid transforms canonicalize the hand with negligible artifacts (Section II.G).
Point clouds are segmented by proximity to skeleton edges and moved with per-bone rigid transformations; the paper asserts artifacts are negligible without quantitative analysis.
domain assumption The CVAE can disentangle identity from pose so that Eq. (2) holds (Section II.C).
The equality of latent codes for the same person's different hands is a training objective, not an architectural guarantee; appearance transfer quality depends on it.
ad hoc to paper For interacting hands, encoding occupancy as pr if pr>pl else -pl is a valid dominance heuristic (Implementation Details, Occupancy Conditioning).
This sign convention is introduced specifically for this pipeline and is not ablated or justified beyond the statement that it identifies the hand with maximum probability.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HandOcc: NeRF-based Hand Rendering with Occupancy Networks." pith.science (2026). https://pith.science/paper/4BZLT7QX

@misc{pith2026250502079,
  author       = {Pith},
  title        = {Pith review of: HandOcc: NeRF-based Hand Rendering with Occupancy Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4BZLT7QX}},
  note         = {Machine review of arXiv:2505.02079}
}

read the original abstract

We propose HandOcc, a novel framework for hand rendering based upon occupancy. Popular rendering methods such as NeRF are often combined with parametric meshes to provide deformable hand models. However, in doing so, such approaches present a trade-off between the fidelity of the mesh and the complexity and dimensionality of the parametric model. The simplicity of parametric mesh structures is appealing, but the underlying issue is that it binds methods to mesh initialization, making it unable to generalize to objects where a parametric model does not exist. It also means that estimation is tied to mesh resolution and the accuracy of mesh fitting. This paper presents a pipeline for meshless 3D rendering, which we apply to the hands. By providing only a 3D skeleton, the desired appearance is extracted via a convolutional model. We do this by exploiting a NeRF renderer conditioned upon an occupancy-based representation. The approach uses the hand occupancy to resolve hand-to-hand interactions further improving results, allowing fast rendering, and excellent hand appearance transfer. On the benchmark InterHand2.6M dataset, we achieved state-of-the-art results.

Figures

Figures reproduced from arXiv: 2505.02079 by the authors.

**Figure 2.** This figure demonstrates the results of the CVAE [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Comparison of single hand rendering to LiveHnad [ [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Comparison of interacting hands with Animatable-NeRF [ [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Figure (a) shows: ground truth, output from the NeRF, output from the upscaling CNN. Figure (b) demonstrates [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: The left figure shows a collage of different hand [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: Pipeline demonstrating the rendering process. The [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 53 canonical work pages

[1]

Alldieck, H

T. Alldieck, H. Xu, and C. Sminchisescu. imghum: Implicit generative models of 3d human shape and articulated pose. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 5441– 5450, 2021. 1

work page 2021
[2]

Atzmon and Y

M. Atzmon and Y . Lipman. Sal: Sign agnostic learning of shapes from raw data. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. 1

work page 2020
[3]

J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin- Brualla, and P. P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 5835–5844, 2021. 8

work page 2021
[4]

Z. Cao, I. Radosavovic, A. Kanazawa, and J. Malik. Reconstructing hand-object interactions in the wild. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 12397–12406, 2021. 2

work page 2021
[5]

R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77–85, 2017. 6

work page 2017
[6]

P. Chen, Y . Chen, D. Yang, F. Wu, Q. Li, Q. Xia, and Y . B. Tan. I2uv- handnet: Image-to-uv prediction network for accurate and high-fidelity 3d hand mesh modeling. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 12909–12918, 2021. 2

work page 2021
[7]

X. Chen, B. Wang, and H.-Y . Shum. Hand avatar: Free-pose hand animation and rendering from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 6, 7, 8, 11

work page 2023
[8]

Corona, T

E. Corona, T. Hodan, M. V o, F. Moreno-Noguer, C. Sweeney, R. New- combe, and L. Ma. Lisa: Learning implicit shape and appearance of hands. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 6

work page 2022

Show all 57 references

[9]

B. Deng, J. Lewis, T. Jeruzalski, G. Pons-Moll, G. Hinton, M. Norouzi, and A. Tagliasacchi. Neural articulated shape approximation. In The European Conference on Computer Vision (ECCV) . Springer, August
[10]

L. Ge, Z. Ren, Y . Li, Z. Xue, Y . Wang, J. Cai, and J. Yuan. 3d hand shape and pose estimation from a single rgb image. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10825–10834, 2019. 1, 2

2019
[11]

Z. Guo, W. Zhou, M. Wang, L. Li, and H. Li. Handnerf: Neural radiance fields for animatable interacting hands. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21078–21087, June 2023. 2, 5, 6, 7, 8, 11

2023
[12]

Hasson, B

Y . Hasson, B. Tekin, F. Bogo, I. Laptev, M. Pollefeys, and C. Schmid. Leveraging photometric consistency over time for sparsely supervised hand-object reconstruction. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 568–577, 2020. 1

2020
[13]

K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 770–778, 2016. 6

2016
[14]

Huang, Y

Z. Huang, Y . Chen, D. Kang, J. Zhang, and Z. Tu. Phrit: Parametric hand representation with implicit template. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , pages 14974–14984, October 2023. 2

2023
[15]

Jiang, Y

B. Jiang, Y . Hong, H. Bao, and J. Zhang. Selfrecon: Self reconstruction your digital avatar from monocular video. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2022. 8

2022
[16]

Jiang, K

W. Jiang, K. M. Yi, G. Samei, O. Tuzel, and A. Ranjan. Neuman: Neural human radiance field from a single video. In Proceedings of the European conference on computer vision (ECCV) , 2022. 7, 8

2022
[17]

J. M. Joyce. Kullback-Leibler Divergence, pages 720–722. Springer Berlin Heidelberg, Berlin, Heidelberg, 2011. 4

2011
[18]

Karunratanakul, S

K. Karunratanakul, S. Prokudin, O. Hilliges, and S. Tang. Harp: Personalized hand reconstruction from a monocular rgb video. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

2023
[19]

Karunratanakul, A

K. Karunratanakul, A. Spurr, Z. Fan, O. Hilliges, and S. Tang. A skeleton-driven neural occupancy representation for articulated hands. In International Conference on 3D Vision (3DV) , 2021. 2

2021
[20]

Karunratanakul, J

K. Karunratanakul, J. Yang, Y . Zhang, M. J. Black, K. Muandet, and S. Tang. Grasping field: Learning implicit representations for human grasps. 2020 International Conference on 3D Vision (3DV) , pages 333–344, 2020. 1

2020
[21]

D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. CoRR, abs/1412.6980, 2014. 6

2014 arXiv
[22]

D. P. Kingma and M. Welling. Auto-encoding variational bayes, 2022. 4

2022
[23]

Krizhevsky, I

A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C. Burges, L. Bottou, and K. Weinberger, editors,Advances in Neural Information Processing Systems, volume 25. Curran Associates, Inc., 2012. 11

2012
[24]

Kulon, R

D. Kulon, R. A. Guler, I. Kokkinos, M. M. Bronstein, and S. Zafeiriou. Weakly-supervised mesh-convolutional hand reconstruction in the wild. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. 4

2020
[25]

Ledig, L

C. Ledig, L. Theis, F. Husz ´ar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. Photo-realistic single image super-resolution using a generative adversarial network. In 2017 IEEE Conference on Computer Vision and Pattern Recognition...

2017
[26]

J. Li, C. Xu, Z. Chen, S. Bian, L. Yang, and C. Lu. Hybrik: A hybrid analytical-neural inverse kinematics solution for 3d human pose and shape estimation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 2

2021
[27]

M. Li, L. An, H. Zhang, L. Wu, F. Chen, T. Yu, and Y . Liu. Interacting attention graph for single image two-hand reconstruction. In IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR), June 2022. 4

2022
[28]

T. Li, T. Bolkart, M. J. Black, H. Li, and J. Romero. Learning a model of facial shape and expression from 4D scans. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) , 36(6):194:1–194:17, 2017. 2

2017
[29]

Liu and W

S. Liu and W. Deng. Very deep convolutional neural network based image classification using small training sample size. In 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR) , pages 730– 734, 2015. 11

2015
[30]

Loper, N

M. Loper, N. Mahmood, J. Romero, G. Pons-Moll, and M. J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia) , 34(6):248:1–248:16, Oct. 2015. 2

2015
[31]

T. Luan, Y . Zhai, J. Meng, Z. Li, Z. Chen, Y . Xu, and J. Yuan. High fidelity 3d hand shape reconstruction via scalable graph frequency decomposition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 16795– 16804, June 2023. 2 9

2023
[32]

Mescheder, M

L. Mescheder, M. Oechsle, M. Niemeyer, S. Nowozin, and A. Geiger. Occupancy networks: Learning 3d reconstruction in function space. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. 1

2019
[33]

Mildenhall, P

B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. In European Conference on Computer Vision (ECCV),
[34]

Moon, S.-I

G. Moon, S.-I. Yu, H. Wen, T. Shiratori, and K. M. Lee. In- terhand2.6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. In European Conference on Computer Vision (ECCV) , 2020. 1, 2, 9, 11

2020
[35]

Mundra, M

A. Mundra, M. B R, J. Wang, M. Habermann, C. Theobalt, and M. Elgharib. Livehand: Real-time and photorealistic neural hand rendering, October 2023. 2, 5, 6, 7, 11

2023
[36]

J. J. Park, P. Florence, J. Straub, R. Newcombe, and S. Lovegrove. Deepsdf: Learning continuous signed distance functions for shape representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019. 1

2019
[37]

K. Park, U. Sinha, J. T. Barron, S. Bouaziz, D. B. Goldman, S. M. Seitz, and R. Martin-Brualla. Nerfies: Deformable neural radiance fields. The IEEE International Conference on Computer Vision (ICCV), 2021. 1

2021
[38]

Pavlakos, V

G. Pavlakos, V . Choutas, N. Ghorbani, T. Bolkart, A. A. A. Osman, D. Tzionas, and M. J. Black. Expressive body capture: 3d hands, face, and body from a single image. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10967– 10977, 2019. 2

2019
[39]

S. Peng, J. Dong, Q. Wang, S. Zhang, Q. Shuai, X. Zhou, and H. Bao. Animatable neural radiance fields for modeling dynamic human bodies. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021. 7, 8

2021
[40]

R. A. Potamias, S. Ploumpis, S. Moschoglou, V . Triantafyllou, and S. Zafeiriou. Handy: Towards a high fidelity 3d hand shape and appearance model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 4670–4680, June 2023. 2

2023
[41]

Prokudin, M

S. Prokudin, M. J. Black, and J. Romero. Smplpix: Neural avatars from 3d human models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1810–1819,
[42]

Pumarola, E

A. Pumarola, E. Corona, G. Pons-Moll, and F. Moreno-Noguer. D- nerf: Neural radiance fields for dynamic scenes. arXiv preprint arXiv:2011.13961, 2020. 1

2011 arXiv
[43]

Romero, D

J. Romero, D. Tzionas, and M. J. Black. Embodied hands: Modeling and capturing hands and bodies together. ACM Transactions on Graphics, (Proc. SIGGRAPH Asia) , 36(6), Nov. 2017. 1, 2

2017
[44]

Spurr, A

A. Spurr, A. Dahiya, X. Wang, X. Zhang, and O. Hilliges. Self- supervised 3d hand pose estimation from monocular rgb via contrastive learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 11230–11239, 2021. 1

2021
[45]

Spurr, J

A. Spurr, J. Song, S. Park, and O. Hilliges. Cross-modal deep varia- tional hand pose estimation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2018. 1

2018
[46]

S.-Y . Su, F. Yu, M. Zollh ¨ofer, and H. Rhodin. A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose. In Advances in Neural Information Processing Systems , 2021. 6

2021
[47]

P. Wang, L. Liu, Y . Liu, C. Theobalt, T. Komura, and W. Wang. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. arXiv preprint arXiv:2106.10689 , 2021. 1

2021 arXiv
[48]

S. Wang, A. Geiger, and S. Tang. Locally aware piecewise trans- formation fields for 3d human mesh registration. In Conference on Computer Vision and Pattern Recognition (CVPR) , 2021. 1, 2

2021
[49]

C.-Y . Weng, B. Curless, P. P. Srinivasan, J. T. Barron, and I. Kemelmacher-Shlizerman. HumanNeRF: Free-viewpoint rendering of moving people from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16210–16220, Ju...

2022
[50]

Zhang, J

J. Zhang, J. Jiao, M. Chen, L. Qu, X. Xu, and Q. Yang. 3d hand pose tracking and estimation using stereo matching. CoRR, abs/1610.07214,

arXiv
[51]

Zhang, P

R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 5

2018
[52]

Zhang, Q

X. Zhang, Q. Li, H. Mo, W. Zhang, and W. Zheng. End-to-end hand mesh recovery from a monocular rgb image. 2019. 1

2019
[53]

Zimmermann, M

C. Zimmermann, M. Argus, and T. Brox. Contrastive representation learning for hand shape estimation. In arxive, 2021. 6, 8, 9, 11

2021
[54]

Zimmermann, D

C. Zimmermann, D. Ceylan, J. Yang, B. Russell, M. Argus, and T. Brox. Freihand: A dataset for markerless capture of hand pose and shape from single rgb images. In IEEE International Conference on Computer Vision (ICCV) , 2019. 6, 8, 9, 11

2019
[55]

Zimmermann, D

C. Zimmermann, D. Ceylan, J. Yang, B. C. Russell, M. Argus, and T. Brox. Freihand: A dataset for markerless capture of hand pose and shape from single rgb images. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 813–822, 2019. 2, 4

2019
[56]

Zuffi, A

S. Zuffi, A. Kanazawa, D. W. Jacobs, and M. J. Black. 3d menagerie: Modeling the 3d shape and pose of animals. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 5524–5532,

2017
[2016]

train/capture0

2 10 VI. I MPLEMENTATION DETAILS Volumetric Hand Cloud. The point cloud extraction process is visualized in Fig. 6. We generate a point cloud and project it onto the multi-view images. By checking color consistency and using hand segmentation masks, we can filter out backgroun...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Alldieck, H

T. Alldieck, H. Xu, and C. Sminchisescu. imghum: Implicit generative models of 3d human shape and articulated pose. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 5441– 5450, 2021. 1

work page 2021

[2] [2]

Atzmon and Y

M. Atzmon and Y . Lipman. Sal: Sign agnostic learning of shapes from raw data. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020. 1

work page 2020

[3] [3]

J. T. Barron, B. Mildenhall, M. Tancik, P. Hedman, R. Martin- Brualla, and P. P. Srinivasan. Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 5835–5844, 2021. 8

work page 2021

[4] [4]

Z. Cao, I. Radosavovic, A. Kanazawa, and J. Malik. Reconstructing hand-object interactions in the wild. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 12397–12406, 2021. 2

work page 2021

[5] [5]

R. Q. Charles, H. Su, M. Kaichun, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 77–85, 2017. 6

work page 2017

[6] [6]

P. Chen, Y . Chen, D. Yang, F. Wu, Q. Li, Q. Xia, and Y . B. Tan. I2uv- handnet: Image-to-uv prediction network for accurate and high-fidelity 3d hand mesh modeling. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 12909–12918, 2021. 2

work page 2021

[7] [7]

X. Chen, B. Wang, and H.-Y . Shum. Hand avatar: Free-pose hand animation and rendering from monocular video. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 6, 7, 8, 11

work page 2023

[8] [8]

Corona, T

E. Corona, T. Hodan, M. V o, F. Moreno-Noguer, C. Sweeney, R. New- combe, and L. Ma. Lisa: Learning implicit shape and appearance of hands. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 6

work page 2022