{"id":"0f7cdb16-31ff-4b50-8dc2-6d24623a59c9","arxiv_id":"2505.02079","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"HandOcc renders articulated hands with a NeRF conditioned on skeletal occupancy instead of a MANO mesh, achieving competitive results on InterHand2.6M.","lead":"This paper introduces HandOcc, a hand rendering pipeline that combines a NeRF with an occupancy network and a skeleton input, removing the need for a parametric mesh model like MANO. It reports rendering quality competitive with mesh-based state-of-the-art on the InterHand2.6M benchmark across several evaluation protocols.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Occupancy pseudo-labels are circularly validated: the 80% IoU cannot be computed without ground truth, yet ray bounds, sampling, and interaction handling all inherit any bias. Compare against MANO-derived geometry on InterHand.","rationale":"The paper's strongest claim is that a MANO-free, skeleton-conditioned occupancy NeRF reaches state-of-the-art hand rendering. The critical dependency is the occupancy field: it is the only geometric prior replacing the mesh, it defines the surface bounds and the hierarchical sampling, and the NeRF is conditioned on its probabilities and features. The training signal for this field is pseudo point clouds obtained by color-consistency and mask filtering (§II.A, §III.A). There is no independent geometric verification. The reported IoU is circular because 'ground truth point clouds are unavailable'—the same paragraph's 'ground truth hand occupancy' can only refer to the pseudo-labels. A biased pseudo-label (e.g., consistent over- or under-estimation of finger thickness, or holes in occluded regions) would be absorbed into the occupancy network and then into ray bounds, sampling, and interaction logic; the NeRF might correct some errors, but the method's core claim is that the occupancy field stands in for the mesh, so the field's accuracy is load-bearing. I do not find a stronger internal inconsistency: the CVAE disentanglement is under-validated but is partially supported by the with/without appearance ablation (Table V), and the SOTA claim is qualified by the actual tables, which show mixed metric wins. The reader's CONDITIONAL verdict is appropriate; the concrete test against MANO-derived geometry is the fastest way to decide whether the pseudo-label concern actually lands.","tokens_in":15553,"tokens_out":9380,"duration_ms":128972,"concrete_test":"On a held-out subset of InterHand2.6M, use the dataset's provided MANO parameters to fit mesh surfaces, voxelize them into binary occupancy, and compute IoU and Chamfer distance against both (a) the color-consistency pseudo point clouds and (b) the learned occupancy field used by HandOcc. If the agreement is substantially below the claimed 80% IoU, or if the errors are spatially structured (e.g., consistently thicker fingers, fused webbing, missing occluded surfaces), the pseudo-label concern is confirmed and the meshless-rendering claim is not yet supported. If the MANO comparison agrees to within the reported margin, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III.A states 'ground truth point clouds are unavailable' and validates the extracted clouds only by 'visual verification on a random subset,' yet the same paragraph reports an 'intersection over union accuracy of our occupancy network is 80% on the validation set.' Since no independent geometry exists, this IoU is necessarily measured against the same pseudo-labels that trained the network (or against an unspecified surrogate); it cannot validate the occupancy field that the whole pipeline relies on. That field is not a peripheral component: it sets the ray bounds (§II.E), drives hierarchical sampling (§II.F), separates interacting hands via per-hand probabilities (§II.B), and its features condition the NeRF (§II.D). If the color-consistency/mask extraction is systematically biased (e.g., surfaces that are self-occluded in all available views are missing, or skin-color background points survive normalization), the bias propagates into the occupancy network, then into the ray sampling and interaction handling. The NeRF can partially compensate because it also sees multi-view images, but the central claim—that a learned occupancy field can replace the parametric MANO mesh—is exactly as strong as the pseudo-labels it was trained on. With code and data withheld, the reader cannot distinguish a trustworthy geometry prior from a geometrically wrong but visually adequate one.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents HandOcc, a meshless hand-rendering framework. Given a 3D hand skeleton and a single RGB image, a pretrained skeleton-conditioned occupancy network predicts per-point occupancy probabilities and features; a CVAE extracts an appearance embedding; a NeRF conditioned on occupancy features, probabilities, and appearance renders a low-resolution image; and an SRResNet CNN upsamples and refines the result. The occupancy field is also used to set ray bounds, guide hierarchical sampling, and separate interacting hands. Experiments are reported on InterHand2.6M under the LiveHand, HandNeRF, and HandAvatar protocols, plus HanCo, with ablations on occupancy-probability and appearance-embedding conditioning. The authors claim to be the first to render a dynamic hand with a pose-conditioned NeRF without an underlying MANO mesh, and they claim state-of-the-art results on InterHand2.6M.","tokens_in":15818,"tokens_out":5388,"duration_ms":60253,"significance":"If the pipeline is sound, the contribution is useful: it replaces the MANO mesh with a skeleton-conditioned occupancy field, which could generalize to objects without parametric models; it introduces an occupancy-guided sampling scheme that accelerates NeRF; the CVAE embedding enables identity and appearance transfer; and the low-resolution NeRF plus CNN upsampling offers a practical speed/quality trade-off. Strengths include evaluation on external benchmark images rather than self-generated renderings, ablations showing that occupancy probabilities and appearance embeddings each improve metrics, and a clearly described modular pipeline. The main reservation is that the occupancy network, which is load-bearing for the whole pipeline, is validated only against self-generated pseudo-labels, so the strength of the central geometric claim is unclear. The paper also reports no error bars, and its 'state-of-the-art' claim extends beyond what the tables support. Code and data are not released, which further limits reproducibility of the pseudo-labelling stage.","major_comments":[{"comment":"The reported 80% IoU for the occupancy network is unverifiable as stated. The paper says 'ground truth point clouds are unavailable' and that the extracted clouds are validated only by 'visual verification on a random subset,' yet the same paragraph reports an 'intersection over union accuracy of our occupancy network is 80% on the validation set.' Because the occupancy network is trained on these pseudo-labels, an IoU computed against the same or similarly derived pseudo-labels cannot validate the geometry; it only measures self-consistency with the extraction pipeline. This matters because the occupancy field sets the ray bounds (Section II.E), drives hierarchical sampling (Section II.F), separates interacting hands (Section II.B), and conditions the NeRF (Section II.D). Any systematic bias in the color-consistency/mask extraction (e.g., self-occluded surfaces missing or background points surviving) propagates into the occupancy network and then into ray sampling and interaction handling. I request an independent validation: compare the predicted occupancy surface against MANO meshes available in InterHand2.6M (e.g., Chamfer distance or voxelized IoU), or against a held-out multi-view reconstruction procedure that does not rely on the same pseudo-labelling, and report the resulting numbers.","section":"Section III.A, with II.B, II.D, II.E, II.F"},{"comment":"The blanket claim of state-of-the-art results is not supported by the tables. In Table I, our LPIPS (27.92) is worse than LiveHand (25.73) and SMPLpix (26.57); in Table II, for interacting hands our SSIM is consistently lower than HandNeRF (0.931 vs 0.955 on 4 views and 0.939 vs 0.956 on 10 views), and on 10-view single hands our LPIPS (0.038) is marginally worse than HandNeRF (0.037); in Table III, on all three HandAvatar splits our LPIPS is worse than HandAvatar (0.107 vs 0.103, 0.117 vs 0.107, 0.112 vs 0.106). The abstract and conclusion should be revised to state the specific metrics and protocols on which the method is superior, competitive, or worse, rather than claiming state-of-the-art results without qualification.","section":"Abstract, Conclusions, Tables I-III"},{"comment":"No error bars, confidence intervals, or repeated-run statistics are reported. Given that several reported margins are very small (e.g., Table I PSNR 32.38 vs 32.37 for SMPLpix; Table III val/Capture0 SSIM 0.895 vs 0.890), the superiority claims cannot be assessed. I ask the authors to report the mean and standard deviation over at least three training runs, or to state explicitly that only a single run was performed, so the reader can calibrate the comparison.","section":"All experiments, Tables I-VI"},{"comment":"The evaluation uses ground-truth images masked with the MANO mesh, which may disadvantage meshless methods. The authors note this for LPIPS, but the same masking can affect PSNR and SSIM because the masked ground truth excludes image content outside the mesh. Please report the fraction of masked pixels, and provide an additional comparison on unmasked ground-truth images, or explain why the masking is protocol-mandated and quantify how it affects each metric.","section":"Section III.C and Table I"}],"minor_comments":[{"comment":"The dataset name is written as 'INTER HAND 2.6M' and should be 'InterHand2.6M'.","section":"Abstract"},{"comment":"There is a typo: 'oportunties' should be 'opportunities'.","section":"Section I.B"},{"comment":"The caption contains 'LiveHnad' and should read 'LiveHand'.","section":"Figure 3 caption"},{"comment":"The method labeled 'Pose-NeRF' cites reference [3], which is Mip-NeRF by Barron et al.; this label/citation mismatch should be corrected.","section":"Table II and references"},{"comment":"The text cites HanCo as [53,54], but those references are 'Contrastive Representation Learning for Hand Shape Estimation' and FreiHand; the correct HanCo citation should be provided.","section":"References [53,54]"},{"comment":"The notation in Eq. (2) is confusing: H^X_i and I^X_Hj need explicit definitions distinguishing the skeleton index, the hand image, and the person identity; the equality should be stated as an objective the CVAE is trained to approximate rather than an identity that holds exactly.","section":"Section II.C, Eq. (2)"},{"comment":"The occupancy threshold p_min and the maximum-saturation/minimum-distance parameters are not given exact values in the implementation details; please specify them.","section":"Section II.E and Implementation Details"},{"comment":"For the HanCo experiments, please clarify what 'persons 26 and 29' refers to, why only those subjects were used, and whether HandAvatar was retrained on the same data and protocol.","section":"Table IV"},{"comment":"Code and data are not released; given the complexity of the point-cloud pseudo-labelling and the difficulty of reproducing it, please include a reproducibility statement or release the occupancy-network training data and extraction scripts.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the validation of the occupancy pseudo-labels, and I agree with the stress-test assessment that this is a load-bearing issue rather than a peripheral one. The paper's stated novelty, 'first to render a dynamic hand using a pose-conditioned NeRF without relying on an underlying MANO model,' should also be checked carefully against HALO [19], which already uses a skeleton-driven neural occupancy representation for hands; the claim should at least distinguish shape reconstruction from NeRF-based rendering. Finally, Tables I-III do not support the unqualified 'state-of-the-art' language in the abstract and conclusion; the authors should be asked to align their claims with the metric-by-metric results."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: HandOcc is the first hand-rendering pipeline in the cited literature that feeds NeRF with a skeleton-conditioned occupancy field plus a CVAE appearance embedding, with no MANO or other parametric mesh. That is a genuine new capability, and the paper shows it produces images that are competitive with mesh-based methods on InterHand2.6M and HanCo across three different evaluation protocols. The ablation tables are useful: removing occupancy probabilities or appearance embeddings measurably hurts, especially on interacting hands. The pipeline is described in enough detail to re-implement (point cloud extraction, occupancy network, per-bone deformation, hierarchical sampling, CNN upsampling).\n\nWhere it goes soft. First, the abstract says 'state-of-the-art results,' but the tables tell a mixed story: on the LiveHand protocol LPIPS is worse than LiveHand; on the HandNeRF single-hand protocol SSIM trails HandNeRF (0.967 vs 0.974); on interacting hands SSIM is clearly worse (0.939 vs 0.956). 'State-of-the-art' only holds for PSNR and sometimes LPIPS. That needs to be fixed.\n\nSecond, the occupancy network is validated with an '80% IoU' but the paper also says ground-truth point clouds are unavailable and the extracted clouds are only visually verified. You cannot compute IoU without a ground truth; the 80% must be measured against the same color-consistency pseudo-labels that trained the network. The stress-test note is right that this is circular, and it matters because the occupancy field sets ray bounds, drives hierarchical sampling, and separates interacting hands. The end-to-end rendering is still evaluated against real benchmark images, so the central rendering claim is not circular—but the paper's internal validation of the occupancy model is. They should compare against MANO-derived geometry from InterHand (which is available) to give an honest outside reference.\n\nThird, no code or data release, and no error bars. For a paper whose headline is a new pipeline, that limits reproducibility; two runs would at least bound the noise.\n\nOverall: this is a solid systems contribution with one inflated claim and one genuinely circular validation. The idea is worth taking seriously, and the benchmark numbers are close enough that a careful referee should engage. Recommend accepting for peer review with the expectation that the authors add an independent occupancy check, temper the SOTA claim, and release code.","headline":"HandOcc is a real new capability—meshless, skeleton-only hand rendering with competitive numbers—but the SOTA claim outruns the tables and the occupancy validation is circular.","tokens_in":16350,"tokens_out":4626,"would_cite":true,"duration_ms":45885,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Hand rendering goes mesh-free with a pose-conditioned NeRF","keywords":["neural radiance fields","hand rendering","occupancy networks","skeleton conditioning","variational autoencoder","meshless rendering","interacting hands","novel view synthesis"],"falsifier":"Compare the learned occupancy field against the MANO mesh ground truth that InterHand2.6M already provides: if the intersection-over-union against MANO-derived volumes on held-out subjects is far below the reported 80 percent, or if rendering quality collapses when the point-cloud extraction is perturbed, then the pseudo-labeling bias is the cause and the mesh-free claim is weakened.","tokens_in":15333,"feed_emoji":"🖐️","tokens_out":5003,"duration_ms":48457,"temperature":0.7,"pith_summary":"This paper tries to establish that a dynamic, photorealistic hand can be rendered from a single image using only an estimated 3D skeleton, with no parametric mesh such as MANO anywhere in the loop. The proposed pipeline trains an occupancy network to encode hand shape from multi-view point clouds, uses a convolutional variational autoencoder to pull appearance and identity from one RGB image into a latent vector, and conditions a NeRF renderer on the resulting occupancy probabilities, features, and appearance embedding. The authors report state-of-the-art or comparable PSNR, SSIM, and LPIPS on the InterHand2.6M benchmark against mesh-based NeRF hand renderers, while also gaining faster ray sampling and a mechanism for handling two interacting hands. The broader point is that meshless neural rendering is not a compromise: the skeleton alone can carry enough shape and appearance information to compete with parametric models.","feed_headline":"Hand rendering goes mesh-free with a pose-conditioned NeRF","feed_subtitle":"Skeleton-only pipeline matches MANO-based renderers on InterHand2.6M and transfers appearance between identities.","key_machinery":"The load-bearing object is a skeleton-conditioned occupancy field: a PointNet encoder reads the sparse 3D hand skeleton, and a decoder assigns each query point a surface probability and a feature embedding. That field does four jobs at once: it defines tight ray bounds for NeRF sampling, supplies extra hierarchical surface samples so no separate fine NeRF is needed, provides per-point conditioning features to the radiance MLP, and, for two hands, labels each point by whichever hand's probability is larger so the renderer can separate interacting hands. Around it, a CVAE supplies the appearance vector, a bone-based rigid deformation layer canonicalizes posed points before the NeRF, and a CNN upsamples the low-resolution NeRF output.","core_discovery":"The central claim is that a pose-conditioned NeRF, guided by a learned occupancy field rather than by a MANO mesh, can render animatable hands at state-of-the-art quality. Given only the 3D skeleton, an occupancy network returns, for any 3D point, the probability that the point lies inside the hand plus per-point features that encode spatial structure; these probabilities supply the NeRF's ray bounds and an extra sampling distribution, and they also disambiguate which of two interacting hands a point belongs to. Appearance and identity are injected through a CVAE latent vector extracted from the input image, which the authors show can be swapped to transfer one person's hand appearance onto another person's skeleton. The paper reports that this mesh-free pipeline outperforms the mesh-based baselines on PSNR under the LiveHand, HandNeRF, and HandAvatar protocols, with a small LPIPS gap that it attributes to the ground-truth masks being derived from MANO meshes.","pith_inferences":["If the unvalidated point-cloud pseudo-labelling proves unbiased, the same recipe should transfer to non-hand objects with articulated skeletons, such as animal paws or soft robots, where no MANO-like parametric model exists.","A direct test of the mesh-free claim would be to compare occupancy-derived surface point clouds against the MANO meshes that InterHand2.6M already provides; if the occupancy field reproduces MANO-like volumes without ever seeing them, the 'no mesh needed' claim is strengthened beyond rendering metrics.","The method's dependence on known camera calibration and hand masks during point-cloud extraction means that 'skeleton-only input' applies at inference; a harder, testable extension is training on monocular video where calibration and masks must be estimated jointly."],"forward_implications":["Mesh-free rendering generalizes beyond hands: any articulated object with a skeleton and multi-view training data can be rendered without a bespoke parametric model.","Because ray bounds come from occupancy probabilities, rendering cost tracks the actual hand surface instead of a fixed volume, which is why the method needs only 16 samples per ray and runs at 7 fps unoptimized.","Interacting hands are handled by occupancy-based assignment, so contact and intersection regions can be rendered by one network rather than separate per-hand models.","Appearance transfer between identities is a byproduct of conditioning the NeRF on CVAE embeddings, as demonstrated by swapping embeddings while keeping the skeleton fixed.","The method's input is a skeleton, so it can consume pose estimates from a standard 3D keypoint detector; the paper shows small Gaussian noise on the skeleton has negligible effect on accuracy."],"supporting_citations":[{"why":"Defines the parametric MANO hand model that the paper deliberately avoids and treats as the baseline limitation.","marker":"[43]"},{"why":"Supplies the volumetric rendering formulation, ray accumulation, and hierarchical sampling that HandOcc adapts.","marker":"[33]"},{"why":"Provides the occupancy-network representation on which the skeleton-conditioned shape field is built.","marker":"[32]"},{"why":"Prior skeleton-driven hand occupancy model that validates the idea of conditioning implicit shape on hand skeletons.","marker":"[19]"},{"why":"The InterHand2.6M benchmark dataset and protocol on which the state-of-the-art claims are evaluated.","marker":"[34]"},{"why":"The primary mesh-based NeRF hand-rendering baseline and evaluation protocol used for comparison.","marker":"[35]"},{"why":"The interacting-hands NeRF baseline whose training and evaluation splits the paper reproduces.","marker":"[11]"},{"why":"Supplies the variational autoencoder formulation used to extract the appearance embedding from a single image.","marker":"[22]"},{"why":"Provides the point-cloud encoder architecture used to condition the occupancy network on the sparse skeleton.","marker":"[5]"}],"fun_headline_variants":["Mesh-free hand rendering: just a skeleton and a NeRF","Skeleton-only NeRF beats mesh-based hand renderers","Occupancy-guided NeRF renders hands without meshes","From skeleton to hand: NeRF with occupancy fields","Meshless hand rendering via skeleton-conditioned NeRF"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The occupancy network is trained on point clouds mined from multi-view images by color consistency and hand masks, with no ground-truth geometry and only visual checks, so a systematic bias in that pseudo-labeling would degrade the occupancy field, the NeRF conditioning, and the ray bounds on which the whole pipeline rests.","fun_headline_variants_meta":{"raw":{"variants":["Mesh-free hand rendering: just a skeleton and a NeRF","Skeleton-only NeRF beats mesh-based hand renderers","Occupancy-guided NeRF renders hands without meshes","From skeleton to hand: NeRF with occupancy fields","Meshless hand rendering via skeleton-conditioned NeRF"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000755,"raw_usage":{"total_tokens":3352,"prompt_tokens":932,"completion_tokens":2420,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":548,"completion_tokens_details":{"reasoning_tokens":2341}},"tokens_in":548,"tokens_out":2420,"duration_ms":16019,"temperature":1.0,"reasoning_tokens":2341,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T01:02:42.467903+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the learned occupancy field against the MANO mesh ground truth that InterHand2.6M already provides: if the intersection-over-union against MANO-derived volumes on held-out subjects is far below the reported 80 percent, or if rendering quality collapses when the point-cloud extraction is perturbed, then the pseudo-labeling bias is the cause and the mesh-free claim is weakened.","supporting_citations":[{"cited_title":"Romero, D","cited_arxiv_id":null,"evidence_quote":"Defines the parametric MANO hand model that the paper deliberately avoids and treats as the baseline limitation."},{"cited_title":"Mildenhall, P","cited_arxiv_id":null,"evidence_quote":"Supplies the volumetric rendering formulation, ray accumulation, and hierarchical sampling that HandOcc adapts."},{"cited_title":"Mescheder, M","cited_arxiv_id":null,"evidence_quote":"Provides the occupancy-network representation on which the skeleton-conditioned shape field is built."},{"cited_title":"Karunratanakul, A","cited_arxiv_id":null,"evidence_quote":"Prior skeleton-driven hand occupancy model that validates the idea of conditioning implicit shape on hand skeletons."},{"cited_title":"Moon, S.-I","cited_arxiv_id":null,"evidence_quote":"The InterHand2.6M benchmark dataset and protocol on which the state-of-the-art claims are evaluated."},{"cited_title":"Mundra, M","cited_arxiv_id":null,"evidence_quote":"The primary mesh-based NeRF hand-rendering baseline and evaluation protocol used for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The interacting-hands NeRF baseline whose training and evaluation splits the paper reproduces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the variational autoencoder formulation used to extract the appearance embedding from a single image."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the point-cloud encoder architecture used to condition the occupancy network on the sparse skeleton."}],"review_version":1}