{"id":"f900d870-4e7e-40ce-9947-9ef20be56629","arxiv_id":"2509.01242","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A single linear layer applied to per-joint Gaussian noise gives a correlation-aware covariance for 3D hand pose uncertainty, improving calibration metrics on two benchmarks.","lead":"This paper adds an uncertainty head to a 3D hand pose estimator, using one learned linear layer on sampled Gaussian noise to model correlations between joints. On two benchmarks it reports better calibrated uncertainty while keeping pose accuracy, though the training objective for that layer appears problematic.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MSE objective drives W to zero; no term couples W to residual covariance, so the claimed correlation learning is unsupported.","rationale":"The reader's weakest assumption precisely identifies the load-bearing flaw: the training loss L_MSE = E||y − (μ + Wz)||², with z drawn independently from N(0, diag(σ²)), has expected gradient 2W diag(σ²) with respect to W, driving W toward zero. The diagonal NLL term (Eq. 2) and L_DETER do not involve W, so the objective provides no signal for W to fit the true residual covariance. The paper claims that sampling and linear transformation 'naturally capture the correlation between hand joints,' but the mathematics shows otherwise. The empirical ablation (Tables 4–5) shows small improvements when adding the linear layer, but these are not accompanied by error bars or an analysis of W's learned values. Given the tiny learning rate, W could remain near its initialization (likely identity) rather than learning meaningful correlations. The AMVUR comparison and other baselines do not address this core issue. Therefore the central contribution is unsupported as stated, and rejection is appropriate. The concern is internal inconsistency in the training objective, not merely a disagreement with community consensus.","tokens_in":12970,"tokens_out":7627,"duration_ms":95910,"concrete_test":"Re-derive the expected gradient of Eq. (6) w.r.t. W; if it equals 2W diag(σ²), then train the model with W initialized to identity and monitor ||W−I||_F and off-diagonal entries over training. Then compare uncertainty metrics (AUSC, AUSE, Pearson ρ) for the learned W versus W frozen at identity. If metrics are statistically indistinguishable, or if W stays at identity/shrinks toward zero, the linear layer is not learning joint correlations.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim—that the single linear layer W learns joint correlations—is not supported by the training objective. With z ~ N(0, diag(σ²)) independent of the residual (y−μ), the expected gradient of Eq. (6) with respect to W is ∇_W E||y−(μ+Wz)||² = 2W diag(σ²), because E[zzᵀ]=diag(σ²) and E[(y−μ)zᵀ]=0. This is a contraction toward W=0; the only fixed point is W=0. The diagonal NLL term (Eq. 2) and L_DETER do not involve W, so nothing in Eq. (7) rewards W for matching the true residual covariance. The paper provides no gradient analysis or control experiment to rule out this collapse. In practice, the very small learning rate (1e-6) and λ_MSE=5e-4 may keep W near its initialization (likely identity), so the reported improvements over the no-linear-layer ablation could stem from a near-identity linear transform rather than learned correlations. This is internally inconsistent: the model's predictive covariance is W diag(σ²)Wᵀ, but the NLL term assumes diag(σ²), so the objective does not correspond to the claimed distribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses aleatoric uncertainty estimation for 3D hand pose estimation. The authors propose to model the predictive distribution as N(μ, W diag(σ²) W^T), where μ is the mean hand pose, σ² are per-joint variances estimated by a diagonal uncertainty head, and W is a single learned linear layer intended to capture correlations among hand joints. Training combines the deterministic HaMeR losses, a diagonal negative log-likelihood term (Eq. 2), and an MSE loss on samples drawn from the noise model (Eq. 6). Experiments on FreiHAND and HO3Dv2 report improved uncertainty metrics (AUSC, AUSE, Pearson's ρ) over diagonal and full-covariance baselines, and competitive pose accuracy. The central claim is that the linear layer W learns intrinsic hand joint correlations and provides a favorable efficiency/expressiveness trade-off.","tokens_in":13300,"tokens_out":8223,"duration_ms":92840,"significance":"If the proposed mechanism were sound, the parameterization would be a useful intermediate between diagonal and full covariance modeling: it introduces O(d_o^2) parameters but avoids full covariance's O(d_o^2) per-input cost and optimization instability. The paper also demonstrates a practical application of uncertainty estimation to hand pose estimation. However, the significance is conditional on the training objective actually fitting W to the residual covariance, and the manuscript does not establish that. The gradient analysis below shows that the MSE loss drives W toward zero and no term in the objective rewards W for capturing correlations, so the central claim is not supported. The empirical improvements, while present, are not explained by the proposed mechanism. The paper's internal inconsistency between the claimed predictive distribution and the training loss make the contribution, as presented, unsound.","major_comments":[{"comment":"The MSE loss does not train W to model joint correlations. Let z ~ N(0, diag(σ²)) be the synthetic noise and r = y − μ. Then L_MSE = E||r − Wz||². Since z is conditionally independent of the data residual r given x, E[r z^T] = 0 and E[z z^T] = diag(σ²), so ∇_W L_MSE = 2 λ_MSE W diag(σ²). This is a contraction toward W = 0; no term in Eq. (7)—neither the diagonal NLL (Eq. 2) nor L_DETER—involves W. Therefore, the training objective provides no mechanism for W to learn the residual covariance. The paper's claim that 'we naturally capture the correlation between hand joints' by this sampling and linear transformation is unsupported. No gradient analysis or control experiment is provided to rule out this collapse.","section":"§3.2, Eq. (6)-(7)"},{"comment":"The model defines the predictive covariance as Σ = W diag(σ²) W^T (Eq. 5), but the NLL term used in training is the diagonal NLL with covariance diag(σ²) (Eq. 2). The full-covariance NLL (Eq. 4) is used only for the Full baseline, not for the proposed model. Consequently, the training objective does not correspond to the negative log-likelihood of the distribution p(y|x) = N(μ, W diag(σ²) W^T) that the paper claims to model. The learned σ² and W are therefore not maximum-likelihood estimates of the proposed probabilistic model, and the uncertainty reported at inference time is not the quantity optimized during training. This internal inconsistency undermines the principled basis of the method.","section":"§3.2, Eq. (5) vs Eq. (2)"},{"comment":"The evidence that W captures 'intrinsic hand joint correlations' is self-referential. The only support is the improvement in AUSC/AUSE/Pearson on the same benchmarks. No analysis of the learned W is presented—for example, no visualization of the implied correlation matrix W W^T, no comparison to empirical residual covariances, and no comparison to anatomical priors. The ablation 'Ours w/o linear layer' does not control for W initialization or training dynamics. Given the gradient analysis above, the reported improvements could stem from the stochastic noise injection acting as a regularizer on the variance head, or from a near-identity W that is slowly shrinking, rather than from learned correlations. A control with W fixed to identity (or to a random constant) under the same training pipeline is needed to attribute the improvements to correlation learning.","section":"§4.4, Table 4; §4.2, Table 2"}],"minor_comments":[{"comment":"The equations are written as L_NLL = log p(...), but the text says these are negative log-likelihood losses. Without a minus sign, minimizing L_NLL would maximize the likelihood, which is the opposite of the stated intent. Please add the minus sign or a note that the constant is absorbed.","section":"Eq. (2) and Eq. (4)"},{"comment":"References [26] and [27] cite the same work (Kendall and Gal, 2017) in two different formats. These duplicates should be merged.","section":"References [26] and [27]"},{"comment":"The expectation in Eq. (6) is written without a subscript. It should be E_z or E_{z~N(0,diag(σ²))} to make clear that the expectation is over the sampled noise.","section":"Eq. (6)"}],"recommendation":"reject","confidential_remarks":"The central theoretical flaw is decisive: the MSE loss in Eq. (6) provably drives W to zero, and no term in the loss couples W to the residual covariance. This is not a minor presentation issue but a fundamental mismatch between the claimed mechanism and the training objective. The reported empirical improvements over the diagonal baseline are, absent further analysis, unexplained and may be attributed to experimental factors rather than the proposed correlation model. A revision would require a fundamentally different training loss (e.g., the full covariance NLL in Eq. 4) or a rigorous demonstration that the existing objective somehow fits W despite the gradient analysis. As the paper stands, the core contribution is unsupported, and I cannot recommend acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Chae-Yeon et al. propose a nice add-on head for 3D hand pose aleatoric uncertainty: a linear layer W applied to per-joint Gaussian noise to give a covariance W diag(sigma^2) W^T. That is a sensible mid-representation between diagonal and full covariance, and the paper is honest that it is an incremental method step. The experiments on FreiHAND and HO3D are extensive, and the reported AUSC/AUSE/Pearson's rho numbers consistently beat the diagonal and full baselines. I also give credit for the ablation showing that adding the linear layer helps.\n\nThe problem is the training objective. The model is trained with L = L_DET + λ_NLL * L_NLL_diag + λ_MSE * E_z ||y - (mu + W z)||^2. For a given input, the gradient of that MSE term with respect to W is 2 W diag(sigma^2) because z has zero mean and is independent of the residual. That pushes W toward zero. The diagonal NLL does not involve W at all. So nothing in the loss fits W to the actual residual covariance. The paper's claim that W learns joint correlations is not supported by the equations. With learning rate 1e-6, W may stay close to its initialization, so the observed gains over the diagonal baseline could come from a near-identity scaling or from the sampling as an implicit regularizer, not from learned correlations. The authors do not provide a gradient analysis or a control experiment testing this.\n\nThe other issues are secondary: no error bars, no code, and the AMVUR comparison is not matched to the same training setup. The AUSC metric is fine, but it is an indirect measure of correlation quality.\n\nThis is a load-bearing flaw. The method as described cannot do what it claims. But it is also a fixable flaw: replace the diagonal NLL with the full NLL for the structured covariance, which would couple W to the residual, or add an explicit covariance-matching loss. I would like to see a revision with a correct training objective and a controlled test. As is, I would not recommend acceptance but I think the paper deserves refereeing rather than instant desk rejection.","headline":"The correlation layer's training objective drives it to zero, so the paper's central claim is unsupported despite clean experiments.","tokens_in":13756,"tokens_out":4984,"would_cite":false,"duration_ms":55739,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes an uncertainty head for 3D hand pose estimation whose covariance is W diag(σ²)W^T, claiming it captures inter-joint correlations better than diagonal or full-covariance baselines while keeping pose accuracy.","keywords":["3D hand pose estimation","aleatoric uncertainty","heteroscedastic uncertainty","joint correlation","structured covariance","negative log-likelihood","sparsification curves","transformer head"],"falsifier":"Train the proposed head on one of the benchmarks, then for each test image compare the average outer product of residuals y − µ3D with the average predicted covariance W diag(σ²3D)W^T. If the predicted covariance does not approximate the empirical residual covariance—for example, if W collapses toward zero while pose accuracy stays fixed—the claim that the linear layer learns joint correlations is not supported. A second check: fix the mean µ3D and train only the uncertainty terms; if the uncertainty ranking gains vanish, the gains come from improved pose estimates rather than from correlation","tokens_in":12888,"feed_emoji":"🖐","tokens_out":8726,"duration_ms":92048,"temperature":0.7,"pith_summary":"The paper tackles a blind spot in 3D hand pose estimation: models say where joints are but not how sure they are, and standard uncertainty heads treat joints independently even though fingers move together. It introduces aleatoric (data-driven) uncertainty modeling into an existing hand pose transformer and proposes a middle-ground covariance parametrization: a diagonal per-joint variance matrix sandwiched by a shared linear layer, giving W diag(σ²)W^T. This captures joint correlations with far fewer parameters than a full covariance matrix and more expressiveness than a diagonal one. Experiments on two standard benchmarks show the resulting uncertainty scores track actual per-joint errors better than diagonal or full covariance baselines, while 3D pose accuracy is maintained. The practical payoff is an uncertainty signal a downstream system can trust when occlusion, blur, or self-similarity make a hand pose genuinely ambiguous.","feed_headline":"One linear layer captures correlated hand-joint uncertainty","feed_subtitle":"A covariance head that links joint uncertainties ranks errors better on FreiHAND and HO3Dv2.","key_machinery":"The central object is the low-rank structured covariance W diag(σ²3D)W^T. Per-joint variances σ²3D are produced by an extra transformer head trained with a diagonal Gaussian NLL loss, which defines a zero-mean probabilistic output space p(z|x) = N(0, diag(σ²3D)). Samples z are fed through a single linear layer W; because linear maps preserve Gaussianity, the resulting covariance is exactly W diag(σ²3D)W^T, and the mean µ3D is added afterward. W is shared across inputs and is the mechanism that injects inter-joint correlation into the uncertainty estimate, giving an analytically tractable covariance without sampling-based covariance estimation. This single matrix is what the paper calls the m","core_discovery":"The paper proposes a new parametrization for aleatoric uncertainty in 3D hand pose estimation. Instead of predicting only per-joint variances or a full covariance matrix, the network regresses per-joint variances under an independence assumption, then passes samples from that zero-mean Gaussian through one learned linear layer W. Because linear maps preserve Gaussianity, the predictive distribution becomes N(µ3D, W diag(σ²3D)W^T). The shared W is the only extra correlation-carrying component, turning the cheap diagonal model into a correlated model with d_o² additional parameters. On FreiHAND and HO3Dv2, the paper reports that this head beats diagonal and full covariance baselines on sparsif","pith_inferences":["Inference beyond the paper: if W is trained across many datasets, its off-diagonal entries may encode anatomical hand synergies (e.g., adjacent fingers moving together); this could be tested by clustering the learned W or comparing it to known joint-coupling matrices.","Inference beyond the paper: the same W diag(σ²)W^T device should transfer to any structured output with correlated components—body pose, hand-object interaction, facial landmarks—and the paper's efficiency argument would be strongest there.","Inference beyond the paper: the reported gains could be stratified by occlusion and blur; if the uncertainty advantage concentrates on occluded or blurred joints, the method's practical case for in-the-wild deployment becomes concrete.","Inference beyond the paper: a direct test of whether W diag(σ²)W^T matches the empirical residual covariance (rather than acting mainly as a learned rescaling of per-joint variance) would separate correlation learning from pose-accuracy improvements."],"forward_implications":["The proposed head mounts on top of an existing pretrained hand pose model, so uncertainty estimation can be added without redesigning the feature extractor or regressor.","Correlation-aware uncertainty improves uncertainty quality metrics (AUSC, AUSE, Pearson's ρ) over both diagonal and full covariance heads on FreiHAND and HO3Dv2.","Adding the linear layer improves pose accuracy slightly compared to the same head without it, so correlation modeling helps rather than hurts regression.","The parametrization requires far fewer parameters than a full covariance matrix, making it practical for the 63-dimensional 3D joint output space.","In the 25-sample ablation, more samples consistently improve uncertainty estimates, so performance can be traded against sampling cost."],"supporting_citations":[{"why":"Supplies the pretrained HaMeR feature extractor and regressor onto which the uncertainty transformer head is added.","marker":"[42]"},{"why":"Provides the diagonal-covariance Gaussian NLL formulation used to train the per-joint variance head.","marker":"[27]"},{"why":"FreiHAND is one of the two standard 3D hand pose benchmarks used to evaluate uncertainty and pose accuracy.","marker":"[59]"},{"why":"HO3Dv2/HOnnotate is the second benchmark used for quantitative comparison of uncertainty and 3D pose estimates.","marker":"[18]"},{"why":"AMVUR is the open-sourced probabilistic 3D reconstruction method compared directly on uncertainty metrics.","marker":"[24]"},{"why":"MANO supplies the parametric hand model through which mean 3D joint positions are regressed.","marker":"[45]"},{"why":"Defines the sparsification curve and AUSE metric used to evaluate how well uncertainty ranks joint errors.","marker":"[22]"},{"why":"Supplies the AUSC metric used for quantitative uncertainty evaluation.","marker":"[21]"}],"fun_headline_variants":["Single layer turns hand-pose uncertainty into correlated model","Cheap uncertainty head learns hand-joint correlations","Correlation-aware aleatoric uncertainty for hand pose","Hand pose uncertainty model links joints with one weight matrix","Better hand pose uncertainty without full covariance cost"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes that the training loss teaches the linear transformation to encode real correlations between joint errors, rather than simply shrinking toward zero, because nothing in the loss forces the learned covariance to match the actual pattern of prediction errors.","fun_headline_variants_meta":{"raw":{"variants":["Single layer turns hand-pose uncertainty into correlated model","Cheap uncertainty head learns hand-joint correlations","Correlation-aware aleatoric uncertainty for hand pose","Hand pose uncertainty model links joints with one weight matrix","Better hand pose uncertainty without full covariance cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000542,"raw_usage":{"total_tokens":2447,"prompt_tokens":772,"completion_tokens":1675,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":1612}},"tokens_in":516,"tokens_out":1675,"duration_ms":14918,"temperature":1.0,"reasoning_tokens":1612,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:44:38.813375+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the proposed head on one of the benchmarks, then for each test image compare the average outer product of residuals y − µ3D with the average predicted covariance W diag(σ²3D)W^T. If the predicted covariance does not approximate the empirical residual covariance—for example, if W collapses toward zero while pose accuracy stays fixed—the claim that the linear layer learns joint correlations is not supported. A second check: fix the mean µ3D and train only the uncertainty terms; if the uncertainty ranking gains vanish, the gains come from improved pose estimates rather than from correlation","supporting_citations":[{"cited_title":"Reconstructing hands in 3d with transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained HaMeR feature extractor and regressor onto which the uncertainty transformer head is added."},{"cited_title":"What uncertainties do we need in bayesian deep learning for computer vision? In NIPS, 2017","cited_arxiv_id":null,"evidence_quote":"Provides the diagonal-covariance Gaussian NLL formulation used to train the per-joint variance head."},{"cited_title":"Freihand: A dataset for markerless capture of hand pose and shape from single rgb images","cited_arxiv_id":null,"evidence_quote":"FreiHAND is one of the two standard 3D hand pose benchmarks used to evaluate uncertainty and pose accuracy."},{"cited_title":"Honnotate: A method for 3d annotation of hand and object poses","cited_arxiv_id":null,"evidence_quote":"HO3Dv2/HOnnotate is the second benchmark used for quantitative comparison of uncertainty and 3D pose estimates."},{"cited_title":"Uncertainty estimates and multi-hypotheses networks for optical flow","cited_arxiv_id":null,"evidence_quote":"Defines the sparsification curve and AUSE metric used to evaluate how well uncertainty ranks joint errors."},{"cited_title":"A quantitative evaluation of confidence measures for stereo vision","cited_arxiv_id":null,"evidence_quote":"Supplies the AUSC metric used for quantitative uncertainty evaluation."}],"review_version":1}