Pith. sign in

REVIEW 5 major objections 5 minor 12 references

CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that learning a deep uncertainty score end-to-end with a transformer-based human pose-shape estimator, then using that score as a weighted-conformal conformity score, yields state-of-the-art accuracy on 3DPW…

desk verdict Solid empirical pose-shape results, but the new coverage bound (Theorem 3) is numerically false, so the theoretical novelty needs a major fix or removal. read the letter →

arxiv 2412.10431 v1 pith:F3X3VFYP submitted 2024-12-11 cs.CV cs.RO

Harry Zhang , Luca Carlone This is my paper · ORCID
classification cs.CVcs.RO
keywords 3Dhumanposeestimationshapeconformalpredictionuncertaintyquantificationnon-exchangeabledatadeepfunctionvideo-basedmeshrecoverySMPL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CUPS is a method for recovering 3D human shape and pose from RGB video that pairs an end-to-end learned uncertainty score with conformal prediction under non-exchangeable data. The paper claims that the same uncertainty function that ranks multiple training-time hypotheses also serves as a conformity score at test time, producing deep uncertainty conformal sets whose empirical coverage sits near the target 1-α (about 86-89% at α=0.1 on three benchmarks) while improving reconstruction accuracy, e.g., PA-MPJPE 48.7 mm on 3DPW versus 50.6 mm for the GLoT baseline. If correct, this gives safety-critical users such as robotics and AR/VR systems a principled way to know when to trust a mesh estimate from video. The theoretical backing consists of two upper bounds on the miscoverage gap for non-exchangeable conformal prediction, one adapted from prior work and one derived assuming Beta-distributed conformity scores.

What carries the argument

The load-bearing object is the Deep Uncertainty Function S_θ(X,Y)=σ(MLP(φ_gl(X),θ,β))∈[0,1], trained with an adversarial discrimination loss plus training-time ensemble augmentation, where each input video is randomly masked H times to generate multiple SMPL hypotheses. At calibration, this function supplies the conformity scores, and the feature-distance weight w_i=exp(-||φ_pred_i-φ_GT_i||^2/T) is used in the weighted quantile to handle non-exchangeability. The conformal set is the DUCS defined above, and the paper proves two practical upper bounds on the miscoverage gap: Theorem 2 bounds it by ρ^k under periodic change, and Theorem 3 bounds it by $\sqrt$(2-2(1-2k/(n+k))^(k/2)) assuming the scores and their swapped counterparts follow Beta distributions with parameters differing by at most k.

What would settle it

Run CUPS's calibration on training-distribution videos but test on a deliberately shifted distribution, such as a held-out subject filmed with a different camera and background, then compare empirical coverage with 1-α minus the promised bound; if coverage falls below it, either the periodic-change assumption or the Beta-distribution assumption fails. Alternatively, apply a goodness-of-fit test such as Kolmogorov-Smirnov to the calibration scores S_θ(Z) and S_θ(Z^i) from Appendix C.2: rejecting the Beta family would falsify Theorem 3's premise.

Watch

Extended reading notes

Core claim

The central claim is that a deep uncertainty function—an MLP that scores SMPL pose-shape hypotheses from the global-local transformer embedding—can be learned jointly with the human reconstructor, and can then be reused as the conformity score in weighted conformal prediction without sacrificing reconstruction quality. On the paper's own terms, CUPS outperforms existing baselines across PA-MPJPE, MPJPE, and MPVPE on 3DPW, MPI-INF-3DHP, and Human3.6M, with 3DPW PA-MPJPE dropping to 48.7 mm from GLoT's 50.6 mm. The calibrated prediction set is the Deep Uncertainty Conformal Set C_θ(X)={Y : S_θ(X,Y)≤τ*}, where τ* is a weighted quantile of calibration scores, and the empirical coverage experiments report weighted CP coverage of 86.2±2.1%, 87.3±2.2%, and 89.0±1.5% at α=0.1, all above the unweighted baseline.

Load-bearing premise

The central guarantee rests on the calibration videos being close enough to the test videos—despite both being non-exchangeable—that the weighting correction closes the coverage gap; in particular, Theorem 3 assumes the uncertainty scores of the original and swapped calibration sets follow the same family of bell-shaped distributions with parameters that differ by at most k.

Editorial extensions

If this is right

  • Training-time ensemble augmentation is not just for calibration: generating more hypotheses per input during training reduces test error, with gains saturating around H=30.
  • The learned deep uncertainty function itself improves reconstruction accuracy, not only uncertainty quality, as shown by CUPS outperforming GLoT and other baselines on nearly every reported metric.
  • Weighted conformal calibration with the deep uncertainty score achieves empirical coverage near the nominal level on three standard benchmarks, and higher coverage than unweighted regular conformal prediction.
  • Monte Carlo Dropout at test time makes the conformal set explicit: the method can emit multiple plausible SMPL meshes and check each one's membership in C_θ(X).
  • When their assumptions hold, the two bounds on the miscoverage gap convert the conformal guarantee into a computable lower bound on coverage, which is the missing safety certificate for video-based human reconstruction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension, not stated by the authors, is that the same recipe—learned uncertainty score plus weighted conformal calibration—could transfer to single-image pose estimation or other sequence regression tasks where exchangeability fails, since neither the architecture nor the weighting scheme depends on the 16-frame video input.
  • The feature-distance weight design suggests a testable improvement: explicitly calibrating with weights derived from the learned embedding distance rather than by score ranking, and measuring whether empirical coverage improves at small calibration-set sizes.
  • Because the paper reports that increasing the number of proposals from 10 to 20 raises GPU memory by about 30%, a practical follow-up would distill the ensemble augmentation into a single forward pass, decoupling the accuracy gain from the compute cost.
  • A deployment check suggested by Theorem 3: measure k empirically on the calibration set as the average number of subject or activity changes after swapping a data point, verify the Beta-distribution assumption on the scores, and only then use the strengthened lower bound as a safety certificate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes CUPS, a video-based 3D human pose and shape estimator built on GLoT, augmented with a learned deep uncertainty function trained end-to-end via hypothesis ensembling and an adversarial-style loss. After training, this score is used as a conformity score for conformal prediction, with weights derived from feature distances to handle non-exchangeable video data. The paper claims state-of-the-art accuracy on 3DPW, MPI-INF-3DHP, and Human3.6M, and reports empirical coverage around 86--89% with two theoretical bounds on the miscoverage gap, one adapted from Barber et al. (2023) and one new Beta-distribution-based bound (Theorem 3).

Significance. If the empirical accuracy holds and the calibration procedure is properly connected to the stated guarantee, the integration of a learned uncertainty score with weighted conformal prediction for video-based human mesh recovery would be a useful contribution. The reported reductions over GLoT (e.g., PA-MPJPE 48.7 mm vs. 50.6 mm on 3DPW) are practically meaningful, and the paper includes a detailed appendix, comparisons across three datasets, and ablations of ensemble size, score function, and loss weight. However, the novel theoretical bound (Theorem 3) is not established, and there are serious gaps in connecting the implemented calibration procedure to the stated conformal guarantee. These issues must be fixed before the paper's uncertainty-quantification claims can be accepted.

major comments (5)
  1. [Section 4.3 and Appendix C.2, Theorem 3 and Eq. (27)] The proof of Theorem 3 is invalid. For n=100, k=2, a1=10, a2=12, a=11, the left-hand side of Eq. (27) evaluates to sqrt(10!^2 88!^2 / (9! 11! 89! 87!)) ≈ 0.948, while the claimed right-hand side ((n−k)/(n+k))^{k/2} equals 0.9608, so the inequality is numerically false. The derivation also moves from a bound of order ((n−k)/(n+k))^k to the stronger exponent k/2 without justification for a base less than 1. In addition, Assumption 2 states that the n-vector Sθ(Z) 'follows a Beta distribution,' which is not well-defined for a vector, and no empirical or architectural evidence is provided for the Beta form or for |a1−a2|≤k. Consequently, the 'Theorem 3 Bound' row in Table 3 is unsupported.
  2. [Section 4.3, Theorem 1 and Appendix B] The theorem statement and the proof are inconsistent. Theorem 1 defines Sθ(Z) = [Sθ(Z_i)]_{i=1}^n with n calibration points, whereas the proof in Appendix B constructs Z = (Z_1,...,Z_{n+1}) with n+1 points and sums over n+1 terms. The proof line following Eq. (15) is also incomplete, ending with '⇔ Q_{1−α}(∑_{i=1}^n ... + \tilde w_{n+1} δ_{+∞})' and no right-hand side. As written, the theorem does not follow from the proof, and the coverage guarantee for the method actually implemented is not established.
  3. [Section 4.3, Definition 3 and Eq. (5)] The feature-distance weight w_i = exp(−||ϕ_pred_i − ϕ_GT_i||^2/T) requires the ground-truth SMPL embedding ϕ_GT_i, which is not available for a new test point. The paper never specifies the test-point weight \tilde w_{n+1} that the Appendix B proof needs for the weighted quantile, nor does it explain how the threshold-only prediction set in Definition 2 corresponds to the weighted conformal set used in Theorem 1. This is a load-bearing gap: the implemented calibration procedure may not be the procedure for which the stated coverage guarantee holds.
  4. [Appendix D and Section 5.1] CUPS is trained on approximately 1500 fewer datapoints than the baselines because of the calibration holdout, about 2.5% of the training data, yet Tables 1 and 2 compare directly against baseline numbers trained on the full dataset. To support the state-of-the-art claim, the authors should either retrain the baselines on the reduced training set or provide evidence that a 2.5% reduction cannot account for the reported gains. Without this, the headline comparisons are confounded.
  5. [Section 5.1 and Table 1] The main accuracy results are reported as single numbers without error bars or significance tests. Since the central empirical claim is a consistent improvement over GLoT (e.g., 1.9 mm PA-MPJPE on 3DPW), the authors should report variance across multiple random seeds or provide a field-standard justification for omitting it. The same issue applies to the coverage numbers in Table 3, where the ± values do not indicate the number of runs or the source of variation.
minor comments (5)
  1. [Section 4.3, Eq. (7)] The displayed definition of Z^i is malformed; it should be the sequence Z with the i-th and n-th entries swapped, not '(Z^1, ..., Z^{i−1}, Z^n, ..., Z^{n−1}, Z^i)'.
  2. [Section 4.3, Theorem 2 and Appendix C.1] The theorem statement uses DTV(Z_i ∥ Z_n), while the assumption and proof use DTV(Sθ(Z_{n+1}) ∥ Sθ(Z_i)); the notation should be aligned across the statement and the proof.
  3. [Section 3 and Definition 1] The symbol θ overloads the SMPL pose parameters and the network parameters of fθ and Sθ; distinct symbols would clarify the exposition.
  4. [Appendix B, Lemma 4] The domain '[0, ∞]' should be '[0, ∞)', and the permutation argument in the proof would be easier to follow if written out explicitly.
  5. [Appendix D] The dataset description consists largely of chains of citations and does not directly state the sizes and sources of each dataset; please provide a clear description of the data composition.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the conformal coverage guarantee is imported from Barber et al. (2023), and the learned-score design does not make the coverage claim reduce to its training objective.

full rationale

The central coverage guarantee (Theorem 1, Eq. 9) is explicitly Theorem 2 of Barber et al. (2023), an external result; the weighted-quantile construction holds for arbitrary weights, so using the learned deep uncertainty score Sθ as the conformity score (Definition 1, Eq. 2) does not make the coverage statement depend on the training objective by construction. The feature-distance weights in Definition 3 are a design choice within Barber's framework, not a fitted parameter renamed as a prediction. Table 3 reports empirical coverage, which is an evaluation rather than a prediction forced by the fit. Theorem 2 is explicitly adapted from Barber et al. (2023, Sec. 4.4). Theorem 3 is stated conditionally on Assumption 2 (Beta-distributed scores with parameter gap at most k); although that assumption is not empirically justified and the Appendix C.2 derivation contains a numerically false inequality (Eq. 27), this is a correctness or missing-proof defect, not a self-referential reduction, because the bound is not defined in terms of the quantity it purports to predict. The self-citations to Zhang and Carlone (2024) motivate learned conformity scores and an adversarial-loss detail, but they are not load-bearing: the coverage theorem and the periodic-change bound come from Barber et al. (2023), and the accuracy gains are benchmarked against external baselines (Tables 1 and 2). Appendix F's limitations (GPU memory, lack of joint-level information) are acknowledged limitations rather than circularity. No enumerated circularity pattern can be exhibited with the required quote-and-reduction evidence.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The main method relies on the Barber et al. (2023) weighted conformal prediction theorem and the GLoT architecture. The so-called new bound adds an unverified Beta distribution assumption and contains a proof error, so it contributes no valid free-standing support.

free parameters (5)
  • lambda (uncertainty loss weight) = 0.6
    Chosen by ablation in Section 5.2; balances score loss against SMPL loss.
  • Htrain (training-time ensemble size) = 20
    Number of random maskings per video during training; ablation Figure 3 suggests saturation after 30, but the implementation uses 20.
  • T (temperature in feature distance weight, Eq. 8) = not reported
    Controls the scale of weights in weighted conformal calibration; not disclosed.
  • rho (decay hyperparameter, Eq. 10) = not reported
    Controls the decay of weights in the periodic-change bound; no value given, so Theorem 2 bound in Table 3 cannot be reproduced.
  • MC dropout rate at test time = not reported
    Used to construct the deep uncertainty conformal set; rate not specified.
assumptions (4)
  • standard math Lemma 4 (Harrison 2012) bounds the weighted sum of indicators used in the conformal coverage proof.
    Used in Appendix B to bound the probability of non-coverage.
  • standard math Barber et al. (2023) Theorem 2 provides the nonexchangeable conformal coverage guarantee.
    Theorem 1 of CUPS is stated as a direct consequence.
  • ad hoc to paper Assumption 1: the video dataset undergoes a periodic distribution shift with a changepoint k steps ago.
    Used for Theorem 2; not verified on the datasets, and k is measured post hoc.
  • ad hoc to paper Assumption 2: conformity scores Stheta(Z) and Stheta(Zi) follow Beta distributions with parameters (a1, n-a1) and (a2, n-a2).
    The new bound rests on this unverified distributional assumption; no goodness-of-fit evidence is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty." pith.science (2026). https://pith.science/paper/F3X3VFYP

@misc{pith2026241210431,
  author       = {Pith},
  title        = {Pith review of: CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F3X3VFYP}},
  note         = {Machine review of arXiv:2412.10431}
}
read the original abstract

We introduce CUPS, a novel method for learning sequence-to-sequence 3D human shapes and poses from RGB videos with uncertainty quantification. To improve on top of prior work, we develop a method to generate and score multiple hypotheses during training, effectively integrating uncertainty quantification into the learning process. This process results in a deep uncertainty function that is trained end-to-end with the 3D pose estimator. Post-training, the learned deep uncertainty model is used as the conformity score, which can be used to calibrate a conformal predictor in order to assess the quality of the output prediction. Since the data in human pose-shape learning is not fully exchangeable, we also present two practical bounds for the coverage gap in conformal prediction, developing theoretical backing for the uncertainty bound of our model. Our results indicate that by taking advantage of deep uncertainty with conformal prediction, our method achieves state-of-the-art performance across various metrics and datasets while inheriting the probabilistic guarantees of conformal prediction.

Figures

Figures reproduced from arXiv: 2412.10431 by the authors.

Figure 1
Figure 1. CUPS sample results obtained on in-the-wild videos collected from TikTok. Given a sequence of 2D RGB frames, CUPS reconstructs a sequence of 3D human meshes, and then a conformal predictor calibrated using a deep uncertainty func￾tion —trained end-to-end with the human pose-shape estimator— quantifies the uncertainty of the output SMPL parameters. way by directly regressing SMPL parameters from video inputs. At the … view at source ↗
Figure 2
Figure 2. CUPS Overview. CUPS takes as input a sequence of input RGB video frames. The RGB video frames get encoded and fed into a global-local transformer human reconstruction model to produce SMPL parameters representing the human pose and shape in 3D as well as a decoupled global-local embedding. The output of the human reconstructor is supervised via SMPL loss. While training, we also learn a deep uncertainty function tha… view at source ↗
Figure 5
Figure 5. Comparison of strength of un￾certainty loss in the total training loss. Input Single Prediction MC Prediction Set [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figures from the paper (1 more)
Figure 6
Figure 6. Figure 6: In the wild video SMPL predictions with both single hypothesis and multiple hypotheses using MC Dropout. are collected from TikTok. For 3D visualization, please refer to this anonymized website to interact with CUPS predictions in 3D. 5.5. Empirical Coverage Here we te…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 8 canonical work pages

  1. [6]

    and V ovk, V

    3 Shafer, G. and V ovk, V . A tutorial on conformal prediction. J. of Machine Learning Research, 9(3), 2008. 2, 3, 4 Shan, W., Liu, Z., Zhang, X., Wang, Z., Han, K., Wang, S., Ma, S., and Gao, W. Diffusion-based 3d human pose estimation with multi-hypothesis aggregation. In IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), pp. 14761–14771, 2...

    work page 2008

  2. [7]

    C., Beaufays, F., Benard, A., Guliani, D., Kabel, A., Khare, N., Lucassen, T., Zadrazil, P., Zhang, H., Johnson, L., et al

    3, 4, 6, 7, 8, 17 Sim, K. C., Beaufays, F., Benard, A., Guliani, D., Kabel, A., Khare, N., Lucassen, T., Zadrazil, P., Zhang, H., Johnson, L., et al. Personalization of end-to-end speech recognition on mobile devices for named entities. In 2019 IEEE Automatic Speech Recognition and Under- standing Workshop (ASRU), pp. 23–30. IEEE, 2019. 17 Stutz, D., Cemg...

    work page 2019

  3. [8]

    Learning Correspondence for Deformable Objects

    3 Sun, X., Xiao, B., Wei, F., Liang, S., and Wei, Y . Integral human pose regression. In European Conf. on Computer Vision (ECCV), pp. 529–545, 2018. 2 Sundaresan, P., Ganapathi, A., Zhang, H., and Devgon, S. Learning correspondence for deformable objects. arXiv preprint arXiv:2405.08996, 2024. 17 V on Marcard, T., Henschel, R., Black, M. J., Rosenhahn, B...

    work page Pith review arXiv 2018

  4. [12]

    Note that our training dataset is about 2.5% smaller than previous works because we hold out a small portion (∼ 1500 datapoints) for calibration

    where the preprocessed data is provided by (Shen et al., 2023; Devgon et al., 2020; Lim et al., 2021; 2022), (Choi et al., 2021; Wang et al., 2024; Sundaresan et al., 2024), and (Kocabas et al., 2020), and evaluated on 3DPW, Human3.6M, MPII-3DHP. Note that our training dataset is about 2.5% smaller than previous works because we hold out a small portion (...

    work page 2023

  5. [1792]

    17 Pavlakos, G., Zhou, X., Derpanis, K

    PMLR, 2023. 17 Pavlakos, G., Zhou, X., Derpanis, K. G., and Daniilidis, K. Coarse-to-fine volumetric prediction for single-image 3d human pose. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 7025–7034, 2017. 2 Rempe, D., Birdal, T., Hertzmann, A., Yang, J., Sridhar, S., and Guibas, L. J. Humor: 3d human motion model for robust pose e...

    work page 2023

  6. [2013]

    J., Jacobs, D

    6, 7, 17 Kanazawa, A., Black, M. J., Jacobs, D. W., and Malik, J. End-to-end recovery of human shape and pose. In IEEE Conf. on Computer Vision and Pattern Recogni- tion (CVPR), pp. 7122–7131, 2018. 1, 2, 3, 4, 6, 7 9 CUPS: Improving Human Pose-Shape Estimators with Conformalized Deep Uncertainty Kanazawa, A., Zhang, J. Y ., Felsen, P., and Malik, J. Lear...

    arXiv 2018

  7. [2016]

    CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimators

    17 Zhang, H. and Carlone, L. CHAMP: Conformalized 3D human multi-hypothesis pose estimators.arXiv preprint: 2407.06141, 2024. 2, 4, 5 Zhang, H., Ichnowski, J., Avigal, Y ., Gonzales, J., Stoica, I., and Goldberg, K. Dex-net ar: Distributed deep grasp planning using a commodity cellphone and augmented reality app. In 2020 IEEE International Conference on R...

    work page Pith review arXiv 2024

  8. [2020]

    Robots of the lost arc: Self- supervised learning to dynamically manipulate fixed- endpoint cables

    17 Zhang, H., Ichnowski, J., Seita, D., Wang, J., Huang, H., and Goldberg, K. Robots of the lost arc: Self- supervised learning to dynamically manipulate fixed- endpoint cables. In 2021 IEEE International Conference on Robotics and Automation (ICRA) , pp. 4560–4567. IEEE, 2021. 17 Zhang, H., Eisner, B., and Held, D. Flowbot++: Learn- ing generalized artic...

    arXiv 2021

Show all 12 references
  1. [2021]

    Optimizing net- work structure for 3d human pose estimation

    2, 6, 7, 8, 17 Ci, H., Wang, C., Ma, X., and Wang, Y . Optimizing net- work structure for 3d human pose estimation. In Intl. Conf. on Computer Vision (ICCV), pp. 2262–2271, 2019. 2 Devgon, S., Ichnowski, J., Balakrishna, A., Zhang, H., and Goldberg, K. Orienting novel 3d objec...

    2019 arXiv

  2. [2022]

    L., Xu, Y ., Winkler, A., Kadle- cek, P., Tang, S., and Bogo, F

    2 Zhang, S., Bhatnagar, B. L., Xu, Y ., Winkler, A., Kadle- cek, P., Tang, S., and Bogo, F. Rohm: Robust human motion reconstruction via diffusion. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , pp. 14606–14617, 2024. 3 Zhang, W., Zhu, M., and Derpanis, K. G...

    2023

  3. [2023]

    3 Ionescu, C., Papava, D., Olaru, V ., and Sminchisescu, C. Human3. 6m: Large scale datasets and predictive meth- ods for 3d human sensing in natural environments.IEEE Trans. Pattern Anal. Machine Intell., 36(7):1325–1339,

  4. [2024]

    3 Angelopoulos, A. N. and Bates, S. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511, 2021. 2, 3, 4 Angelopoulos, A. N., Kohli, A. P., Bates, S., Jordan, M., Malik, J., Alshaabi, T., Upadhyayula, S....

    2021 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.