Pith. sign in

REVIEW 3 major objections 6 minor 12 references

ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A learned quality scorer, not human annotators, supplies the preference signal that aligns diffusion-based human mesh recovery, letting ten predictions beat two hundred.

desk verdict Solid DPO-for-HMR paper with a real novelty (learned scorer instead of human preferences), but the scorer is never validated on the real base-model outputs it ranks—an addressable gap that should not block acceptance. read the letter →

arxiv 2505.10250 v2 pith:HWFXR2VQ submitted 2025-05-15 cs.CV

Wenhao Shen , Wanqi Yin , Xiaofeng Yang , Cheng Chen , Chaoyue Song , Zhongang Cai , Lei Yang , Hao Wang
show 1 more author
Guosheng Lin
This is my paper · ORCID
classification cs.CV
keywords diffusionmodelhumanmeshrecoverydirectpreferenceoptimizationrewardprobabilisticposeestimationdatacleaning3Dandshapescoreprediction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a probabilistic human mesh recovery model, which generates many plausible 3D body estimates from a single image, can be improved by finetuning it with direct preference optimization on winner–loser prediction pairs ranked not by humans but by an automatic quality scorer. The scorer, HMR-Scorer, is trained to approximate standard reconstruction metrics using only image cues and projected keypoints, so it can rank predictions even for in-the-wild images that lack clean 3D annotations. If this holds, probabilistic HMR models no longer need to generate hundreds of candidates to be accurate, and the same scorer can filter poorly annotated training images to improve other state-of-the-art HMR models. The paper supports the claim with benchmark results showing that the aligned model's ten-prediction results beat the base model's two-hundred-prediction results in PVE and MPJPE on the 3DPW benchmark.

What carries the argument

The load-bearing object is HMR-Scorer, a transformer-encoder-based reward model that takes, for each candidate mesh, the UVD keypoint coordinates (3D joints projected into image space through the predicted camera) together with multi-scale image features: global ViT tokens plus pixel-aligned local features sampled at the reprojected joint positions. It outputs a scalar quality score and is trained with a RankNet-style pairwise probabilistic ranking loss whose quality labels are derived from four standard HMR metrics (PVE, MPJPE, PA-PVE, PA-MPJPE) on synthetic predictions formed by adding joint-wise Gaussian noise to ground-truth SMPL poses. The second load-bearing object is the Diffusion-DPO objective, which contrasts, at a random timestep along the diffusion trajectory, the denoising error on a winner sample against the error on a loser sample, weighted by the signal-to-noise ratio $\lambda_t$, while penalizing deviation from the frozen reference denoiser. Together these convert an unlabeled preference signal into gradient updates of the base denoiser.

What would settle it

Take the base model's actual multi-sample outputs on a benchmark with ground-truth meshes, such as the 3DPW test set, compute HMR-Scorer's scores and the true PVE and MPJPE errors for each prediction, and measure their rank correlation; if the correlation collapses toward zero on real outputs while it sits near 0.6 on the synthetic noise-perturbed test sets, the preference pairs that drive ADHMR do not track reconstruction quality. A complementary ablation finetunes the base model on preference pairs ranked by true metrics instead of by the scorer; identical gains would mean the scorer itself contributes nothing beyond approximating the metric.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the knowledge of a quality-assessment model can be distilled into a diffusion-based human mesh reconstruction model: the base denoiser HypoNet is finetuned with the Diffusion-DPO objective so that denoising trajectories toward scorer-ranked winner meshes are favored over loser trajectories, with a KL constraint keeping the aligned model close to the frozen base model. The resulting model, ADHMR, and its in-the-wild variant ADHMR (ITW), outperform prior probabilistic methods on 3DPW and Human3.6M across the reported metrics; at $M=10$ predictions, ADHMR (ITW) reaches MPJPE 61.3 mm and PVE 71.3 mm on 3DPW, surpassing the base model's $M=200$ result of 61.8 mm and 71.9 mm. The paper also shows that HMR-Scorer's predicted scores correlate with ground-truth reconstruction errors, with PLCC and SRCC near 0.6 on GTA-Human and DNA-Rendering synthetic noise-perturbed test sets, and that filtering training data by scorer score improves models such as SMPLer-X, OSX, and Hand4Whole even when fewer training samples remain.

Load-bearing premise

HMR-Scorer, the model that ranks every winner–loser pair in the preference dataset, is trained and validated only on synthetic predictions formed by adding Gaussian noise to ground-truth poses (Section 5.2), never on the actual outputs of the base diffusion model it is asked to rank in Sections 4.3 and 4.5; if its ranking of real outputs diverges from true reconstruction quality, the preference signal that drives ADHMR is misaligned.

Editorial extensions

If this is right

  • At $M=10$ predictions, ADHMR (ITW) already beats the base HypoNet at $M=200$ on 3DPW in both PVE and MPJPE, so aligned models can cut the number of candidate predictions by an order of magnitude.
  • Preference finetuning handles noisy pseudo-labels better than supervised finetuning: on the in-the-wild InstaVariety data, ADHMR improves 3DPW accuracy while supervised finetuning on the same pseudo-labels degrades it (Tables 3–4).
  • HMR-Scorer doubles as a data-cleaning tool: keeping only pseudo-labeled training samples scored above a threshold ($\tau=0.6$) improves Hand4Whole, OSX, and SMPLer-X-Base on 3DPW, Human3.6M, and EHF even though fewer training samples remain (Table 5).
  • The preference dataset requires no human annotations: HMR-Scorer ranks base-model predictions automatically, and for studio datasets with ground truth the ordering can be computed directly from reconstruction error (Section 4.3).

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If HMR-Scorer's ranking holds on real rather than only synthetic model outputs, the same score–rank–align recipe should transfer to other ill-posed inverse problems, such as hand or face mesh recovery, monocular depth estimation, or single-view reconstruction, wherever a metric can label synthetic perturbations.
  • The data-cleaning results raise the possibility that curation beats scale for in-the-wild HMR training; an adaptive per-dataset score threshold instead of the fixed $\tau=0.6$ would test how far this can be pushed.
  • The $M=10$ result implies that reporting min-error over hundreds of hypotheses may overstate what probabilistic HMR actually needs, and error-versus-$M$ curves of the kind this paper reports give a fairer basis for comparing probabilistic methods.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ADHMR, a pipeline that (i) trains a mesh-quality scorer, HMR-Scorer, on synthetic noisy SMPL predictions, (ii) uses that scorer to rank predictions of a diffusion-based HMR base model (HypoNet) and construct winner/loser preference pairs, (iii) fine-tunes the base model with Diffusion-DPO, and (iv) uses the same scorer to clean noisy pseudo-labeled training data. On 3DPW and Human3.6M the method reports consistent improvements over the base model and over prior probabilistic HMR methods; ablations indicate that preference fine-tuning outperforms supervised fine-tuning; and data cleaning improves several HMR models. The central claim is that a learned scorer can replace human preference labels in aligning diffusion-based HMR models.

Significance. If the results hold, the paper provides a practical way to apply preference optimization to human mesh recovery without manual preference annotation, and it demonstrates both a DPO-based alignment gain and a reusable data-cleaning tool. The study is well positioned in the literature, with clear baselines, ablations against supervised fine-tuning, and a code release. The main caveat is that the quality of the learned scorer, which is the linchpin of the pipeline, is only measured on the same synthetic noise-perturbation distribution used for training; its behavior on real base-model outputs is the critical unverified link.

major comments (3)
  1. [Sec. 4.2, 4.3, 5.2; Table 2] The HMR-Scorer is trained and evaluated exclusively on predictions formed by adding Gaussian noise to ground-truth SMPL poses, but in Sec. 4.3 it is used to rank actual HypoNet predictions while building the preference dataset. Real HypoNet errors are not additive Gaussian perturbations of the ground truth: they contain systematic pose bias, camera errors, and shape errors whose distribution differs from the scorer's training distribution. The paper reports no PLCC/SRCC, nor pairwise ranking accuracy, of the scorer on real base-model outputs, so the quality of the preference pairs used for ADHMR (ITW) in Table 2 is unverified. The authors should validate the scorer on real predictions of the base model (e.g., on a held-out portion of 3DPW or Human3.6M) by comparing scorer scores with true PVE/MPJPE, and should report the fraction of preference pairs where the scorer agrees with ground-truth error ordering. If the correlation is weak, the scorer should be retrained or augmented with real model outputs before the headline M=10 improvement can be attributed to the learned scorer.
  2. [Sec. 4.3, Table 2] The description of preference construction is ambiguous about which rows of Table 2 actually rely on HMR-Scorer. The text says that for 'studio-captured datasets with precise human mesh annotations' the ordering is computed directly from ground-truth reconstruction errors, but the ADHMR rows in Table 2 are labeled as fine-tuned on 'the target benchmark dataset,' which includes 3DPW (an in-the-wild dataset) as well as Human3.6M. The paper should specify, for each row, whether preferences come from HMR-Scorer or from ground-truth errors. This matters because the abstract and introduction claim that the method works 'without 3D annotations'; if the non-ITW rows use ground-truth errors for ranking, the only unlabeled-setting evidence is the ITW row, which in turn depends on the unvalidated scorer.
  3. [Sec. 4.5, Table 5] The data-cleaning contribution inherits the same validation gap. The scorer is used to filter pseudo-labeled samples from in-the-wild data, but there is no experiment showing that low scorer scores actually correspond to poor pseudo labels or to high 3D error. The improvements in Table 5 could in principle come from removing hard or atypical images rather than from removing mislabeled ones. The authors should report the correlation between scorer scores and pseudo-label errors on a held-out set, or compare against random filtering of the same number of samples, to establish that the cleaning signal is specifically annotation quality.
minor comments (6)
  1. [Sec. 4.4, Eq. (5)] In Eq. (5), the loser term is missing the norm notation on the epsilon terms; please fix the typesetting so that the two squared-norm differences are clearly displayed.
  2. [Sec. 4.2] The notation {s_m}_{i=0}^M uses i instead of m in the subscript, and F_g is reused for both the global image feature and the low-channel global feature; please rename one of these to avoid confusion.
  3. [Sec. 5.1, 5.2] The paper states that HMR-Scorer is trained on five datasets, including GTA-Human and DNA-Rendering, and then evaluates on the 'original test set' of those same datasets; please clarify the exact train/test split to rule out overlap.
  4. [Sec. 4.3, 4.5] The values of the free parameters K, beta, tau, and the noise magnitudes used for scorer training are not reported in the experiments; please list them and ideally ablate K, beta, and tau.
  5. [Table 2] The table header reports MPJPE and PVE, but the text in Sec. 5.3 states that the probabilistic methods report min-MPJPE and min-PVE over M samples; please make this explicit in the table caption and mark the deterministic methods accordingly.
  6. [Tables 2-5] No variance or multiple-seed results are reported; given that some of the headline margins are small (e.g., ADHMR (ITW) M=10 vs base M=200 on 3DPW MPJPE is 61.3 vs 61.8), the authors should provide at least a sensitivity check or error bars.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the final evaluation is ground-truth based, and the HMR-Scorer is an external proxy whose outputs are not used as the evaluation metric.

full rationale

The derivation chain is self-contained and anchored to external ground truth at every evaluation point. HMR-Scorer is trained on synthetic Gaussian-perturbed ground-truth SMPL poses with quality labels computed from standard HMR metrics (Sec. 4.2), but it is used only as a proxy to construct preference pairs; the headline results in Table 2 and the ablations in Tables 3-4 are evaluated with ground-truth PVE, MPJPE, PA-PVE, and PA-MPJPE on 3DPW and Human3.6M, not with scorer scores. DPO finetuning is adapted from external Diffusion-DPO (Wallace et al., 2024), and the base model HypoNet is external prior work (Xu et al., 2024b), so there is no load-bearing self-citation chain. The fact that HMR-Scorer is validated only on Gaussian-perturbed ground-truth poses rather than on actual base-model outputs is a robustness or generalization concern, not a circularity concern, because the scorer's ranking is not definitionally identical to the final error metrics. No step in the derivation reduces by construction to its own input, and the central claim of improved M=10 accuracy is measured against independent ground-truth annotations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four free parameters and four assumptions. The free parameters are all empirical choices not justified by sensitivity analysis. The most fragile assumption is the scorer's generalization to real model outputs, which is unvalidated. No new physical or conceptual entities are introduced.

free parameters (4)
  • DPO KL weight beta
    Controls the strength of KL regularization to the base model in the DPO objective (Eq. 4-5). Value not reported in the paper.
  • Number of preference pairs K per image
    Number of winner-loser pairs sampled per image in Section 4.3. Value not reported.
  • Data cleaning threshold tau = 0.6
    Samples with HMR-Scorer score below 0.6 are removed from training sets (Section 5.4). Chosen empirically.
  • Noise magnitudes for scorer training
    Magnitudes of joint-wise Gaussian noise added to ground truth SMPL poses to create prediction-quality training pairs (Section 4.2). Described as 'empirically determined', not specified.
assumptions (4)
  • domain assumption HMR-Scorer generalizes from synthetic noise-perturbed ground truths to actual base model outputs, including in-the-wild images.
    The preference dataset and data cleaning depend on scorer rankings of real predictions, but the scorer is only evaluated on synthetic perturbations (Section 5.2).
  • domain assumption The swing/twist pose decomposition and latent-space denoiser of HypoNet are compatible with the Diffusion-DPO objective.
    Eq. (5) operates on the denoiser latent space of the base model; the paper does not discuss any modifications needed for this representation.
  • standard math The Diffusion-DPO derivation of Wallace et al. (2024) applies unchanged to the HMR setting.
    The paper adopts Eq. (5) directly from prior work without re-derivation, assuming the assumptions of DPO for diffusion models hold.
  • domain assumption Ground-truth annotations in the five scorer training datasets are accurate enough to serve as quality anchors.
    The scorer's quality labels are computed against these ground truths (Section 5.1), so any label noise transfers to the scorer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization." pith.science (2026). https://pith.science/paper/HWFXR2VQ

@misc{pith2026250510250,
  author       = {Pith},
  title        = {Pith review of: ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HWFXR2VQ}},
  note         = {Machine review of arXiv:2505.10250}
}
read the original abstract

Human mesh recovery (HMR) from a single image is inherently ill-posed due to depth ambiguity and occlusions. Probabilistic methods have tried to solve this by generating numerous plausible 3D human mesh predictions, but they often exhibit misalignment with 2D image observations and weak robustness to in-the-wild images. To address these issues, we propose ADHMR, a framework that Aligns a Diffusion-based HMR model in a preference optimization manner. First, we train a human mesh prediction assessment model, HMR-Scorer, capable of evaluating predictions even for in-the-wild images without 3D annotations. We then use HMR-Scorer to create a preference dataset, where each input image has a pair of winner and loser mesh predictions. This dataset is used to finetune the base model using direct preference optimization. Moreover, HMR-Scorer also helps improve existing HMR models by data cleaning, even with fewer training samples. Extensive experiments show that ADHMR outperforms current state-of-the-art methods. Code is available at: https://github.com/shenwenhao01/ADHMR.

Figures

Figures reproduced from arXiv: 2505.10250 by the authors.

Figure 1
Figure 1. Overview of ADHMR. We aim to finetune a probabilistic HMR base model that generates multiple human mesh predictions conditioned on the input image. We first train the HMR-Scorer that assesses the reconstruction quality given an image and corresponding human mesh predictions. The reconstruction quality annotations Q ∗ are computed using standard HMR metrics, including PVE Q pve , MPJPE Q mpjpe, PA-MPJPE Q pajpe, and … view at source ↗
Figure 2
Figure 2. Qualitative comparison of the state-of-the-art probabilistic model ScoreHypo (Xu et al., 2024b) and our ADHMR. Our framework significantly improves image alignment and in-the-wild robustness. (a) ∼ (f) are from the 3DPW (Von Marcard et al., 2018) dataset, and (g) ∼ (h) are challenging in-the-wild images. Method 3DPW PVE ↓ MPJPE ↓ PA-MPJPE ↓ HypoNet (Base Model) 71.9 61.8 36.1 Supervised finetuning 69.9 59.7 35.2 ADH… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 4 canonical work pages

  1. [4]

    Lin, T.-Y ., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- manan, D., Doll´ar, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13 , pp. 740–

    2014

  2. [7]

    Novel view synthesis of human interactions from sparse multi-view videos

    Shuai, Q., Geng, C., Fang, Q., Peng, S., Shen, W., Zhou, X., and Bao, H. Novel view synthesis of human interactions from sparse multi-view videos. InACM SIGGRAPH 2022 Conference Proceedings, pp. 1–10,

    work page 2022

  3. [8]

    Imagereward: Learning and evaluating hu- man preferences for text-to-image generation

    Xu, J., Liu, X., Wu, Y ., Tong, Y ., Li, Q., Ding, M., Tang, J., and Dong, Y . Imagereward: Learning and evaluating hu- man preferences for text-to-image generation. Advances in Neural Information Processing Systems , 36, 2024a. Xu, Y ., Ma, X., Su, J., Zhu, W., Qiao, Y ., and Wang, Y . Scorehypo: Probabilistic human mesh estimation with hypothesis scorin...

    arXiv

  4. [9]

    Unify3D: An Augmented Holistic End-to-end Monocular 3D Human Reconstruction via Anatomy Shaping and Twins Negotiating

    Yao, N., Zhang, G., Shen, W., Shu, J., and Wang, H. Unify3d: An augmented holistic end-to-end monocular 3d human reconstruction via anatomy shaping and twins negotiating. arXiv preprint arXiv:2504.18215,

    work page Pith review arXiv

  5. [11]

    E., Zhang, M., et al

    11 ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization Yin, W., Cai, Z., Wang, R., Zeng, A., Wei, C., Sun, Q., Mei, H., Wang, Y ., Pang, H. E., Zhang, M., et al. Smplest-x: Ultimate scaling for expressive human pose and shape estimation. arXiv preprint arXiv:2501.09782,

    arXiv

  6. [12]

    M., Stiennon, N., Wu, J., Brown, T

    Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593,

    arXiv 1909

  7. [2015]

    Monocular 3d human pose estimation in the wild using improved cnn supervision

    Mehta, D., Rhodin, H., Casas, D., Fua, P., Sotnychenko, O., Xu, W., and Theobalt, C. Monocular 3d human pose estimation in the wild using improved cnn supervision. In 2017 international conference on 3D vision (3DV) , pp. 506–516. IEEE,

    work page 2017

  8. [2017]

    Lee, K., Liu, H., Ryu, M., Watkins, O., Du, Y ., Boutilier, C., Abbeel, P., Ghavamzadeh, M., and Gu, S. S. Aligning text- to-image models using human feedback. arXiv preprint arXiv:2302.12192,

    arXiv

Show all 12 references
  1. [2021]

    Hybrik-x: Hybrid analytical-neural inverse kine- matics for whole-body mesh recovery

    Li, J., Bian, S., Xu, C., Chen, Z., Yang, L., and Lu, C. Hybrik-x: Hybrid analytical-neural inverse kine- matics for whole-body mesh recovery. arXiv preprint arXiv:2304.05690,

    arXiv

  2. [2023]

    and Shen, W

    9 ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization Hong, Z. and Shen, W. Free-viewpoint video in the wild using a flying camera. In ECCV 2024 Workshop on Wild 3D: 3D Modeling, Reconstruction, and Generation in the Wild,

    2024

  3. [2024]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347,

    arXiv

  4. [2025]

    Dreamreward: Text-to-3d generation with human preference

    Ye, J., Liu, F., Li, Q., Wang, Z., Wang, Y ., Wang, X., Duan, Y ., and Zhu, J. Dreamreward: Text-to-3d generation with human preference. arXiv preprint arXiv:2403.14613 ,

    arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.