{"id":"b34451d1-8826-4faf-bfdf-07ab0aa39632","arxiv_id":"2505.10250","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ADHMR aligns diffusion-based human mesh recovery with a learned scorer and direct preference optimization, improving accuracy and in-the-wild robustness without human preference labels.","lead":"This paper trains a judge model that scores how well a predicted 3D human mesh matches an image, then uses those scores to finetune a diffusion-based mesh generator with preference optimization. The method improves accuracy and robustness on standard benchmarks, and the same judge can clean noisy training data for other human mesh recovery models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"HMR-Scorer is trained and evaluated only on Gaussian-perturbed GT poses (Sec. 4.2, 5.2), never validated on actual base-model predictions; if its ranking of real HypoNet outputs is weak, the DPO preference pairs in Sec. 4.3 are mislabeled and the headline M=10 improvement could be spurious.","rationale":"The paper is well-executed: the DPO adaptation is technically sound, the ablations are informative, and the reported gains are consistent with a genuine improvement. The load-bearing assumption, however, is that HMR-Scorer can reliably rank the actual outputs of the base diffusion model, including on in-the-wild images, in order of metric-based reconstruction quality. The evidence in Sec. 5.2 is restricted to synthetic Gaussian perturbations of ground-truth poses, which do not match the error distribution of real model predictions. This is a domain-shift problem, not an internal inconsistency. The proposed concrete test is cheap, decisive, and directly targets the weakest link: if the scorer's rank correlation on real HypoNet outputs is low, the preference dataset is noisy and the central mechanism fails, regardless of how well the rest of the pipeline is built. I therefore agree with the reader's weakest_assumption and recommend keeping the conditional verdict pending that validation; I do not see grounds to reject the paper outright, and I would not fully accept it without the test.","tokens_in":15890,"tokens_out":3345,"duration_ms":34145,"concrete_test":"Build a real-prediction benchmark: on the 3DPW test set, run the frozen base HypoNet to draw M=100 predictions per image, compute ground-truth PVE/MPJPE for each prediction and HMR-Scorer's score, and report SRCC/PLCC over all (image, prediction) pairs. Then, as a causal check, construct two DPO preference datasets on 3DPW training images, one ranked by GT errors and one by HMR-Scorer; finetune with identical hyperparameters and compare final 3DPW MPJPE/PVE at M=10. If scorer-ranked finetuning is substantially worse than GT-ranked finetuning, the scorer is the bottleneck; if they match, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central pipeline is a chain: HMR-Scorer ranks base-model predictions (Sec. 4.3), those preferences drive DPO finetuning (Sec. 4.4), and the resulting model is claimed to beat the base at M=10 (Table 2). The weakest link is the scorer. In Sec. 4.2 it is trained on synthetic predictions formed by adding joint-wise Gaussian noise to ground-truth SMPL poses; in Sec. 5.2 it is evaluated on the same type of Gaussian perturbations on GTA-Human and DNA-Rendering. Real HypoNet predictions are not Gaussian perturbations around GT: they contain systematic pose and shape bias, self-occlusion and depth errors, and camera misestimation, so their error distribution differs from the scorer's training distribution. The paper reports no PLCC or SRCC of the scorer on actual base-model outputs. If the scorer's ranking on real outputs is only weakly correlated with true PVE/MPJPE, many (winner, loser) pairs in the preference dataset are inverted and DPO optimizes a poor reward. That would not undermine the supervised-finetuning comparison in Table 3, but it would invalidate the specific claim that a learned scorer can replace ground-truth labels and yield the reported cross-model gains.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ADHMR, a pipeline that (i) trains a mesh-quality scorer, HMR-Scorer, on synthetic noisy SMPL predictions, (ii) uses that scorer to rank predictions of a diffusion-based HMR base model (HypoNet) and construct winner/loser preference pairs, (iii) fine-tunes the base model with Diffusion-DPO, and (iv) uses the same scorer to clean noisy pseudo-labeled training data. On 3DPW and Human3.6M the method reports consistent improvements over the base model and over prior probabilistic HMR methods; ablations indicate that preference fine-tuning outperforms supervised fine-tuning; and data cleaning improves several HMR models. The central claim is that a learned scorer can replace human preference labels in aligning diffusion-based HMR models.","tokens_in":16190,"tokens_out":6923,"duration_ms":68026,"significance":"If the results hold, the paper provides a practical way to apply preference optimization to human mesh recovery without manual preference annotation, and it demonstrates both a DPO-based alignment gain and a reusable data-cleaning tool. The study is well positioned in the literature, with clear baselines, ablations against supervised fine-tuning, and a code release. The main caveat is that the quality of the learned scorer, which is the linchpin of the pipeline, is only measured on the same synthetic noise-perturbation distribution used for training; its behavior on real base-model outputs is the critical unverified link.","major_comments":[{"comment":"The HMR-Scorer is trained and evaluated exclusively on predictions formed by adding Gaussian noise to ground-truth SMPL poses, but in Sec. 4.3 it is used to rank actual HypoNet predictions while building the preference dataset. Real HypoNet errors are not additive Gaussian perturbations of the ground truth: they contain systematic pose bias, camera errors, and shape errors whose distribution differs from the scorer's training distribution. The paper reports no PLCC/SRCC, nor pairwise ranking accuracy, of the scorer on real base-model outputs, so the quality of the preference pairs used for ADHMR (ITW) in Table 2 is unverified. The authors should validate the scorer on real predictions of the base model (e.g., on a held-out portion of 3DPW or Human3.6M) by comparing scorer scores with true PVE/MPJPE, and should report the fraction of preference pairs where the scorer agrees with ground-truth error ordering. If the correlation is weak, the scorer should be retrained or augmented with real model outputs before the headline M=10 improvement can be attributed to the learned scorer.","section":"Sec. 4.2, 4.3, 5.2; Table 2"},{"comment":"The description of preference construction is ambiguous about which rows of Table 2 actually rely on HMR-Scorer. The text says that for 'studio-captured datasets with precise human mesh annotations' the ordering is computed directly from ground-truth reconstruction errors, but the ADHMR rows in Table 2 are labeled as fine-tuned on 'the target benchmark dataset,' which includes 3DPW (an in-the-wild dataset) as well as Human3.6M. The paper should specify, for each row, whether preferences come from HMR-Scorer or from ground-truth errors. This matters because the abstract and introduction claim that the method works 'without 3D annotations'; if the non-ITW rows use ground-truth errors for ranking, the only unlabeled-setting evidence is the ITW row, which in turn depends on the unvalidated scorer.","section":"Sec. 4.3, Table 2"},{"comment":"The data-cleaning contribution inherits the same validation gap. The scorer is used to filter pseudo-labeled samples from in-the-wild data, but there is no experiment showing that low scorer scores actually correspond to poor pseudo labels or to high 3D error. The improvements in Table 5 could in principle come from removing hard or atypical images rather than from removing mislabeled ones. The authors should report the correlation between scorer scores and pseudo-label errors on a held-out set, or compare against random filtering of the same number of samples, to establish that the cleaning signal is specifically annotation quality.","section":"Sec. 4.5, Table 5"}],"minor_comments":[{"comment":"In Eq. (5), the loser term is missing the norm notation on the epsilon terms; please fix the typesetting so that the two squared-norm differences are clearly displayed.","section":"Sec. 4.4, Eq. (5)"},{"comment":"The notation {s_m}_{i=0}^M uses i instead of m in the subscript, and F_g is reused for both the global image feature and the low-channel global feature; please rename one of these to avoid confusion.","section":"Sec. 4.2"},{"comment":"The paper states that HMR-Scorer is trained on five datasets, including GTA-Human and DNA-Rendering, and then evaluates on the 'original test set' of those same datasets; please clarify the exact train/test split to rule out overlap.","section":"Sec. 5.1, 5.2"},{"comment":"The values of the free parameters K, beta, tau, and the noise magnitudes used for scorer training are not reported in the experiments; please list them and ideally ablate K, beta, and tau.","section":"Sec. 4.3, 4.5"},{"comment":"The table header reports MPJPE and PVE, but the text in Sec. 5.3 states that the probabilistic methods report min-MPJPE and min-PVE over M samples; please make this explicit in the table caption and mark the deterministic methods accordingly.","section":"Table 2"},{"comment":"No variance or multiple-seed results are reported; given that some of the headline margins are small (e.g., ADHMR (ITW) M=10 vs base M=200 on 3DPW MPJPE is 61.3 vs 61.8), the authors should provide at least a sensitivity check or error bars.","section":"Tables 2-5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong and consistent experimental story, and the DPO-vs-supervised-finetuning ablation is a useful comparison. The main risk is the scorer-validation gap: the central 'learned scorer replaces labels' claim is not yet established because the scorer is never tested on real base-model outputs. If the authors can add the requested real-output validation, clarify which table rows use scorer-based vs ground-truth-based preferences, and strengthen the data-cleaning evidence, the paper could become a solid contribution. I do not see a reason for rejection, but the current manuscript is not yet publishable as is."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a legitimate, well-executed paper. The core idea is to replace human preference labels in DPO with a learned scorer for human mesh recovery, and that idea mostly holds up. The main weakness is that the scorer's ranking ability is only tested on synthetic Gaussian perturbations of ground truth, never on the actual outputs of the base model it is used to rank. That is a real gap, but an addressable one, not a fatal flaw.\n\nWhat is new: this is the first adaptation of DPO to probabilistic HMR, and it uses a learned scorer to synthesize preference pairs without human annotation. The HMR-Scorer has some thoughtful design choices (UVD coordinates, multi-scale pixel-aligned features) and outperforms the ScoreHypo reward model as a quality predictor. The ablations are clean: DPO beats supervised finetuning both on the target benchmark and on noisy in-the-wild data, and the extra-data ablation rules out the trivial explanation that gains just come from seeing more images. The data cleaning result is a nice bonus and works across several base models.\n\nWhere it gets soft: Section 4.3 builds the preference dataset by ranking base-model predictions with HMR-Scorer. But the scorer was trained on Gaussian-perturbed ground truth poses (Section 4.2) and evaluated only on those same perturbations (Section 5.2). Real HypoNet outputs contain systematic pose bias, occlusion errors, and camera misestimation—error patterns that look nothing like additive Gaussian noise. If the scorer's ranking on real outputs is weak, many preference pairs are mislabeled and DPO is optimizing a noisy reward. The paper reports no PLCC/SRCC of the scorer on actual base-model predictions. This is the load-bearing assumption and it is unvalidated. It affects the strong cross-model claim (M=10 beating base M=200) more than the DPO-vs-supervised ablation, which is internally controlled. Several hyperparameters (beta, K, cleaning threshold, noise magnitudes) are also unreported, which makes reproduction harder even with the code.\n\nThe central argument holds up conditionally. The results are consistent and the method is sound enough to justify serious referee time. I would recommend acceptance with a request for scorer validation on real base-model outputs and full hyperparameter disclosure. This paper is for HMR researchers and for anyone working on learned rewards for generative models.","headline":"Solid DPO-for-HMR paper with a real novelty (learned scorer instead of human preferences), but the scorer is never validated on the real base-model outputs it ranks—an addressable gap that should not block acceptance.","tokens_in":16732,"tokens_out":3093,"would_cite":true,"duration_ms":25890,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A learned quality scorer, not human annotators, supplies the preference signal that aligns diffusion-based human mesh recovery, letting ten predictions beat two hundred.","keywords":["diffusion model","human mesh recovery","direct preference optimization","reward model","probabilistic pose estimation","data cleaning","3D human pose and shape estimation","score prediction"],"falsifier":"Take the base model's actual multi-sample outputs on a benchmark with ground-truth meshes, such as the 3DPW test set, compute HMR-Scorer's scores and the true PVE and MPJPE errors for each prediction, and measure their rank correlation; if the correlation collapses toward zero on real outputs while it sits near 0.6 on the synthetic noise-perturbed test sets, the preference pairs that drive ADHMR do not track reconstruction quality. A complementary ablation finetunes the base model on preference pairs ranked by true metrics instead of by the scorer; identical gains would mean the scorer itself contributes nothing beyond approximating the metric.","tokens_in":15699,"feed_emoji":"🧍","tokens_out":12585,"duration_ms":101036,"temperature":0.7,"pith_summary":"The paper claims that a probabilistic human mesh recovery model, which generates many plausible 3D body estimates from a single image, can be improved by finetuning it with direct preference optimization on winner–loser prediction pairs ranked not by humans but by an automatic quality scorer. The scorer, HMR-Scorer, is trained to approximate standard reconstruction metrics using only image cues and projected keypoints, so it can rank predictions even for in-the-wild images that lack clean 3D annotations. If this holds, probabilistic HMR models no longer need to generate hundreds of candidates to be accurate, and the same scorer can filter poorly annotated training images to improve other state-of-the-art HMR models. The paper supports the claim with benchmark results showing that the aligned model's ten-prediction results beat the base model's two-hundred-prediction results in PVE and MPJPE on the 3DPW benchmark.","feed_headline":"Ten predictions beat two hundred for 3D human mesh recovery","feed_subtitle":"A learned scorer supplies preference labels that align diffusion mesh recovery, and cleans noisy training data.","key_machinery":"The load-bearing object is HMR-Scorer, a transformer-encoder-based reward model that takes, for each candidate mesh, the UVD keypoint coordinates (3D joints projected into image space through the predicted camera) together with multi-scale image features: global ViT tokens plus pixel-aligned local features sampled at the reprojected joint positions. It outputs a scalar quality score and is trained with a RankNet-style pairwise probabilistic ranking loss whose quality labels are derived from four standard HMR metrics (PVE, MPJPE, PA-PVE, PA-MPJPE) on synthetic predictions formed by adding joint-wise Gaussian noise to ground-truth SMPL poses. The second load-bearing object is the Diffusion-DPO objective, which contrasts, at a random timestep along the diffusion trajectory, the denoising error on a winner sample against the error on a loser sample, weighted by the signal-to-noise ratio $\\lambda_t$, while penalizing deviation from the frozen reference denoiser. Together these convert an unlabeled preference signal into gradient updates of the base denoiser.","core_discovery":"On its own terms, the paper establishes that the knowledge of a quality-assessment model can be distilled into a diffusion-based human mesh reconstruction model: the base denoiser HypoNet is finetuned with the Diffusion-DPO objective so that denoising trajectories toward scorer-ranked winner meshes are favored over loser trajectories, with a KL constraint keeping the aligned model close to the frozen base model. The resulting model, ADHMR, and its in-the-wild variant ADHMR (ITW), outperform prior probabilistic methods on 3DPW and Human3.6M across the reported metrics; at $M=10$ predictions, ADHMR (ITW) reaches MPJPE 61.3 mm and PVE 71.3 mm on 3DPW, surpassing the base model's $M=200$ result of 61.8 mm and 71.9 mm. The paper also shows that HMR-Scorer's predicted scores correlate with ground-truth reconstruction errors, with PLCC and SRCC near 0.6 on GTA-Human and DNA-Rendering synthetic noise-perturbed test sets, and that filtering training data by scorer score improves models such as SMPLer-X, OSX, and Hand4Whole even when fewer training samples remain.","pith_inferences":["If HMR-Scorer's ranking holds on real rather than only synthetic model outputs, the same score–rank–align recipe should transfer to other ill-posed inverse problems, such as hand or face mesh recovery, monocular depth estimation, or single-view reconstruction, wherever a metric can label synthetic perturbations.","The data-cleaning results raise the possibility that curation beats scale for in-the-wild HMR training; an adaptive per-dataset score threshold instead of the fixed $\\tau=0.6$ would test how far this can be pushed.","The $M=10$ result implies that reporting min-error over hundreds of hypotheses may overstate what probabilistic HMR actually needs, and error-versus-$M$ curves of the kind this paper reports give a fairer basis for comparing probabilistic methods."],"forward_implications":["At $M=10$ predictions, ADHMR (ITW) already beats the base HypoNet at $M=200$ on 3DPW in both PVE and MPJPE, so aligned models can cut the number of candidate predictions by an order of magnitude.","Preference finetuning handles noisy pseudo-labels better than supervised finetuning: on the in-the-wild InstaVariety data, ADHMR improves 3DPW accuracy while supervised finetuning on the same pseudo-labels degrades it (Tables 3–4).","HMR-Scorer doubles as a data-cleaning tool: keeping only pseudo-labeled training samples scored above a threshold ($\\tau=0.6$) improves Hand4Whole, OSX, and SMPLer-X-Base on 3DPW, Human3.6M, and EHF even though fewer training samples remain (Table 5).","The preference dataset requires no human annotations: HMR-Scorer ranks base-model predictions automatically, and for studio datasets with ground truth the ordering can be computed directly from reconstruction error (Section 4.3)."],"supporting_citations":[{"why":"Supplies the Diffusion-DPO objective that ADHMR adapts to aligned human mesh recovery.","marker":"(Wallace et al., 2024)"},{"why":"Origin of direct preference optimization, the alignment principle the method ports to HMR.","marker":"(Rafailov et al., 2024)"},{"why":"Supplies HypoNet, the base diffusion model ADHMR finetunes, and ScoreNet, the reward-model baseline it is compared against.","marker":"(Xu et al., 2024b)"},{"why":"RankNet, whose pairwise probabilistic ranking loss is used to train HMR-Scorer.","marker":"(Burges et al., 2005)"},{"why":"InstaVariety, the in-the-wild pseudo-labeled dataset used to build the ADHMR (ITW) preference set.","marker":"(Kanazawa et al., 2019)"},{"why":"SMPL, the parametric body model whose pose and shape parameters the method predicts.","marker":"(Loper et al., 2015)"},{"why":"ProHMR, a normalizing-flow probabilistic baseline compared in the main results.","marker":"(Kolotouros et al., 2021)"},{"why":"HMDiff, the diffusion-based HMR baseline compared in the main results.","marker":"(Foo et al., 2023)"}],"fun_headline_variants":["Ten predictions beat two hundred for 3D human mesh","Preference-aligned diffusion model recovers better 3D meshes","Scorer-based preference optimization sharpens human mesh recovery","Diffusion-DPO aligns mesh recovery: 10 samples outperform 200","Better 3D meshes from fewer predictions via preference alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"HMR-Scorer, the model that ranks every winner–loser pair in the preference dataset, is trained and validated only on synthetic predictions formed by adding Gaussian noise to ground-truth poses (Section 5.2), never on the actual outputs of the base diffusion model it is asked to rank in Sections 4.3 and 4.5; if its ranking of real outputs diverges from true reconstruction quality, the preference signal that drives ADHMR is misaligned.","fun_headline_variants_meta":{"raw":{"variants":["Ten predictions beat two hundred for 3D human mesh","Preference-aligned diffusion model recovers better 3D meshes","Scorer-based preference optimization sharpens human mesh recovery","Diffusion-DPO aligns mesh recovery: 10 samples outperform 200","Better 3D meshes from fewer predictions via preference alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000869,"raw_usage":{"total_tokens":3792,"prompt_tokens":1004,"completion_tokens":2788,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":2715}},"tokens_in":620,"tokens_out":2788,"duration_ms":21147,"temperature":1.0,"reasoning_tokens":2715,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:12:58.171013+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the base model's actual multi-sample outputs on a benchmark with ground-truth meshes, such as the 3DPW test set, compute HMR-Scorer's scores and the true PVE and MPJPE errors for each prediction, and measure their rank correlation; if the correlation collapses toward zero on real outputs while it sits near 0.6 on the synthetic noise-perturbed test sets, the preference pairs that drive ADHMR do not track reconstruction quality. A complementary ablation finetunes the base model on preference pairs ranked by true metrics instead of by the scorer; identical gains would mean the scorer itself contributes nothing beyond approximating the metric.","supporting_citations":[],"review_version":1}