{"id":"de912aa1-5957-4610-a0eb-051d52f1feeb","arxiv_id":"2411.19824","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"Scale-adaptive tokens allocate high-resolution processing only to small-scale individuals, yielding SOTA-comparable multi-person 3D mesh accuracy at real-time speed.","lead":"At a glance, the paper introduces scale-adaptive tokens for real-time multi-person 3D human mesh estimation from a single image. It dynamically assigns higher-resolution image tokens to smaller people and pools background tokens, achieving accuracy on par with much slower state-of-the-art methods while running at 24 FPS.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'preserves accuracy' claim is not tested against the authors' own full-resolution baseline on the main benchmark; on BEDLAM/AGORA validation the Res-1288 baseline beats the proposed method by 2.8/3.6 mm MVE.","rationale":"The paper is a well-executed empirical study with strong internal evidence: the scale-map ablation (Table B5) shows the predicted scale map is near the GT oracle (56.0 vs 55.8 MVE), and the iso-compute comparison in Table A1 shows the proposed token allocation improves MVE from 63.3 to 56.0 at essentially the same runtime as the Res-644 baseline. These are real merits. My concern is different from the reader's identified weakest assumption. The reader focused on scale-map generalization; my focus is the absence of a same-backbone, same-training full-resolution baseline on the main AGORA test benchmark. The ablations on BEDLAM and AGORA validation show a 2.8-3.6 mm MVE gap between the full-resolution 1288 baseline and the proposed method, so the abstract's claim that the method 'preserves the accuracy benefits of high-resolution processing' is stronger than the evidence supports. On the AGORA test table, the comparison to Multi-HMR is confounded by ViT-L versus ViT-B and SMPL-X versus SMPL, so it cannot establish preservation either. This is a load-bearing concern because it targets the central accuracy claim directly, not an auxiliary component. The reader's scale-map concern is real but substantially mitigated by the near-oracle ablation; my concern is not mitigated anywhere in the paper. I therefore keep the verdict at CONDITIONAL (UNCHANGED) and would add a specific request: report the same-backbone Res-1288 baseline on AGORA test, or explicitly reframe the contribution as a speed-accuracy tradeoff. I do not see grounds for rejection, because the real-time efficiency advantage over ROMP/BEV is large and the method is clearly useful even if the 'preserves' phrasing is softened.","tokens_in":18612,"tokens_out":10162,"duration_ms":94433,"concrete_test":"Evaluate the authors' own ViT-B baseline at input resolution 1288 (the model used for ablation in Table A1/B2) on the AGORA test set with the same protocol, and report MVE, MPJPE, MACs, and runtime. If the 1288 baseline beats the proposed 644* model by more than roughly 3 mm MVE (the gap observed on BEDLAM and AGORA validation), then the central claim that scale-adaptive tokens 'preserve' high-resolution accuracy is not supported on the main benchmark, and the contribution should be framed as a deliberate speed-accuracy tradeoff. Additionally, report the per-scale MVE for this baseline to confirm whether the degradation is concentrated in large-scale individuals. If a ViT-L variant of the proposed method exists (the implementation details mention ViT-L is 74.2 ms), report its MVE on AGORA test to separate the effect of backbone size from the token-allocation design.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim has two parts: scale-adaptive tokens preserve high-resolution accuracy, and they do so at greatly reduced cost. The accuracy-preservation half is only directly tested against the authors' own full-resolution baseline in ablations on BEDLAM and AGORA validation (Tables A1 and B2). There, the Res-1288 baseline achieves 53.2 MVE on BEDLAM versus 56.0 for the proposed 644* model, and 61.9 versus 65.5 on AGORA validation. The main AGORA test comparison (Table 1) is against Multi-HMR with ViT-L and SMPL-X, i.e., a different backbone and a different output representation, so the headline 'comparable to SOTA' conflates the token-allocation mechanism with architecture and representation choices. The paper does not report its own ViT-B Res-1288 baseline on the AGORA test set, the exact benchmark where the headline number is produced. Moreover, the per-scale breakdown in Table B2 shows the largest absolute gap between 644* and 1288 occurs for large-scale people (30%+ scale: 48.2 vs 52.7 on AGORA validation), which weakens the motivation that high resolution gives only marginal improvements for large-scale individuals. This does not invalidate the efficiency contribution, but it means the 'preserves the accuracy benefits of high-resolution processing' claim is not established on the main benchmark and may be better characterized as a speed-accuracy tradeoff with a modest, partly scale-dependent accuracy cost.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SAT-HMR, a one-stage DETR-style multi-person 3D human mesh estimator that replaces uniformly partitioned image tokens with scale-adaptive tokens. A patch-level scale map predicted from low-resolution tokens classifies patches into background, small-scale, and large-scale categories; small-scale patches are replaced by higher-resolution tokens, background tokens are pooled, and large-scale tokens are kept unchanged. Experiments on AGORA, 3DPW, MuPoTS, and CMU Panoptic report performance comparable to recent SOTA methods at about 24 FPS (42 ms) with 133 G MACs, versus 4-5 FPS for high-resolution baselines. The paper includes ablations on BEDLAM and AGORA validation analyzing per-scale accuracy, background pooling, the scale threshold, and the accuracy of the scale map.","tokens_in":18927,"tokens_out":4966,"duration_ms":42485,"significance":"If the speed-accuracy tradeoff is taken as the contribution, the paper is a solid engineering contribution: it demonstrates a controlled speedup from about 5 FPS to 24 FPS at a modest accuracy cost relative to the full-resolution baseline (e.g., 53.2 vs 56.0 MVE on BEDLAM), with a thorough per-scale analysis and a public code release. The central conceptual claim, however, that the method 'preserves the accuracy benefits of high-resolution processing' is not supported by the authors' own comparisons; the method incurs a systematic MVE gap of roughly 3-4 mm relative to the full-resolution baseline. The paper is strongest when framed as an efficiency-accuracy tradeoff and weakest when framed as accuracy-preserving. The empirical design is not circular: the scale threshold is a hyperparameter tuned on validation data, and the GT-scale-map ablation (Table B5) shows that scale prediction is accurate (F1=0.98) and not a bottleneck.","major_comments":[{"comment":"The claim in the Abstract and in the Contributions that the method 'preserves the accuracy benefits of high-resolution processing' is not supported by the authors' own single-resolution baselines. On BEDLAM validation, the Res-1288 baseline achieves 53.2 MVE versus 56.0 for the proposed 644* model (Table A1); on AGORA validation, the corresponding numbers are 61.9 versus 65.5 (Table B2). These gaps are comparable to the differences among SOTA methods in Table 1 (e.g., 61.1 for Multi-HMR 1288 versus 63.3 for Ours), so the accuracy cost is not negligible. The main AGORA test comparison in Table 1 does not include the authors' own ViT-B Res-1288 baseline, so the headline 'comparable to SOTA' conflates the token-allocation mechanism with backbone and output-representation choices. The paper should either report the own-baseline result on the AGORA test set or explicitly recast the claim as a speed-accuracy tradeoff with a modest, scale-dependent accuracy cost.","section":"§4.4, Tables A1 and B2"},{"comment":"The per-scale breakdown in Table B2 shows that the largest absolute gap between the proposed 644* model and the Res-1288 baseline occurs for large-scale individuals, not small-scale ones. On AGORA validation, the 30%+ scale range shows MVE of 52.7 for Ours versus 48.2 for Res-1288, a 4.5 mm gap, while the 0-10% range shows 84.6 versus 82.2, a 2.4 mm gap. For large individuals, Ours essentially matches the low-resolution Res-644 baseline (53.0 MVE) rather than preserving the high-resolution benefit. This weakens the motivation stated in §1 that high-resolution input gives only marginal improvements for large-scale individuals, and it raises a question about where the resolution benefit is actually lost. Please discuss this per-scale behavior explicitly and, if possible, provide an analysis of why the large-scale range loses the most accuracy.","section":"Table B2"},{"comment":"The efficiency comparison to Multi-HMR is not apples-to-apples: Multi-HMR uses a ViT-L backbone and regresses SMPL-X, while the proposed method uses ViT-B and regresses SMPL. The 5x speedup claim in the Contributions therefore conflates the scale-adaptive token mechanism with architecture and output-representation choices. The controlled comparison in Fig. 2 and Table A1 (same architecture, different resolutions) is the appropriate evidence for the efficiency-accuracy tradeoff, and the SOTA table should be interpreted with this caveat. The paper should state this limitation directly when reporting the speedup.","section":"§4.3, Table 1"}],"minor_comments":[{"comment":"The paper's own limitation statement that the method is not age- or height-aware is directly relevant to the small-scale population the method is designed to handle; Fig. B5 shows a child depth error. This limitation should be moved to the main text, since it tempers the claim about small-scale individuals.","section":"Sec. B.3"},{"comment":"The word 'interence' appears in the sentence 'decreases efficiency with longer interence time cost'; it should be 'inference'.","section":"Supplementary, Table B4 paragraph"},{"comment":"There is a missing space in 'set toλmap = 4' in Sec. A.2; please fix the formatting of the loss weights.","section":"Sec. 3.4 and Sec. A.2"},{"comment":"The phrase 'achieving an unparalleled balance' is promotional and not supported by the data, since the method sacrifices a few mm of accuracy relative to the full-resolution baseline; please replace it with a neutral statement.","section":"§4.3, last paragraph"}],"recommendation":"major_revision","confidential_remarks":"The technical work is competent and the scale-adaptive token mechanism is a reasonable contribution to real-time multi-person mesh estimation. The main risk is the mismatch between the abstract's 'preserves accuracy' claim and the authors' own ablation tables; the revision should make the speed-accuracy tradeoff explicit and add the missing own-baseline comparison on AGORA test. The release of code and models is a positive factor."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The scale-adaptive token idea is the real thing: use a learned patch-level scale map to route high-resolution tokens to small-scale people, keep low-res tokens for large people, and pool background tokens. The per-scale error analysis in Fig. 2 is convincing and the ablations are thorough—threshold sweeps, GT scale map comparison, token counts, runtime. The 5x speedup at 24 FPS with accuracy close to much slower SOTA methods is a solid engineering contribution to multi-person HMR.\n\nThat said, the paper's headline claim is a bit stronger than the data. On BEDLAM and AGORA validation, the authors' own full-resolution baseline (Res-1288) beats the proposed 644* model by 2.8 mm and 3.6 mm MVE, respectively. So it is not really 'preserving the accuracy benefits of high-resolution processing'; it is a speed-accuracy trade-off with a modest, partly scale-dependent accuracy cost. The stress-test note is right: the main AGORA test comparison is against Multi-HMR with ViT-L and SMPL-X, so the headline 'comparable to SOTA' conflates the token mechanism with architecture and representation differences. They also do not report their own ViT-B Res-1288 baseline on the AGORA test set, which is the exact benchmark for the headline number. And the per-scale breakdown shows the largest absolute gap for large-scale people (30%+: 48.2 vs 52.7 on AGORA validation), which weakens the motivation that high resolution barely helps large individuals.\n\nThese are real but not fatal. The central mechanism is sound: the GT scale map ablation improves average MVE by only 0.2 mm, so scale prediction is accurate. The paper is honest about its limitations (no height/age awareness, body-only), and the missing apples-to-apples baselines and error bars are standard revision items, not red flags. The efficiency claim is well supported: 42 ms versus 231.7 ms for Multi-HMR at 1288, with a 3.4 mm MVE gap on AGORA test—a reasonable trade for real-time use.\n\nI would send this to peer review. The authors should be asked to report their own Res-1288 ViT-B baseline on AGORA test, add multiple-seed variance, and soften the 'preserves accuracy' wording to 'maintains competitive accuracy at substantially lower cost.' The paper is for researchers working on efficient DETR-style mesh estimation and adaptive tokenization; it deserves a serious referee.","headline":"A genuine efficiency win with a slightly over-stated 'preserves accuracy' claim; the scale-adaptive token mechanism is new, well-ablated, and worth refereeing.","tokens_in":19536,"tokens_out":1865,"would_cite":true,"duration_ms":16822,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Scale-adaptive tokens—high-res for small people, low-res for large, pooled background—let a DETR-style mesh estimator keep high-resolution accuracy at 24 FPS.","keywords":["3D human mesh estimation","multi-person","scale-adaptive tokens","DETR","real-time inference","SMPL","token pruning","single RGB image"],"falsifier":"Take the trained model, swap in the ground-truth scale map at test time, and compare per-scale-range mean vertex error on AGORA validation; if mispredicted small-scale patches show errors near the low-resolution baseline, the scale predictor is the bottleneck, whereas if the gap stays small (the BEDLAM ablation shows only 0.2 mm average), the residual error lies in mesh regression rather than token allocation.","tokens_in":18330,"feed_emoji":"🧍","tokens_out":9050,"duration_ms":66985,"temperature":0.7,"pith_summary":"The paper is trying to establish that the accuracy benefit of high-resolution input in one-stage multi-person 3D mesh estimation can be preserved without processing the entire image at high resolution. Its proposal, scale-adaptive tokens, predicts a patch-level scale map and spends resolution where it matters: small-scale people (far from the camera, children, or in closed poses) get high-resolution tokens, large-scale people keep low-resolution tokens, and background regions are pooled into fewer tokens. On the AGORA test set the method reports 63.3 mm mean vertex error at 42 ms per image, comparable to Multi-HMR's 61.1 mm at 231.7 ms with 1288-resolution input, and it runs at roughly 24 FPS. If the claim holds, real-time multi-person mesh recovery no longer has to sacrifice accuracy on small or distant people to stay fast.","feed_headline":"Adaptive tokens make 3D mesh recovery real-time at near-SOTA accuracy","feed_subtitle":"Giving distant people high-res tokens and pooling the rest keeps error near SOTA at 42 ms per image.","key_machinery":"The load-bearing object is the scale-adaptive token set $T_{\\text{SA}} = \\{T'_B, T_{\\text{LARGE}}, T_{\\text{HR}}\\}$, assembled from a predicted patch-level scale map $S(i,j) = (c,s)$. Here $c$ is the confidence that a patch overlaps a person and $s$ is the diagonal of that person's bounding box normalized by the longer side of the high-resolution image. A shallow three-layer transformer and an MLP scale head predict the map from low-resolution tokens; patches below the scale threshold $\\alpha_s = 0.5$ are re-tokenized at four times the resolution by a parallel shallow encoder, large-scale patches remain low-resolution, and background tokens are pooled four-to-one to cut their number. This conditional allocation is what carries the argument: it concentrates computation on the difficult small-scale cases that previously forced one-stage methods to run the whole image at high resolution.","core_discovery":"The central discovery is that uniform high-resolution tokenization of the whole image is wasteful because different people need different levels of detail. The paper's own baseline analysis shows that raising input resolution from 518 to 1288 cuts mean vertex error by nearly 35 mm for individuals in the 0-10% scale range, while giving almost nothing to people covering more than 30% of the image. SAT-HMR therefore builds a scale-adaptive token set from a patch-level scale map: patches classified as small-scale are replaced by high-resolution tokens extracted from a 1288-resolution image, large-scale patches keep base 644-resolution tokens, and background tokens are spatially pooled. These tokens feed a transformer encoder and a DAB-DETR-style decoder that regresses SMPL pose, shape, and translation. The result is accuracy close to high-resolution state-of-the-art models at a fraction of the cost—133.1 G MACs and 42 ms per image versus 6104.6 G and 231.7 ms for Multi-HMR at 1288 resolution.","pith_inferences":["Because the model is not age- or height-aware, children and atypically small adults can be mis-scaled; adding an explicit height or age prior to the scale head or depth head is a natural testable extension that targets the failure mode the paper itself illustrates.","The 35 mm gap between low- and high-resolution baselines for small-scale individuals suggests small-instance feature detail, not depth ambiguity, is the dominant bottleneck in one-stage mesh estimation; the same adaptive allocation could be applied to face and hand regions for whole-body estimation, which the authors name as future work.","The scale map is predicted from only three transformer layers on low-resolution tokens, so its 0.98 F1 may not transfer to unusual camera angles or extremely dense crowds; stress-testing on those distributions would show whether allocation or regression deserves the next round of capacity.","Since the ground-truth scale map closes only about 0.2 mm of average error, spending the saved compute on stronger decoders or SMPL priors may buy more accuracy per FLOP than further improving scale prediction."],"forward_implications":["One-stage mesh estimators can run at real-time speed without dropping to low-resolution accuracy: 63.3 mm MVE on AGORA test at 42 ms per image, within 2.2 mm of Multi-HMR's 1288-resolution result while running about 5.5 times faster.","Because the change is confined to token construction in the encoder, the scale-adaptive recipe can be transplanted into other DETR-style pose and mesh estimators without altering their decoders or losses.","Background tokens carry context that matters: pooling them preserves accuracy for large or truncated people, while dropping them entirely hurts the largest-scale range on BEDLAM.","Scale prediction is accurate enough in aggregate (0.98 F1 on BEDLAM) that replacing it with ground-truth scale changes average MVE only from 56.0 to 55.8 mm, so most remaining error is in mesh regression rather than token allocation.","Increasing the base resolution of the adaptive scheme from 644* to 896* continues to cut small-scale error (from 84.6 to 76.5 mm for 0-10% scale in AGORA validation), so the approach gains from extra compute without changing the mechanism."],"supporting_citations":[{"why":"Multi-HMR is the main one-stage DETR-style baseline whose high-resolution accuracy and high runtime are the comparison point for the paper's speed-accuracy claim.","marker":"[2]"},{"why":"DETR supplies the end-to-end query-based transformer pipeline (encoder, decoder, Hungarian matching) that the method extends with scale-adaptive tokens.","marker":"[5]"},{"why":"DAB-DETR provides the decoder-query design of content embeddings plus anchor boxes used to regress SMPL parameters.","marker":"[23]"},{"why":"DINOv2 pretrained weights initialize the transformer encoder, giving the feature backbone that is re-tokenized at varying resolutions.","marker":"[32]"},{"why":"AGORA is the main benchmark whose test set supplies the headline MVE, F1, and runtime numbers.","marker":"[33]"},{"why":"AiOS is the other one-stage SOTA method compared against, and its high-resolution cost motivates the efficiency goal.","marker":"[44]"},{"why":"BEDLAM is the dataset used for the ablations of background pooling, scale thresholds, and scale-map quality.","marker":"[3]"}],"fun_headline_variants":["Scale-aware tokens cut 3D mesh cost 45x, keep accuracy","Small people get high-res tokens for real-time mesh estimation","Real-time 3D mesh: give distant folks high-res, pool rest","SAT-HMR: adaptive tokens slash compute 45x, keep SOTA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the learned patch-level scale map correctly identifies which image patches contain small-scale people, so high-resolution tokens are placed where they matter; if the scale prediction is wrong for a person, that person effectively gets only low-resolution features and the accuracy gain disappears.","fun_headline_variants_meta":{"raw":{"variants":["Scale-aware tokens cut 3D mesh cost 45x, keep accuracy","Small people get high-res tokens for real-time mesh estimation","Real-time 3D mesh: give distant folks high-res, pool rest","SAT-HMR: adaptive tokens slash compute 45x, keep SOTA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000208,"raw_usage":{"total_tokens":1411,"prompt_tokens":957,"completion_tokens":454,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":573,"tokens_out":454,"duration_ms":4372,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:46:43.018730+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained model, swap in the ground-truth scale map at test time, and compare per-scale-range mean vertex error on AGORA validation; if mispredicted small-scale patches show errors near the low-resolution baseline, the scale predictor is the bottleneck, whereas if the gap stays small (the BEDLAM ablation shows only 0.2 mm average), the residual error lies in mesh regression rather than token allocation.","supporting_citations":[{"cited_title":"Multi-hmr: Multi-person whole-body human mesh recovery in a single shot","cited_arxiv_id":null,"evidence_quote":"Multi-HMR is the main one-stage DETR-style baseline whose high-resolution accuracy and high runtime are the comparison point for the paper's speed-accuracy claim."},{"cited_title":"End- to-end object detection with transformers","cited_arxiv_id":null,"evidence_quote":"DETR supplies the end-to-end query-based transformer pipeline (encoder, decoder, Hungarian matching) that the method extends with scale-adaptive tokens."},{"cited_title":"DAB-DETR: Dynamic anchor boxes are better queries for DETR","cited_arxiv_id":null,"evidence_quote":"DAB-DETR provides the decoder-query design of content embeddings plus anchor boxes used to regress SMPL parameters."},{"cited_title":"Dinov2: Learning robust visual features without supervision","cited_arxiv_id":null,"evidence_quote":"DINOv2 pretrained weights initialize the transformer encoder, giving the feature backbone that is re-tokenized at varying resolutions."},{"cited_title":"Agora: Avatars in geography optimized for regression analysis","cited_arxiv_id":null,"evidence_quote":"AGORA is the main benchmark whose test set supplies the headline MVE, F1, and runtime numbers."},{"cited_title":"Aios: All-in-one-stage expressive human pose and shape estimation","cited_arxiv_id":null,"evidence_quote":"AiOS is the other one-stage SOTA method compared against, and its high-resolution cost motivates the efficiency goal."},{"cited_title":"Bedlam: A synthetic dataset of bodies exhibiting de- tailed lifelike animated motion","cited_arxiv_id":null,"evidence_quote":"BEDLAM is the dataset used for the ablations of background pooling, scale thresholds, and scale-map quality."}],"review_version":1}