REVIEW 8 minor 2 cited by
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
T0 review · 0 major / 8 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read HigherHRNet claims that bottom-up pose estimation can match top-down accuracy by predicting heatmaps from a high-resolution feature pyramid.
desk verdict HigherHRNet is a solid empirical contribution with credible SOTA numbers, but the scale-awareness story is softer than the headline and should be read as interpretation rather than demonstrated mechanism. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the high-resolution feature pyramid, which reverses the usual direction of feature pyramids: instead of starting at 1/32 and upsampling to 1/4, it starts at the backbone's 1/4 resolution and generates a 1/2-resolution level. The level is produced by a deconvolution module consisting of a 4x4 transposed convolution, BatchNorm, ReLU, and four residual blocks, taking as input the concatenation of lower-level features and predicted heatmaps. Multi-resolution supervision trains each level with same-width Gaussian targets, and multi-resolution heatmap aggregation averages all levels at inference. This machinery is what lets the network localize small-person keypoints more precisely while retaining large-person performance.
What would settle it
Run the same architecture comparisons on a held-out validation set that was not used in any design choice—for example, a new split of COCO or a different scale-heavy benchmark—and check whether one deconvolution module, multi-resolution supervision, and heatmap aggregation still improve AP; if the gains vanish or reverse, the scale-aware explanation is unsupported. A more direct check on COCO is to compare inference using only the 1/2-resolution heatmap against inference using only the 1/4-resolution heatmap: if the medium-person gain disappears, the higher-resolution feature maps themselves, not the aggregation, are the cause.
Extended reading notes
Core claim
The discovery, on the authors' terms, is that scale variation in bottom-up pose estimation is best addressed by learning scale-aware high-resolution heatmaps rather than by relying on a single feature resolution or image pyramids. HigherHRNet takes HRNet's highest-resolution 1/4 feature maps and adds a deconvolution module that doubles the resolution to 1/2, producing a high-resolution feature pyramid. Training uses multi-resolution supervision with the same Gaussian standard deviation at both scales, and inference aggregates all predicted heatmaps after bilinear upsampling. This yields 70.5 AP on COCO2017 test-dev without any post-processing, including 66.6 AP for medium persons, and 67.6 AP on CrowdPose test, surpassing all existing methods there.
Load-bearing premise
The load-bearing premise is that the performance gaps measured on the COCO validation set are stable effects of the proposed mechanisms, not artifacts of picking the number of deconvolution modules, the training image size, and the aggregation strategy using that same validation set.
Editorial extensions
If this is right
- Bottom-up pose estimation can now reach 70.5 AP on COCO test-dev without refinement, substantially narrowing the gap to top-down methods while keeping the efficiency advantage.
- Most of the improvement is concentrated in medium persons, with large-person AP unchanged or slightly better, supporting the claim that the model is genuinely scale-aware rather than simply benefiting from more computation.
- On CrowdPose, a crowded-scene benchmark, HigherHRNet surpasses all top-down methods, suggesting bottom-up approaches are better suited to crowded images where person detectors often fail.
- The number of deconvolution levels should track the dataset's person-scale distribution; adding a second level helped only marginally on COCO and hurt large-person performance, so the architecture is intentionally adjustable.
- Larger training image sizes (640) and wider backbones (HRNet-W48) both improve results, and the method still trains and tests in a single forward pass without post-processing.
Reading between the lines
- The choice to keep the Gaussian standard deviation fixed across resolutions is an unexamined assumption; the paper itself notes a smaller standard deviation is desirable at higher resolution, so per-level standard deviations could further separate small- and large-person performance.
- COCO's keypoint task has no small-person subset, so the paper's small-person claims are tested only through medium persons; a benchmark with official small-person annotations would be the direct test of the scale-variation hypothesis.
- If the scale-specialization story is correct, per-level recall curves should show that the 1/2-resolution heatmap alone recovers keypoints missed by the 1/4-resolution heatmap; this direct diagnostic is not presented in the paper.
- The CrowdPose result suggests that bottom-up methods are robust in crowded scenes because they do not rely on single-person crops, but the grouping tags are trained only at low resolution; higher-resolution tag training, if made to converge, could push crowded-scene accuracy further.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces HigherHRNet, a bottom-up multi-person 2D pose estimation method that augments HRNet with a high-resolution feature pyramid built from transposed-convolution deconvolution modules, trained with multi-resolution supervision and combined at inference by multi-resolution heatmap aggregation. The authors report 70.5 AP on COCO 2017 test-dev without refinement, exceeding previous bottom-up methods, and 67.6 AP on CrowdPose test, exceeding all compared top-down and bottom-up methods. Ablations on COCO val (Tables 3-6) decompose the gains from the deconvolution module, feature concatenation, heatmap aggregation, residual blocks, training resolution, and backbone width.
Significance. If the reported results are correct, the paper is a clear advance: it establishes a simple, efficient high-resolution feature-pyramid design as the state of the art in bottom-up pose estimation and provides a strong bottom-up result on CrowdPose. The paper's strengths are that it follows standard COCO/CrowdPose evaluation protocols, reports single-model numbers, compares against a wide set of prior methods, includes component-level ablations, and ships code and pretrained models. The conceptual claim that the gains are due to scale-aware multi-resolution representation is plausible and supported by the monotonic trend in the ablations, although the component-level evidence is less tight than the benchmark result itself; I agree with the stress-test assessment that the scale-aware attribution is the softest point, but it does not undermine the empirical core.
minor comments (8)
- [4.2, Tables 4-6] The component ablations are single-run, and several design choices (number of deconvolution modules, training image size) are selected on the same COCO val set used for the ablations; because the individual deltas are small (e.g., feature concatenation improves AP by only 0.3 in Table 4), please state this limitation explicitly and either provide multi-seed variance or temper the causal language so that the cumulative monotonic trend, rather than any single delta, is presented as the evidence for the design.
- [Abstract and Section 4.1] The phrase "especially for small person" overstates what can be measured, since the COCO keypoint task has no small-person annotations; the supporting numbers are for medium persons (APM), so please rephrase to "smaller/medium persons" or otherwise clarify the proxy used.
- [Section 3.4] The statement "We believe the model has the ability to automatically focus on specific scales" is an assertion rather than a demonstrated property; please either add evidence (for example, per-scale heatmap visualizations) or mark it clearly as a conjecture.
- [Table 4 text] The sentence "Comparing method (d) and (e), the gain of heatmap aggregation comes from large person" should compare methods (c) and (d); as written, it attributes to heatmap aggregation the step that is actually the addition of residual blocks.
- [Tables 5 and 6] Please specify the training and testing image size used for the backbone comparison in Table 6 (the text does not state that it uses the 640x640 setting), and add a footnote clarifying the testing protocol for each ablation table.
- [Abstract and Table 1] The phrase "without any post processing" is qualified by the use of flip testing and multi-scale test; please clarify whether multi-scale test is considered test-time augmentation rather than post-processing, so that readers do not infer that no test-time operations are used.
- [Section 4.3] The statement "All training parameters follow COCO exactly" conflicts with the later statement that a crop size of 640x640 is used; please state that the crop size is the only intentional difference.
- [Throughout] There are several small typos: "netork" in Section 2, "concatennation" in Figure 3's caption, "muilt" in Section 5, "MSR" for "MRS" in Table 4's caption, and "headmaps" in Section 3.1; these should be corrected.
Circularity Check
No significant circularity: benchmark results are evaluated on held-out test sets, and ablations are standard validation-set model selection.
full rationale
The paper's central claim is an empirical benchmark comparison: COCO test-dev AP 70.5 and CrowdPose test AP 67.6, both evaluated on held-out test sets with a released model, not derived from equations or fitted constants. The architectural choices (one deconvolution module, training size, Gaussian standard deviation) are justified by ablations on COCO val; selecting hyperparameters on a validation set and reporting test-set performance is standard practice, not circular prediction. Citations to HRNet, SimpleBaseline, and associative embedding are independent prior work with external evaluations; they are used as building blocks, not invoked as self-referential theorems that force the conclusions. No equation or parameter in the paper reduces to the target result by construction, so no circularity is present.
Assumptions & free parameters
free parameters (5)
- Number of deconvolution modules =
1
- Gaussian kernel standard deviation in ground-truth heatmaps =
2
- Heatmap and grouping loss weights =
1 (heatmap), 1e-3 (grouping)
- Number of residual blocks after deconvolution =
4
- Training image crop size =
512 (W32), 640 (final W48 and CrowdPose)
assumptions (5)
- domain assumption COCO and CrowdPose annotations with OKS/AP constitute a valid evaluation protocol for multi-person pose estimation.
- domain assumption Associative embedding with a scalar tag per keypoint is sufficient for grouping.
- ad hoc to paper Each feature pyramid level can automatically specialize to different person scales even without explicit scale assignment.
- domain assumption Training with Adam for 300 epochs with the stated schedule yields converged models.
- domain assumption Train/test distribution shift between COCO train and test-dev is small enough for benchmark comparisons.
Cite this review
Pith. "Pith review of HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation." pith.science (2026). https://pith.science/paper/5FLYTQDE
@misc{pith2026190810357,
author = {Pith},
title = {Pith review of: HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/5FLYTQDE}},
note = {Machine review of arXiv:1908.10357}
}
read the original abstract
Bottom-up human pose estimation methods have difficulties in predicting the correct pose for small persons due to challenges in scale variation. In this paper, we present HigherHRNet: a novel bottom-up human pose estimation method for learning scale-aware representations using high-resolution feature pyramids. Equipped with multi-resolution supervision for training and multi-resolution aggregation for inference, the proposed approach is able to solve the scale variation challenge in bottom-up multi-person pose estimation and localize keypoints more precisely, especially for small person. The feature pyramid in HigherHRNet consists of feature map outputs from HRNet and upsampled higher-resolution outputs through a transposed convolution. HigherHRNet outperforms the previous best bottom-up method by 2.5% AP for medium person on COCO test-dev, showing its effectiveness in handling scale variation. Furthermore, HigherHRNet achieves new state-of-the-art result on COCO test-dev (70.5% AP) without using refinement or other post-processing techniques, surpassing all existing bottom-up methods. HigherHRNet even surpasses all top-down methods on CrowdPose test (67.6% AP), suggesting its robustness in crowded scene. The code and models are available at https://github.com/HRNet/Higher-HRNet-Human-Pose-Estimation.
Figures
Forward citations
Cited by 2 Pith papers
-
On the Robustness of Human Pose Estimation
Human pose estimation models are relatively robust to single-step attacks, but heatmap-based and structure-aware models resist attacks better than direct-regression models, and universal perturbations can still break them.
-
Improving Skeleton-based Action Recognition with Interactive Object Information
By adding detected object nodes to skeleton graphs and training with a variable-graph network plus a random node attack regularizer, the authors report large accuracy gains over skeleton-only baselines on action recog...
Reference graph
Works this paper leans on
-
[1]
Segnet: A deep convolutional encoder-decoder architecture for image segmentation
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE TPAMI, 2017. 3
work page 2017
-
[2]
A unified multi-scale deep convolutional neural network for fast object detection
Zhaowei Cai, Quanfu Fan, Rogerio S Feris, and Nuno Vas- concelos. A unified multi-scale deep convolutional neural network for fast object detection. In ECCV, 2016. 3
work page 2016
-
[3]
Realtime multi-person 2d pose estimation using part affinity fields
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In CVPR, 2017. 1, 2, 5, 6, 8
work page 2017
-
[4]
Liang-Chieh Chen, Maxwell D. Collins, Yukun Zhu, George Papandreou, Barret Zoph, Florian Schroff, Hartwig Adam, and Jonathon Shlens. Searching for efficient multi-scale ar- chitectures for dense image prediction. In NeurIPS, 2018. 3
work page 2018
-
[5]
Semantic image segmen- tation with deep convolutional nets and fully connected crfs
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Semantic image segmen- tation with deep convolutional nets and fully connected crfs
-
[6]
Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolu- tion, and fully connected crfs. IEEE TPAMI, 2018. 3
work page 2018
-
[7]
Rethinking atrous convolution for seman- tic image segmentation
Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for seman- tic image segmentation. arXiv preprint arXiv:1706.05587 ,
-
[8]
Encoder-decoder with atrous separable convolution for semantic image segmentation
Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In ECCV, 2018. 3
2018
Show all 44 references
-
[9]
Cascaded pyramid network for multi-person pose estimation
Yilun Chen, Zhicheng Wang, Yuxiang Peng, Zhiqiang Zhang, Gang Yu, and Jian Sun. Cascaded pyramid network for multi-person pose estimation. In CVPR, 2018. 1, 2, 3, 5, 6
2018
-
[10]
Spgnet: Semantic prediction guidance for scene parsing
Bowen Cheng, Liang-Chieh Chen, Yunchao Wei, Yukun Zhu, Zilong Huang, Jinjun Xiong, Thomas S Huang, Wen- Mei Hwu, and Honghui Shi. Spgnet: Semantic prediction guidance for scene parsing. In ICCV, 2019. 3
2019
-
[11]
Panoptic-deeplab
Bowen Cheng, Maxwell D Collins, Yukun Zhu, Ting Liu, Thomas S Huang, Hartwig Adam, and Liang-Chieh Chen. Panoptic-deeplab. arXiv preprint arXiv:1910.04751 , 2019. 3
1910 arXiv
-
[12]
Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation
Bowen Cheng, Maxwell D Collins, Yukun Zhu, Ting Liu, Thomas S Huang, Hartwig Adam, and Liang-Chieh Chen. Panoptic-deeplab: A simple, strong, and fast baseline for bottom-up panoptic segmentation. In CVPR, 2020. 3
2020
-
[13]
Decoupled classification refinement: Hard false positive suppression for object detec- tion
Bowen Cheng, Yunchao Wei, Honghui Shi, Rogerio Feris, Jinjun Xiong, and Thomas Huang. Decoupled classification refinement: Hard false positive suppression for object detec- tion. arXiv preprint arXiv:1810.04002, 2018. 2
2018 arXiv
-
[14]
Revisiting rcnn: On awakening the classification power of faster rcnn
Bowen Cheng, Yunchao Wei, Honghui Shi, Rogerio Feris, Jinjun Xiong, and Thomas Huang. Revisiting rcnn: On awakening the classification power of faster rcnn. In ECCV,
-
[15]
Rmpe: Regional multi-person pose estimation
Hao-Shu Fang, Shuqin Xie, Yu-Wing Tai, and Cewu Lu. Rmpe: Regional multi-person pose estimation. In ICCV,
-
[16]
Mask r-cnn
Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In ICCV, 2017. 1, 2, 3, 6, 8
2017
-
[17]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR,
-
[18]
A coarse- fine network for keypoint localization
Shaoli Huang, Mingming Gong, and Dacheng Tao. A coarse- fine network for keypoint localization. In ICCV, 2017. 2, 6
2017
-
[19]
Deepercut: A deeper, stronger, and faster multi-person pose estimation model
Eldar Insafutdinov, Leonid Pishchulin, Bjoern Andres, Mykhaylo Andriluka, and Bernt Schiele. Deepercut: A deeper, stronger, and faster multi-person pose estimation model. In ECCV, 2016. 2
2016
-
[20]
Multi-person pose estimation with local joint-to-person associations
Umar Iqbal and Juergen Gall. Multi-person pose estimation with local joint-to-person associations. In ECCV, 2016. 2
2016
-
[21]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,
-
[22]
Pifpaf: Composite fields for human pose estimation
Sven Kreiss, Lorenzo Bertoni, and Alexandre Alahi. Pifpaf: Composite fields for human pose estimation. InCVPR, 2019. 1, 2, 3, 6
2019
-
[23]
Cornernet: Detecting objects as paired keypoints
Hei Law and Jia Deng. Cornernet: Detecting objects as paired keypoints. In ECCV, 2018. 4
2018
-
[24]
Crowdpose: Efficient crowded scenes pose estimation and a new benchmark
Jiefeng Li, Can Wang, Hao Zhu, Yihuan Mao, Hao-Shu Fang, and Cewu Lu. Crowdpose: Efficient crowded scenes pose estimation and a new benchmark. In CVPR, 2019. 2, 8
2019
-
[25]
Refinenet: Multi-path refinement networks for high- resolution semantic segmentation
Guosheng Lin, Anton Milan, Chunhua Shen, and Ian Reid. Refinenet: Multi-path refinement networks for high- resolution semantic segmentation. In CVPR, 2017. 3
2017
-
[26]
Feature pyramid networks for object detection
Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In CVPR, 2017. 2, 3, 5
2017
-
[27]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, 2014. 2, 4, 5, 6, 8
2014
-
[28]
Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation
Chenxi Liu, Liang-Chieh Chen, Florian Schroff, Hartwig Adam, Wei Hua, Alan Yuille, and Li Fei-Fei. Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation. In CVPR, 2019. 3
2019
-
[29]
Ssd: Single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, 2016. 3
2016
-
[30]
Associa- tive embedding: End-to-end learning for joint detection and grouping
Alejandro Newell, Zhiao Huang, and Jia Deng. Associa- tive embedding: End-to-end learning for joint detection and grouping. In NeurIPS. 2017. 1, 2, 3, 4, 5, 6
2017
-
[31]
Stacked hour- glass networks for human pose estimation
Alejandro Newell, Kaiyu Yang, and Jia Deng. Stacked hour- glass networks for human pose estimation. In ECCV, 2016. 2, 3
2016
-
[32]
Single-stage multi-person pose machines
Xuecheng Nie, Jiashi Feng, Jianfeng Zhang, and Shuicheng Yan. Single-stage multi-person pose machines. In ICCV,
-
[33]
Person- lab: Person pose estimation and instance segmentation with a part-based geometric embedding model
George Papandreou, Tyler Zhu, Liang chieh Chen, Spyros Gidaris, Jonathan Tompson, and Kevin Murphy. Person- lab: Person pose estimation and instance segmentation with a part-based geometric embedding model. In ECCV, 2018. 1, 2, 5, 6 9
2018
-
[34]
Towards accurate multi-person pose estimation in the wild
George Papandreou, Tyler Zhu, Nori Kanazawa, Alexander Toshev, Jonathan Tompson, Chris Bregler, and Kevin Mur- phy. Towards accurate multi-person pose estimation in the wild. In CVPR, 2017. 1, 2, 5, 6
2017
-
[35]
Deepcut: Joint subset partition and labeling for multi person pose estimation
Leonid Pishchulin, Eldar Insafutdinov, Siyu Tang, Bjoern Andres, Mykhaylo Andriluka, Peter V Gehler, and Bernt Schiele. Deepcut: Joint subset partition and labeling for multi person pose estimation. In CVPR, 2016. 2
2016
-
[36]
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In NeurIPS, 2015. 2
2015
-
[37]
U-net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 3
2015
-
[38]
Deep high-resolution representation learning for human pose esti- mation
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose esti- mation. In CVPR, 2019. 1, 2, 3, 4, 6, 7
2019
-
[39]
Integral human pose regression
Xiao Sun, Bin Xiao, Fangyin Wei, Shuang Liang, and Yichen Wei. Integral human pose regression. In ECCV, 2018. 1, 6
2018
-
[40]
Deep high-resolution representation learning for visual recogni- tion
Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. Deep high-resolution representation learning for visual recogni- tion. CoRR, abs/1908.07919, 2019. 1, 2, 3, 4, 6, 7
1908 arXiv
-
[41]
The devil is in the decoder
Zbigniew Wojna, Vittorio Ferrari, Sergio Guadarrama, Nathan Silberman, Liang-Chieh Chen, Alireza Fathi, and Jasper Uijlings. The devil is in the decoder. In BMVC, 2017. 3
2017
-
[42]
Simple baselines for human pose estimation and tracking
Bin Xiao, Haiping Wu, and Yichen Wei. Simple baselines for human pose estimation and tracking. In ECCV, 2018. 1, 2, 3, 4, 5, 6
2018
-
[43]
Deeperlab: Single-shot image parser
Tien-Ju Yang, Maxwell D Collins, Yukun Zhu, Jyh-Jing Hwang, Ting Liu, Xiao Zhang, Vivienne Sze, George Pa- pandreou, and Liang-Chieh Chen. Deeperlab: Single-shot image parser. arXiv preprint arXiv:1902.05093, 2019. 3
1902 arXiv
-
[44]
Multi-scale context aggregation by dilated convolutions
Fisher Yu and Vladlen Koltun. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122, 2015. 3 10
2015 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.