REVIEW 5 major objections 6 minor 26 references
DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints
T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read DETRPose claims to be the first real-time, end-to-end transformer family for multi-person 2D pose estimation, with a small model matching YOLO-scale baselines at a fraction of the parameters and latency.
desk verdict Genuinely new training recipe for real-time transformer pose, but the abstract's YOLO comparison is contradicted by the paper's own Table 5. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the keypoint-similarity metric, $KS = \exp\left(-d^2/(2s^2\kappa_i^2)\right)$, which measures how close a perturbed keypoint is to ground truth in normalized coordinates. Inverting this metric gives a perturbation radius $\hat{\alpha}_{\mathrm{pose}} = s\kappa\sqrt{-2\ln(KS)}$, so the paper can sample positive queries from $KS \in U(0.5,1)$ and negative queries from $KS \in U(0.1,0.5)$ and translate ground-truth keypoints by exactly that amount. This creates a supervised denoising task that accelerates convergence, and the denoising queries are discarded at inference. The rest of the machinery is the modified decoder: GroupPose-style self-attention that groups keypoints by person and by joint, D-FINE's pre-pose and fine-grained distribution refinement layers for offset refinement, a Pose-LQE classification head that reuses predicted keypoint locations, and the Keypoint Similarity VariFocal loss that combines similarity and confidence.
What would settle it
Run DETRPose-S and YOLOv8-Pose-X / YOLO11-Pose-X on the same GPU with the same inference engine version and measure end-to-end latency and AP on COCO val; if DETRPose-S does not show roughly the reported 2.39 ms versus about 5 ms advantage and 67.0 AP, the central claim fails.
Extended reading notes
Core claim
The central claim is that the decoder, not the encoder, is what makes transformer pose estimators slow, and that a decoder built around keypoints instead of bounding boxes can remove that gap. DETRPose keeps GroupPose's per-person grouping of keypoint queries and adds D-FINE's hybrid encoder, pre-pose layer, and fine-grained distribution refinement heads. Its new training technique, pose denoising, perturbs ground-truth keypoints by a controlled keypoint-similarity radius, labels high-similarity samples as positive queries and low-similarity samples as negative queries, and teaches the decoder to refine noisy keypoints during training at no inference cost. A Pose-LQE head uses features sampled at the predicted keypoints to sharpen classification, and the Keypoint Similarity VariFocal loss ties predicted confidence to keypoint similarity. On COCO test-dev the authors report DETRPose-L at 72.5 AP with 32.5 ms latency on a V100, and on CrowdPose DETRPose-X reaches 75.1 AP, ahead of every compared model except an ED-Pose variant trained with additional data.
Load-bearing premise
The load-bearing premise is that the reported speed and accuracy numbers are comparable across the different GPUs and inference-software versions used for DETRPose and its baselines; if that comparison is not apples-to-apples, the real-time and matching-YOLO conclusions could fail.
Editorial extensions
If this is right
- Transformer-based pose estimators can compete with YOLO-style one-stage detectors on both accuracy and latency, removing a major objection to end-to-end DETR designs.
- Training real-time pose models becomes dramatically cheaper: DETRPose converges in 48 to 96 epochs where RTMO needs hundreds, so iterating on datasets or deployment scenarios is faster.
- Because all keypoints are estimated simultaneously in one forward pass, latency stays roughly constant as the number of people grows, unlike top-down two-stage methods whose cost scales with person count.
- Pose denoising is a training-only addition, so the accuracy and convergence gains come without any increase in inference time or parameter count.
- On crowded scenes, DETRPose-X's 75.1 AP on CrowdPose exceeds all compared baselines except an ED-Pose model trained with extra data, using 82.3M parameters versus 218M.
Reading between the lines
- The same keypoint-similarity denoising recipe should transfer to other keypoint tasks—face alignment, hand pose, animal pose—where a ground-truth keypoint and a similarity radius can be defined, potentially giving them the same convergence speedup.
- If the speed advantage survives a unified benchmark on one GPU and one inference-engine version, attention-based decoders may become the default for real-time keypoint tasks, putting pressure on YOLO-style detectors to adopt transformer decoders rather than optimize convolutional heads.
- The KSVF idea of modulating classification with a geometric similarity could be imported back into object detection, using box IoU in place of keypoint similarity, to improve query selection in DETR-style detectors.
- Because DETRPose keeps constant latency as the number of people grows, video pose estimation and multi-person tracking are natural next testbeds; the current model does not estimate boxes, so tracking support would require adding a lightweight box head.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes DETRPose, a family of transformer-based end-to-end multi-person pose estimators. The method modifies the GroupPose decoder with D-FINE-style pre-pose and FDR layers, introduces a pose-specific denoising procedure, an auxiliary Pose-LQE classification layer, and a Keypoint Similarity VariFocal (KSVF) loss. The models are evaluated on COCO test-dev, CrowdPose, and COCO val, with claims of real-time latency, competitive accuracy, and 5–10x faster training convergence than RTMO. The advertised headline is that DETRPose-S matches YOLOv8-Pose-X and YOLO11-Pose-X on COCO at 67.0 AP with 81% fewer parameters and 52% faster inference.
Significance. If the reported numbers are corrected and independently verified, the paper would be a useful empirical contribution: it demonstrates that a DETR-like decoder can be made fast enough for real-time multi-person pose estimation, and the 5–10x training-convergence speedup is practically valuable. The code release, the multi-dataset evaluation, and the detailed ablation study are strengths. However, the central advertised comparison is internally inconsistent, and the benchmarking protocol is not clearly specified, which prevents acceptance as written.
major comments (5)
- [Abstract vs. Appendix A, Table 5] The abstract's headline claim is contradicted by the paper's own Table 5. The abstract states DETRPose-S matches YOLOv8-Pose-X and YOLO11-Pose-X with AP 67.0 vs 67.3 and 67.2, but Table 5 lists YOLOv8-X at AP 69.2 and YOLO11-X at AP 69.5. The Appendix A text itself says DETRPose-M surpasses all YOLO-based models except YOLOv11-X by 0.1 AP, which agrees with Table 5 (69.4 vs 69.5) and not with the abstract. This is a load-bearing contradiction: by the paper's own data, DETRPose-S trails the YOLO-X models by 2.2–2.5 AP. The authors must reconcile the abstract and Table 5 and restate the accuracy comparison that is actually supported.
- [Section 4.1, Table 5, Fig. 4] The latency comparison is not apples-to-apples. Main-text latencies are measured with TensorRT 8.6.4 on a V100, while Table 5 and Fig. 4 use TensorRT 10.11 on an A100. The abstract's YOLO baseline latencies of 5.23 ms and 4.93 ms appear in neither Table 5 nor Fig. 4 (Table 5 reports 6.42 ms and 6.48 ms for YOLOv8-X and YOLO11-X, respectively). The claimed 52% speed advantage therefore cannot be reproduced from the reported measurements. Every latency claim should be tied to one explicit GPU/TensorRT protocol, or the comparison should be restricted to runs made under the same protocol.
- [Table 3, CrowdPose row for DETRPose-S] The CrowdPose APH value for DETRPose-S is reported as 5.93, which is implausible given that all other models in Table 3 have APH values between roughly 59 and 68. This appears to be a typo (likely 59.3 or 60.9), but as printed it invalidates the CrowdPose comparison for that row and must be corrected.
- [Abstract and body (OCHuman)] The abstract claims that "On the OCHuman dataset, DETRPose-S surpasses all previous models," but no OCHuman experiments, result table, evaluation protocol, or comparison appear anywhere in the submitted manuscript. Either add the missing OCHuman evaluation with full details, or remove this claim from the abstract.
- [Appendix B] Appendix B states that "DETRPose has equal or less GFLOPS than YOLOv8, but it has a higher latency," which is in direct tension with the abstract's framing that DETRPose is 52% faster than YOLOv8-X/YOLO11-X. The Appendix B observation is consistent with Table 5 for most model sizes (e.g., DETRPose-M at 3.67 ms vs YOLOv8-M at 2.85 ms), so the abstract should be revised to identify precisely which DETRPose variant achieves which speed advantage over which YOLO baseline, rather than implying a family-wide speed superiority.
minor comments (6)
- [Table 1] The column header contains a stray 'D' before DETRPose-L; it should read 'DETRPose-L'.
- [Table 4 and text after it] The text states that the final configuration decreased GFLOPs by 10%, but Table 4 shows a decrease from 115.2 to 107.2 GFLOPs, which is about 6.9%; please reconcile the percentage or the baseline used.
- [Introduction vs. abstract] The introduction reports a 12.5x training speedup relative to RTMO (48 vs 700 epochs), while the abstract says 5–10 times fewer epochs; with 700/48 ≈ 14.6, the two statements are inconsistent and should be aligned.
- [Abstract vs. Table 5 (parameters)] The abstract lists DETRPose-S as having 11.5M parameters, while Table 5 reports 11.9M; please make the parameter counts consistent.
- [Fig. 2 caption] The caption says 'the right image' twice when describing the proposed pose denoising and the box denoising; based on the figure and the surrounding text, one of these should say 'left image.'
- [Appendix A first paragraph] The paragraph begins with the stray character '6' before the sentence 'We provide comparisons against YOLO-based methods'; this appears to be a numbering artifact and should be removed.
Circularity Check
No significant circularity: DETRPose is an empirical architecture paper evaluated on external benchmarks.
full rationale
The derivation chain in DETRPose is not circular. The paper's contributions are architectural and training-technical: a modified GroupPose-style decoder, a keypoint denoising scheme, a Pose-LQE classification layer, and a Keypoint Similarity VariFocal loss. The denoising construction (Eqs. 1-4) defines a similarity metric and samples perturbations from it; this is a training-data augmentation strategy, not a quantity fitted to the benchmark results that are later reported. The query-selection, decoder, and loss procedures are specified before evaluation and are not tuned against the COCO, CrowdPose, or OCHuman test metrics in a way that would make the reported numbers predictions of their own inputs. The paper does not invoke a uniqueness theorem or a load-bearing self-citation chain; its baselines are external published methods, and its own models are evaluated on held-out public benchmarks. The inconsistent numbers between the abstract and Appendix A Table 5 (e.g., YOLOv8-X and YOLO11-X AP values) are a correctness and reproducibility concern, not a circularity concern, because the comparison is to external models rather than to quantities constructed from the paper's own outputs. Therefore, no circular step is present, and the score is 0.
Assumptions & free parameters
free parameters (6)
- lambda_box =
0.5
- KS sampling ranges =
positive U(0.5,1), negative U(0.1,0.5)
- kappa_i (fall-off per keypoint) =
not specified
- alpha and gamma in KSVF loss =
not specified
- s (scaling factor) =
not specified
- top-k pixels =
not specified
assumptions (5)
- domain assumption GroupPose decoder is an effective base for pose estimation and can be modified for real-time use.
- domain assumption D-FINE backbone and hybrid encoder pretrained on Object365 provide a strong feature extractor for pose estimation.
- domain assumption Training with square 640x640 images is sufficient for competitive pose estimation.
- domain assumption COCO and CrowdPose evaluation protocols are reliable and follow standard practice.
- domain assumption The KS metric with Gaussian form is a valid measure of keypoint similarity for generating denoising queries.
Cite this review
Pith. "Pith review of DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints." pith.science (2026). https://pith.science/paper/XEDMTC6U
@misc{pith2026250613027,
author = {Pith},
title = {Pith review of: DETRPose: Real-Time End-to-End Multi-Person Pose Estimation via Modified Transformer Decoder and Novel Denoising Keypoints},
year = {2026},
howpublished = {\url{https://pith.science/paper/XEDMTC6U}},
note = {Machine review of arXiv:2506.13027}
}
abstract
Multi-person pose estimation (MPPE), which involves detecting body joint positions (keypoints) for every person in an image, is a fundamental task in computer vision. Despite recent advances, no transformer-based model currently achieves real-time performance. This work addresses the latency challenge by introducing DETRPose, the first family of real-time, end-to-end transformer models for multi-person 2D pose estimation. DETRPose significantly enhances the GroupPose decoder, enabling real-time inference. For training, a novel denoising keypoint technique is proposed to accelerate convergence. The varifocal loss is also extended for keypoints, termed Keypoint Similarity VariFocal loss, to improve query quality. Extensive evaluation demonstrates that DETRPose models achieve accuracy comparable to or exceeding that of leading alternatives while requiring five to ten times fewer training epochs. DETRPose-S matches the accuracy of YOLOv8-Pose-X and YOLO11-Pose-X on the COCO dataset (67.0 vs 67.3 and 67.2 in AP) with 81% fewer parameters (11.5M vs 69.4M and 58.8M) and 52% faster inference speed (2.39ms vs 5.23ms and 4.93ms). On the CrowdPose dataset, DETRPose-X has $13.1\times$ fewer FLOPs (232.3G vs 3048.1G) and only $2%$ fewer precision (75.1 vs 76.6 in AP) than ED-Pose-SwinL-5S. On the OCHuman dataset, DETRPose-S surpasses all previous models, showing the robustness of DETRPose on out-of-distribution datasets. Code is available at https://github.com/SebastianJanampa/DETRPose
Figures
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
End-to-end object detection with transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Proceedings of the European Conference on Computer Vision, pages 213--229, 2020
work page 2020
-
[3]
Lw-detr: A transformer replacement to yolo for real-time detection
Qiang Chen, Xiangbo Su, Xinyu Zhang, Jian Wang, Jiahui Chen, Yunpeng Shen, Chuchu Han, Ziliang Chen, Weixiang Xu, Fanrong Li, et al. Lw-detr: A transformer replacement to yolo for real-time detection. arXiv preprint arXiv:2406.03459, 2024
arXiv 2024
-
[4]
Ultralytics yolo11, 2024
Glenn Jocher and Jing Qiu. Ultralytics yolo11, 2024
2024
-
[5]
Ultralytics yolov8, 2023
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralytics yolov8, 2023
2023
-
[6]
Dn-detr: Accelerate detr training by introducing query denoising
Feng Li, Hao Zhang, Shilong Liu, Jian Guo, Lionel M Ni, and Lei Zhang. Dn-detr: Accelerate detr training by introducing query denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13619--13627, 2022
work page 2022
-
[7]
Crowdpose: Efficient crowded scenes pose estimation and a new benchmark
Jiefeng Li, Can Wang, Hao Zhu, Yihuan Mao, Hao-Shu Fang, and Cewu Lu. Crowdpose: Efficient crowded scenes pose estimation and a new benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10863--10872, 2019
work page 2019
-
[8]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \'a r, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Proceedings of the European Conference on Computer Vision, pages 740--755, 2014
work page 2014
Show all 26 references
-
[9]
Group pose: A simple baseline for end-to-end multi-person pose estimation
Huan Liu, Qiang Chen, Zichang Tan, Jiangjiang Liu, Jian Wang, Xiangbo Su, Xiaolong Li, Kun Yao, Junyu Han, Errui Ding, Yao Zhao, and Jingdong Wang. Group pose: A simple baseline for end-to-end multi-person pose estimation. In Proceedings of the IEEE International Conference on...
2023
-
[10]
RTMO : Towards high-performance one-stage real-time multi-person pose estimation, 2023
Peng Lu, Tao Jiang, Yining Li, Xiangtai Li, Kai Chen, and Wenming Yang. RTMO : Towards high-performance one-stage real-time multi-person pose estimation, 2023
2023
-
[11]
Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss
Debapriya Maji, Soyeb Nagori, Manu Mathew, and Deepak Poddar. Yolo-pose: Enhancing yolo for multi person pose estimation using object keypoint similarity loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2637--2646, 2022
2022
-
[12]
Fcpose: Fully convolutional multi-person pose estimation with dynamic instance-aware convolutions
Weian Mao, Zhi Tian, Xinlong Wang, and Chunhua Shen. Fcpose: Fully convolutional multi-person pose estimation with dynamic instance-aware convolutions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9034--9043, 2021
2021
-
[13]
Rethinking keypoint representations: Modeling keypoints and poses as objects for multi-person human pose estimation
William McNally, Kanav Vats, Alexander Wong, and John McPhee. Rethinking keypoint representations: Modeling keypoints and poses as objects for multi-person human pose estimation. arXiv preprint arXiv:2111.08557, 2021
2021 arXiv
-
[14]
D-fine: Redefine regression task in detrs as fine-grained distribution refinement, 2024
Yansong Peng, Hebei Li, Peixi Wu, Yueyi Zhang, Xiaoyan Sun, and Feng Wu. D-fine: Redefine regression task in detrs as fine-grained distribution refinement, 2024
2024
-
[15]
Isaac Robinson, Peter Robicheaux, and Matvei Popov. Rf-detr. https://github.com/roboflow/rf-detr, 2025. SOTA Real-Time Object Detection Model
2025
-
[16]
Objects365: A large-scale, high-quality dataset for object detection
Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Jing Li, Xiangyu Zhang, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 8425--8434, 2019
2019
-
[17]
Inspose: instance-aware networks for single-stage multi-person pose estimation
Dahu Shi, Xing Wei, Xiaodong Yu, Wenming Tan, Ye Ren, and Shiliang Pu. Inspose: instance-aware networks for single-stage multi-person pose estimation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 3079--3087, 2021
2021
-
[18]
End-to-end multi-person pose estimation with transformers
Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan. End-to-end multi-person pose estimation with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11069--11078, 2022
2022
-
[19]
Deim: Detr with improved matching for fast convergence, 2025
Xiaodong Cun Yongjun Yu Xiao Zhou Shihua Huang, Zhichao Lu and Xi Shen. Deim: Detr with improved matching for fast convergence, 2025
2025
-
[20]
Directpose: Direct end-to-end multi-person pose estimation
Zhi Tian, Hao Chen, and Chunhua Shen. Directpose: Direct end-to-end multi-person pose estimation. arXiv preprint arXiv:1911.07451, 2019
1911 arXiv
-
[21]
Contextual instance decoupling for robust multi-person pose estimation
Dongkai Wang and Shiliang Zhang. Contextual instance decoupling for robust multi-person pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11060--11068, 2022
2022
-
[22]
Querypose: Sparse multi-person pose regression via spatial-aware part-level query
Yabo Xiao, Kai Su, Xiaojuan Wang, Dongdong Yu, Lei Jin, Mingshu He, and Zehuan Yuan. Querypose: Sparse multi-person pose regression via spatial-aware part-level query. In Advances in Neural Information Processing Systems, pages 1--14, 2022
2022
-
[23]
Explicit box detection unifies end-to-end multi-person pose estimation
Jie Yang, Ailing Zeng, Shilong Liu, Feng Li, Ruimao Zhang, and Lei Zhang. Explicit box detection unifies end-to-end multi-person pose estimation. In International Conference on Learning Representations, pages 1--17, 2023
2023
-
[24]
Dino: Detr with improved denoising anchor boxes for end-to-end object detection
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel Ni, and Harry Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. In International Conference on Learning Representations, pages 1--18, 2022
2022
-
[25]
Detrs beat yolos on real-time object detection, 2023
Yian Zhao, Wenyu Lv, Shangliang Xu, Jinman Wei, Guanzhong Wang, Qingqing Dang, Yi Liu, and Jie Chen. Detrs beat yolos on real-time object detection, 2023
2023
-
[26]
a henb \
Xingyi Zhou, Dequan Wang, and Philipp Kr \"a henb \"u hl. Objects as points. arXiv preprint arXiv:1904.07850, 2019
1904 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.