REVIEW 3 major objections 6 minor 31 references
Self-supervised One-Stage Learning for RF-based Multi-Person Pose Estimation
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read A lightweight one-stage transformer can estimate multi-person poses directly from raw RF signals, outperforming larger raw-RF baselines by up to 15 PCKh@0.5 points in untrained rooms while using about 2% of the parameters.
desk verdict A genuinely new one-stage RF-based MPPE architecture with a sensible SSL scheme, but the evaluation rests on unvalidated pseudo-labels, so the headline gains are unanchored to true pose accuracy. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is channel subgroup patch embedding: the 64 RF channels are arranged into four subgroups of 16, each subgroup is embedded by the same 1D CNN into patches, and the patches are concatenated with fixed positional embeddings before entering a four-block transformer encoder. This is what lets a small model replace a large deep CNN: the subgroup structure makes the attention's job easier than embedding all 64 channels at once. The second mechanism is the self-supervised objective, a siamese encoder pair with shared weights; one unmasked subgroup provides the key/value memory, masked patches from the other subgroups form the queries, and a transformer decoder with an extra self-attention block reconstructs the masked latent patches against a no-gradient target from a fully unmasked pass. The reconstruction is computed at the latent-feature level, not the raw-signal level, and an ablation shows that both the cross-attention and the latent-level target contribute to the final gain.
What would settle it
Take a held-out set of RF frames from an untrained room, collect human-annotated or motion-captured keypoints for the same moments, and recompute PCKh@0.5 for RPET, OS, SOS, and the image-based teacher on identical frames; the central claim fails if SOS's clean-label score is not close to the teacher's or does not beat RPET by a similar margin.
Extended reading notes
Core claim
The central claim is that sub-grouping raw RF channels is the key to making one-stage RF-based pose estimation work: instead of feeding all 64 signals at once through a deep CNN, the model embeds each subgroup of 16 channels with a shared single-layer CNN and then lets a transformer encoder and multi-head attention integrate the subgroups. The one-stage design uses 15 learnable person queries in a transformer decoder; each query is matched to a person by bipartite matching and predicts class and all keypoints, so training needs keypoint labels only. The self-supervised pretraining step takes one unmasked subgroup as an anchor, masks 75% of the patches in the remaining subgroups, and uses a cross-attention decoder to reconstruct the masked patches at the latent-feature level, with reconstruction targets produced by an unmasked pass through the same encoder. After fine-tuning for pose estimation, SOS outperforms OS on every reported joint, with the largest improvements for wrists, new rooms, obstacles, and four-person scenes. The authors report a reduction from about 94 million to 1.3 million learnable parameters.
Load-bearing premise
The whole training and evaluation pipeline depends on pseudo-keypoint labels produced by an image-based pose model applied to RGB frames (Section 4.3); if those labels are biased for side views, occluded people, or unusual poses, the reported PCKh@0.5 numbers measure agreement with that teacher rather than anatomical accuracy, and no human-annotated validation set is provided.
Editorial extensions
If this is right
- RF-based multi-person pose estimation can be a one-stage, end-to-end pipeline: no separate person detector, no ROI cropping, and no non-maximum suppression.
- Because the model uses about 2% of the parameters of the prior raw-RF method and roughly one-sixth the training time, it is realistic for embedded or battery-powered radar systems.
- Self-supervised pretraining on masked RF subgroups improves accuracy most in precisely the situations where RF sensing is valuable: new rooms, occluding obstacles in front of the antennas, and larger numbers of people.
- The model degrades gracefully when fewer subgroups are supplied at inference, so a radar with a reduced or partially failed antenna array can still estimate poses with limited accuracy loss.
- Releasing the dataset and code makes raw-RF MPPE a reproducible benchmark, which has been missing compared with image-based pose estimation.
Reading between the lines
- Editorial inference: the reported numbers are only as trustworthy as the pseudo-keypoint labels produced by an image-based model; a human-annotated evaluation set could shift absolute PCKh values, though it would not necessarily change the ranking of the compared RF methods.
- Editorial inference: because the subgroups are arbitrary partitions of the 64 channels, the same architecture could treat each physical antenna pair as a subgroup, which would let the method scale to other array geometries or heterogeneous sensors without architectural change.
- Editorial inference: the cross-attention between one clean subgroup and masked others resembles a feature-distillation loop; a direct test would be to measure how SOS's gains scale when the mask ratio or the number of subgroups is varied.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a one-stage, end-to-end trainable model for multi-person pose estimation (MPPE) from raw ultra-wideband radar signals. The method divides the 64 RF channels into subgroups, embeds each subgroup with a shared single-layer 1D CNN, and then uses a transformer encoder and a query-based decoder to produce person keypoints directly. A self-supervised pretraining step reconstructs masked latent feature patches using cross-attention between one unmasked subgroup and masked other subgroups, with the target being the unmasked full-encoder output. Experiments on a dataset collected with an 8x8 antenna array report that the proposed OS (one-stage) and SOS (self-supervised one-stage) models outperform the prior raw-RF method RPET by up to 15 PCKh@0.5 points in a new-room scenario, while using about 2% of RPET's learnable parameters. The authors release their code and dataset.
Significance. If the reported results are reliable, this is a meaningful advance: it demonstrates that a lightweight, one-stage architecture can beat a much larger two-stage baseline on raw RF signals, and the proposed subgroup-embedding scheme plus self-supervised cross-attention pretraining appears to yield consistent gains across scenarios. The open-source release of the dataset and code is a valuable community resource. However, the empirical claims hinge on the quality of the pseudo-labels used for both training and evaluation, which are generated by an image-based model (Person-in-WiFi) without human verification; this threatens the external validity of the reported PCKh numbers and, in particular, the magnitude of the claimed improvement over RPET.
major comments (3)
- [Section 7.1, Table 6] The pseudo-label validity is the load-bearing concern. Section 4.3 states that keypoint labels for pose estimation are obtained by running Person-in-WiFi [26] on RGB frames, and all evaluation in Tables 6-11 computes PCKh@0.5 against these same pseudo-labels. Because every compared model (Baseline, RPET, OS, SOS) is trained to imitate the same teacher, the reported numbers measure agreement with the teacher's predictions, not independently verified anatomical accuracy. If the teacher is systematically biased for side views, occluded people, or unfamiliar rooms, the relative gains of SOS over RPET could reflect better imitation of the teacher rather than better pose estimation. The paper provides no human-annotated validation set, so the absolute accuracy of OS/SOS and the claimed up-to-15-point improvement are unanchored. The authors should add a human-annotated evaluation on a subset of test frames (or, at minimum, report per-joint agreement with the teacher separately) to support the central empirical claim.
- [Section 7] The subgroup-size hyperparameter is selected on the test scenario itself. Table 6 reports OS performance for 16, 4, and 1 subgroup(s) in test scenario (A), and the paper then adopts the best configuration (four subgroups of 16 signals) as the OS model. Since the selection is made on the same test scenario used to report final results, the reported numbers are optimistic. The gap between the chosen configuration (83.7) and the runner-up (82.2) is small but non-negligible. The authors should either use a validation split for this choice or clearly disclose the selection process; adding multiple seeds and error bars would also help assess whether the differences are meaningful.
- [Section 7] The evaluation metric PCKh@0.5 is incompletely specified. The paper defines the threshold as 'half of the head size' but does not define how head size is computed for the pseudo-labels, nor does it state the keypoint set (number of keypoints, K) or the exact keypoint definitions used. This is not merely a presentation issue: without a precise head-size normalization and keypoint set, the absolute PCKh values are not reproducible, and comparisons with RPET, which may use a different keypoint convention, are difficult to interpret. The authors should specify the keypoint set, the head-size definition (e.g., the distance between specific keypoints), and the coordinate normalization (e.g., pixel coordinates or normalized coordinates).
minor comments (6)
- [Figure 5] The figure contains a typo: 'Psudolabel' should be 'Pseudo label'.
- [Section 3.2] The notation for the patch embedding is inconsistent: the text first says x in R^{C x S} is transformed to x_p in R^{N x (c x s)}, but later says x is transformed to x in R^{(C/N) x N x S}. Please clarify the exact tensor shape and variable meanings.
- [Section 4.1] The hardware description says '8 pairs of Rx and Tx antennas' but also refers to an '8x8 antenna' array; clarify that each Rx receives from all Tx, yielding 64 channels, so the reader does not infer only 8 channels.
- [Section 7] The description of the Baseline and RPET models is brief; since the paper compares against these methods, please report their training details (e.g., exact architecture, loss weights, and whether they were retrained with the same data split and augmentations) to ensure a fair comparison.
- [Tables 8 and 9] The tables report mean PCKh @0.5 without any variance or number of test frames; adding standard deviations and frame counts would help gauge the reliability of the differences between models.
- [Section 3.3] The self-supervised target is described as the unmasked output of the RF encoder, but the relation between the siamese encoder and the target encoder should be stated more explicitly (the text notes stop-gradient, which is good; please make clear that the target encoder shares weights and the target is detached).
Circularity Check
No significant circularity; the empirical claims are evaluated on held-out scenarios against an external pseudo-label teacher, and the cited prior work is used as a baseline, not as a load-bearing proof step.
full rationale
The paper's derivation chain is empirical rather than mathematical, and no equation or construction reduces the final pose-estimation claim to its own inputs. The supervised training and evaluation labels are pseudo-keypoint labels obtained by running the external image-based model Person-in-WiFi [26] on RGB frames (Section 4.3: 'we utilized the image collection and performed inference using [26] to obtain pseudo keypoint labels'). Because the proposed model is trained on RF signals and evaluated against an independent image-based teacher, the reported PCKh numbers measure agreement with that teacher, but this is a standard cross-modal supervision setup and not a self-definitional loop. The self-supervised pretraining target is the unmasked output of the same RF encoder with stop-gradient (Section 3.3), yet the paper's reported claim is downstream MPPE accuracy after fine-tuning on the pseudo-labels, so the SSL component is not circularly evaluated on its own reconstruction loss. The paper does cite the authors' prior work [14] for the dataset and the RPET baseline, and the dataset is the same as in [14], but this is a descriptive setup choice rather than a load-bearing citation of an unverified uniqueness theorem or ansatz. All compared methods are trained on the same labels and evaluated on held-out test scenarios (different antenna locations, obstacles, and a new room), so the relative comparisons are meaningful for the stated empirical claim. The lack of human-annotated validation is a correctness or label-quality caveat, not a circularity defect, and the subgroup-size selection on the test scenario is a potential overfitting concern rather than a circular derivation.
Assumptions & free parameters
free parameters (5)
- Channel subgroup configuration =
4 subgroups of 16 signals each
- SSL mask ratio =
0.75
- Loss weights =
lambda_cls=1, lambda_pose=50
- Number of encoder/decoder blocks, embedding dimension =
4 blocks, dim 128, 4 heads, 2 decoder blocks
- Number of person queries =
15
assumptions (3)
- domain assumption Reflected UWB RF signals from an 8x8 antenna array contain sufficient spatial information to estimate 2D poses of up to 4 people.
- domain assumption Pseudo-keypoint labels generated by an image-based model (Person-in-WiFi [26]) are accurate enough to serve as ground truth for training and evaluating pose estimators.
- domain assumption The self-collected dataset of 176,000 frames from 12 volunteers at four locations generalizes to new rooms, new radar positions, and obstacles.
Cite this review
Pith. "Pith review of Self-supervised One-Stage Learning for RF-based Multi-Person Pose Estimation." pith.science (2026). https://pith.science/paper/W5AEK2VC
@misc{pith2026250605420,
author = {Pith},
title = {Pith review of: Self-supervised One-Stage Learning for RF-based Multi-Person Pose Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/W5AEK2VC}},
note = {Machine review of arXiv:2506.05420}
}
read the original abstract
In the field of Multi-Person Pose Estimation (MPPE), Radio Frequency (RF)-based methods can operate effectively regardless of lighting conditions and obscured line-of-sight situations. Existing RF-based MPPE methods typically involve either 1) converting RF signals into heatmap images through complex preprocessing, or 2) applying a deep embedding network directly to raw RF signals. The first approach, while delivering decent performance, is computationally intensive and time-consuming. The second method, though simpler in preprocessing, results in lower MPPE accuracy and generalization performance. This paper proposes an efficient and lightweight one-stage MPPE model based on raw RF signals. By sub-grouping RF signals and embedding them using a shared single-layer CNN followed by multi-head attention, this model outperforms previous methods that embed all signals at once through a large and deep CNN. Additionally, we propose a new self-supervised learning (SSL) method that takes inputs from both one unmasked subgroup and the remaining masked subgroups to predict the latent representations of the masked data. Empirical results demonstrate that our model improves MPPE accuracy by up to 15 in PCKh@0.5 compared to previous methods using raw RF signals. Especially, the proposed SSL method has shown to significantly enhance performance improvements when placed in new locations or in front of obstacles at RF antennas, contributing to greater performance gains as the number of people increases. Our code and dataset is open at Github. https://github.com/sshnan7/SOSPE .
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[26]
Fei Wang, Sanping Zhou, Stanislav Panev, Jinsong Han, and Dong Huang. 2019. Person-in-WiFi: Fine-grained person perception using WiFi. In Proceedings of the IEEE/CVF International Conference on Computer Vision . 5452–5461
work page 2019
-
[1]
Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3D tracking via body radio reflections. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14) . 317–329
work page 2014
-
[2]
Alexei Baevski, Arun Babu, Wei-Ning Hsu, and Michael Auli. 2023. Efficient self-supervised learning with contextualized target representations for vision, speech and language. In International Conference on Machine Learning . PMLR, 1416–1429
work page 2023
-
[3]
Alexei Baevski, Wei-Ning Hsu, Qiantong Xu, Arun Babu, Jiatao Gu, and Michael Auli. 2022. Data2vec: A general framework for self-supervised learning in speech, vision and language. In International Conference on Machine Learning . PMLR, 1298–1312
2022
-
[4]
Navaneeth Bodla, Bharat Singh, Rama Chellappa, and Larry S Davis. 2017. Soft- NMS–improving object detection with one line of code. InProceedings of the IEEE international conference on computer vision . 5561–5569
work page 2017
-
[5]
Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2017. Realtime multi- person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7291–7299
work page 2017
-
[6]
Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexan- der Kirillov, and Sergey Zagoruyko. 2020. End-to-end object detection with transformers. In European conference on computer vision . Springer, 213–229
2020
-
[7]
Xinlei Chen and Kaiming He. 2021. Exploring simple siamese representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 15750–15758
2021
Show all 31 references
-
[8]
Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S Huang, and Lei Zhang. 2020. Higherhrnet: Scale-aware representation learning for bottom-up human pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5386–5395
2020
-
[9]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv prepri...
2020 arXiv
-
[10]
Agrim Gupta, Jiajun Wu, Jia Deng, and Li Fei-Fei. 2023. Siamese Masked Autoen- coders. arXiv preprint arXiv:2305.14344 (2023)
2023 arXiv
-
[11]
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick
-
[12]
Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. 2017. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision . 2961–2969
2017
-
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition . 770–778
2016
-
[14]
Seunghyun Kim, Seunghwan Shin, Sangwon Lee, Kaewon Choi, and Yusung Kim. 2024. Learning Visual Clue for UWB-based multi-person pose estimation. Knowledge-Based Systems 284 (2024), 111289
2024
-
[15]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classifi- cation with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)
2012
-
[16]
Ke Li, Shijie Wang, Xiang Zhang, Yifan Xu, Weijian Xu, and Zhuowen Tu. 2021. Pose recognition with cascade transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 1944–1953
2021
-
[17]
Huan Liu, Qiang Chen, Zichang Tan, Jiang-Jiang Liu, Jian Wang, Xiangbo Su, Xiaolong Li, Kun Yao, Junyu Han, Errui Ding, et al. 2023. Group Pose: A Simple Baseline for End-to-End Multi-person Pose Estimation. In Proceedings of the IEEE/CVF International Conference on Computer V...
2023
-
[18]
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin transformer: Hierarchical vision transformer us- ing shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision. 10012–10022
2021
-
[19]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 11976–11986
2022
-
[20]
Alejandro Newell, Zhiao Huang, and Jia Deng. 2017. Associative embedding: End- to-end learning for joint detection and grouping. Advances in neural information processing systems 30 (2017)
2017
-
[21]
Dahu Shi, Xing Wei, Liangqi Li, Ye Ren, and Wenming Tan. 2022. End-to-end multi-person pose estimation with transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition . 11069–11078
2022
-
[22]
Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
2014 arXiv
-
[23]
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. 2019. Deep high-resolution representation learning for human pose estimation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition . 5693–5703
2019
-
[24]
Taylor and Dag T
James D. Taylor and Dag T. Wisland. 2017. Novelda Nanoscale Impulse Radar
2017
-
[25]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)
2017
-
[27]
Bin Xiao, Haiping Wu, and Yichen Wei. 2018. Simple baselines for human pose estimation and tracking. In Proceedings of the European conference on computer vision (ECCV). 466–481
2018
-
[28]
Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. 2022. Vitpose: Simple vision transformer baselines for human pose estimation. Advances in Neural Information Processing Systems 35 (2022), 38571–38584
2022
-
[29]
Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. 2018. Through-wall human pose estimation using radio signals. In Proceedings of the IEEE conference on computer vision and pattern recognition . 7356–7365
2018
-
[30]
Zhijie Zheng, Jun Pan, Zhikang Ni, Cheng Shi, Diankun Zhang, Xiaojun Liu, and Guangyou Fang. 2022. Recovering human pose and shape from through-the-wall radar images. IEEE Transactions on Geoscience and Remote Sensing 60 (2022), 1–15
2022
-
[2022]
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition
Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 16000–16009
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.