REVIEW 3 major objections 3 minor 32 references
Periodic-MAE: Periodic Video Masked Autoencoder for rPPG Estimation
T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Periodic-MAE claims that self-supervised pretraining with rhythm-aware frame masking and frequency bandlimit losses improves rPPG heart-rate estimation, especially on unseen datasets.
desk verdict A novel periodic-masking idea that is worth a proper look, but the current manuscript has unfinished text and its cross-dataset evidence is contaminated by pre-training on the test distributions. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is periodic frame masking: frames are removed at a regular step size t randomly chosen from 2 to 5 in each epoch, simulating different downsampling rates and thus different effective pulse rhythms during reconstruction. This is paired with a VideoMAE-style asymmetric encoder–decoder that reconstructs masked RGB tokens, and with two frequency-domain losses (Lb and Ls) computed from a 1D FFT of the decoded signal: Lb penalizes spectral power outside 0.66–3 Hz, while Ls penalizes power within that band that lies away from the detected spectral peak. A fusion stem that concatenates raw frames with frame differences focuses attention on subtle skin-tone changes rather than static facial features.
What would settle it
Retrain Periodic-MAE under the paper's protocol but remove the target dataset from the pretraining corpus (for example, when fine-tuning on PURE and testing on UBFC-rPPG, pretrain only on MMPD and other non-UBFC data) and compare cross-dataset error to RhythmFormer. If the improvements shrink or vanish, the gains come from exposure to unlabeled test-domain video rather than from a generic periodic-signal representation.
Extended reading notes
Core claim
Periodic-MAE demonstrates that a VideoMAE-style pretraining task, applied directly to raw RGB face videos with a periodic frame masking strategy and physiological bandlimit losses, produces a feature extractor that transfers better to rPPG estimation than models trained from scratch or with random, frame, or tube masking. The authors argue that exploiting the rhythmic structure of the pulse during pretraining is what yields generic spatio-temporal representations, evidenced by cross-dataset improvements on PURE, UBFC-rPPG, MMPD, and V4V, and by ablations in which periodic masking and the peak-sparsity loss each contribute to the gains.
Load-bearing premise
The reported cross-dataset gains assume that pretraining on unlabeled videos from the test datasets does not materially improve downstream cross-dataset performance; if that assumption is false, only the V4V results remain a clean test of generic representation learning.
Editorial extensions
If this is right
- The pre-trained encoder can be fine-tuned on a small labeled rPPG dataset while retaining strong generalization to new subjects, cameras, and lighting conditions.
- Directly reconstructing RGB video avoids the STMap preprocessing used by prior rPPG-MAE work, preserving fine-grained spatial detail.
- The periodic masking strategy acts as a data augmentation that exposes the model to a range of pulse rhythms during pretraining.
- The frequency bandlimit losses can be applied as self-supervised constraints in other video pretraining tasks aimed at periodic physiological signals.
- Strong results on V4V, which is not used in pretraining, suggest the learned representations scale to large, diverse, unconstrained facial video corpora.
Reading between the lines
- Because pretraining includes PURE, UBFC-rPPG, and MMPD, the reported cross-dataset numbers on those sets may partly reflect exposure to unlabeled video from the test distribution; V4V is the only fully held-out domain, and the largest relative gains appear there.
- The ablation showing that Lb alone degrades performance while Ls helps suggests the bandwidth restriction is only useful when combined with peak-sparsity guidance, implying that the frequency loss contribution is carried mainly by Ls.
- The step-size range 2–5 combined with 160-frame clips covers a limited set of effective pulse frequencies; a testable extension would be to adapt the masking step to the observed heart-rate distribution of each video rather than choosing it uniformly.
- The same pretraining recipe could plausibly transfer to other quasi-periodic signals from video (respiratory rate, blood pressure), a direction the authors name as future work.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Periodic-MAE, a self-supervised masked autoencoder for learning spatiotemporal representations from unlabeled facial videos, targeting remote photoplethysmography (rPPG) estimation. The method introduces a periodicity-aware frame masking strategy based on video resampling and incorporates frequency-domain bandlimit losses into the MAE pre-training objective. The pre-trained encoder is then fine-tuned for rPPG signal regression. Experiments are reported on PURE, UBFC-rPPG, MMPD, and V4V, with claims of improved performance especially in cross-dataset settings.
Significance. If the claims are supported, the method could offer a useful self-supervised pre-training recipe for rPPG, where labeled data are scarce. The V4V results are encouraging, and the availability of code is a strength. However, the experimental design currently undermines the central claim of learning data-agnostic and generic representations: pre-training uses the same datasets that later serve as test targets, so the reported cross-dataset gains on PURE, UBFC-rPPG, and MMPD are confounded. The V4V dataset is the only clean held-out target, but one dataset is too narrow to support the broad conclusions. The abstract also promises a real-world dataset experiment that is entirely absent from the full text.
major comments (3)
- The cross-dataset evaluation is contaminated. As stated in §5.1, the PURE, UBFC-rPPG, and MMPD datasets are used for pretraining, finetuning, and testing, while V4V is used for testing only. In the cross-dataset protocol of Table 1, the encoder is pre-trained on unlabeled videos from the same datasets that later appear as test domains. Even if only the training splits are used for pre-training, the encoder has been adapted to the target domain's recording conditions, subjects, and compression artifacts, independent of the proposed periodic masking and frequency losses. Therefore, the large improvements on MMPD (e.g., MAE 9.08 to 7.85 when trained on UBFC) cannot be attributed to learning a generic periodic representation. V4V is the only clean held-out set, and while Table 2 shows gains, a single dataset without multiple seeds or error bars is insufficient to support the paper's broad claims of data-agnostic and generic representations.
- The abstract promises evaluation on "a real-world rPPG dataset collected under unconstrained lighting conditions and subject motion," but no such dataset, description, or results appear anywhere in the manuscript. Section 5 describes only PURE, UBFC-rPPG, MMPD, and V4V. This missing promised experiment is a load-bearing discrepancy between the claimed contributions and the actual content, and it cannot be overlooked in a journal submission.
- The manuscript contains a first "Introduction" section filled with placeholder lorem ipsum text ("Nam dui ligula, fringilla a, euismod sodales, ...") and a second "Introduction" section that contains the actual content. This is not a simple typographical error; it shows that the manuscript is an incomplete draft. The paper is not in a state suitable for peer review, and the central technical presentation is compromised by this incomplete state.
minor comments (3)
- Several entries in Table 1 appear to be concatenated without proper spacing or separators, e.g., "0.998.69" and "0.53-7.73", making the values difficult to read. The table should be reformatted to clearly separate metric values, and Section 5.3 refers to "Table 4.2" when it evidently means Table 1.
- The text in Section 5.4 contains two nearly identical paragraphs describing the intra-dataset results, likely a copy-paste error. One of the repetitions should be removed.
- The pre-processing description states that the first frame is selected and fixed for subsequent frames, but the details of how the face region is tracked or whether a fixed bounding box is used for all frames are not specified. Adding these details would aid reproducibility.
Circularity Check
No significant circularity; the pre-training and fine-tuning losses are defined independently of the downstream rPPG targets, and no prediction reduces to a fitted input by construction.
full rationale
The proposed pre-training objective is L_MAE = L_f + L_r, where L_r is an MSE reconstruction (Eq. 3) and L_f = L_b + L_s are frequency-band constraints (Eqs. 1-2) taken from prior external work [20]. These losses are computed on the reconstructed decoder output and impose only the physiological prior that pulse spectra lie in 0.66-3 Hz with a spectral peak padding of Delta_f = 6 bpm; they do not incorporate downstream HR/PPG labels. Downstream rPPG estimation (Section 4.2) uses L_time + lambda_1 L_freq (Eqs. 4-5) on the fine-tuned encoder, so the encoder is not optimized to reproduce the evaluation metric. The periodic masking strategy is a new frame-selection schedule, not a relabeling of the test quantity. No part of the method is defined in terms of the reported MAE, RMSE, rho, or SNR metrics, and no fitted parameter is renamed as a prediction. The only self-references ([17], [32]) supply dataset splits and prior architecture or tooling and are not load-bearing evidence for the central claim. A separate validity concern is that Section 5.1 includes PURE, UBFC-rPPG and MMPD in pre-training, so the cross-dataset numbers in Table 1 may reflect domain-adaptive pre-training rather than transfer to unseen domains; this is an experimental confound, not a circular derivation, and the V4V results in Table 2 provide independent evidence because V4V was not pre-trained. Overall, the derivation is self-contained and not circular.
Assumptions & free parameters
free parameters (4)
- periodic masking step size range =
t in [2,5]
- masking ratio =
50% and 80%
- fine-tuning frequency loss weight =
lambda_1 = 100
- spectral peak padding =
Delta f = 6 bpm
assumptions (3)
- domain assumption Pulse signals are quasi-periodic and sparse in the 0.66-3 Hz band
- domain assumption Reconstructing periodically-masked frames forces representation of pulse periodicity
- ad hoc to paper Pre-training on unlabeled videos from the same datasets later used for fine-tuning and testing does not invalidate cross-dataset claims
Cite this review
Pith. "Pith review of Periodic-MAE: Periodic Video Masked Autoencoder for rPPG Estimation." pith.science (2026). https://pith.science/paper/XB3H4WNM
@misc{pith2026250621855,
author = {Pith},
title = {Pith review of: Periodic-MAE: Periodic Video Masked Autoencoder for rPPG Estimation},
year = {2026},
howpublished = {\url{https://pith.science/paper/XB3H4WNM}},
note = {Machine review of arXiv:2506.21855}
}
read the original abstract
In this paper, we propose Periodic-MAE, a self-supervised framework for learning generalizable spatio-temporal representations of periodic physiological signals from unlabeled facial videos. The proposed method leverages a masked autoencoder (MAE), which learns high-dimensional facial representations by reconstructing masked video tokens without relying on remote photoplethysmography (rPPG) specific supervision. To explicitly align representation learning with the characteristics of rPPG, we introduce a periodicity-aware frame masking strategy based on video resampling, enabling the encoder to learn representations that capture quasi-periodic temporal patterns relevant to pulse signal estimation. In addition, physiological bandlimit constraints are integrated into the MAE pre-training framework, exploiting the sparsity of pulse signals in the frequency domain to guide the learned representations toward physiologically meaningful patterns. After pre-training, the learned representations are transferred to downstream rPPG estimation, where the encoder serves as a generic feature extractor for recovering pulse-related signals from facial videos. We conduct extensive experiments on four benchmark datasets, including PURE, UBFC-rPPG, MMPD, and V4V. Moreover, we evaluate the proposed approach on a real-world rPPG dataset collected under unconstrained lighting conditions and subject motion. Experimental results demonstrate that Periodic-MAE consistently improves rPPG estimation performance, particularly in challenging cross-dataset and real-world evaluation settings. Our code is available at https://github.com/ziiho08/Periodic-MAE.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Detecting pulse from head motions in video,
G. Balakrishnan, F. Durand, and J. Guttag, “Detecting pulse from head motions in video,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2013, pp. 3430–3437
work page 2013
-
[2]
Non-contact, automated cardiac pulse measurements using video imaging and blind source separation
M.-Z. Poh, D. J. McDuff, and R. W. Picard, “Non-contact, automated cardiac pulse measurements using video imaging and blind source separation.”Optics express, vol. 18, no. 10, pp. 10 762–10 774, 2010
work page 2010
-
[3]
Robust pulse rate from chrominance-based rppg,
G. De Haan and V . Jeanne, “Robust pulse rate from chrominance-based rppg,”IEEE transactions on biomedical engineering, vol. 60, no. 10, pp. 2878–2886, 2013
work page 2013
-
[4]
Algorithmic principles of remote ppg,
W. Wang, A. C. Den Brinker, S. Stuijk, and G. De Haan, “Algorithmic principles of remote ppg,”IEEE Transactions on Biomedical Engineering, vol. 64, no. 7, pp. 1479–1491, 2016
work page 2016
-
[5]
An image is worth 16x16 words: Transformers for image recognition at scale,
A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,”arXiv preprint arXiv:2010.11929, 2020
arXiv 2010
-
[6]
A simple framework for contrastive learning of visual representations,
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInternational conference on machine learning. PMLR, 2020, pp. 1597–1607
2020
-
[7]
Masked autoencoders are scalable vision learners,
K. He, X. Chen, S. Xie, Y . Li, P. Dollár, and R. Girshick, “Masked autoencoders are scalable vision learners,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000–16 009
2022
-
[8]
Z. Sun and X. Li, “Contrast-phys+: Unsupervised and weakly-supervised video-based remote physiological measurement via spatiotemporal contrast,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
work page 2024
Show all 32 references
-
[9]
Facial video-based remote physiological measurement via self-supervised learning,
Z. Yue, M. Shi, and S. Ding, “Facial video-based remote physiological measurement via self-supervised learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023
2023
-
[10]
Tranphys: Spatiotemporal masked transformer steered remote photoplethysmography estimation,
H. Shao, L. Luo, J. Qian, S. Chen, C. Hu, and J. Yang, “Tranphys: Spatiotemporal masked transformer steered remote photoplethysmography estimation,”IEEE Transactions on Circuits and Systems for Video Technology, 2023
2023
-
[11]
rppg-mae: Self-supervised pretraining with masked autoencoders for remote physiological measurements,
X. Liu, Y . Zhang, Z. Yu, H. Lu, H. Yue, and J. Yang, “rppg-mae: Self-supervised pretraining with masked autoencoders for remote physiological measurements,”IEEE Transactions on Multimedia, 2024
2024
-
[12]
Videomae: Masked autoencoders are data-efficient learners for self- supervised video pre-training,
Z. Tong, Y . Song, J. Wang, and L. Wang, “Videomae: Masked autoencoders are data-efficient learners for self- supervised video pre-training,”Advances in neural information processing systems, vol. 35, pp. 10 078–10 093, 2022
2022
-
[13]
Remote photoplethysmograph signal measurement from facial videos using spatio- temporal networks,
Z. Yu, X. Li, and G. Zhao, “Remote photoplethysmograph signal measurement from facial videos using spatio- temporal networks,”arXiv preprint arXiv:1905.02419, 2019
1905 arXiv
-
[14]
Physformer: Facial video-based physiological measurement with temporal difference transformer,
Z. Yu, Y . Shen, J. Shi, H. Zhao, P. H. Torr, and G. Zhao, “Physformer: Facial video-based physiological measurement with temporal difference transformer,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4186–4196. 10
2022
-
[15]
Rhythmformer: Extracting patterned rppg signals based on periodic sparse attention,
B. Zou, Z. Guo, J. Chen, J. Zhuo, W. Huang, and H. Ma, “Rhythmformer: Extracting patterned rppg signals based on periodic sparse attention,”Pattern Recognition, vol. 164, p. 111511, 2025
2025
-
[16]
Calibrationphys: Self-supervised video-based heart and respiratory rate measurements by calibrating between multiple cameras,
Y . Akamatsu, T. Umematsu, and H. Imaoka, “Calibrationphys: Self-supervised video-based heart and respiratory rate measurements by calibrating between multiple cameras,”IEEE Journal of Biomedical and Health Informatics, 2023
2023
-
[17]
Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video,
G. Hwang and S. J. Lee, “Phase-shifted remote photoplethysmography for estimating heart rate and blood pressure from facial video,”arXiv preprint arXiv:2401.04560, 2024
2024 arXiv
-
[18]
Autohr: A strong end-to-end baseline for remote heart rate measurement with neural searching,
Z. Yu, X. Li, X. Niu, J. Shi, and G. Zhao, “Autohr: A strong end-to-end baseline for remote heart rate measurement with neural searching,”IEEE Signal Processing Letters, vol. 27, pp. 1245–1249, 2020
2020
-
[19]
And-rppg: A novel denoising-rppg network for improving remote heart rate estimation,
B. Lokendra and G. Puneet, “And-rppg: A novel denoising-rppg network for improving remote heart rate estimation,”Computers in biology and medicine, vol. 141, p. 105146, 2022
2022
-
[20]
Non-contrastive unsupervised learning of physiological signals from video,
J. Speth, N. Vance, P. Flynn, and A. Czajka, “Non-contrastive unsupervised learning of physiological signals from video,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 14 464–14 474
2023
-
[21]
Deepphys: Video-based physiological measurement using convolutional attention networks,
W. Chen and D. McDuff, “Deepphys: Video-based physiological measurement using convolutional attention networks,” inProceedings of the european conference on computer vision (ECCV), 2018, pp. 349–365
2018
-
[22]
Multi-task temporal shift attention networks for on-device contactless vitals measurement,
X. Liu, J. Fromm, S. Patel, and D. McDuff, “Multi-task temporal shift attention networks for on-device contactless vitals measurement,”Advances in Neural Information Processing Systems, vol. 33, pp. 19 400–19 411, 2020
2020
-
[23]
Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement,
X. Liu, B. Hill, Z. Jiang, S. Patel, and D. McDuff, “Efficientphys: Enabling simple, fast and accurate camera-based cardiac measurement,” inProceedings of the IEEE/CVF winter conference on applications of computer vision, 2023, pp. 5008–5017
2023
-
[24]
Non-contact video-based pulse rate measurement on a mobile service robot,
R. Stricker, S. Müller, and H.-M. Gross, “Non-contact video-based pulse rate measurement on a mobile service robot,” inThe 23rd IEEE International Symposium on Robot and Human Interactive Communication. IEEE, 2014, pp. 1056–1062
2014
-
[25]
Unsupervised skin tissue segmentation for remote photoplethysmography,
S. Bobbia, R. Macwan, Y . Benezeth, A. Mansouri, and J. Dubois, “Unsupervised skin tissue segmentation for remote photoplethysmography,”Pattern Recognition Letters, vol. 124, pp. 82–90, 2019
2019
-
[26]
Mmpd: multi-domain mobile video physiology dataset,
J. Tang, K. Chen, Y . Wang, Y . Shi, S. Patel, D. McDuff, and X. Liu, “Mmpd: multi-domain mobile video physiology dataset,” in2023 45th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). IEEE, 2023, pp. 1–5
2023
-
[27]
The first vision for vitals (v4v) challenge for non-contact video-based physiological estimation,
A. Revanur, Z. Li, U. A. Ciftci, L. Yin, and L. A. Jeni, “The first vision for vitals (v4v) challenge for non-contact video-based physiological estimation,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 2760–2767
2021
-
[28]
Siamese-rppg network: Remote photoplethysmography signal estimation from face videos,
Y .-Y . Tsou, Y .-A. Lee, C.-T. Hsu, and S.-H. Chang, “Siamese-rppg network: Remote photoplethysmography signal estimation from face videos,” inProceedings of the 35th annual ACM symposium on applied computing, 2020, pp. 2066–2073
2020
-
[29]
Dual-gan: Joint bvp and noise modeling for remote physiological measurement,
H. Lu, H. Han, and S. K. Zhou, “Dual-gan: Joint bvp and noise modeling for remote physiological measurement,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 12 404–12 413
2021
-
[30]
Efficient remote photoplethysmography with temporal derivative modules and time-shift invariant loss,
J. Comas, A. Ruiz, and F. Sukno, “Efficient remote photoplethysmography with temporal derivative modules and time-shift invariant loss,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 2182–2191
2022
-
[31]
Learning motion-robust remote photoplethysmography through arbitrary resolution videos,
J. Li, Z. Yu, and J. Shi, “Learning motion-robust remote photoplethysmography through arbitrary resolution videos,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 1, 2023, pp. 1334–1342
2023
-
[32]
Lstc-rppg: Long short-term convolutional network for remote photoplethysmography,
J. S. Lee, G. Hwang, M. Ryu, and S. J. Lee, “Lstc-rppg: Long short-term convolutional network for remote photoplethysmography,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 6015–6023. 11
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.