REVIEW 3 major objections 6 minor 39 references
Contactless Cardiac Pulse Monitoring Using Event Cameras
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Event-camera face recordings preserve cardiac pulse well enough for contactless heart-rate monitoring.
desk verdict First face-based event-camera pulse extraction with a CNN, and the code release is real; but the headline 120 FPS advantage over RGB rests on ten subjects and one split, so treat the comparison as preliminary. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the event frame: events, each carrying timestamp, coordinates, and polarity, are binned into non-overlapping windows of fixed duration and summed by polarity into a single-channel 2D image, making the asynchronous event stream compatible with frame-based network architectures. A modified TS-CAN, a two-branch convolutional attention network with temporal shift modules, takes the same single-channel event frame into both branches and learns to output a normalized signal difference that post-processing integrates into a pulse waveform. On the label side, the ECG signal is inverted to emphasize s-wave peaks, smoothed, bandpass filtered, clipped, resampled to the frame rate, and differentiated, so the supervision target is designed to align with the blood-pulse timing the event stream is assumed to encode.
What would settle it
Simultaneously record an event camera, a synchronized RGB camera, and both ECG and fingertip SpO2 from the same subjects; train the same model with ECG-derived labels, then compare its predicted pulse peaks against SpO2 pulse peaks beat by beat. If the event-frame predictions do not track SpO2 timing within a fraction of a cardiac cycle while the RGB model does, the claim that cardiac information is preserved in event streams would be falsified.
Extended reading notes
Core claim
The paper claims that physiological cardiac information in the facial region is effectively preserved within the event stream, and that a simple 2D event-frame representation lets a standard convolutional network recover it. On its local dataset, the event-frame model at 120 FPS reaches RMSE 2.13 bpm, MAE 1.58 bpm, and Pearson correlation 0.97, outperforming the 30 FPS RGB model (RMSE 2.92) and the 60 FPS event model (RMSE 2.54). The 30 FPS event model is slightly worse than RGB but still within the accepted error range for remote photoplethysmography. The authors also report that an event-count-only method from prior work fails on face data, with RMSE above 25 bpm, while their spatial event-frame method does not.
Load-bearing premise
The paper's central claim rests on the assumption that the processed ECG signal used as the training label tracks the timing and shape of the optical blood-volume pulse that the event camera sees in the face; if that mapping is wrong, the reported errors measure something other than cardiac information in the event stream.
Editorial extensions
If this is right
- Contactless pulse monitoring can run on event-camera hardware, which offers microsecond-level latency and lower power than frame-based video capture.
- Existing frame-based remote-PPG network architectures can be reused for event data simply by feeding them temporally binned event frames, lowering the barrier to neuromorphic health sensing.
- Increasing the event-frame sampling rate from 30 to 120 FPS improves heart-rate accuracy in the reported experiments, so shorter temporal bins appear to be preferable for this task.
- The failure of event-count-only methods on face data suggests that spatial information is necessary for face-based event pulse estimation.
- The achieved RMSE values fall within the accepted threshold of 5 bpm for remote-PPG methods, so the accuracy is in the clinically relevant range.
Reading between the lines
- If the finding generalizes to natural head motion, event cameras could be built into vehicle cabins or wearables without some of the illumination constraints that affect RGB remote-PPG, because event cameras have high dynamic range.
- A direct test of the label assumption would be to compare the model's predicted pulse peaks against a simultaneous fingertip SpO2 waveform, whose timing reflects blood arrival in the extremities rather than electrical heart activity.
- Synthetic generation of event streams from public RGB remote-PPG datasets could be used to scale training data and to test whether the model learns the same optical blood-volume signal that RGB methods exploit.
- Beat-by-beat waveform comparison, not just heart-rate error, would clarify whether event frames preserve pulse shape or only periodicity.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper investigates whether event cameras can capture cardiac pulse information from face recordings. The authors collect a proprietary dataset of 64 subjects with simultaneous event, RGB, and ECG data, represent event streams as 2D event frames, and train a modified TS-CAN to predict a processed ECG signal. They compare models trained on RGB frames and on event frames at 30, 60, and 120 FPS, evaluating heart rate via FFT of the post-processed prediction. Reported RMSEs are 2.92 bpm for RGB 30 FPS, and 3.32, 2.54, and 2.13 bpm for event frames at 30, 60, and 120 FPS, respectively. The abstract and conclusion interpret the 120 FPS event result as outperforming the RGB baseline and supporting event cameras as a viable low-power contactless cardiac monitoring sensor.
Significance. If the result holds, this is a useful first demonstration: it extends event-camera vitals monitoring from wrist and chest to the face, uses end-to-end learning rather than hand-crafted event-count features, and provides code and trained weights for reproducibility. The design choices (held-out subject evaluation, no subject exclusion, standard r-PPG error metrics) are appropriate for a feasibility study. The main limitation is statistical: the headline comparison rests on 10 test subjects, one split, and one training run per configuration, with highly variable per-subject errors.
major comments (3)
- [Section 4.4, Tables 3 and 4] The claim that the 120 FPS event model outperforms the RGB baseline (RMSE 2.13 vs. 2.92 bpm) is not statistically established. The evaluation uses a single random split (44/10/10 subjects), one training run per configuration, and a test set of only 10 subjects. Per-subject RMSE in Table 4 varies from 0.00 to 5.05 for the 120 FPS event model, so the aggregate difference could be dominated by one or two subjects. Please report confidence intervals, a paired significance test over subjects, and ideally multiple splits or seeds before drawing the comparative conclusion in the abstract.
- [Section 3.2] The label pre-processing inverts the ECG to emphasize s-wave peaks and then applies smoothing, band-pass filtering, clipping, resampling, and differentiation, but no evidence is given that the resulting signal tracks the facial blood-volume pulse observed by the event camera. Since heart rate is derived from the FFT peak of the post-processed output, waveform phase details may be less critical, but the claim that the model extracts 'physiological cardiac information' would be considerably stronger if the processed ECG were validated against an SpO2 or standard r-PPG reference, or if an ablation without the inversion step were reported.
- [Section 5, Conclusion] The manuscript concludes that event cameras are a viable low-power alternative to RGB for remote heart rate monitoring, but the current evidence supports a narrower feasibility claim: on a stationary, proprietary 10-subject test set, a 120 FPS event-frame model achieves competitive aggregate RMSE. Given the acknowledged lack of motion scenarios, the small test set, and the single-split evaluation, I recommend softening the conclusion to a feasibility demonstration until broader evaluation is available.
minor comments (6)
- [Section 3.1.1] Equation (1) defines Wj with an ambiguous set-builder notation; please clarify that i and k are the first and last event indices in the window and write the set condition explicitly.
- [Section 3.1.1] The pre-processing description says frames are normalized between -8 and 8 and then multiplied by 255 and cast to 8-bit unsigned integers; negative values cannot be represented in this conversion, so please describe the actual normalization and typecasting procedure.
- [Figure 3] The caption says 'A 10-second sample is inverted and smoothed' while the text says the figure shows a 1-second sample; these statements are inconsistent.
- [Section 4.3] The paper states that validation loss was used to select optimal weights, but no validation metrics or selected epoch numbers are reported; please include them.
- [Section 5, Table 5] The implementation details of the event-count baseline from [26] (filter order, cutoff frequencies, FFT windowing) are not provided, which limits reproducibility of the comparison.
- [Section 4.1] The sentence about some subjects providing an additional minute of break data is vague; clarify how recording durations varied across the 64 subjects.
Circularity Check
No significant circularity: the event-to-pulse mapping is learned from independent ECG labels and evaluated on held-out subjects.
full rationale
The paper's derivation chain is empirical rather than definitional: event frames are generated from raw event streams, a modified TS-CAN is trained with MSE against an ECG-derived label signal, and heart rate is then computed from the post-processed model output via FFT. The labels come from an independent ECG sensor, not from the event stream or from the model's own output, so the learned mapping is tested rather than assumed. The ECG preprocessing in Section 3.2 (inversion, smoothing, bandpass filtering, clipping, resampling, and differentiation) is a modeling assumption about label suitability, but it does not encode the event-frame content or the final heart-rate value, so it is not circular. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and the self-citations (e.g., Refs. [2], [22], [24]) serve as literature context or external comparison baselines rather than load-bearing justification for the central claim. The limitations acknowledged in Sections 4.4 and 5 (proprietary data, stationary subjects, small test set, single split) are concerns about statistical strength and external validity, not about circularity. The honest finding is therefore a non-finding on circularity, with score 0.
Assumptions & free parameters
free parameters (8)
- Event frame window period L =
33333 us (30 FPS), 16666 us (60 FPS), 8333 us (120 FPS)
- Event downsampling factor df =
5
- RGB face crop and downsample size =
400x400 crop, then 144x144
- ECG Savitzky-Golay smoothing window =
101 samples, quadratic fit
- ECG bandpass filter cutoffs =
0.0015 and 0.0048 normalized (0.75 Hz and 2.5 Hz)
- ECG clipping percentiles =
Top and bottom 1% clipped
- Event frame normalization range =
[-8, 8], then multiplied by 255 and cast to uint8
- Training hyperparameters =
learning rate 18e-5 or 9e-5; frame depth 10, 20, or 40; batch size 8; 30 epochs
assumptions (4)
- standard math FFT-based heart rate extraction from a bandpass-filtered signal identifies the dominant cardiac frequency.
- domain assumption The processed and inverted ECG signal is a valid proxy for the blood-volume pulse signal visible in the face.
- domain assumption A fixed-index face crop captures the face without per-frame tracking, which is adequate for the recorded stationary subjects.
- domain assumption Accumulating event polarities into 2D frames preserves enough spatial-temporal structure for a CNN to extract cardiac information.
Cite this review
Pith. "Pith review of Contactless Cardiac Pulse Monitoring Using Event Cameras." pith.science (2026). https://pith.science/paper/AIVT7BBR
@misc{pith2026250509529,
author = {Pith},
title = {Pith review of: Contactless Cardiac Pulse Monitoring Using Event Cameras},
year = {2026},
howpublished = {\url{https://pith.science/paper/AIVT7BBR}},
note = {Machine review of arXiv:2505.09529}
}
read the original abstract
Time event cameras are a novel technology for recording scene information at extremely low latency and with low power consumption. Event cameras output a stream of events that encapsulate pixel-level light intensity changes within the scene, capturing information with a higher dynamic range and temporal resolution than traditional cameras. This study investigates the contact-free reconstruction of an individual's cardiac pulse signal from time event recording of their face using a supervised convolutional neural network (CNN) model. An end-to-end model is trained to extract the cardiac signal from a two-dimensional representation of the event stream, with model performance evaluated based on the accuracy of the calculated heart rate. The experimental results confirm that physiological cardiac information in the facial region is effectively preserved within the event stream, showcasing the potential of this novel sensor for remote heart rate monitoring. The model trained on event frames achieves a root mean square error (RMSE) of 3.32 beats per minute (bpm) compared to the RMSE of 2.92 bpm achieved by the baseline model trained on standard camera frames. Furthermore, models trained on event frames generated at 60 and 120 FPS outperformed the 30 FPS standard camera results, achieving an RMSE of 2.54 and 2.13 bpm, respectively.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[26]
Niklas Huhs. Non-invasive vital parameter detection using neuromorphic cameras exploring the use of neuromor- phic cameras for biomedical applications. 11 2024
work page 2024
-
[1]
Backpropagation applied to handwritten zip code recognition
Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541– 551, 1989
1989
-
[2]
Event cameras in automotive sensing: A review
Waseem Shariff, Mehdi Sefidgar Dilmaghani, Paul Kielty, Mohamed Moustafa, Joe Lemley, and Peter Corcoran. Event cameras in automotive sensing: A review. IEEE Access, 2024
2024
-
[3]
Pushing the limits of asynchronous graph-based object detection with event cameras
Daniel Gehrig and Davide Scaramuzza. Pushing the limits of asynchronous graph-based object detection with event cameras. arXiv preprint arXiv:2211.12324, 2022
arXiv 2022
-
[4]
Spiking-fer: spiking neural network for facial expression recognition with event cameras
Sami Barchid, Benjamin Allaert, Amel Aissaoui, José Mennesson, and Chaabane C Djeraba. Spiking-fer: spiking neural network for facial expression recognition with event cameras. In Proceedings of the 20th International Conference on Content-based Multimedia Indexing, pages 1–7, 2023
work page 2023
-
[5]
E-gaze: Gaze estimation with event camera
Nealson Li, Muya Chang, and Arijit Raychowdhury. E-gaze: Gaze estimation with event camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
work page 2024
-
[6]
Event-based vision: A survey
Guillermo Gallego, Tobi Delbrück, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(1):154–180, 2020
2020
-
[7]
Recent event camera innovations: A survey
Bharatesh Chakravarthi, Aayush Atul Verma, Kostas Daniilidis, Cornelia Fermuller, and Yezhou Yang. Recent event camera innovations: A survey. arXiv preprint arXiv:2408.13627, 2024
arXiv 2024
Show all 39 references
-
[8]
Remote plethysmographic imaging using ambient light
Wim Verkruysse, Lars O Svaasand, and J Stuart Nelson. Remote plethysmographic imaging using ambient light. Optics express, 16(26):21434–21445, 2008
2008
-
[9]
Multi-task temporal shift attention networks for on-device contactless vitals measurement
Xin Liu, Josh Fromm, Shwetak Patel, and Daniel McDuff. Multi-task temporal shift attention networks for on-device contactless vitals measurement. Advances in Neural Information Processing Systems, 33:19400–19411, 2020
2020
-
[10]
Non-contact ppg signal and heart rate estimation with multi-hierarchical convolutional network
Bin Li, Panpan Zhang, Jinye Peng, and Hong Fu. Non-contact ppg signal and heart rate estimation with multi-hierarchical convolutional network. Pattern Recognition, 139:109421, 2023
2023
-
[11]
Robust pulse rate from chrominance-based rppg
Gerard De Haan and Vincent Jeanne. Robust pulse rate from chrominance-based rppg. IEEE Transactions on Biomedical Engineering, 60(10):2878–2886, 2013
2013
-
[12]
Deepphys: Video-based physiological measurement using convolutional attention networks
Weixuan Chen and Daniel McDuff. Deepphys: Video-based physiological measurement using convolutional attention networks. In Proceedings of the european conference on computer vision (ECCV), pages 349–365, 2018
2018
-
[13]
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017
2017
-
[14]
Tsm: Temporal shift module for efficient video understanding
Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7083–7093, 2019
2019
-
[15]
Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks
Zitong Yu, Xiaobai Li, and Guoying Zhao. Remote photoplethysmograph signal measurement from facial videos using spatio-temporal networks. arXiv preprint arXiv:1905.02419, 2019. 13 A PREPRINT - AUGUST 21, 2025
1905 arXiv
-
[16]
A logical calculus of the ideas immanent in nervous activity
Warren S McCulloch and Walter Pitts. A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 5:115–133, 1943
1943
-
[17]
Rhythmformer: Extracting rppg signals based on hierarchical temporal periodic transformer
Bochao Zou, Zizheng Guo, Jiansheng Chen, and Huimin Ma. Rhythmformer: Extracting rppg signals based on hierarchical temporal periodic transformer. arXiv preprint arXiv:2402.12788, 2024
2024 arXiv
-
[18]
Instantaneous physiological estimation using video transformers
Ambareesh Revanur, Ananyananda Dasari, Conrad S Tucker, and László A Jeni. Instantaneous physiological estimation using video transformers. In Multimodal AI in healthcare: A paradigm shift in health intelligence , pages 307–319. Springer, 2022
2022
-
[19]
Physformer: Facial video-based physiological measurement with temporal difference transformer
Zitong Yu, Yuming Shen, Jingang Shi, Hengshuang Zhao, Philip HS Torr, and Guoying Zhao. Physformer: Facial video-based physiological measurement with temporal difference transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 41...
2022
-
[20]
Physformer++: Facial video-based physiological measurement with slowfast temporal difference transformer
Zitong Yu, Yuming Shen, Jingang Shi, Hengshuang Zhao, Yawen Cui, Jiehua Zhang, Philip Torr, and Guoy- ing Zhao. Physformer++: Facial video-based physiological measurement with slowfast temporal difference transformer. International Journal of Computer Vision, 131(6):1307–1330, 2023
2023
-
[21]
Eddd: Event-based drowsiness driving detection through facial motion analysis with neuromorphic vision sensor
Guang Chen, Lin Hong, Jinhu Dong, Peigen Liu, Jörg Conradt, and Alois Knoll. Eddd: Event-based drowsiness driving detection through facial motion analysis with neuromorphic vision sensor. IEEE Sensors Journal , 20(11):6170–6181, 2020
2020
-
[22]
Real-time face & eye tracking and blink detection using event cameras
Cian Ryan, Brian O’Sullivan, Amr Elrasad, Aisling Cahill, Joe Lemley, Paul Kielty, Christoph Posch, and Etienne Perot. Real-time face & eye tracking and blink detection using event cameras. Neural Networks, 141:87–97, 2021
2021
-
[23]
Spikepoint: An efficient point-based spiking neural network for event cameras action recognition.arXiv preprint arXiv:2310.07189, 2023
Hongwei Ren, Yue Zhou, Yulong Huang, Haotian Fu, Xiaopeng Lin, Jie Song, and Bojun Cheng. Spikepoint: An efficient point-based spiking neural network for event cameras action recognition.arXiv preprint arXiv:2310.07189, 2023
2023 arXiv
-
[24]
Heart rate detection using an event camera
Aniket Jagtap, RamaKrishna Venkatesh Saripalli, Joe Lemley, Waseem Shariff, and Alan F Smeaton. Heart rate detection using an event camera. In 2023 IEEE International Symposium on Multimedia (ISM), pages 243–246. IEEE, 2023
2023
-
[25]
Periodicity intensity reveals insights into time series data: three use cases
Alan F Smeaton and Feiyan Hu. Periodicity intensity reveals insights into time series data: three use cases. Algorithms, 16(2):119, 2023
2023
-
[27]
The Fourier transform and its applications, volume 31999
Ronald Newbold Bracewell and Ronald N Bracewell. The Fourier transform and its applications, volume 31999. McGraw-Hill New York, 1986
1986
-
[28]
A 128×128 120 db 15µs latency asynchronous temporal contrast vision sensor
Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. A 128×128 120 db 15µs latency asynchronous temporal contrast vision sensor. IEEE journal of solid-state circuits, 43(2):566–576, 2008
2008
-
[29]
Heart rate estimation network from facial videos using spatiotem- poral feature image
Kokila Bharti Jaiswal and Toshanlal Meenpal. Heart rate estimation network from facial videos using spatiotem- poral feature image. Computers in Biology and Medicine, 151:106307, 2022
2022
-
[30]
Smoothing and differentiation of data by simplified least squares procedures
Abraham Savitzky and Marcel JE Golay. Smoothing and differentiation of data by simplified least squares procedures. Analytical chemistry, 36(8):1627–1639, 1964
1964
-
[31]
Deep physiological sensing toolbox
Xin Liu, Xiaoyu Zhang, Girish Narayanswamy, Yuzhe Zhang, Yuntao Wang, Shwetak Patel, and Daniel McDuff. Deep physiological sensing toolbox. arXiv preprint arXiv:2210.00716, 2022
2022 arXiv
-
[32]
Plux biosignals
PLUX Biosignals. Plux biosignals. https://www.pluxbiosignals.com/. Accessed: 2024-07-23
2024
-
[33]
Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017
Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5, 2017
2017 arXiv
-
[34]
Super-convergence: Very fast training of neural networks using large learning rates
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pages 369–386. SPIE, 2019
2019
-
[35]
Non-contact, automated cardiac pulse measurements using video imaging and blind source separation
Ming-Zher Poh, Daniel J McDuff, and Rosalind W Picard. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics express, 18(10):10762–10774, 2010
2010
-
[36]
Zenodo, September 2023
Facial Camera-Based Heart Rate Estimation Using r-PPG Convolutional Neural Networks. Zenodo, September 2023
2023
-
[37]
Unsupervised skin tissue segmentation for remote photoplethysmography
Serge Bobbia, Richard Macwan, Yannick Benezeth, Alamin Mansouri, and Julien Dubois. Unsupervised skin tissue segmentation for remote photoplethysmography. Pattern Recognition Letters, 124:82–90, 2019. 14 A PREPRINT - AUGUST 21, 2025
2019
-
[38]
Non-contact video-based pulse rate measurement on a mobile service robot
Ronny Stricker, Steffen Müller, and Horst-Michael Gross. Non-contact video-based pulse rate measurement on a mobile service robot. InThe 23rd IEEE International Symposium on Robot and Human Interactive Communication, pages 1056–1062. IEEE, 2014
2014
-
[39]
Remote photoplethysmography for heart rate measurement: A review
Hanguang Xiao, Tianqi Liu, Yisha Sun, Yulin Li, Shiyi Zhao, and Alberto Avolio. Remote photoplethysmography for heart rate measurement: A review. Biomedical Signal Processing and Control, 88:105608, 2024. 15
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.