Pith

open record

sign in

arxiv: 2606.17376 · v1 · pith:IUUR6QCR · submitted 2026-06-16 · cs.RO · cs.CV

Contactless Respiratory Monitoring on Heterogeneous Mobile Robots: A Multimodal Edge-Computing Framework

Milind Rampure , Shadman Sakib , Haley Patel , Zahid Hasan , Nirmalya Roy This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-06-27 01:36 UTCgrok-4.3pith:IUUR6QCRrecord.jsonopen to challenge →

classification cs.RO cs.CV
keywords contactless respiratory monitoringmobile robotsmultimodal frameworkedge computingrespiratory ratesearch and rescuecamera modalitieskeypoint detection
0
0 comments X

The pith

A single multimodal framework lets heterogeneous mobile robots perform contactless respiratory monitoring without platform-specific retuning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a system that uses multiple camera types on mobile robots to measure breathing rates from a distance. It adapts sensor choice based on brightness and uses keypoint detection to locate the chest area reliably even when subjects change posture. Signal quality filtering ensures accurate estimates. Tests on different robots show the same software works across platforms, with RGB cameras effective up to 8 meters, near-infrared to 6 meters, thermal at short distances, and low-light cameras in darkness up to 8 meters. This approach addresses challenges in emergency situations where direct contact is unsafe.

Core claim

The proposed modality-adaptive contactless RR monitoring framework for heterogeneous mobile robots with onboard edge computing generalizes across platforms without per-platform algorithmic retuning. It combines brightness-adaptive sensor selection across RGB, thermal, near-infrared, and low-light cameras with keypoint-guided chest ROI extraction and SQI-based filtering, revealing that RGB covers up to 8m, NIR up to 6m, thermal only at short range, and low-light supports complete darkness up to 8m.

What carries the argument

Brightness-adaptive sensor selection across four camera modalities combined with keypoint-guided chest ROI extraction and signal-quality-index filtering for respiratory signal processing.

If this is right

  • The framework supports deployment on both quadruped and wheeled robots for remote monitoring.
  • RGB modality provides the widest operational range up to 8 meters.
  • Near-infrared remains effective up to 6 meters.
  • Thermal imaging is limited to short ranges.
  • Low-light cameras enable monitoring in complete darkness up to 8 meters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The results suggest the system could form the basis for autonomous systems that scan multiple victims in disaster zones.
  • Modality-specific ranges indicate that hybrid camera setups could optimize coverage in varied environments.
  • Edge processing on the robot supports real-time decisions during field operations.

Load-bearing premise

Keypoint-guided chest ROI extraction combined with SQI-based filtering remains reliable across posture changes, variable illumination, and varying robot-to-subject distances without platform-specific retuning.

What would settle it

Demonstrating that the respiratory rate estimation fails to maintain accuracy on a fourth robotic platform or under a new combination of lighting and distance without any algorithmic adjustments would falsify the generalization claim.

Figures

Figures reproduced from arXiv: 2606.17376 by Haley Patel, Milind Rampure, Nirmalya Roy, Shadman Sakib, Zahid Hasan.

Figure 1
Figure 1. Figure 1: Adaptive multi-modal respiratory-rate (RR) monitoring on a mobile [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: System methodology workflow illustrating detection, ROI extraction, temporal signal processing, and RR estimation pipeline. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Representative experimental configurations: (left to right) Platform A [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Bandpass filtered respiratory signals (8–35 BPM) across platforms and modalities. Red-shaded regions indicate SQI-rejected windows. Panels at 4 m [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Platform computational performance during RR pipeline execution: [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
read the original abstract

Respiratory-rate (RR) monitoring is a critical component of remote triage and victim assessment in emergency response, disaster recovery, and infectious-disease scenarios, where minimizing physical contact can reduce responder risk and improve operational safety. However, field deployment of contactless RR monitoring remains challenging due to variable illumination, posture changes, platform heterogeneity, and the impracticality of wearable sensors in hazardous environments. In this paper, we present a modality-adaptive contactless RR monitoring framework for heterogeneous mobile robots with onboard edge computing. The proposed system combines brightness-adaptive sensor selection across RGB, thermal, near-infrared (NIR), and low-light cameras, keypoint-guided chest ROI extraction for posture-robust monitoring, and a signal-quality-index (SQI)-based filtering mechanism for reliable respiratory estimation. We implement and evaluate the framework on three robotic platforms spanning quadruped and wheeled locomotion and multiple edge-computing architectures. Experiments conducted across diverse lighting conditions, subject poses, and robot-to-subject distances demonstrate that the framework generalizes across platforms without per-platform algorithmic retuning, while revealing modality-specific operational boundaries. RGB provides the broadest coverage up to 8m, NIR remains effective up to 6m, thermal is reliable only at short range, and low-light sensing supports monitoring in complete darkness up to 8m. Overall, the results demonstrate the feasibility of multimodal contactless RR monitoring on mobile robots and support its use as a foundation for autonomous triage and victim assessment in hazardous search-and-rescue settings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper presents a modality-adaptive contactless respiratory rate (RR) monitoring framework for heterogeneous mobile robots with onboard edge computing. It combines brightness-adaptive sensor selection across RGB, thermal, NIR, and low-light cameras, keypoint-guided chest ROI extraction, and SQI-based filtering. The framework is implemented on three robotic platforms (quadruped and wheeled) and evaluated across lighting conditions, poses, and distances; the central claim is that it generalizes across platforms without per-platform retuning while revealing modality-specific boundaries (RGB to 8 m, NIR to 6 m, thermal short range only, low-light to 8 m in darkness).

Significance. If the generalization and boundary claims hold, the work would offer a practical engineering contribution for contactless vital-sign monitoring in hazardous search-and-rescue and triage scenarios. The explicit multimodal design and deployment on three heterogeneous platforms with edge computing are strengths that could inform future autonomous systems.

major comments (2)
  1. [Abstract and experimental evaluation sections] Abstract and experimental evaluation sections: the manuscript asserts that experiments demonstrate generalization without retuning and the listed modality-specific ranges, yet supplies no quantitative metrics (e.g., RR estimation error, detection success rates at each distance, or cross-platform statistical comparisons) to support these claims.
  2. [Method sections describing keypoint-guided ROI extraction and SQI filtering] Method sections describing keypoint-guided ROI extraction and SQI filtering: these components are presented as the mechanism enabling posture-robust performance without platform-specific retuning, but no equations, pseudocode, threshold values, or ablation results on posture/illumination changes are provided to substantiate reliability across conditions.
minor comments (2)
  1. [Figures] Figure captions and axis labels should explicitly state the robot platform, distance, and lighting condition for each example to aid interpretation.
  2. [Platform description] The manuscript would benefit from a short table summarizing the three robotic platforms (locomotion type, compute hardware, camera suite) for quick reference.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the referee's insightful comments. We address the major concerns below by committing to specific revisions that will strengthen the quantitative and methodological support in the manuscript.

read point-by-point responses

  1. Referee: [Abstract and experimental evaluation sections] Abstract and experimental evaluation sections: the manuscript asserts that experiments demonstrate generalization without retuning and the listed modality-specific ranges, yet supplies no quantitative metrics (e.g., RR estimation error, detection success rates at each distance, or cross-platform statistical comparisons) to support these claims.

    Authors: We agree that additional quantitative metrics are necessary to support the claims. The revised manuscript will include detailed tables with RR estimation errors (MAE and RMSE), detection success rates at varying distances for each modality, and cross-platform statistical comparisons to demonstrate generalization without retuning. revision: yes

  2. Referee: [Method sections describing keypoint-guided ROI extraction and SQI filtering] Method sections describing keypoint-guided ROI extraction and SQI filtering: these components are presented as the mechanism enabling posture-robust performance without platform-specific retuning, but no equations, pseudocode, threshold values, or ablation results on posture/illumination changes are provided to substantiate reliability across conditions.

    Authors: We will expand the method sections in the revision to include the mathematical equations for keypoint-guided ROI extraction and SQI computation, pseudocode for the overall process, the specific threshold values used, and ablation results showing the impact of posture and illumination variations on performance. revision: yes

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is an applied system description of a multimodal RR monitoring framework evaluated experimentally on three heterogeneous robots under varied conditions. No mathematical derivations, equations, fitted parameters, or self-referential predictions appear in the provided content. Central claims of platform generalization and modality boundaries rest on direct experimental results rather than any reduction to inputs, self-citations, or ansatzes. The work is self-contained against external benchmarks with no load-bearing circular steps.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No mathematical model, derivations, or new physical entities; the work is an engineering system paper relying on standard computer vision and signal processing assumptions not detailed in the abstract.

pith-pipeline@v0.9.1-grok · 5812 in / 1114 out tokens · 32121 ms · 2026-06-27T01:36:24.628324+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 2 canonical work pages

  1. [1]

    Wearable sensors for remote health monitoring,

    S. Majumder, T. Mondal, and M. J. Deen, “Wearable sensors for remote health monitoring,”Sensors, vol. 17, no. 1, p. 130, 2017

    2017

  2. [2]

    Contactless methods for measuring respiratory rate: A review,

    C. Massaroni, A. Nicolo, M. Sacchetti, and E. Schena, “Contactless methods for measuring respiratory rate: A review,”IEEE Sensors Jour- nal, vol. 21, no. 11, pp. 12 821–12 839, 2020

    2020

  3. [3]

    A state-of-the-art survey of remote photoplethysmography for contactless health parameters sensing,

    S. Sakib, Z. Hasan, and N. Roy, “A state-of-the-art survey of remote photoplethysmography for contactless health parameters sensing,”Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, vol. 15, no. 3, p. e70039, 2025

    2025

  4. [4]

    Contact-based methods for measuring respiratory rate,

    C. Massaroni, A. Nicol `o, D. Lo Presti, M. Sacchetti, S. Silvestri, and E. Schena, “Contact-based methods for measuring respiratory rate,” Sensors, vol. 19, no. 4, p. 908, 2019

    2019

  5. [5]

    Comparison of two methods for estimating respiratory waveforms from videos without con- tact,

    C. Massaroni, E. Schena, S. Silvestri, and S. Maji, “Comparison of two methods for estimating respiratory waveforms from videos without con- tact,” in2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA). IEEE, 2019, pp. 1–6

    2019

  6. [6]

    Respformer: A motion-guided temporal-frequency multimodal fusion transformer for contactless respiratory monitoring,

    S. Sakib, G. Shinde, S. Chugh, M. S. Anwar, and N. Roy, “Respformer: A motion-guided temporal-frequency multimodal fusion transformer for contactless respiratory monitoring,” in2025 International Conference on Machine Learning and Applications (ICMLA). IEEE, 2025, pp. 631–638

    2025

  7. [7]

    Video- based breathing rate monitoring in sleeping subjects,

    L. Queiroz, H. Oliveira, S. Yanushkevich, and R. Ferber, “Video- based breathing rate monitoring in sleeping subjects,” in2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 2020, pp. 2458–2464

    2020

  8. [8]

    Triage Challenge,

    Defense Advanced Research Projects Agency, “Triage Challenge,” https://triagechallenge.darpa.mil/, n.d., accessed: Jul. 27, 2025

    2025

  9. [9]

    Automatic infant respiration estimation from video: A deep flow-based algorithm and a novel public benchmark,

    S. K. R. Manne, S. Zhu, S. Ostadabbas, and M. Wan, “Automatic infant respiration estimation from video: A deep flow-based algorithm and a novel public benchmark,” inInternational Workshop on Preterm, Perinatal and Paediatric Image Analysis. Springer, 2023, pp. 111–120

    2023

  10. [10]

    E2respunet: End-to-end respiratory signal reconstruction and rate prediction using a unified attention-enhanced u-net,

    S. Sakib, G. Shinde, E. Dev, and N. Roy, “E2respunet: End-to-end respiratory signal reconstruction and rate prediction using a unified attention-enhanced u-net,” in2025 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 2025, pp. 154–161

    2025

  11. [11]

    Autonomous thermal vision robotic system for victims recognition in search and rescue missions,

    C. Cruz Ulloa, G. Prieto S ´anchez, A. Barrientos, and J. Del Cerro, “Autonomous thermal vision robotic system for victims recognition in search and rescue missions,”Sensors, vol. 21, no. 21, p. 7346, 2021

    2021

  12. [12]

    Video-based real-time monitoring for heart rate and respiration rate,

    M. Alnaggar, A. I. Siam, M. Handosa, T. Medhat, and M. Rashad, “Video-based real-time monitoring for heart rate and respiration rate,” Expert Systems with Applications, vol. 225, p. 120135, 2023

    2023

  13. [13]

    Anymal parkour: Learning agile navigation for quadrupedal robots,

    D. Hoeller, N. Rudin, D. Sako, and M. Hutter, “Anymal parkour: Learning agile navigation for quadrupedal robots,”Science Robotics, vol. 9, no. 88, p. eadi7566, 2024

    2024

  14. [14]

    Mobile robotic platform for contactless vital sign monitoring,

    H.-W. Huang, J. Chen, P. R. Chai, C. Ehmke, P. Rupp, F. Z. Dadabhoy, A. Feng, C. Li, A. J. Thomas, M. da Silvaet al., “Mobile robotic platform for contactless vital sign monitoring,”Cyborg and Bionic Systems, 2022

    2022

  15. [15]

    Mixed-reality for quadruped-robotic guidance in sar tasks,

    C. Cruz Ulloa, J. del Cerro, and A. Barrientos, “Mixed-reality for quadruped-robotic guidance in sar tasks,”Journal of Computational Design and Engineering, vol. 10, no. 4, pp. 1479–1489, 2023

    2023

  16. [16]

    From sensors to safety: Internet of emergency services (ioes) for emergency response and disaster management,

    R. Dama ˇseviˇcius, N. Bacanin, and S. Misra, “From sensors to safety: Internet of emergency services (ioes) for emergency response and disaster management,”Journal of sensor and actuator networks, vol. 12, no. 3, p. 41, 2023

    2023

  17. [17]

    Real-time vision based respiration monitoring system,

    K. S. Tan, R. Saatchi, H. Elphick, and D. Burke, “Real-time vision based respiration monitoring system,” in2010 7th International Symposium on Communication Systems, Networks & Digital Signal Processing (CSNDSP 2010). IEEE, 2010, pp. 770–774

    2010

  18. [18]

    Video-based respiration monitoring with automatic region of interest detection,

    R. Janssen, W. Wang, A. Moc ¸o, and G. De Haan, “Video-based respiration monitoring with automatic region of interest detection,” Physiological measurement, vol. 37, no. 1, p. 100, 2015

    2015

  19. [19]

    Motion based detection of respiration rate in infants using video,

    D. M. Tveit, K. Engan, I. Austvoll, and Ø. Meinich-Bache, “Motion based detection of respiration rate in infants using video,” in2016 IEEE International Conference on Image Processing (ICIP). IEEE, 2016, pp. 1225–1229

    2016

  20. [20]

    Contactless monitoring of respiratory rate and breathing absence from head move- ments using an rgb camera,

    M. Gwak, K. Vatanparvar, L. Zhu, J. Kuang, and A. Gao, “Contactless monitoring of respiratory rate and breathing absence from head move- ments using an rgb camera,”IEEE Engineering in Medicine & Biology Society Conference, 2023

    2023

  21. [21]

    Motion-based respiratory rate estimation with motion artifact removal using video of face and upper body,

    M. Gwak, K. Vatanparvar, J. Kuang, and A. Gao, “Motion-based respiratory rate estimation with motion artifact removal using video of face and upper body,”44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), p. 1961, 2022

    1961

  22. [22]

    Non-contact video-based vital sign monitoring using ambient light and auto-regressive models,

    L. Tarassenko, M. Villarroel, A. Guazzi, J. Jorge, D. Clifton, and C. Pugh, “Non-contact video-based vital sign monitoring using ambient light and auto-regressive models,”Physiological measurement, vol. 35, no. 5, p. 807, 2014

    2014

  23. [23]

    Contactless vital signs monitoring from videos recorded with digital cameras: An overview,

    N. Molinaro, E. Schena, S. Silvestri, F. Bonotti, D. Aguzzi, E. Viola, F. Buccolini, and C. Massaroni, “Contactless vital signs monitoring from videos recorded with digital cameras: An overview,”Frontiers in physiology, vol. 13, p. 801709, 2022

    2022

  24. [24]

    Contactless monitoring system versus gold standard for respiratory rate monitoring in emergency department patients: pilot comparison study,

    C. E. Goldfine, M. F. T. Oshim, B. P. Chapman, D. Ganesan, T. Rahman, and S. P. Carreiro, “Contactless monitoring system versus gold standard for respiratory rate monitoring in emergency department patients: pilot comparison study,”JMIR Formative Research, vol. 8, p. e44717, 2024

    2024

  25. [25]

    Contactless measurement of vital signs using thermal and rgb cameras: A study of covid 19-related health monitoring,

    F. Yang, S. He, S. Sadanand, A. Yusuf, and M. Bolic, “Contactless measurement of vital signs using thermal and rgb cameras: A study of covid 19-related health monitoring,”Sensors, vol. 22, no. 2, p. 627, 2022

    2022

  26. [26]

    Remote photoplethysmography in real-world and extreme lighting scenarios,

    H. Shao, L. Luo, J. Qian, M. Yan, S. Chen, and J. Yang, “Remote photoplethysmography in real-world and extreme lighting scenarios,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025, pp. 10 858–10 867

    2025

  27. [27]

    Remote photoplethysmography (rppg) in the wild: Remote heart rate imaging via online webcams,

    D. D. Lernia, G. Finotti, M. Tsakiris, G. Riva, and M. Naber, “Remote photoplethysmography (rppg) in the wild: Remote heart rate imaging via online webcams,”Behavior Research Methods, pp. 6904–6914, 2024

    2024

  28. [28]

    Optimal signal quality index for remote photoplethysmogram sensing,

    M. Elgendi, I. Martinelli, and C. Menon, “Optimal signal quality index for remote photoplethysmogram sensing,”npj Biosensing, vol. 1, no. 1, p. 5, 2024

    2024

  29. [29]

    Pqr signal quality indexes: A method for real-time photoplethysmogram signal quality estimation based on noise interferences,

    J. Song, D. Li, X. Ma, G. Teng, and J. Wei, “Pqr signal quality indexes: A method for real-time photoplethysmogram signal quality estimation based on noise interferences,”Biomedical Signal Processing and Control, vol. 47, pp. 88–95, 2019

    2019

  30. [30]

    Rrpips: Respiratory waveform reconstruction using persistent independent particles tracking from video,

    Z. Hasan, M. Ahmed, S. Sakib, S. Chugh, M. A. Khan, A. Z. M. Faridee, and N. Roy, “Rrpips: Respiratory waveform reconstruction using persistent independent particles tracking from video,” inProceed- ings of the ACM/IEEE International Conference on Connected Health: Applications, Systems and Engineering Technologies, 2025, pp. 13–24

    2025

  31. [31]

    Artemis: Ai-driven robotic triage labeling and emergency medical information system,

    R. K. Senthilkumaran, M. Prashanth, H. Viswanath, S. Kotha, K. Tiwari, and A. Bera, “Artemis: Ai-driven robotic triage labeling and emergency medical information system,”arXiv preprint arXiv:2309.08865, 2023

    work page arXiv 2023

  32. [32]

    Team cerberus wins the darpa subterranean challenge: Technical overview and lessons learned,

    M. Tranzatto, M. Dharmadhikari, L. Bernreiter, M. Camurri, S. Khattak, F. Mascarich, P. Pfreundschuh, D. Wisth, S. Zimmermann, M. Kulkarni et al., “Team cerberus wins the darpa subterranean challenge: Technical overview and lessons learned,”arXiv preprint arXiv:2207.04914, vol. 11, 2022

    work page arXiv 2022