{"id":"a23b1d5d-557f-4873-83d9-1a659c4ecac2","arxiv_id":"2506.22460","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A 3D convolutional neural network estimates heart rate and respiratory rate from noisy smartphone fingertip videos with lower RMS error than an EEMD-PCA baseline on a small private real-world dataset.","lead":"This paper tests whether a deep learning model can estimate heart rate and breathing rate from smartphone videos of a fingertip recorded in daily life. It reports that the model cuts heart-rate error by about 68 percent and breathing-rate error by about 75 percent compared with a standard signal-processing method.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Best-of-four-fold selection on 15 test videos makes the headline 68%/75% error reductions statistically unsecured; no confidence intervals are reported.","rationale":"Reader's weakest assumption was ground-truth accuracy, especially self-counted RR. That is a real concern for absolute values, but the central relative claim (deep regressor beats EEMD-PCA) would survive random label noise because both methods are scored on the same labels. The more immediate threat is that the reported error reductions are unquantified small-sample point estimates: one selected fold, 15 test videos, no CIs. The paper's own Section 4.2 describes the best-validation-fold selection, and no per-video residuals or per-fold test results are given. The dataset is also private, and no deep-learning rPPG baseline is compared, but those are secondary. The correct verdict remains CONDITIONAL: the approach is plausible and the tables are internally consistent, but the headline claim needs confidence intervals, all-fold test statistics, and ideally open-data validation before it can be accepted. Since the reader already recommended CONDITIONAL, no change is needed.","tokens_in":11156,"tokens_out":4667,"duration_ms":49927,"concrete_test":"Re-run the reported protocol and report all four folds' test RMS for HR and RR, not just the best-validation fold; pool the 4×15 per-video predictions and compute paired bootstrap 95% CIs for the RMS difference between DVR and EEMD-PCA on the same 15 videos. If the CI excludes zero and every fold shows a large margin, the headline claim is secure. Additionally, repeat the subject-level 20% holdout with 50 random seeds; if the median improvement is much smaller than 68%/75% or overlaps zero, best-fold selection on 15 videos was load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing gap is statistical reliability, not the labels. Section 4.2 holds out 20% of subjects (15/71 videos) and then takes 'the fold that performs the best on the validation set' as the final model, characterizing it on the 15 test videos. The headline RMS values (HR 2.92, RR 4.80 in Tables 3 and 5) are therefore a best-of-four-fold point estimate on 15 clips. With n=15, a single difficult video can move RMS by several BPM, and selecting the best validation fold adds optimistic bias. The EEMD-PCA baseline is measured on the same 15 clips, so the comparison is not unfair, but no confidence interval, per-fold breakdown, or repeated split is reported. The claimed 68%/75% reductions could be driven by which videos happened to be in the test set and which fold looked best on a small validation set. This is more load-bearing than the label issue: even with perfect labels, the relative improvement is not statistically quantified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes a deep 3D convolutional architecture (DVR 1/2/3) for estimating heart rate (HR) and respiratory rate (RR) from 30-second smartphone fingertip videos recorded in uncontrolled daily-life settings. A dataset of 161 videos was collected from participants who self-counted RR while an expert measured HR by pulse palpation; after automated quality filtering, 71 videos from 46 subjects remained. The authors hold out 15 videos by subject, train with four-fold validation on the remaining data, and select the validation-best fold as the final model. They report that the proposed method reduces HR RMS error from 9.02 to 2.92 beats/min (68%) and RR RMS error from 19.62 to 4.80 breaths/min (75%) relative to an EEMD-PCA baseline, and they additionally report single-task and simultaneous HR/RR prediction results with Bland-Altman and correlation plots.","tokens_in":11344,"tokens_out":5696,"duration_ms":59460,"significance":"If the point estimates are reliable, this is a useful real-world stress test of camera-based vital-sign estimation: it shows that a classical EEMD-PCA method degrades sharply on uncontrolled smartphone video and that a purpose-built 3D CNN can recover much of that gap. The subject-disjoint split and the direct comparison against a published signal-processing baseline are strengths, and the paper is explicit about label-adjustment procedures and augmentation choices. However, the central quantitative claim rests on 15 test videos with best-of-four-fold model selection and no uncertainty quantification, and the RR ground truth is self-counted with no validation. The result is therefore promising but not yet statistically secured; the contribution is closer to a benchmark report than a definitive demonstration of the stated generalization to daily life.","major_comments":[{"comment":"The headline 68% and 75% error reductions are point estimates from the validation-best of four cross-validation folds evaluated on 15 held-out test videos. With n = 15, the RMS is highly sensitive to individual clips, and selecting the fold with the best validation loss adds optimistic selection bias. No per-fold test errors, confidence intervals, bootstrap estimates, or significance tests are reported, so the central claim is not statistically quantified. Please report per-fold test results, confidence intervals for the test RMS differences, and ideally repeated subject-disjoint splits or leave-one-subject-out validation.","section":"Section 4.2, Tables 3 and 5"},{"comment":"The RR ground truth is self-counted by participants and doubled, with no validation of this procedure, while the HR label uncertainty is at least quantified by the cited MAE of slightly over 4 BPM. Because these labels serve as both training targets and the evaluation reference, label noise or bias in RR directly contaminates the reported 75% RR reduction. A label-quality assessment (e.g., expert re-counting on a subset, duplicate measurements, or a sensitivity analysis) is needed before the RR comparison can be considered reliable. In addition, the participant counts are inconsistent: the abstract states N=111, Section 3.1 states N=105, and Section 4.2 uses 52 subjects while Section 3.3 reports 46 unique subjects.","section":"Section 3.1"},{"comment":"The automated quality filter discarded 90 of 161 videos (56%) for lacking a detectable pulsatile pattern, and the evaluation is performed only on the remaining 71 high-quality clips. The conclusion that the method works on noisy real-world daily-life video is therefore limited to the 44% of recordings that pass quality filtering. The manuscript should either report performance on the unfiltered set or explicitly characterize the deployment-time filtering step, its accuracy, and the failure rate; otherwise the abstract's generalization claim overstates what is demonstrated.","section":"Sections 3.2 and 5 (Discussion)"},{"comment":"The text states that the simultaneous predictor improves RR performance slightly (RMS of 5.20 vs. 4.80 breaths/min) compared with predicting RR alone, but 5.20 is worse than 4.80, and Table 5 reports the single-task red-channel model at 4.80. This contradicts the reported direction of the comparison and should be corrected, along with a clearer explanation of why simultaneous training is beneficial when it appears to hurt HR performance.","section":"Section 4.3, Tables 5 and 7"}],"minor_comments":[{"comment":"The number of participants differs between the abstract (N=111) and the data collection section (N=105); this should be reconciled.","section":"Abstract and Section 3.1"},{"comment":"The Discussion says the RR RMS of 5 breaths/min is shown in Table 3, but the RR single-task results appear in Table 5; the cross-reference is incorrect.","section":"Discussion"},{"comment":"Table 1 repeats 'DVR 1' three times in the header; the three columns should be labeled DVR 1, DVR 2, and DVR 3 so that the parameter and training-time comparisons are interpretable.","section":"Table 1"},{"comment":"The statement that K=4 was chosen to provide approximately a 55-25-20 train-validation-test split is arithmetically unclear; a four-fold split on the 80% training pool gives roughly 60-20-20 splits, not 55-25-20, and the sampling procedure from HR×RR tiers should be described more precisely.","section":"Section 4.2"},{"comment":"The phrase 'fingerprint videos' appears in the Introduction and should be 'fingertip videos' for consistency with the rest of the paper.","section":"Throughout"},{"comment":"The table captions say 'DVR 2 and 3' but the architecture described in the text is DVR 3; no separate results for DVR 2 versus DVR 3 are presented, so the captions should state which architecture produced the numbers.","section":"Tables 3, 5, and 7"},{"comment":"The training section says both 'Lookahead' and 'DP-SGD' were used as the optimizer; unless these are composed in a specific way, the description is contradictory and should be clarified.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper reads like a preliminary report with substantial copy-editing needs, but the underlying comparison is potentially publishable if the statistical presentation is strengthened. The most important request is per-fold and uncertainty-aware reporting of the 15-video test results; without that, the 68%/75% headline is not defensible. The self-counted RR labels are a separate correctness risk that also needs at least a sensitivity analysis. I would not reject the paper outright, but I would require a revised version that directly addresses these points before considering it for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a look for the dataset alone: 71 quality-filtered fingertip videos collected in daily life with HR and RR labels, plus the finding that EEMD-PCA degrades badly on this data (HR RMS 9.02 vs 0.69 in the original paper, RR RMS 19.62 vs 2.7). That is a real observation about generalization. The DVR 3 architecture, a temporally focused C3D variant, is a reasonable design for a counting task, and the reported HR RMS of 2.92 BPM for the grayscale model approaches the expected label error of about 4 BPM. The arithmetic in Tables 3 and 5 supports the claimed percentage reductions on the test set. I also agree with the paper's decision to hold out subjects, not videos, which avoids the worst leakage.\n\nThe soft spots are in proportion. The load-bearing one is statistical reliability, and the stress-test note gets it right: the model is chosen as the best of four validation folds and evaluated on 15 held-out videos. With n=15, a single difficult clip can move RMS by several units, and best-fold selection adds optimistic bias. No confidence intervals, no per-fold breakdown, no repeated splits. So the 68%/75% reductions are a point estimate, not a secured result. The label issue is real but secondary: RR is self-counted, and the paper does not validate those labels independently. Also, the only baseline is EEMD-PCA; no comparison to existing deep learning rPPG methods (PhysNet, TS-CAN, etc.), so the claim that deep regression is the right approach is supported only against a weak signal-processing baseline. The private dataset and code mean nobody can reproduce the numbers without the authors' cooperation. The 56% discard rate is also important: the pipeline only works on the quality-filtered subset, and the paper does not address how a deployed system would detect and handle the bad videos.\n\nNet: this is a solid pilot study with a useful dataset, not a definitive result. The likely conclusion - that regressor-based deep learning beats EEMD-PCA on noisy real-world fingertip video - is plausible but not yet demonstrated with statistical rigor. I would send it to peer review because the question and dataset matter, but I would expect major revision: confidence intervals or bootstrap, comparison to at least one deep learning baseline, RR label validation, and ideally a data release or open-code for a subset. mHealth researchers working on smartphone vital signs should read it; others can wait for the strengthened version.","headline":"A genuinely new real-world fingertip video dataset and a plausible deep learning win over EEMD-PCA, but the headline 68%/75% error reductions rest on a best-of-four-fold point estimate over 15 test videos with no confidence intervals.","tokens_in":11881,"tokens_out":2291,"would_cite":false,"duration_ms":25867,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"On noisy, real-world smartphone fingertip videos, a purpose-built 3D convolutional network estimates heart rate and respiratory rate more accurately than a standard signal-processing baseline, cutting RMS error by 68% and 75%.","keywords":["vital signs","deep learning","regression","mobile phones","mHealth","photoplethysmography","3D convolutional neural network","fingertip video"],"falsifier":"Collect a new uncontrolled smartphone-fingertip dataset where heart rate is recorded by ECG and respiratory rate by capnography or respiratory inductance plethysmography, then run DVR 3 against EEMD-PCA under the paper's training protocol. If the respiratory-rate RMS advantage over EEMD-PCA disappears once labels are reference-grade, or if HR RMS rises above the ~4 beats/min label-noise floor, the central claim would be refuted.","tokens_in":10947,"feed_emoji":"📱","tokens_out":8780,"duration_ms":80836,"temperature":0.7,"pith_summary":"Mobile fingertip videos are a cheap source of vital signs, but past accuracy claims were mostly measured in controlled lab conditions. This paper tests that assumption with smartphone videos collected in everyday life, finds that a standard signal-processing method performs much worse there, and shows that a purpose-built 3D convolutional network (DVR 3) cuts heart-rate RMS error from 9.02 to 2.92 beats/min and respiratory-rate RMS error from 19.62 to 4.80 breaths/min—a 68% and 75% reduction. The result suggests regressor-based deep learning, rather than hand-crafted spectral analysis, should be the basis for real-world smartphone vital-sign estimation. The paper also introduces the quality-filtered dataset of 71 videos from 46 subjects used for training and testing.","feed_headline":"Deep 3D CNN cuts heart-rate and breathing-rate errors by 68-75%","feed_subtitle":"A 3D regressor network predicts heart and respiratory rates from noisy daily-life phone video, beating a standard baseline.","key_machinery":"The central object is the Deep Video Regression 3 (DVR 3) network, a C3D-inspired 3D convolutional neural network built for temporal counting. It takes 360 frames (24 seconds at 30 fps) of $32\\times32\\times1$ video—either the red channel or a grayscale composite—and stacks 3D convolutions and max pools whose temporal-depth kernels shrink from 90 to 10, giving the network a long temporal context in which to count color oscillations. Splitting each convolution into two layers with $1\\times1$ filters cuts trainable parameters from 470.6M in DVR 2 to 157.5M in DVR 3 while keeping the same error. The work it does is replacing hand-crafted spectral decomposition (EEMD-PCA) with a learned mapping from raw spatiotemporal color variation to beats and breaths per minute.","core_discovery":"On the quality-filtered subset of a new real-world dataset—71 fingertip videos from 46 participants, recorded without experimenter supervision before and after exercise—the Deep Video Regression 3 (DVR 3) network predicts heart rate with a root-mean-square error of 2.92 beats/min and respiratory rate with an RMS error of 4.80 breaths/min. Against the EEMD-PCA signal-processing baseline on the same test videos, those are reductions of 68% and 75% respectively (from 9.02 beats/min and 19.62 breaths/min). The paper's central claim is that this gap reflects the failure of laboratory-tuned spectral methods to generalize to noisy daily-life video, and that a 3D CNN that learns to count color-variation frequency from raw frames is the more robust approach. The HR error approaches the roughly 4 beats/min error of the manual pulse-count labels themselves, while RR remains the harder target.","pith_inferences":["Beyond the paper: because the reported numbers come from the best of four validation folds evaluated on 15 held-out videos, a single fixed model would likely show a smaller gain on a new population; a pre-registered single-model evaluation would give the deployable expectation.","Beyond the paper: the 44% retention rate means the reported accuracies apply only to videos with a clear pulsatile signal; end-to-end performance including rejected videos is likely lower and should be measured.","Beyond the paper: a natural testable extension is the same architecture on 30-second windows, which the paper itself suggests; if temporal context is the limiting factor for RR, longer input should raise RR accuracy.","Beyond the paper: the learned color-variation-frequency representation might transfer to other PPG-like inputs such as webcam facial video or wearable optical sensors, since the network does not assume a particular sensor geometry."],"forward_implications":["A regressor-based deep network should replace spectral decomposition as the default estimator for HR and RR from fingertip phone video in uncontrolled settings.","Heart-rate predictions at 2.92 BPM RMS are close to the ~4 BPM error of the 30-second palpation labels, so further HR improvements will likely require better ground truth, not just better networks.","Respiratory rate remains the limiting vital sign; the paper's analysis points to longer windows (30 seconds) and explicitly modeling respiratory sinus arrhythmia and baseline/amplitude modulation as next steps.","Any practical deployment needs an automated quality gate, since 56% of collected videos lacked a detectable pulse and were discarded.","Predicting HR and RR jointly reduces the RR error bias but costs HR accuracy (RMS rises from 2.92 to 6.36 BPM), so the choice between single-task and multi-task models depends on the use case."],"supporting_citations":[{"why":"Supplies the EEMD-PCA baseline method and the controlled-condition error values (HR RMS 0.69, RR RMS 2.7) that this paper's real-world results are contrasted against.","marker":"(25)"},{"why":"Justifies the heart-rate ground-truth protocol, reporting a mean absolute error of just over 4 beats/min for 30-second radial pulse counts.","marker":"(15)"},{"why":"Provides the automated signal-processing check used to decide which videos contain a detectable pulse and therefore enter the 71-video dataset.","marker":"(24)"},{"why":"Reports near-perfect RR accuracy from smartphone-camera PPG in controlled settings, the expectation that real-world data fail to meet.","marker":"(20)"},{"why":"Documents controlled fingertip-image HR errors of 1.8–7.5 BPM, the benchmark that motivates the real-world test.","marker":"(37)"},{"why":"Establishes the smartphone camera as a PPG sensor, the data source for the whole study.","marker":"(17)"},{"why":"Demonstrates deep regression in vision tasks, supporting the choice of a CNN regressor over hand-crafted spectral features.","marker":"(2)"}],"fun_headline_variants":["3D CNN cuts heart-rate and breathing-rate errors by 68-75%","Phone video vital signs: deep learning cuts errors 68-75%","3D CNN improves real-world HR/RR estimates by 68-75%","Deep 3D CNN predicts HR and RR from noisy phone video with 68-75% less error"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the manual labels are accurate enough to train and evaluate against: heart rate was counted as radial/carotid pulse over 30 seconds (a method with about 4 beats/min error), and respiratory rate was self-counted by participants with no independent validation.","fun_headline_variants_meta":{"raw":{"variants":["3D CNN cuts heart-rate and breathing-rate errors by 68-75%","Phone video vital signs: deep learning cuts errors 68-75%","3D CNN improves real-world HR/RR estimates by 68-75%","Deep 3D CNN predicts HR and RR from noisy phone video with 68-75% less error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000943,"raw_usage":{"total_tokens":4034,"prompt_tokens":956,"completion_tokens":3078,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":2988}},"tokens_in":572,"tokens_out":3078,"duration_ms":21824,"temperature":1.0,"reasoning_tokens":2988,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:48:14.591393+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect a new uncontrolled smartphone-fingertip dataset where heart rate is recorded by ECG and respiratory rate by capnography or respiratory inductance plethysmography, then run DVR 3 against EEMD-PCA under the paper's training protocol. If the respiratory-rate RMS advantage over EEMD-PCA disappears once labels are reference-grade, or if HR RMS rises above the ~4 beats/min label-noise floor, the central claim would be refuted.","supporting_citations":[],"review_version":2}