{"id":"db96c1ee-5eb6-4a6d-b10c-57ac8563dc17","arxiv_id":"2501.01691","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"An rPPG network combining 3DCNN local features with Transformer global features, with spatiotemporal attention and cross-branch fusion, claims state-of-the-art heart rate estimation across five datasets.","lead":"VidFormer is an end-to-end rPPG framework that fuses a 3DCNN branch with a Transformer branch and reports very low heart-rate estimation errors on five public datasets. The authors claim sub-1 bpm MAE on datasets where prior SOTA methods report 3 to 5 bpm, and the evaluation protocol is under-specified with no code released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified; the reported SOTA results rest on an unverifiable evaluation protocol, but the paper shows no internal contradiction.","rationale":"The reader's verdict REJECT is based on the claim being unsupported due to missing protocol details, but the paper's central claim is not internally inconsistent; it is simply not verifiable from the manuscript alone. The reader's strongest_claim is the empirical superiority, and the weakest assumption is the fairness of the evaluation protocol. The missing split details are a genuine load-bearing gap, but they do not constitute evidence of fraud or internal error; they indicate that the claim is unverified. The reader's own rationale also notes the lack of code release and clear protocol as the main issue, which is a reproducibility gap rather than a proven flaw. Therefore, the appropriate disposition is UNVERDICTED (or at most CONDITIONAL on providing the protocol), not REJECT, because the paper does not provide enough information to falsify or confirm the central claim. My concrete test asks for the protocol and a reproduction on one dataset, which would settle whether the concern lands.","tokens_in":27023,"tokens_out":1066,"duration_ms":9637,"concrete_test":"Request the authors to release the exact evaluation protocol: subject-independent split details, number of windows per video, overlap across train/test, random seed, and the code or a detailed pseudocode. Then reproduce Table I for a single dataset (e.g., PURE) with the reported setting; if the reproduced MAE differs by more than 0.5 bpm or if the protocol is revealed to be subject-dependent, the headline claim of SOTA performance is not supported.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The paper claims state-of-the-art performance (e.g., MAE of 0.42 bpm on PURE, 0.75 bpm on DEAP, 0.64 bpm on ECG-fitness) but does not specify the train/test split protocol, whether it is subject-independent, or how windows are sampled, which are essential for interpreting these results. The reader's concern about potential subject leakage and data processing artifacts is not contradicted by the manuscript, which only says that one segment is randomly selected and sliced with a window of 250 frames and step size of 50. The abstract's claim that VidFormer outperforms SOTA methods is therefore not independently verifiable from the provided text. However, this is a reproducibility/verification gap, not a demonstrated internal inconsistency.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes VidFormer, a dual-branch architecture combining 3DCNN and Transformer branches for video-based remote photoplethysmography (rPPG). The method introduces a 3DCNN branch with spatial and temporal attention (GA-3DCNN), a Transformer branch with separated spatial and temporal multi-head self-attention (ST-MHSA), and a CTIM module for cross-branch information exchange. The authors also present an 'enhanced skin reflection model' in Eq. (2), which reformulates the dichromatic reflection model with reflection terms that depend on the blood volume pulse, pixel location, and time. The paper reports state-of-the-art heart-rate estimation results on five public datasets (UBFC-rPPG, PURE, DEAP, ECG-fitness, COHFACE), with MAE values below 1 bpm on DEAP and ECG-fitness, along with HRV/RF results, ablations, and discussions on ethnicity, makeup, and exercise.","tokens_in":27218,"tokens_out":6290,"duration_ms":55604,"significance":"If the reported results hold under a fair, reproducible evaluation protocol, VidFormer would be a strong empirical contribution to the rPPG literature. The architecture is plausible and the paper covers a broad set of benchmarks and ablations, which is valuable. The paper also clearly attempts to address known limitations of both CNN and Transformer models for this task. However, the load-bearing claim of state-of-the-art performance is currently unverifiable because the train/test protocol is not specified, and the comparison to prior methods is not controlled. The proposed analytical model is not used to derive any testable prediction, so the theoretical contribution is overstated. These issues must be resolved before the empirical claims can be assessed.","major_comments":[{"comment":"The intra-dataset evaluation protocol is not specified. Section IV-B states only that \"we randomly select one segment and slice it using a window with length of 250 frames and a step size of 50 frames,\" with no statement about subject-independent train/test splits, the number of windows extracted per video, whether overlapping windows from the same subject can appear in both training and test sets, or the random seed used. Since the central claim is state-of-the-art performance, this omission makes Tables I-V unverifiable and creates a real risk of subject/segment leakage. The authors must specify the exact protocol for every dataset, including subject-independent splits and window sampling details.","section":"Section IV-B and IV-C"},{"comment":"The comparison with prior methods is not controlled. It is unclear whether all numbers in Table I were produced under the same protocol as VidFormer or quoted from original papers that may use different splits and preprocessing. The reported gaps (e.g., MAE 0.42 vs 1.10 bpm on PURE and 0.75 vs 3.03 bpm on DEAP relative to Physformer) are implausibly large without a shared evaluation setup. The authors should either re-run all comparison methods under an identical subject-independent protocol or clearly state the protocol and source for each entry in the table.","section":"Table I and Section IV-D1"},{"comment":"All tables report single-point metrics without error bars, confidence intervals, or multiple-seed statistics. Given the random segment selection described in Section IV-B, the results may be sensitive to the chosen segments. The paper should report mean and standard deviation over at least three random seeds and, where possible, use paired statistical tests to support the claimed improvements over prior methods.","section":"Tables I-X"},{"comment":"The claimed 'enhanced skin reflection model' in Eq. (2) is a notational restatement of Eq. (1) with ℓ_s and ℓ_d declared to depend on y_t, ρ, and t. No prediction from this model is derived, no parameter is estimated from it, and it does not constrain the architecture or loss. The statement that VidFormer is 'based on this improved model' is therefore not supported. The authors should either use the model to derive a testable component or remove the claim that the architecture is based on it.","section":"Section III-A, Eq. (2)"}],"minor_comments":[{"comment":"The MAE value for HRCNN is printed as '14 , 48' (likely 14.48); the formatting should be corrected.","section":"Table I, HRCNN row"},{"comment":"The column heading 'COHFACE-African-wM' is inconsistent with Section VI-B, which describes the subset as 'COHFACE with makeup'; rename the column to 'COHFACE-wM'.","section":"Table XII"},{"comment":"Figures 17, 18, and 19 contain the literal placeholder string '121312312大苏打撒旦'; these should be replaced with proper figure content or removed.","section":"Figures 17-19"},{"comment":"The description 'P_os is a random number that satisfies a Gaussian distribution with a mean of 0 and a variance of 1' is ambiguous; please specify whether the positional encoding is fixed after initialization, learned, or re-sampled at every forward pass, and state its dimension.","section":"Section III-D, Eq. (7)"},{"comment":"The cross-dataset protocol is underspecified; the paper should describe how frame rates, ROIs, and signal sampling rates are aligned across datasets, and how the model is adapted when moving from one dataset to another.","section":"Section IV-C"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about the evaluation protocol is well-founded and is the main reason I cannot support acceptance. I chose major_revision rather than reject because the missing protocol details could in principle be supplied and the experiments re-run under a subject-independent split. However, if the authors cannot provide such splits or if the results change materially under a fair protocol, the paper should be rejected. The manuscript also appears rushed, as evidenced by placeholder text in figures and formatting errors in tables."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, VidFormer is a genuine architectural contribution: a dual-branch 3DCNN+Transformer with bidirectional cross-attention fusion (CTIM), and the ablations show both branches and the fusion matter. Second, the headline result—sub-1 bpm MAE on DEAP and ECG-fitness—is not supportable as reported, because the paper never says whether the train/test split is subject-independent, how many windows are drawn per video, or whether overlapping windows leak across splits. The reader's concern is not pedantry; it's the load-bearing assumption.\n\nWhat is new: the specific combination of a 3DCNN branch with spatiotemporal attention (GA-3DCNN) and a Transformer branch with separated spatial/temporal MHSA (ST-MHSA), fused through CTIM, is not in the cited literature. The paper is also honest about the role of each module, and the cross-dataset results (e.g., UBFC→PURE MAE 1.79) are more plausible than the intra-dataset numbers. The ethnicity and makeup discussion is a nice extra.\n\nWhere it falls down: the evaluation protocol is underspecified to the point of non-reproducibility. No error bars, no seeds, no code release. Table I shows VidFormer beating PhysFormer by 2-5x on several datasets; that is the kind of jump that demands protocol details, not just a table. The skin reflection model in Eq (2) is a notational restatement of Eq (1) and does no analytical work, so the 'enhanced model' is really just motivation. The manuscript also has clear incompleteness: placeholder text in figure captions ('121312312大苏打撒旦'), mislabeled Table X (title says ST-MHSA but the content is CTIM ablation), and a wrong header in Table XII. A reviewer would spend time on these, but they're surface issues compared to the protocol gap.\n\nBottom line: the architecture deserves a serious look, but the empirical claims are not yet verifiable. The paper is for someone working on rPPG fusion architectures who wants ideas, not for someone who needs reliable performance comparisons. I'd send it to peer review only if the authors can be required to release code and specify the split; otherwise desk reject with encourage-resubmit. My recommendation: treat it as a reject in current form, but invite a resubmission with full protocol and code.","headline":"Plausible new fusion architecture, but the headline sub-1-bpm results rest on an underspecified evaluation protocol and the manuscript is incomplete.","tokens_in":27725,"tokens_out":2655,"would_cite":false,"duration_ms":25530,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"VidFormer claims to set a new state of the art in video-based remote heart-rate measurement by fusing a 3DCNN with a Transformer, reaching mean absolute errors below 1 bpm on large and motion-rich datasets.","keywords":["remote photoplethysmography","rPPG","heart rate estimation","3DCNN","Transformer","dual-branch fusion","spatiotemporal attention","video-based vital signs"],"falsifier":"Re-run VidFormer with a strict subject-exclusive split on DEAP and ECG-fitness, using the same window length and step size, and report the resulting MAE. If the mean absolute error rises above 1 bpm or the margin over PhysFormer narrows substantially, the paper's central SOTA claim would be falsified.","tokens_in":26863,"feed_emoji":"🫀","tokens_out":4140,"duration_ms":41346,"temperature":0.7,"pith_summary":"VidFormer is an end-to-end framework for remote photoplethysmography (rPPG) that reconstructs blood-volume-pulse signals from ordinary facial videos and then derives heart rate, heart rate variability, and respiration frequency from them. The paper's central claim is that by running a 3DCNN branch for local spatiotemporal features alongside a Transformer branch for global features, with explicit cross-branch information exchange, the model outperforms previously published methods on all five tested public datasets. The headline result is a mean absolute heart-rate error below 1 bpm on the large DEAP dataset and on the motion-heavy ECG-fitness dataset, which the authors attribute to the joint local-global modeling and to an improved skin reflection model that treats the face-to-signal mapping as time-dependent.","feed_headline":"Fused 3DCNN and Transformer cut heart-rate error below 1 bpm","feed_subtitle":"A dual-branch model that blends local and global video features outperforms prior methods on five public datasets.","key_machinery":"The core mechanism is a dual-branch fusion architecture composed of five modules: a Stem for initial feature extraction, a Local Convolution Branch built on BS-3DCNN blocks augmented by a Global Attention 3DCNN (GA-3DCNN) with separate Spatial Attention and Time Attention, a Global Transformer Branch using a Spatial-Time Multi-headed Self-attention (ST-MHSA) that splits attention into spatial and temporal streams, a CNN-Transformer Interaction Module (CTIM) with Trans-Conv and Conv-Trans blocks that reshape and exchange features between the branches, and an rPPG Generation Module (RGM) that turns each branch's features into BVP estimates. The two outputs are optimized separately with a combined negative Pearson and Smooth L1 loss, and their heart-rate estimates are averaged.","core_discovery":"The paper claims that a dual-branch fusion of 3DCNN and Transformer, named VidFormer, is the first such end-to-end architecture specifically designed for rPPG, and that it achieves superior heart-rate estimation accuracy on UBFC-rPPG, PURE, COHFACE, ECG-fitness, and DEAP. On the author's own terms, the discovery is that separately extracting local features (via a convolutional branch with spatial and temporal attention) and global features (via a Transformer branch with split spatial and temporal self-attention), and then exchanging information between the branches at multiple levels, lets the model reconstruct BVP signals accurately enough to keep mean absolute error below 1 bpm even on datasets with complex lighting, electrode patches, and exercise-induced motion. The improved skin reflection model, which rewrites the dichromatic model so that the mapping from blood-volume changes to skin color depends explicitly on time and pixel location, serves as the design rationale for combining local and global modeling rather than relying on either alone.","pith_inferences":["A reader should treat the sub-1 bpm figures as contingent on the evaluation protocol: if the random segment selection allowed windows from the same subject to appear in both training and testing, the model could memorize per-subject skin appearance and inflate accuracy.","The split spatial/temporal attention design could transfer to other video-understanding tasks where short-range texture and long-range dynamics both matter.","The ethnicity analysis suggests a testable extension: the reported lower accuracy on African subjects could be probed with cameras or preprocessing that linearize sensor response in dark skin tones.","A stricter test of the paper's central claim would be to fix the train/test split to be subject-independent and to report the random-segment seed, so other groups can reproduce the exact protocol."],"forward_implications":["If correct, non-contact heart-rate monitoring can hold mean absolute error below 1 bpm on large, varied datasets, not just on small controlled ones.","The cross-dataset results suggest the model generalizes across recording conditions, lighting, and subject populations without per-dataset retraining.","Ablations indicate that both the local convolutional branch and the global Transformer branch are individually necessary, and that the interaction module substantially improves accuracy over either branch alone.","The framework also produces competitive respiratory-frequency and heart-rate-variability estimates, broadening its potential use beyond heart rate.","The improved skin reflection model offers a principled reason to pair local and global feature extractors, a design choice other rPPG systems could adopt."],"supporting_citations":[{"why":"Supplies the dichromatic skin reflection model that the paper analyzes and refines into its time-dependent mapping.","marker":"[5]"},{"why":"Provides the POS skin-reflection baseline and the MAE/RMSE/r evaluation convention the paper follows.","marker":"[20]"},{"why":"DeepPhys is a central CNN attention baseline that VidFormer compares against and builds on conceptually.","marker":"[25]"},{"why":"PhysNet establishes the end-to-end 3DCNN rPPG approach and the Pearson-correlation loss that VidFormer adopts.","marker":"[9]"},{"why":"PhysFormer is the Transformer-based state-of-the-art baseline that VidFormer aims to surpass, and the direct comparison on UBFC-rPPG.","marker":"[47]"},{"why":"Provides the standard Transformer architecture that the Global Transformer Branch is built upon.","marker":"[31]"},{"why":"ViViT supplies the cube-patch decomposition and spatiotemporal attention-splitting scheme that ST-MHSA adapts.","marker":"[55]"},{"why":"RhythmNet supplies the negative Pearson loss formulation used to align predicted and ground-truth BVP signals.","marker":"[13]"},{"why":"HeartPy is the toolbox used to compute heart rate, respiration frequency, and HRV metrics from estimated BVP.","marker":"[60]"},{"why":"Meta-rppg is a supervised baseline used in cross-dataset heart-rate comparisons, anchoring the generalization claims.","marker":"[15]"}],"fun_headline_variants":["3DCNN+Transformer fusion nails rPPG with sub-1 bpm error","Hybrid attention model from 3DCNN and Transformer beats prior rPPG methods","VidFormer dual-branch fusion cuts rPPG error to <1 bpm on five datasets","Fused local and global video features slash rPPG error below 1 bpm"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported superiority assumes that the train/test protocol keeps the same subject's randomly selected video windows out of the test set, but the paper never states whether the split is subject-independent or how the random segment selection is seeded.","fun_headline_variants_meta":{"raw":{"variants":["3DCNN+Transformer fusion nails rPPG with sub-1 bpm error","Hybrid attention model from 3DCNN and Transformer beats prior rPPG methods","VidFormer dual-branch fusion cuts rPPG error to <1 bpm on five datasets","Fused local and global video features slash rPPG error below 1 bpm"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001455,"raw_usage":{"total_tokens":5883,"prompt_tokens":1000,"completion_tokens":4883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":4789}},"tokens_in":616,"tokens_out":4883,"duration_ms":33084,"temperature":1.0,"reasoning_tokens":4789,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:22:26.670325+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run VidFormer with a strict subject-exclusive split on DEAP and ECG-fitness, using the same window length and step size, and report the resulting MAE. If the mean absolute error rises above 1 bpm or the margin over PhysFormer narrows substantially, the paper's central SOTA claim would be falsified.","supporting_citations":[{"cited_title":"Deepphys: Video-based physiological mea- surement using convolutional attention networks,","cited_arxiv_id":null,"evidence_quote":"DeepPhys is a central CNN attention baseline that VidFormer compares against and builds on conceptually."},{"cited_title":"Remote heart rate measurement from highly compressed facial videos: an end-to-end deep learning solution with video enhancement,","cited_arxiv_id":null,"evidence_quote":"PhysNet establishes the end-to-end 3DCNN rPPG approach and the Pearson-correlation loss that VidFormer adopts."},{"cited_title":"Robust pulse rate from chrominance-based rppg,","cited_arxiv_id":null,"evidence_quote":"Supplies the dichromatic skin reflection model that the paper analyzes and refines into its time-dependent mapping."},{"cited_title":"Algorithmic principles of remote ppg,","cited_arxiv_id":null,"evidence_quote":"Provides the POS skin-reflection baseline and the MAE/RMSE/r evaluation convention the paper follows."},{"cited_title":"Physformer: Facial video-based physiological measurement with temporal difference transformer,","cited_arxiv_id":null,"evidence_quote":"PhysFormer is the Transformer-based state-of-the-art baseline that VidFormer aims to surpass, and the direct comparison on UBFC-rPPG."},{"cited_title":"Vivit: A video vision transformer,","cited_arxiv_id":null,"evidence_quote":"ViViT supplies the cube-patch decomposition and spatiotemporal attention-splitting scheme that ST-MHSA adapts."},{"cited_title":"Rhythmnet: End-to-end heart rate estimation from face via spatial-temporal representation,","cited_arxiv_id":null,"evidence_quote":"RhythmNet supplies the negative Pearson loss formulation used to align predicted and ground-truth BVP signals."},{"cited_title":"Analysing noisy driver physiology real-time using off-the-shelf sensors: Heart rate analysis software from the taking the fast lane project,","cited_arxiv_id":null,"evidence_quote":"HeartPy is the toolbox used to compute heart rate, respiration frequency, and HRV metrics from estimated BVP."},{"cited_title":"Meta-rppg: Remote heart rate estima- tion using a transductive meta-learner,","cited_arxiv_id":null,"evidence_quote":"Meta-rppg is a supervised baseline used in cross-dataset heart-rate comparisons, anchoring the generalization claims."}],"review_version":1}