{"id":"5018f1bc-21db-434f-8338-aaa2572ba74e","arxiv_id":"2607.21776","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":11,"one_line_summary":"A classifier trained on rPPG waveforms from face videos detects talking-face deepfakes with AUC 0.806, and detection difficulty varies by generator (AUC 0.690–0.985).","lead":"This paper uses heart-rate signals extracted from face videos (rPPG) to detect talking-face deepfakes, reporting AUC 0.806 on a new benchmark subset and finding that detection difficulty varies strongly across generation methods. A generalist should read it because if the method-dependent 'physiological fingerprint' claim holds, it offers an audio-free, complementary way to rank which deepfake generators are hardest to catch.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Per-method AUC ranking rests on unverified positional method labels; mis-assignment would invalidate the paper's primary theoretical contribution.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the per-method evaluation relies on a positional assignment of method labels that cannot be independently verified. This is precisely the foundation of the paper's primary theoretical contribution—the stable, interpretable AUC spread across generators. Without verified labels, the ranking could be an artifact of mis-assignment, and the physiological interpretation would be unfounded. The aggregate real-vs-fake AUC (0.806) is not affected, but the paper's novelty rests on the per-method analysis. The authors themselves flag this limitation in Section 6.4, which is honest but does not resolve the issue. Therefore, the reader's REJECT verdict is appropriate: the central claim is currently unsupported. A corrected version with verified method labels and a proper comparison against SOTA on the same protocol could warrant reconsideration. I see no other concern more load-bearing than this one; other issues (e.g., comparison to Effort, validation identity leakage) are secondary and would not independently invalidate the main claim if the labels were verified.","tokens_in":10472,"tokens_out":3828,"duration_ms":40934,"concrete_test":"Obtain the official Celeb-DF++ directory listing or metadata from the dataset authors and compare the order of TF-subset filenames against the positional assignment used in Section 5.4. Specifically, verify that the first 2,500 videos in the TF folder correspond to the first generator (e.g., Real3DPortrait), the next 2,500 to the second, etc. If the order differs, re-run Table 4 with corrected labels; if the per-method AUC ranking changes, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the method-dependent detectability spread (AUC 0.690–0.985) and its interpretation as a stable, generator-specific physiological property. This claim depends entirely on correctly assigning each fake video to one of seven TF generators. Section 6.4 states that this assignment is positional, based on the 'known Celeb-DF++ generation ordering,' and 'cannot be independently verified at the video level' because method names are not recoverable from filenames. If the positional mapping is wrong—e.g., if the TF videos are not grouped by generator in the assumed order—then Table 4 and Fig. 2 mislabel the methods. The claimed 'interpretable physiological property' (Section 6.3) would then be unsupported, and the paper's primary theoretical contribution collapses. The aggregate real-vs-fake AUC (0.806) would still stand, but the novel per-method analysis would be invalid. The authors explicitly acknowledge this limitation, making it a load-bearing concern rather than a hypothetical.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the first rPPG-based detection framework specifically for talking-face (TF) deepfakes. It extracts 160-sample pulse waveforms with RhythmFormer from face crops and trains lightweight 1D classifiers (ResNet/CNN/Transformer) under a subject-independent identity split on the TF subset of Celeb-DF++. The authors report AUC 0.806 with the 1D ResNet, reproduce DeepFakesON-Phys at AUC 0.622 on the same TF data, and present a method-stratified analysis showing a per-generator AUC spread from 0.690 (IP-LAP) to 0.985 (Real3DPortrait), interpreted as a stable, generator-specific physiological property linked to each method's pixel synthesis strategy.","tokens_in":10748,"tokens_out":8977,"duration_ms":100026,"significance":"If the main claims hold, the paper makes a useful empirical contribution: a physiological-channel detector for a forgery category where image-based detectors are known to be weak, a controlled reproduction of a prior rPPG detector demonstrating distribution shift, and an analysis suggesting that TF generators differ greatly in rPPG detectability. The paper also ships public code, uses identity-disjoint splits, and reports multi-seed means, which are strengths. However, the paper's advertised primary theoretical contribution—the method-dependent detectability ranking—rests on an explicitly unverifiable positional assignment of method labels, and the headline comparison to Effort is made against a different evaluation protocol. The aggregate real-vs-fake result and the reproduction study are more robust, but the central advertised novelty is not adequately supported as written.","major_comments":[{"comment":"The primary theoretical contribution—the stable per-generator AUC ranking (0.690–0.985)—depends entirely on the positional assignment of method labels to fake videos. Section 6.4 admits that this assignment 'cannot be independently verified at the video level' because method names are not recoverable from filenames. If the assumed Celeb-DF++ ordering is even partially wrong, Table 4, Fig. 2, Fig. 3, and the interpretation in §6.3 would all be invalid, even though the aggregate real-vs-fake AUC would still stand. The authors must either provide an independently verifiable per-video label mapping (e.g., a released label file tied to dataset metadata) or remove/reframe the per-method claim. This is a load-bearing limitation, not a cosmetic one.","section":"§6.4, Table 4, Fig. 2"},{"comment":"The headline 'within 2.4 points of Effort (ICML 2025)' compares the proposed TF-only AUC (0.806) with Effort's 0.830 reported on the full Celeb-DF++ benchmark across all forgery categories. Since §2.5 states that TF is the hardest category for existing detectors (AUC 50–70%), this is not a same-protocol comparison and the 'within 2.4 points' claim is not established. The authors should report Effort on the same TF-only split if possible, or explicitly state that no such per-category result exists and soften the claim accordingly.","section":"§5.3, §6.1, §2.5"},{"comment":"The claim that the method ranking is 'perfectly consistent across every evaluation protocol' is not backed by a protocol-by-protocol table; only the 18-identity result is shown. In addition, the mechanistic explanation that detection difficulty is determined by 'pixel synthesis strategy' (warp vs. synthesize) is a post hoc categorization: no direct measure of pixel preservation or temporal color coherence is provided, and the paper does not test this explanation against alternative factors (e.g., video length, compression, identity distribution across methods). The stability claim and the causal-sounding interpretation should be either directly supported with the full AUC matrix and quantitative covariates, or explicitly downgraded to a hypothesis.","section":"§5.4, Table 4, §6.3"},{"comment":"The evaluation unit is ambiguous. Section 3.1 defines the classifier input as a 'per-video rPPG waveform', but §4.3 windows long videos into multiple 160-frame clips at stride 60, and Table 3 reports AUC without stating whether it is window-level or video-level. The DeepFakesON-Phys reproduction in §5.1, by contrast, is explicitly 'video-level'. If the main AUC numbers are window-level, they are not directly comparable to video-level benchmarks and may benefit from within-video autocorrelation. The authors should specify the unit of evaluation, aggregate to video-level if that is the forensic claim, or report both.","section":"§3.1, §4.3, §5.1"}],"minor_comments":[{"comment":"The text says 'yielding 585 real waveforms (585 source videos with stride-60 temporal windowing produce 2,435 waveforms in total)' and then reports 2,435 real training waveforms. The '585 real waveforms' phrase appears to be a typo and should be corrected for clarity.","section":"Abstract, §4.1"},{"comment":"Table 1 lists a 9-identity validation split, but the main result in §5.3 uses an '18-identity test set (val and test identities combined)'. Clarify whether the validation identities were used in any model-selection or early-stopping procedure before being combined into the final evaluation; if they were fully untouched, state this explicitly.","section":"§4.2, §5.3"},{"comment":"The waveforms shown are explicitly 'representative' and appear to have been selected after seeing the results. This is acceptable as illustration, but the caption should state that these are hand-picked examples, not a systematic characterization of each method's output.","section":"Fig. 3"},{"comment":"The statement that 'robust results are limited to FS forgeries (AUC>85%), while FR and TF forgeries yield degraded detection rates (AUC 50–70%)' is not followed by a citation or a table. Please add the source of these numbers.","section":"§2.5"},{"comment":"The term 'theoretical contribution' is used in the abstract and §5.4 for what is an empirical, method-stratified observation. Consider using 'empirical finding' or 'forensic characterization' unless the paper provides a formal theoretical model predicting the ranking.","section":"§5.4"}],"recommendation":"major_revision","confidential_remarks":"The aggregate rPPG detection result and the DeepFakesON-Phys reproduction are genuinely useful and appear to survive the label-verification concern. My recommendation is major revision rather than rejection because the paper could be made publishable by either verifying the per-method labels with external metadata or removing/reframing the unsupported central claim. If the authors cannot supply verifiable per-video method labels, the per-method ranking and the 'primary theoretical contribution' should be removed from the abstract and conclusions; the paper would then stand as a more conventional, still valid, TF-focused rPPG detection study."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper has one solid contribution, one interesting but unverified claim, and one misleading comparison. The solid part is the aggregate result: a 1D ResNet trained on rPPG waveforms from RhythmFormer gets AUC 0.806 on the talking-face subset of Celeb-DF++ with subject-independent splits, and a reproduction of DeepFakesON-Phys drops from 0.999 to 0.622 on the same data. That is a real, useful finding: prior rPPG detectors were evaluated on face-swap data, and this quantifies the distribution shift.\n\nWhat's new: first rPPG-based detector aimed specifically at talking-face forgeries. The per-method stratification (AUC 0.690 to 0.985) is also new in this context, and the stability across protocols is striking if true.\n\nThe soft spots are in proportion. The per-method ranking is load-bearing for the paper's \"primary theoretical contribution\"—the claim that detection difficulty reflects each generator's pixel synthesis strategy. But the labels for the seven generators come from a positional assignment based on assumed Celeb-DF++ generation ordering, and Sec. 6.4 admits this cannot be independently verified. If the mapping is wrong, Table 4 and Fig. 2 mislabel methods and the whole physiological interpretation is unsupported. That is not a hypothetical nit; it's the center of the paper's novelty. The aggregate AUC doesn't depend on those labels, so that part stands, but the main claim doesn't.\n\nThe comparison to Effort (ICML 2025) is apples-to-oranges: Effort's 0.830 is on the full Celeb-DF++ benchmark, while this paper evaluates only the TF subset. The \"within 2.4 points\" phrasing should be dropped or replaced with a matched-protocol comparison.\n\nOne more thing to check: the final evaluation uses 18 identities, merging the val and test splits. That is fine if the val identities were never used for hyperparameter choices, but the paper doesn't say so explicitly. The real pool is only 59 identities, with 9 held out for test, so the real-side estimate has wide error bars even before the label issue.\n\nWho this is for: people working on deepfake detection, especially physiological channels, and anyone using Celeb-DF++ for TF evaluation. The reproduction study alone is worth a look.\n\nRecommendation: don't desk-reject. Send to peer review, but the per-method analysis needs verified labels or it should be cut. If the labels can't be verified, the paper should be revised to present the aggregate result and the reproduction study as its main contributions.","headline":"The aggregate rPPG result and the DeepFakesON-Phys reproduction are solid; the paper's advertised main claim—a stable per-generator physiological ranking—rests on positional method labels the authors admit cannot be verified.","tokens_in":11236,"tokens_out":2729,"would_cite":false,"duration_ms":27262,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Talking-face deepfakes are detectable from the physiological pulse signal alone, nearly matching state-of-the-art general detectors.","keywords":["talking-face deepfake detection","remote photoplethysmography (rPPG)","1D ResNet","subject-independent evaluation","Celeb-DF++","physiological signal forensics","method-dependent detectability","RhythmFormer"],"falsifier":"Obtain ground-truth generator labels for the Celeb-DF++ talking-face videos (for example, by querying the benchmark's original metadata or re-synthesizing videos with known method names) and recompute per-method AUC. If the ordering Real3DPortrait > EDTalk > SadTalker > AniTalker > EchoMimic > FLOAT > IP-LAP does not reproduce under verified labels, the claim that the ranking is an interpretable physiological property of each generator is refuted.","tokens_in":10322,"feed_emoji":"🫀","tokens_out":7186,"duration_ms":63467,"temperature":0.7,"pith_summary":"This paper claims that talking-face deepfakes—videos synthesized from a single still image and an audio track—carry no real physiological substrate, so any apparent heart-pulse signal is an artifact of the generator. The authors therefore propose detecting them by extracting a per-video rPPG waveform from facial skin-color changes and classifying it with a small 1D residual network. On the talking-face subset of Celeb-DF++, evaluated with strict separation of celebrity identities between training and test, the network reaches an AUC of 0.806 (EER 27.8%), within 2.4 points of the best published general-purpose detector while using only the physiological channel. The paper's central discovery is that detection difficulty varies sharply and stably with the generation method—AUC ranges from 0.985 for a 3D-rendering method down to 0.690 for a landmark-warping method—and that this stable ordering reflects an interpretable property of each generator's pixel synthesis strategy rather than evaluation noise.","feed_headline":"Pulse signals expose talking-face deepfakes at AUC 0.806","feed_subtitle":"A tiny 1D network nears full detectors with no audio or texture clues; the per-generator spread is itself a forensic fingerprint.","key_machinery":"The central object is the rPPG (remote photoplethysmography) waveform: a 160-sample cardiac-pulse trace recovered from subtle periodic color variations in facial skin. The pipeline detects faces with a lightweight detector, extracts the waveform with RhythmFormer (a transformer with periodic sparse attention), z-score normalizes each waveform to remove identity-specific amplitude, and feeds it to a 1D ResNet (165K parameters). The per-method analysis treats each generator's output waveform as its physiological signature, using AUC to quantify how well the waveform discriminates real from fake.","core_discovery":"The primary claim is that rPPG-based detection is not just applicable but especially well-motivated for talking-face forgeries, because unlike face-swapping, talking-face synthesis leaves no real video whose blood-flow characteristics could be inherited. The authors show that a 165K-parameter 1D ResNet trained on RhythmFormer pulse waveforms outperforms a reproduced prior rPPG detector (AUC 0.622) by 0.184 points and comes within 2.4 points of the best general-purpose detector on the benchmark. The theoretically significant finding is the per-method AUC spread: Real3DPortrait (0.985), EDTalk (0.950), SadTalker (0.946), AniTalker (0.908), EchoMimic (0.865), FLOAT (0.794), and IP-LAP (0.690),","pith_inferences":["A natural extension the paper leaves implicit: the generator-specific waveform shape (incoherent noise vs near-sinusoidal artifact) could be used for generator attribution or fingerprinting, not just binary real/fake classification, since the physiological artifact encodes the synthesis family.","The paper's dual-use caveat can be pushed further: if adversarial synthesis injects a plausible cardiac waveform into latent-synthesis methods, the detection gap for those methods may widen; a direct experiment would be to add synthetic rPPG to Real3DPortrait outputs and measure the AUC drop.","The acknowledged skin-tone and lighting sensitivity suggests the reported aggregate AUC may not hold uniformly across demographic groups; a stratified evaluation by Fitzpatrick skin type is a testable next step that could revise the per-method ordering for some generators.","The authors' reliance on positional method labels for the per-method analysis is a verification gap independent of the aggregate result; if the Celeb-DF++ ordering is ever corrected, the ranking might shift, and the 'physiological fingerprint' interpretation should be re-tested with verified labels."],"forward_implications":["If the claims are correct, rPPG provides a detection channel for talking-face forgeries that does not depend on audio, texture, or frequency features, so it works even when audio is missing or tampered with.","The stable per-method ranking implies that the forensic difficulty of a future talking-face generator can be anticipated from its synthesis strategy (latent-pixel synthesis vs source-frame warping) without running detection experiments.","The DeepFakesON-Phys reproduction (0.999 on face-swap data, 0.622 on talking-face data) shows that existing rPPG detectors do not transfer to talking-face forgeries; talking-face-specific training is required.","IP-LAP's resistance, including the inversion under isolated training, identifies identity-preserving landmark-based synthesis as the main open challenge for physiological forensics.","The near-parity with general-purpose detectors on a strict identity-disjoint protocol suggests physiological signals generalize across identities, strengthening the case for including rPPG in multi-modal detectors."],"fun_headline_variants":["Pulse signals unmask talking-face deepfakes","Blood flow betrays talking-face fakes","rPPG detector nears best deepfake models","Talking-face fakes leak pulse: AUC 0.806","Per-generator pulse spread fingerprints deepfakes"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The per-method AUC ranking rests on an unverifiable label assignment: because Celeb-DF++ filenames do not carry method names, the authors map each fake video to a generator by its position in the dataset's generation order; if that positional mapping is wrong, the central per-generator ranking collapses, even though the aggregate real-vs-fake detection might survive.","fun_headline_variants_meta":{"raw":{"variants":["Pulse signals unmask talking-face deepfakes","Blood flow betrays talking-face fakes","rPPG detector nears best deepfake models","Talking-face fakes leak pulse: AUC 0.806","Per-generator pulse spread fingerprints deepfakes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0004,"raw_usage":{"total_tokens":2007,"prompt_tokens":908,"completion_tokens":1099,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":652,"completion_tokens_details":{"reasoning_tokens":1023}},"tokens_in":652,"tokens_out":1099,"duration_ms":11396,"temperature":1.0,"reasoning_tokens":1023,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T06:43:02.798762+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Obtain ground-truth generator labels for the Celeb-DF++ talking-face videos (for example, by querying the benchmark's original metadata or re-synthesizing videos with known method names) and recompute per-method AUC. If the ordering Real3DPortrait > EDTalk > SadTalker > AniTalker > EchoMimic > FLOAT > IP-LAP does not reproduce under verified labels, the claim that the ranking is an interpretable physiological property of each generator is refuted.","supporting_citations":[],"review_version":1}