REVIEW 4 major objections 5 minor 26 references
Talking-face deepfakes are detectable from the physiological pulse signal alone, nearly matching state-of-the-art general detectors.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
A classifier trained on rPPG waveforms from face videos detects talking-face deepfakes with AUC 0.806, and detection difficulty varies by generator (AUC 0.690–0.985).
T0 review reviewed 2026-08-01 challenge →
load-bearing objection The aggregate rPPG result and the DeepFakesON-Phys reproduction are solid; the paper's advertised main claim—a stable per-generator physiological ranking—rests on positional method labels the authors admit cannot be verified. the 4 major comments →
Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The primary claim is that rPPG-based detection is not just applicable but especially well-motivated for talking-face forgeries, because unlike face-swapping, talking-face synthesis leaves no real video whose blood-flow characteristics could be inherited. The authors show that a 165K-parameter 1D ResNet trained on RhythmFormer pulse waveforms outperforms a reproduced prior rPPG detector (AUC 0.622) by 0.184 points and comes within 2.4 points of the best general-purpose detector on the benchmark. The theoretically significant finding is the per-method AUC spread: Real3DPortrait (0.985), EDTalk (0.950), SadTalker (0.946), AniTalker (0.908), EchoMimic (0.865), FLOAT (0.794), and IP-LAP (0.690),
What carries the argument
The central object is the rPPG (remote photoplethysmography) waveform: a 160-sample cardiac-pulse trace recovered from subtle periodic color variations in facial skin. The pipeline detects faces with a lightweight detector, extracts the waveform with RhythmFormer (a transformer with periodic sparse attention), z-score normalizes each waveform to remove identity-specific amplitude, and feeds it to a 1D ResNet (165K parameters). The per-method analysis treats each generator's output waveform as its physiological signature, using AUC to quantify how well the waveform discriminates real from fake.
Load-bearing premise
The per-method AUC ranking rests on an unverifiable label assignment: because Celeb-DF++ filenames do not carry method names, the authors map each fake video to a generator by its position in the dataset's generation order; if that positional mapping is wrong, the central per-generator ranking collapses, even though the aggregate real-vs-fake detection might survive.
What would settle it
Obtain ground-truth generator labels for the Celeb-DF++ talking-face videos (for example, by querying the benchmark's original metadata or re-synthesizing videos with known method names) and recompute per-method AUC. If the ordering Real3DPortrait > EDTalk > SadTalker > AniTalker > EchoMimic > FLOAT > IP-LAP does not reproduce under verified labels, the claim that the ranking is an interpretable physiological property of each generator is refuted.
If this is right
- If the claims are correct, rPPG provides a detection channel for talking-face forgeries that does not depend on audio, texture, or frequency features, so it works even when audio is missing or tampered with.
- The stable per-method ranking implies that the forensic difficulty of a future talking-face generator can be anticipated from its synthesis strategy (latent-pixel synthesis vs source-frame warping) without running detection experiments.
- The DeepFakesON-Phys reproduction (0.999 on face-swap data, 0.622 on talking-face data) shows that existing rPPG detectors do not transfer to talking-face forgeries; talking-face-specific training is required.
- IP-LAP's resistance, including the inversion under isolated training, identifies identity-preserving landmark-based synthesis as the main open challenge for physiological forensics.
- The near-parity with general-purpose detectors on a strict identity-disjoint protocol suggests physiological signals generalize across identities, strengthening the case for including rPPG in multi-modal detectors.
Where Pith is reading between the lines
- A natural extension the paper leaves implicit: the generator-specific waveform shape (incoherent noise vs near-sinusoidal artifact) could be used for generator attribution or fingerprinting, not just binary real/fake classification, since the physiological artifact encodes the synthesis family.
- The paper's dual-use caveat can be pushed further: if adversarial synthesis injects a plausible cardiac waveform into latent-synthesis methods, the detection gap for those methods may widen; a direct experiment would be to add synthetic rPPG to Real3DPortrait outputs and measure the AUC drop.
- The acknowledged skin-tone and lighting sensitivity suggests the reported aggregate AUC may not hold uniformly across demographic groups; a stratified evaluation by Fitzpatrick skin type is a testable next step that could revise the per-method ordering for some generators.
- The authors' reliance on positional method labels for the per-method analysis is a verification gap independent of the aggregate result; if the Celeb-DF++ ordering is ever corrected, the ranking might shift, and the 'physiological fingerprint' interpretation should be re-tested with verified labels.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the first rPPG-based detection framework specifically for talking-face (TF) deepfakes. It extracts 160-sample pulse waveforms with RhythmFormer from face crops and trains lightweight 1D classifiers (ResNet/CNN/Transformer) under a subject-independent identity split on the TF subset of Celeb-DF++. The authors report AUC 0.806 with the 1D ResNet, reproduce DeepFakesON-Phys at AUC 0.622 on the same TF data, and present a method-stratified analysis showing a per-generator AUC spread from 0.690 (IP-LAP) to 0.985 (Real3DPortrait), interpreted as a stable, generator-specific physiological property linked to each method's pixel synthesis strategy.
Significance. If the main claims hold, the paper makes a useful empirical contribution: a physiological-channel detector for a forgery category where image-based detectors are known to be weak, a controlled reproduction of a prior rPPG detector demonstrating distribution shift, and an analysis suggesting that TF generators differ greatly in rPPG detectability. The paper also ships public code, uses identity-disjoint splits, and reports multi-seed means, which are strengths. However, the paper's advertised primary theoretical contribution—the method-dependent detectability ranking—rests on an explicitly unverifiable positional assignment of method labels, and the headline comparison to Effort is made against a different evaluation protocol. The aggregate real-vs-fake result and the reproduction study are more robust, but the central advertised novelty is not adequately supported as written.
major comments (4)
- [§6.4, Table 4, Fig. 2] The primary theoretical contribution—the stable per-generator AUC ranking (0.690–0.985)—depends entirely on the positional assignment of method labels to fake videos. Section 6.4 admits that this assignment 'cannot be independently verified at the video level' because method names are not recoverable from filenames. If the assumed Celeb-DF++ ordering is even partially wrong, Table 4, Fig. 2, Fig. 3, and the interpretation in §6.3 would all be invalid, even though the aggregate real-vs-fake AUC would still stand. The authors must either provide an independently verifiable per-video label mapping (e.g., a released label file tied to dataset metadata) or remove/reframe the per-method claim. This is a load-bearing limitation, not a cosmetic one.
- [§5.3, §6.1, §2.5] The headline 'within 2.4 points of Effort (ICML 2025)' compares the proposed TF-only AUC (0.806) with Effort's 0.830 reported on the full Celeb-DF++ benchmark across all forgery categories. Since §2.5 states that TF is the hardest category for existing detectors (AUC 50–70%), this is not a same-protocol comparison and the 'within 2.4 points' claim is not established. The authors should report Effort on the same TF-only split if possible, or explicitly state that no such per-category result exists and soften the claim accordingly.
- [§5.4, Table 4, §6.3] The claim that the method ranking is 'perfectly consistent across every evaluation protocol' is not backed by a protocol-by-protocol table; only the 18-identity result is shown. In addition, the mechanistic explanation that detection difficulty is determined by 'pixel synthesis strategy' (warp vs. synthesize) is a post hoc categorization: no direct measure of pixel preservation or temporal color coherence is provided, and the paper does not test this explanation against alternative factors (e.g., video length, compression, identity distribution across methods). The stability claim and the causal-sounding interpretation should be either directly supported with the full AUC matrix and quantitative covariates, or explicitly downgraded to a hypothesis.
- [§3.1, §4.3, §5.1] The evaluation unit is ambiguous. Section 3.1 defines the classifier input as a 'per-video rPPG waveform', but §4.3 windows long videos into multiple 160-frame clips at stride 60, and Table 3 reports AUC without stating whether it is window-level or video-level. The DeepFakesON-Phys reproduction in §5.1, by contrast, is explicitly 'video-level'. If the main AUC numbers are window-level, they are not directly comparable to video-level benchmarks and may benefit from within-video autocorrelation. The authors should specify the unit of evaluation, aggregate to video-level if that is the forensic claim, or report both.
minor comments (5)
- [Abstract, §4.1] The text says 'yielding 585 real waveforms (585 source videos with stride-60 temporal windowing produce 2,435 waveforms in total)' and then reports 2,435 real training waveforms. The '585 real waveforms' phrase appears to be a typo and should be corrected for clarity.
- [§4.2, §5.3] Table 1 lists a 9-identity validation split, but the main result in §5.3 uses an '18-identity test set (val and test identities combined)'. Clarify whether the validation identities were used in any model-selection or early-stopping procedure before being combined into the final evaluation; if they were fully untouched, state this explicitly.
- [Fig. 3] The waveforms shown are explicitly 'representative' and appear to have been selected after seeing the results. This is acceptable as illustration, but the caption should state that these are hand-picked examples, not a systematic characterization of each method's output.
- [§2.5] The statement that 'robust results are limited to FS forgeries (AUC>85%), while FR and TF forgeries yield degraded detection rates (AUC 50–70%)' is not followed by a citation or a table. Please add the source of these numbers.
- [§5.4] The term 'theoretical contribution' is used in the abstract and §5.4 for what is an empirical, method-stratified observation. Consider using 'empirical finding' or 'forensic characterization' unless the paper provides a formal theoretical model predicting the ranking.
Circularity Check
No significant circularity: the detection pipeline is fit on held-out identities, and the per-method interpretability is post hoc explanation, not a fitted input.
full rationale
The claimed derivation chain is a standard supervised evaluation, not a circular reduction. RhythmFormer is a pretrained external rPPG extractor (UBFC checkpoint), and the 1D ResNet is trained on subject-disjoint splits; the target label (real vs. TF fake) is never used to fit the waveform extractor or to define the generator ranking. The aggregate AUC 0.806 is therefore an independent measurement, and the comparison against Effort is an external benchmark. The per-method AUC spread (0.690-0.985) is a descriptive stratification of the same held-out test set; the explanation in terms of pixel synthesis strategy is a post hoc qualitative interpretation that does not set any parameter, appear in the loss, or enter the ROC computation. The paper's own acknowledged limitation (Section 6.4, "method names are not recoverable from filenames... cannot be independently verified at the video level") concerns the provenance of method labels, not a circular fit; a mislabeled input would be a correctness risk, not a self-referential derivation. There are no self-citations, no imported uniqueness theorem, and no ansatz smuggled in by citation: RhythmFormer and DeepFakesON-Phys are external prior work with released weights. Thus no prediction in the paper reduces by construction to its inputs, and the circularity score is 0.
Axiom & Free-Parameter Ledger
free parameters (11)
- 1D ResNet learning rate =
1e-3
- 1D ResNet weight decay =
5e-4
- 1D ResNet dropout =
0.5
- Transformer learning rate =
5e-4
- Transformer weight decay =
1e-3
- Transformer dropout =
0.1
- Temporal window length =
160 frames
- Temporal stride =
60 frames
- Face bounding-box expansion =
1.5x
- Zero-mask length (augmentation) =
U[10,25] frames
- Z-score epsilon =
1e-6
axioms (5)
- domain assumption RhythmFormer pretrained on UBFC-rPPG transfers to Celeb-DF++ faces and produces waveforms that carry physiological information.
- domain assumption Celeb-DF++ TF subset real/fake labels are correct.
- ad hoc to paper Positional assignment of method labels to fake videos matches the actual generator for each video.
- domain assumption z-score normalization of each waveform removes identity-specific baseline while preserving discriminative temporal structure.
- ad hoc to paper Per-generator detection difficulty is determined by pixel synthesis strategy (warp vs. synthesize).
Cite this review
Pith. "Pith review of Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection." pith.science (2026). https://pith.science/paper/HHLHNTVY
@misc{pith2026260721776,
author = {Pith},
title = {Pith review of: Physiological Signals as a Forensic Modality for Talking-Face Deepfake Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/HHLHNTVY}},
note = {Machine review of arXiv:2607.21776}
}
read the original abstract
Talking-face (TF) deepfake generation synthesizes photore- alistic facial video from a static source image and an au- dio signal, producing forgeries that current image-based detectors consistently fail to identify. Unlike face-swap ma- nipulation, TF synthesis has no underlying real video from which to inherit physiological characteristics, making re- mote photoplethysmography (rPPG) a uniquely motivated detection modality for this forgery category. We propose a detection framework that extracts per-video rPPG wave- forms via RhythmFormer and trains a suite of lightweight classifiers to distinguish real from synthesized physiologi- cal signals. Evaluated on the TF subset of Celeb-DF++ un- der a strict subject-independent protocol, where test identi- ties are completely separated from training identities, our 1D ResNet achieves an AUC of 0.806 and EER of 27.8%, placing it within 2.4 points of the best published general- purpose detector (Effort, ICML 2025) while operating ex- clusively on the physiological channel. We document a con- trolled reproduction study of DeepFakesON-Phys, the rep- resentative prior rPPG detector, demonstrating degrada- tion from AUC 0.999 on legacy face-swap data to 0.622 on the TF subset of Celeb-DF++. We further show that detec- tion difficulty is strongly method-dependent: AUC ranges from 0.985 (Real3DPortrait) to 0.690 (IP-LAP) across the seven TF generators, with the ranking remaining perfectly stable across all evaluation protocols. This spread reflects an interpretable physiological property of each generator rather than evaluation noise, and constitutes the primary theoretical contribution of the work.
Figures
Reference graph
Works this paper leans on
-
[1]
Deepfake video detection through optical flow based CNN
Giuseppe Boccignone et al. Deepfake video detection through optical flow based CNN. InInt. Conf. Pattern Recog., 2022. 1
2022
-
[2]
EchoMimic: Lifelike audio-driven portrait animations through editable landmark conditions
Zhiyuan Chen et al. EchoMimic: Lifelike audio-driven portrait animations through editable landmark conditions. arXiv:2407.08136, 2024. 1, 2
Pith/arXiv arXiv 2024
-
[3]
Fake- Catcher: Detection of synthetic portrait videos using bio- logical signals.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
Umur Aybars Ciftci, Ilke Demir, and Lijun Yin. Fake- Catcher: Detection of synthetic portrait videos using bio- logical signals.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020. 1, 2
2020
-
[4]
Soumyya Kanti Datta, Shan Jia, and Siwei Lyu. Detecting lip-syncing deepfakes: Vision temporal transformer for ana- lyzing mouth inconsistencies.arXiv:2504.01470, 2025. 2
Pith/arXiv arXiv 2025
-
[5]
Robust pulse rate from chrominance-based rPPG.IEEE Transactions on Biomedical Engineering, 60(10):2878–2886, 2013
Gerard de Haan and Vincent Jeanne. Robust pulse rate from chrominance-based rPPG.IEEE Transactions on Biomedical Engineering, 60(10):2878–2886, 2013. 2
2013
-
[6]
Lips don’t lie: A generalisable and robust approach to face forge detection
Alexandros Haliassos et al. Lips don’t lie: A generalisable and robust approach to face forge detection. InIEEE Conf. Comput. Vis. Pattern Recog., 2021. 1, 2
2021
-
[7]
DeepFakesON-Phys: DeepFakes de- tection based on heart rate estimation.arXiv:2010.00400,
Javier Hernandez-Ortega, Ruben Tolosana, Julian Fierrez, and Aythami Morales. DeepFakesON-Phys: DeepFakes de- tection based on heart rate estimation.arXiv:2010.00400,
Pith/arXiv arXiv 2010
-
[8]
FLOAT: Generative motion latent flow matching for audio-driven talking portrait
Taekyung Ki, Dongchan Min, and Gyeongsu Chae. FLOAT: Generative motion latent flow matching for audio-driven talking portrait. InInt. Conf. Comput. Vis., 2025. 1, 2
2025
-
[9]
Yuezun Li, Delong Zhu, Xinjie Cui, and Siwei Lyu. Celeb- DF++: A large-scale challenging video deepfake benchmark for generalizable forensics.arXiv:2507.18015, 2025. 1, 2, 4
Pith/arXiv arXiv 2025
-
[10]
Identifying deepfake videos with optical flow and biological signals.arXiv:2106.11468,
Tianyi Liang and Xin Deng. Identifying deepfake videos with optical flow and biological signals.arXiv:2106.11468,
-
[11]
AniTalker: Animate vivid and diverse talk- ing faces via identity-decoupled facial motion encoding
Tao Liu et al. AniTalker: Animate vivid and diverse talk- ing faces via identity-decoupled facial motion encoding. arXiv:2405.03121, 2024. 1, 2
Pith/arXiv arXiv 2024
-
[12]
Xiaoyi Liu et al. Lips are lying: Spotting the temporal incon- sistency between audio and visual in lip-syncing deepfakes. arXiv:2401.15668, 2024. 1, 2
Pith/arXiv arXiv 2024
-
[13]
A meta-analysis of the impact of skin tone and gender on non-contact photoplethysmography measurements
Ewa M Nowara, Daniel McDuff, and Ashok Veeraraghavan. A meta-analysis of the impact of skin tone and gender on non-contact photoplethysmography measurements. InIEEE Conf. Comput. Vis. Pattern Recog. Worksh., 2020. 7
2020
-
[14]
SpecAugment: A simple data augmen- tation method for automatic speech recognition
Daniel S Park et al. SpecAugment: A simple data augmen- tation method for automatic speech recognition. InInter- speech, 2019. 4
2019
-
[15]
DeepRhythm: Exposing deepfakes with attentional visual heartbeat rhythms
Hanqing Qi et al. DeepRhythm: Exposing deepfakes with attentional visual heartbeat rhythms. InACM Int. Conf. Mul- timedia, 2020. 1, 2
2020
-
[16]
EDTalk: Efficient disentanglement for emo- tional talking head synthesis
Shuai Tan et al. EDTalk: Efficient disentanglement for emo- tional talking head synthesis. InEur. Conf. Comput. Vis.,
-
[17]
Re- mote plethysmographic imaging using ambient light.Optics Express, 16(26):21434–21445, 2008
Wim Verkruysse, Lars O Svaasand, and J Stuart Nelson. Re- mote plethysmographic imaging using ambient light.Optics Express, 16(26):21434–21445, 2008. 1, 2
2008
-
[18]
Algorithmic principles of remote PPG
Wenjin Wang, Albertus C den Brinker, Sander Stuijk, and Gerard de Haan. Algorithmic principles of remote PPG. IEEE Transactions on Biomedical Engineering, 64(7):1479– 1491, 2017. 1, 2
2017
-
[19]
Detecting deepfake videos with temporal dropout 3DCNN
Yue Wu et al. Detecting deepfake videos with temporal dropout 3DCNN. InIJCAI, 2023. 1, 2
2023
-
[20]
Orthogonal subspace decom- position for generalizable AI-generated image detection
Zhiyuan Yan, Jiangming Wang, Peng Jin, Ke-Yue Zhang, Chengchun Liu, Shen Chen, Taiping Yao, Shouhong Ding, Baoyuan Wu, and Li Yuan. Orthogonal subspace decom- position for generalizable AI-generated image detection. In Proceedings of the 42nd International Conference on Ma- chine Learning, pages 70268–70288. PMLR, 2025. 2, 5
2025
-
[21]
Real3D-Portrait: One-shot realistic 3D talking portrait synthesis
Zhenhui Ye et al. Real3D-Portrait: One-shot realistic 3D talking portrait synthesis. InInt. Conf. Learn. Represent.,
-
[22]
Remote photoplethysmograph signal mea- surement from facial videos using spatio-temporal networks
Zitong Yu et al. Remote photoplethysmograph signal mea- surement from facial videos using spatio-temporal networks. InBrit. Mach. Vis. Conf., 2019. 2
2019
-
[23]
PhysFormer++: Facial video-based phys- iological measurement with slowfast temporal difference transformer.Int
Zitong Yu et al. PhysFormer++: Facial video-based phys- iological measurement with slowfast temporal difference transformer.Int. J. Comput. Vis., 2023. 2
2023
-
[24]
SadTalker: Learning realistic 3D mo- tion coefficients for stylized audio-driven single image talk- ing face animation
Wenxuan Zhang et al. SadTalker: Learning realistic 3D mo- tion coefficients for stylized audio-driven single image talk- ing face animation. InIEEE Conf. Comput. Vis. Pattern Recog., 2023. 1, 2
2023
-
[25]
Identity-preserving talking face gen- eration with landmark and appearance priors
Zhimeng Zhong et al. Identity-preserving talking face gen- eration with landmark and appearance priors. InIEEE Conf. Comput. Vis. Pattern Recog., 2023. 1, 2
2023
-
[26]
RhythmFormer: Extracting patterned rPPG signals based on periodic sparse attention.Pattern Recognition, 164, 2025
Bochao Zou, Zizheng Guo, Jiansheng Chen, and Huimin Ma. RhythmFormer: Extracting patterned rPPG signals based on periodic sparse attention.Pattern Recognition, 164, 2025. 1, 2, 3 8
2025
This paper was first reviewed by deepseek-v4-flash on August 1, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.