REVIEW 4 major objections 6 minor 33 references
Multimodal Biometric Authentication Using Camera-Based PPG and Fingerprint Fusion
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper claims that fusing camera-based pulse signals with fingerprint-like ridge images extracted from the same fingertip video lifts smartphone authentication accuracy to 100% with a 0.1% equal error rate in a single session, and to…
desk verdict Novel fusion of camera PPG and fingertip fingerprint frames with SSM encoders and cross-modal attention, but the reported SOTA accuracy is not trustworthy until the evaluation protocol and split are specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is a pair of homogeneous SSM encoders—each a deep sequence model whose core is a discretized linear state-space system $h'(t) = A h(t) + B x(t)$, $y(t) = C h(t) + D x(t)$ with a structured matrix $A$—that map both the PPG beat waveform and the flattened fingerprint image into embeddings. A multi-head cross-modal attention module lets each modality query the other, and a distribution-oriented contrastive loss (InfoNCE on exponentially moving average cluster means plus a normalized variance term) aligns the two distributions in a shared latent space. The SSM encoders are what extract long-range dependencies from the 300-sample beats and 4096-pixel image sequences, and the alignment loss is what makes the fusion additive rather than redundant.
What would settle it
Compare the extracted fingerprint frames from different sessions of the same subject against conventional fingerprint matching algorithms: if the images fail to produce matching minutiae or ridge flow consistent with a real fingerprint, then the claimed identity signal must come from the PPG branch or from dataset-specific artifacts, and the 0.1% single-session EER would collapse under cross-device or cross-pressure testing. A simpler observational check is to print the extracted 'fingerprint' images and see whether ridge structure is visually resolvable at 64×64 resolution.
Extended reading notes
Core claim
The paper claims that treating both modalities as sequences—PPG beats as 300-point waveforms and fingerprint images flattened into 64×64 pixel sequences—and feeding them through two homogeneous structured state-space (SSM) encoders with multi-head cross-modal attention yields a fused latent representation in which the user's identity is far more separable than either modality alone. On the paper's own evaluation, the fused system reaches 0.1% EER in a single-session setting (V1, 40 subjects), improving over the 13.9% EER of the camera-based See Red baseline and matching the 0.1% EER of the best IR-device PPG system; in the two-session setting (V2, 27 subjects, about 7 days apart) it reaches 6.9% EER versus 21.7% for the camera-only baseline. The authors attribute the gain to a distribution-oriented contrastive loss that aligns first-order moments and variances of the two modalities in a shared latent space while keeping other users' distributions apart.
Load-bearing premise
The system assumes the ridge-like images produced by contrast enhancement and edge detection of fingertip video frames contain stable, identity-specific fingerprint structure, even though the paper shows no example images or quality metrics for these frames; if those images mainly reflect contact pressure, illumination, or motion, the large fusion gains will not transfer across sessions or devices.
Editorial extensions
If this is right
- If the reported numbers hold, smartphone-camera authentication becomes competitive with dedicated IR PPG hardware, removing the need for specialized sensors in daily verification tasks.
- The many-to-many pairing of PPG beats with fingerprint images, combined with distribution alignment, suggests the system tolerates within-session variability in either modality without retraining per user.
- The two-session result (94.6% ACC, 6.9% EER) implies the extracted fingerprint frames carry identity information stable across at least a week, not just contact artifacts of a single recording.
- The ablation numbers (PPG-only EER 16.7% vs fused 0.1% in V1) indicate that the fingerprint modality, not the SSM backbone alone, is what drives the accuracy gain.
Reading between the lines
- A direct test of the fingerprint-pipeline assumption would be to run standard fingerprint minutiae matching on the extracted ridge images and compare against a conventional fingerprint database; if the images do not contain genuine ridge structure, the two-session EER should degrade sharply when the recording device, finger pressure, or lighting changes.
- The same architecture could be applied to other paired physiological cues that are jointly captured by a single sensor, such as finger ECG plus fingertip imaging, or iris plus periocular texture, where distribution alignment across modalities might confer similar robustness.
- Because the paper does not report cross-device or cross-subject-transfer experiments, an implicit untested extension is whether the fused representation learned on one phone model transfers to another; the 64×64 downsampled fingerprint images are likely to be device-sensitive.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents a multimodal biometric authentication method that uses a smartphone camera to capture fingertip video and extracts both PPG waveforms and fingerprint-like images from the same recording. The proposed network contains two structured state-space model (SSM) encoders, a multi-head cross-modal attention module, and a distribution-oriented contrastive loss that aligns the two modalities in a shared latent space. The authors collect two datasets, V1 (40 subjects, single session) and V2 (27 subjects, two sessions), and report 100% ACC/0.1% EER on V1 and 94.6% ACC/6.9% EER on V2, surpassing the camera-based See Red baseline and approaching IR-device PPG performance. An ablation study compares the full model with PPG-only, fingerprint-only, CNN-LSTM, and ResNet-18 baselines.
Significance. The paper's intended contribution is timely and potentially valuable: a smartphone-only acquisition setup that achieves IR-level PPG authentication with two modalities would be practically significant. The architecture is reasonably well motivated, and the distribution-alignment loss (Eqs. 5-6) is a sensible way to handle one-to-many correspondences between PPG beats and fingerprint images. However, the manuscript does not provide code, data, or a detailed validation protocol, and the reported empirical gains are not currently interpretable. The claimed state-of-the-art result therefore does not yet constitute a reliable scientific contribution; its significance depends entirely on a leakage-controlled re-evaluation.
major comments (4)
- [III.B, Table III] Section III.B states that the authors duplicate data samples of the true user by allowing PPG beat waveforms and fingerprint images to be randomly matched in a many-to-many relationship, but it does not say whether this augmentation is applied before or after the train/test split. If it is applied to the full dataset, the same fingerprint image or PPG beat can occur in both training and test pairs, allowing the network to memorize identity-specific content rather than learn cross-modal generalization. This is a concrete leakage path, and it is especially serious because the fingerprint-only SSM baseline already reaches 1.31% EER on V1 (Table III), so the reported 0.1% EER could be explained by memorizing fingerprint images. Please specify a subject-disjoint, recording-disjoint split and perform augmentation only within the training partition.
- [Tables I-II] The evaluation protocol is underspecified in several load-bearing ways: the manuscript never states the number of enrolled users, the size and composition of the impostor set, the decision threshold, or whether accuracy is computed per subject, per pair, or per recording. For V2 it is not clear whether enrollment and testing use different sessions or whether both sessions are mixed in a single partition. Tables I and II compare cohorts of 14, 24, 27, 31, 37, and 40 subjects from different devices and datasets, so the claim that the proposed system is comparable to IR-based SOTA is not supported by matched evaluation. Please provide a precise protocol and re-report all comparisons under the same setting.
- [II.A.2] The fingerprint image branch is not validated at all. The paper gives no example images, no ridge-quality or minutiae statistics, and no comparison with conventional fingerprints, so it is unclear whether the CLAHE plus Canny plus averaging plus downsampling pipeline produces stable identity-discriminative ridge structure or merely encodes illumination, contact pressure, and motion artifacts. Because Table III shows that the fingerprint-only SSM already achieves 1.31% EER on V1, the validity of the fingerprint images is load-bearing for the claimed fusion improvement; please include visual examples, quality metrics, and a small matching experiment that demonstrates identity discriminability.
- [III.C, Table III] No error bars, standard deviations, or number of repeated runs are reported for any table. Moreover, the main model is trained for 200 epochs with batch size 256, while the ablation baselines are trained for at most 80 epochs with batch size 32 (Section III.C). These differences confound the architectural comparison in Table III, so the conclusion that SSM encoders are better suited to camera-based PPG than CNN-LSTM is not established. Please report repeated-run statistics with matched training budgets and identical data partitions.
minor comments (6)
- [II.B, Fig. 1] The notation fβ(xu; θβ) should be fβ(xv; θβ), since xv is the fingerprint embedding.
- [II.C, Eq. (7)] The recursive update writes (α, β) ⊙ (ˆµk_v, ˆµk_v); the second component should be ˆµk_u, so that the two modality moment estimates are updated separately.
- [II.C, Eq. (8)] The text refers to penalty factors λA and λC for equation (8), but equation (8) contains λA and λS; λC is the weight of the classification loss, not a penalty factor.
- [Introduction] The phrase 'features such as such as peaks' contains a duplicated phrase and should be corrected.
- [References] Reference [12] is listed in the bibliography but is not cited in the text, and the citation numbers jump from [11] to [13].
- [Tables I-II] The column headers would be easier to interpret if they specified the exact evaluation metric, such as verification accuracy at a fixed false acceptance rate, and the units for EER; as written, ACC and EER are not sufficiently defined.
Circularity Check
No significant circularity; the central result is empirical and independently benchmarked, with only minor non-load-bearing self-citations.
full rationale
This is an empirical learning paper rather than a derivation chain, so there is no equation-level step in which an output is its own input. The preprocessing (CLAHE, Canny, averaging, downsampling) produces fingerprint-like images and PPG beats from the same recordings, but these are different spatial and temporal quantities, not identical by construction. The contrastive losses (5)-(6) and EMA update (7) are supervised training objectives: they align positive pairs and separate negatives using the same labels they are later evaluated against, as in standard metric learning, and this does not make the reported accuracy a tautology. The headline 100% ACC / 0.1% EER is an empirical result on the authors' own V1/V2 datasets with external baselines (See Red [19], IR systems [10], [27], [28]), so the central claim has independent content. The self-citations to [10], [30], [31], [32] are used as baselines and as a source of data augmentation, not as a uniqueness theorem or as justification that the fingerprint extraction is valid; they are not load-bearing for the core result. The underspecified train/test protocol and the many-to-many random pairing in Section III.B create a possible data-leakage risk (same fingerprint or PPG sample could appear on both sides of the split), but that is an evaluation-validity concern, not circularity in the derivation. Accordingly, no specific circular step can be quoted and exhibited, and the score is near zero.
Assumptions & free parameters
free parameters (10)
- InfoNCE temperature tau =
not reported
- EMA decay alpha =
0.9
- EMA decay beta =
0.9
- Alignment penalty lambda_A =
0.8
- Spread penalty lambda_S =
0.05
- SSM hidden state dimension dh =
64
- Number of SSM stacks N =
2
- Number of attention heads =
4
- Input sequence lengths =
300 and 4096 (64x64 pixels)
- BCE class weights =
not reported
assumptions (5)
- standard math SSM discretization relations in Eq. (2) are a valid zero-order-hold discretization of the continuous system in Eq. (1).
- standard math Scaled dot-product attention and InfoNCE contrastive loss behave as claimed.
- domain assumption Fingertip-on-lens video with flashlight contains both a recoverable PPG signal and fingerprint ridge information after CLAHE and Canny processing.
- domain assumption Random many-to-many pairing of PPG beats and fingerprint images from the same user preserves identity and does not leak information across train and test sets.
- ad hoc to paper First-order moments and normalized variance are sufficient statistics for aligning cross-modal identity distributions.
Cite this review
Pith. "Pith review of Multimodal Biometric Authentication Using Camera-Based PPG and Fingerprint Fusion." pith.science (2026). https://pith.science/paper/UVVHF6JI
@misc{pith2026241205660,
author = {Pith},
title = {Pith review of: Multimodal Biometric Authentication Using Camera-Based PPG and Fingerprint Fusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/UVVHF6JI}},
note = {Machine review of arXiv:2412.05660}
}
read the original abstract
Camera-based photoplethysmography (PPG) obtained from smartphones has shown great promise for personalized healthcare and secure authentication. This paper presents a multimodal biometric system that integrates PPG signals extracted from videos with fingerprint data to enhance the accuracy of user verification. The system requires users to place their fingertip on the camera lens for a few seconds, allowing the capture and processing of unique biometric characteristics. Our approach employs a neural network with two structured state-space model (SSM) encoders to manage the distinct modalities. Fingerprint images are transformed into pixel sequences, and along with segmented PPG waveforms, they are input into the encoders. A cross-modal attention mechanism then extracts refined feature representations, and a distribution-oriented contrastive loss function aligns these features within a unified latent space. Experimental results demonstrate the system's superior performance across various evaluation metrics in both single-session and dual-session authentication scenarios.
Figures
Reference graph
Works this paper leans on
-
[1]
Maltoni, D
D. Maltoni, D. Maio, A. K. Jain, S. Prabhakar et al. , Handbook of fingerprint recognition. Springer, 2009, vol. 2
2009
-
[2]
Touch-less palm print biometrics: Novel design and implementation,
G. K. O. Michael, T. Connie, and A. B. J. Teoh, “Touch-less palm print biometrics: Novel design and implementation,”Image Vis. Comput., vol. 26, no. 12, pp. 1551–1560, 2008
work page 2008
-
[3]
Privacy–enhancing face biometrics: A comprehensive survey,
B. Meden, P. Rot, P. Terh ¨orst, N. Damer, A. Kuijper, W. J. Scheirer, A. Ross, P. Peer, and V . ˇStruc, “Privacy–enhancing face biometrics: A comprehensive survey,” IEEE Trans. Inf. Forensics Security, vol. 16, pp. 4147–4183, 2021
work page 2021
-
[4]
Recognition of human iris patterns for biometric identification,
E. Ali, E. Ahmed, and A. Ali, “Recognition of human iris patterns for biometric identification,” J. Eng. Appl. Sci. - Cairo , vol. 54, no. 6, p. 635, 2007
work page 2007
-
[5]
Joint speaker verification and antispoofing in the i-vector space,
A. Sizov, E. Khoury, T. Kinnunen, Z. Wu, and S. Marcel, “Joint speaker verification and antispoofing in the i-vector space,” IEEE Trans. Inf. Forensics Security, vol. 10, no. 4, pp. 821–832, 2015
work page 2015
-
[6]
Car- diovascular biometrics: combining mechanical and electrical signals,
I. Odinaka, J. A. O’Sullivan, E. J. Sirevaag, and J. W. Rohrbaugh, “Car- diovascular biometrics: combining mechanical and electrical signals,” IEEE Trans. Inf. Forensics Security , vol. 10, no. 1, pp. 16–27, 2014
work page 2014
-
[7]
Ecg biometric with abnormal cardiac conditions in remote monitoring system,
K. A. Sidek, I. Khalil, and H. F. Jelinek, “Ecg biometric with abnormal cardiac conditions in remote monitoring system,” IEEE Trans. Syst. Man Cybern. Syst., vol. 44, no. 11, pp. 1498–1509, 2014
work page 2014
-
[8]
Unveiling the biometric potential of finger-based ecg signals,
A. Lourenc ¸o, H. Silva, and A. Fred, “Unveiling the biometric potential of finger-based ecg signals,” Comput. Intell. Neurosci. , vol. 2011, pp. 1–8, 2011
work page 2011
Show all 33 references
-
[9]
A novel biometric approach in human verification by photoplethysmographic signals,
Y . Gu, Y . Zhang, and Y . Zhang, “A novel biometric approach in human verification by photoplethysmographic signals,” in Proc. 4th Int. IEEE EMBS Special Topic Conf. Inf. Technol. Appl. Biomed., 2003, pp. 13–14
2003
-
[10]
Evaluation of the time stability and uniqueness in ppg-based biometric system,
D. Y . Hwang, B. Taha, D. S. Lee, and D. Hatzinakos, “Evaluation of the time stability and uniqueness in ppg-based biometric system,” IEEE Trans. Inf. Forensics Security, vol. 16, pp. 116–130, 2020
2020
-
[11]
Photoplethysmographic authentication through fuzzy logic,
Y . Gu and Y . Zhang, “Photoplethysmographic authentication through fuzzy logic,” in Proc. IEEE EMBS Asian–Pacific Conf. Biomed. Eng. , 2003, pp. 136–137
2003
-
[12]
A pilot study on using derivatives of photoplethysmographic signals as a biometric identifier,
J. Yao, X. Sun, and Y . Wan, “A pilot study on using derivatives of photoplethysmographic signals as a biometric identifier,” in Proc. 29th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. , 2007, pp. 4576–4579
2007
-
[13]
Enhancement in the design of biometric identification system based on photoplethys- mography data,
G. R. S. NS, D. M. N. S. Sadhasivam, and D. A. Samraj, “Enhancement in the design of biometric identification system based on photoplethys- mography data,” in Proc. Int. Conf. Green High Perform. Comput., 2013
2013
-
[14]
Human recognition from photoplethysmography (ppg) based on non-fiducial features,
N. Karimian, Z. Guo, M. Tehranipoor, and D. Forte, “Human recognition from photoplethysmography (ppg) based on non-fiducial features,” in IEEE Int. Conf. Acoust. Speech Signal Process.(ICASSP) , 2017, pp. 4636–4640
2017
-
[15]
An adaptive deep learning approach for ppg-based identification,
V . Jindal, J. Birjandtalab, M. B. Pouyan, and M. Nourani, “An adaptive deep learning approach for ppg-based identification,” in Proc. 38th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. (EMBC) , 2016, pp. 6401–6404
2016
-
[16]
A multimodal biometric identification system based on ecg and ppg signals,
C. Yaacoubi, R. Besrour, and Z. Lachiri, “A multimodal biometric identification system based on ecg and ppg signals,” in Proc. 2nd Int. Conf. Digital Tools & Uses Congress , 2020, pp. 1–6
2020
-
[17]
Cornet: Deep learning framework for ppg-based heart rate estimation and biometric identification in ambulant environment,
D. Biswas, L. Everson, M. Liu, M. Panwar, B.-E. Verhoef, S. Patki, C. H. Kim, A. Acharyya, C. Van Hoof, M. Konijnenburg et al., “Cornet: Deep learning framework for ppg-based heart rate estimation and biometric identification in ambulant environment,” IEEE Trans. Biomed. Circu...
2019
-
[18]
A low-power ppg processor for real-time biometric identification and heart rate estima- tion,
H.-C. Chang, T. Wang, C.-A. Liao, and T.-T. Liu, “A low-power ppg processor for real-time biometric identification and heart rate estima- tion,” IEEE Trans. Circuits Syst. II: Exp. Briefs , vol. 70, no. 10, pp. 3932–3936, 2023
2023
-
[19]
Seeing red: Ppg biometrics using smartphone cameras,
G. Lovisotto, H. Turner, S. Eberz, and I. Martinovic, “Seeing red: Ppg biometrics using smartphone cameras,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops , 2020, pp. 818–819
2020
-
[20]
Efficiently modeling long sequences with structured state spaces,
A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396 , 2021
2021 arXiv
-
[21]
On the parameterization and initialization of diagonal state space models,
A. Gu, K. Goel, A. Gupta, and C. R ´e, “On the parameterization and initialization of diagonal state space models,” Adv. Neural Inf. Process. Syst., vol. 35, pp. 35 971–35 983, 2022
2022
-
[22]
Contrast-limited adaptive histogram equalization: speed and effective- ness,
S. Pizer, R. Johnston, J. Ericksen, B. Yankaskas, and K. Muller, “Contrast-limited adaptive histogram equalization: speed and effective- ness,” in Proc. First Conf. Vis. Biomed. Comput. , 1990, pp. 337–345
1990
-
[23]
Canny edge detection enhancement by scale multiplication,
P. Bao, L. Zhang, and X. Wu, “Canny edge detection enhancement by scale multiplication,” IEEE Trans. Pattern Anal. Mach. Intell. , vol. 27, no. 9, pp. 1485–1490, 2005
2005
-
[24]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Adv. Neural Inf. Process. Syst. , vol. 30, 2017
2017
-
[25]
Representation learning with contrastive predictive coding,
A. v. d. Oord, Y . Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748 , 2018
2018 arXiv
-
[26]
Delving into deep imbalanced regression,
Y . Yang, K. Zha, Y . Chen, H. Wang, and D. Katabi, “Delving into deep imbalanced regression,” in Proc. Int. Conf. Mach. Learn. (ICML) . PMLR, 2021, pp. 11 842–11 851
2021
-
[27]
Non-fiducial ppg-based authentication for healthcare application,
N. Karimian, M. Tehranipoor, and D. Forte, “Non-fiducial ppg-based authentication for healthcare application,” inProc. IEEE EMBS Int. Conf. Biomed. Health Informat. (BHI) , 2017, pp. 429–432
2017
-
[28]
Biometric authentication using the ppg: A long-term feasibility study,
J. Sancho, ´A. Alesanco, and J. Garc ´ıa, “Biometric authentication using the ppg: A long-term feasibility study,” Sensors, vol. 18, no. 5, p. 1525, 2018
2018
-
[29]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 770–778
2016
-
[30]
Pbgan: Learning ppg representations from gan for time-stable and unique verification system,
D. Y . Hwang, B. Taha, and D. Hatzinakos, “Pbgan: Learning ppg representations from gan for time-stable and unique verification system,” IEEE Trans. Inf. Forensics Security , vol. 16, pp. 5124–5137, 2021
2021
-
[31]
Variation-stable fusion for ppg-based biometric system,
D. Y . Hwang, B. Taha, and D. Hatzinakos, “Variation-stable fusion for ppg-based biometric system,” in IEEE Int. Conf. Acoust. Speech Signal Process.(ICASSP). IEEE, 2021, pp. 8042–8046
2021
-
[32]
A new score level fusion approach for stable user verification system using the ppg signal,
D. Y . Hwang, B. Taha, and D. Hatzinakos, “A new score level fusion approach for stable user verification system using the ppg signal,” J. Signal Process. Syst. , vol. 94, no. 8, pp. 787–798, 2022
2022
-
[33]
Biosignal classification for human identification based on convolutional neural networks,
A. I. Siam, A. Sedik, W. El-Shafai, A. A. Elazm, N. A. El-Bahnasawy, G. M. El Banby, A. A. Khalaf, and F. E. Abd El-Samie, “Biosignal classification for human identification based on convolutional neural networks,” International journal of communication systems , vol. 34, no. ...
2021
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.