{"id":"696bcb82-9244-4a5a-889f-1fa3843a327b","arxiv_id":"2412.05660","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":10,"one_line_summary":"Fusing camera-based PPG signals with fingerprint-like images from the same fingertip video gives 0.1% EER in one session and 6.9% EER across two sessions.","lead":"This paper combines a heartbeat signal and a fingertip texture pattern captured from one phone camera video to verify identity. The reported error rates are very low, but the dataset and code are private, so independent verification is not yet possible.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation protocol is underspecified and the many-to-many random pairing in §III.B can leak training samples into test pairs; the reported 0.1% EER may reflect fingerprint-image memorization rather than generalized multimodal authentication.","rationale":"The architecture is internally coherent, and the use of S4D encoders with cross-modal attention is a reasonable design, so the problem is not a mathematical contradiction. The claimed 0.1% EER could in principle be plausible, because the fingerprint-only SSM on the same pipeline already reaches 1.31% EER in V1 (Table III). However, the reported 100% ACC and 0.1% EER cannot be evaluated because the paper gives no train/test split, no subject-level detail, and no leakage control. The many-to-many random pairing in §III.B is the concrete mechanism: if a fingerprint image is sampled for training and then reused in a test pair, the classifier can memorize the exact image, and the fingerprint modality alone is strong enough to drive error rates near zero. Thus the fusion results may represent memorization rather than cross-modal generalization. A strict subject-disjoint re-run, with random pairing confined to the training partition, would settle whether the SOTA numbers transfer. Even if that re-run succeeds, the paper would still need public code/data and error bars to support the SOTA claim, but the leakage-controlled re-run is the most decisive single check. I therefore keep the reader's REJECT verdict: the empirical support as written is not sufficient.","tokens_in":9396,"tokens_out":3626,"duration_ms":37130,"concrete_test":"Obtain or re-run the V1/V2 evaluation under a strict subject-disjoint protocol: train on 39/26 subjects and test the remaining subject's full recordings, or use a fixed 80/20 subject split, with random positive pairing performed only inside the training partition and never crossing into test. Report per-subject EER/ACC with standard deviation and compare against the same protocol for the fingerprint-only SSM. If fused EER stays below ~1% while fingerprint-only is ~1.3%, the fusion claim holds; if the gap collapses or EER jumps above ~5%, the current Tables I–II reflect data leakage rather than generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central SOTA claim rests on an unspecified evaluation protocol, and the data augmentation described in §III.B creates a concrete leakage path. The paper states that for the true user the authors \"duplicate the data samples ... by allowing PPG beat waveforms and fingerprint images to be randomly matched in a many-to-many relationship.\" If this random matching is performed before the train/test partition, the same fingerprint image or PPG beat can appear in both training and test pairs. Since the fingerprint-only SSM baseline already reaches 1.31% EER on V1 (Table III), a classifier can memorize fingerprint images rather than learn identity-generalizing cross-modal features, yielding the reported 0.1% EER. The paper never specifies the split (subject-disjoint or recording-disjoint), the number of enrolled users, the impostor set, or whether the same-session data is used on both sides; Tables I–II mix 14-, 24-, 27-, 31-, 37-, and 40-subject cohorts from different devices and sessions. The fingerprint-image validity concern is real, but even granting that the fingerprint frames are discriminative, the headline accuracy numbers are not interpretable without a leakage-controlled protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents a multimodal biometric authentication method that uses a smartphone camera to capture fingertip video and extracts both PPG waveforms and fingerprint-like images from the same recording. The proposed network contains two structured state-space model (SSM) encoders, a multi-head cross-modal attention module, and a distribution-oriented contrastive loss that aligns the two modalities in a shared latent space. The authors collect two datasets, V1 (40 subjects, single session) and V2 (27 subjects, two sessions), and report 100% ACC/0.1% EER on V1 and 94.6% ACC/6.9% EER on V2, surpassing the camera-based See Red baseline and approaching IR-device PPG performance. An ablation study compares the full model with PPG-only, fingerprint-only, CNN-LSTM, and ResNet-18 baselines.","tokens_in":9675,"tokens_out":8356,"duration_ms":73257,"significance":"The paper's intended contribution is timely and potentially valuable: a smartphone-only acquisition setup that achieves IR-level PPG authentication with two modalities would be practically significant. The architecture is reasonably well motivated, and the distribution-alignment loss (Eqs. 5-6) is a sensible way to handle one-to-many correspondences between PPG beats and fingerprint images. However, the manuscript does not provide code, data, or a detailed validation protocol, and the reported empirical gains are not currently interpretable. The claimed state-of-the-art result therefore does not yet constitute a reliable scientific contribution; its significance depends entirely on a leakage-controlled re-evaluation.","major_comments":[{"comment":"Section III.B states that the authors duplicate data samples of the true user by allowing PPG beat waveforms and fingerprint images to be randomly matched in a many-to-many relationship, but it does not say whether this augmentation is applied before or after the train/test split. If it is applied to the full dataset, the same fingerprint image or PPG beat can occur in both training and test pairs, allowing the network to memorize identity-specific content rather than learn cross-modal generalization. This is a concrete leakage path, and it is especially serious because the fingerprint-only SSM baseline already reaches 1.31% EER on V1 (Table III), so the reported 0.1% EER could be explained by memorizing fingerprint images. Please specify a subject-disjoint, recording-disjoint split and perform augmentation only within the training partition.","section":"III.B, Table III"},{"comment":"The evaluation protocol is underspecified in several load-bearing ways: the manuscript never states the number of enrolled users, the size and composition of the impostor set, the decision threshold, or whether accuracy is computed per subject, per pair, or per recording. For V2 it is not clear whether enrollment and testing use different sessions or whether both sessions are mixed in a single partition. Tables I and II compare cohorts of 14, 24, 27, 31, 37, and 40 subjects from different devices and datasets, so the claim that the proposed system is comparable to IR-based SOTA is not supported by matched evaluation. Please provide a precise protocol and re-report all comparisons under the same setting.","section":"Tables I-II"},{"comment":"The fingerprint image branch is not validated at all. The paper gives no example images, no ridge-quality or minutiae statistics, and no comparison with conventional fingerprints, so it is unclear whether the CLAHE plus Canny plus averaging plus downsampling pipeline produces stable identity-discriminative ridge structure or merely encodes illumination, contact pressure, and motion artifacts. Because Table III shows that the fingerprint-only SSM already achieves 1.31% EER on V1, the validity of the fingerprint images is load-bearing for the claimed fusion improvement; please include visual examples, quality metrics, and a small matching experiment that demonstrates identity discriminability.","section":"II.A.2"},{"comment":"No error bars, standard deviations, or number of repeated runs are reported for any table. Moreover, the main model is trained for 200 epochs with batch size 256, while the ablation baselines are trained for at most 80 epochs with batch size 32 (Section III.C). These differences confound the architectural comparison in Table III, so the conclusion that SSM encoders are better suited to camera-based PPG than CNN-LSTM is not established. Please report repeated-run statistics with matched training budgets and identical data partitions.","section":"III.C, Table III"}],"minor_comments":[{"comment":"The notation fβ(xu; θβ) should be fβ(xv; θβ), since xv is the fingerprint embedding.","section":"II.B, Fig. 1"},{"comment":"The recursive update writes (α, β) ⊙ (ˆµk_v, ˆµk_v); the second component should be ˆµk_u, so that the two modality moment estimates are updated separately.","section":"II.C, Eq. (7)"},{"comment":"The text refers to penalty factors λA and λC for equation (8), but equation (8) contains λA and λS; λC is the weight of the classification loss, not a penalty factor.","section":"II.C, Eq. (8)"},{"comment":"The phrase 'features such as such as peaks' contains a duplicated phrase and should be corrected.","section":"Introduction"},{"comment":"Reference [12] is listed in the bibliography but is not cited in the text, and the citation numbers jump from [11] to [13].","section":"References"},{"comment":"The column headers would be easier to interpret if they specified the exact evaluation metric, such as verification accuracy at a fixed false acceptance rate, and the units for EER; as written, ACC and EER are not sufficiently defined.","section":"Tables I-II"}],"recommendation":"major_revision","confidential_remarks":"I recommend major revision rather than reject because the empirical questions are answerable with a re-run and a detailed protocol. The reader's stronger negative assessment is reasonable given the current text; if the authors cannot provide leakage-free results under a well-specified, subject-disjoint protocol, the paper should not be accepted. The modality-validity concern about the extracted fingerprint images is also central and needs direct evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a sensible architecture paper whose advertised numbers I would not yet trust. The combination—two SSM encoders for PPG beats and flattened fingerprint frames, cross-modal attention, distribution-alignment contrastive loss, all from one fingertip-on-lens capture—is genuinely new relative to the papers it cites, which mostly use CNN-LSTM, score fusion, or single modality. The ablation is helpful: it tells you fingerprint frames carry most of the signal (1.31% EER alone vs 16.7% for PPG alone on V1), and SSM beats CNN-LSTM on this camera PPG. Credit where due: the model is internally coherent and the data collection, 40 and 27 subjects with a two-session subset, is nontrivial.\n\nThe soft spot is the evaluation protocol, and it is load-bearing. Tables I and II compare the proposed system on its own private V1/V2 sets against literature results on different devices, different subject counts, and different protocols. There are no error bars. More seriously, Section III.B describes duplicating true-user samples by randomly matching PPG beats and fingerprint images many-to-many. If that random pairing happens before the train/test split, the same fingerprint image or PPG beat can appear on both sides, so the classifier can memorize fingerprint images rather than learn identity-generalizing fusion. The ablation numbers make the worry concrete: fingerprint-only SSM already reaches 1.31% EER, so the fused 0.1% EER may just be that baseline plus the classifier seeing familiar images. The paper never states the split (subject-disjoint? recording-disjoint?), enrollment size, impostor set, or session overlap. Also, the claimed fingerprint frames are just CLAHE + Canny + averaging, with no sample images or ridge-quality metrics shown, so it is hard to tell what identity information is actually in them.\n\nThis is not a case of bad math. It is a case of unreported protocol details making the central empirical claim uninterpretable. The architecture and the dataset are worth a serious look; the authors should release code and data, add a subject-disjoint evaluation, and compare against baselines trained under the same protocol. I would not hang a SOTA claim on these numbers today.\n\nRecommendation: send to peer review rather than desk reject—the idea is real and the flaw is fixable—but tell the reviewers to insist on a leakage-controlled protocol and public artifacts before accepting. If the accuracy survives that, it is a useful result for the biometrics community.","headline":"Novel fusion of camera PPG and fingertip fingerprint frames with SSM encoders and cross-modal attention, but the reported SOTA accuracy is not trustworthy until the evaluation protocol and split are specified.","tokens_in":10204,"tokens_out":2129,"would_cite":false,"duration_ms":21117,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T10","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that fusing camera-based pulse signals with fingerprint-like ridge images extracted from the same fingertip video lifts smartphone authentication accuracy to 100% with a 0.1% equal error rate in a single session, and to…","keywords":["biometric authentication","photoplethysmography (PPG)","fingerprint recognition","state-space models","cross-modal attention","contrastive learning","smartphone camera sensing","multimodal fusion"],"falsifier":"Compare the extracted fingerprint frames from different sessions of the same subject against conventional fingerprint matching algorithms: if the images fail to produce matching minutiae or ridge flow consistent with a real fingerprint, then the claimed identity signal must come from the PPG branch or from dataset-specific artifacts, and the 0.1% single-session EER would collapse under cross-device or cross-pressure testing. A simpler observational check is to print the extracted 'fingerprint' images and see whether ridge structure is visually resolvable at 64×64 resolution.","tokens_in":9219,"feed_emoji":"🔐","tokens_out":5403,"duration_ms":44096,"temperature":0.7,"pith_summary":"This paper argues that a smartphone camera, with no extra hardware, can authenticate a user by fusing two biometric cues captured from the same fingertip video: the photoplethysmogram (PPG) pulse waveform, extracted from average pixel intensity, and a fingerprint-like ridge image, extracted from contrast-enhanced and edge-detected frames. The paper's central claim is that this multimodal fusion closes the accuracy gap between camera-based and infrared-device PPG authentication, reporting 100% accuracy with a 0.1% equal error rate in a single session and 94.6% accuracy with a 6.9% EER across two sessions separated by about a week. If correct, the result would make secure biometric verification available on ordinary smartphones without dedicated pulse oximeter hardware, and would give convenience-of-use a direct accuracy payoff.","feed_headline":"One fingertip video reaches 100% authentication accuracy","feed_subtitle":"Fusing pulse signals with ridge images lets ordinary smartphone cameras rival dedicated infrared PPG hardware.","key_machinery":"The load-bearing mechanism is a pair of homogeneous SSM encoders—each a deep sequence model whose core is a discretized linear state-space system $h'(t) = A h(t) + B x(t)$, $y(t) = C h(t) + D x(t)$ with a structured matrix $A$—that map both the PPG beat waveform and the flattened fingerprint image into embeddings. A multi-head cross-modal attention module lets each modality query the other, and a distribution-oriented contrastive loss (InfoNCE on exponentially moving average cluster means plus a normalized variance term) aligns the two distributions in a shared latent space. The SSM encoders are what extract long-range dependencies from the 300-sample beats and 4096-pixel image sequences, and the alignment loss is what makes the fusion additive rather than redundant.","core_discovery":"The paper claims that treating both modalities as sequences—PPG beats as 300-point waveforms and fingerprint images flattened into 64×64 pixel sequences—and feeding them through two homogeneous structured state-space (SSM) encoders with multi-head cross-modal attention yields a fused latent representation in which the user's identity is far more separable than either modality alone. On the paper's own evaluation, the fused system reaches 0.1% EER in a single-session setting (V1, 40 subjects), improving over the 13.9% EER of the camera-based See Red baseline and matching the 0.1% EER of the best IR-device PPG system; in the two-session setting (V2, 27 subjects, about 7 days apart) it reaches 6.9% EER versus 21.7% for the camera-only baseline. The authors attribute the gain to a distribution-oriented contrastive loss that aligns first-order moments and variances of the two modalities in a shared latent space while keeping other users' distributions apart.","pith_inferences":["A direct test of the fingerprint-pipeline assumption would be to run standard fingerprint minutiae matching on the extracted ridge images and compare against a conventional fingerprint database; if the images do not contain genuine ridge structure, the two-session EER should degrade sharply when the recording device, finger pressure, or lighting changes.","The same architecture could be applied to other paired physiological cues that are jointly captured by a single sensor, such as finger ECG plus fingertip imaging, or iris plus periocular texture, where distribution alignment across modalities might confer similar robustness.","Because the paper does not report cross-device or cross-subject-transfer experiments, an implicit untested extension is whether the fused representation learned on one phone model transfers to another; the 64×64 downsampled fingerprint images are likely to be device-sensitive."],"forward_implications":["If the reported numbers hold, smartphone-camera authentication becomes competitive with dedicated IR PPG hardware, removing the need for specialized sensors in daily verification tasks.","The many-to-many pairing of PPG beats with fingerprint images, combined with distribution alignment, suggests the system tolerates within-session variability in either modality without retraining per user.","The two-session result (94.6% ACC, 6.9% EER) implies the extracted fingerprint frames carry identity information stable across at least a week, not just contact artifacts of a single recording.","The ablation numbers (PPG-only EER 16.7% vs fused 0.1% in V1) indicate that the fingerprint modality, not the SSM backbone alone, is what drives the accuracy gain."],"supporting_citations":[{"why":"The See Red camera-based PPG system is the primary baseline the proposed method improves upon (86.6% ACC / 13.9% EER re-evaluated).","marker":"[19]"},{"why":"Supplies the IR-device PPG benchmark, the CNN-LSTM PPG baseline in the ablation, and the data augmentation method applied to beats.","marker":"[10]"},{"why":"The S4D diagonal state-space model used as the core of the homogeneous encoders for both modalities.","marker":"[21]"},{"why":"Defines the attention mechanism that the multi-head cross-modal attention module is built from.","marker":"[24]"},{"why":"The InfoNCE contrastive loss that drives the first-order moment alignment between the two modalities.","marker":"[25]"},{"why":"The CLAHE contrast-enhancement method used in the fingerprint extraction pipeline.","marker":"[22]"},{"why":"The Canny edge detector used to extract ridge edges from the fingertip frames.","marker":"[23]"},{"why":"An IR-device non-fiducial PPG baseline whose performance the single-session comparison is measured against.","marker":"[27]"},{"why":"An IR-device long-term PPG feasibility baseline used in the two-session comparison.","marker":"[28]"}],"fun_headline_variants":["Fusing pulse and fingerprint hits 0.1% EER on phone camera","Video PPG plus fingerprint cuts auth error to 0.1%","Smartphone camera fusion reaches 0.1% biometric EER","Camera PPG and fingerprint fusion achieves 0.1% EER","One fingertip video: fused biometrics reach 0.1% EER"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The system assumes the ridge-like images produced by contrast enhancement and edge detection of fingertip video frames contain stable, identity-specific fingerprint structure, even though the paper shows no example images or quality metrics for these frames; if those images mainly reflect contact pressure, illumination, or motion, the large fusion gains will not transfer across sessions or devices.","fun_headline_variants_meta":{"raw":{"variants":["Fusing pulse and fingerprint hits 0.1% EER on phone camera","Video PPG plus fingerprint cuts auth error to 0.1%","Smartphone camera fusion reaches 0.1% biometric EER","Camera PPG and fingerprint fusion achieves 0.1% EER","One fingertip video: fused biometrics reach 0.1% EER"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1331,"prompt_tokens":899,"completion_tokens":432,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":515,"completion_tokens_details":{"reasoning_tokens":335}},"tokens_in":515,"tokens_out":432,"duration_ms":5025,"temperature":1.0,"reasoning_tokens":335,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:29:19.526983+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare the extracted fingerprint frames from different sessions of the same subject against conventional fingerprint matching algorithms: if the images fail to produce matching minutiae or ridge flow consistent with a real fingerprint, then the claimed identity signal must come from the PPG branch or from dataset-specific artifacts, and the 0.1% single-session EER would collapse under cross-device or cross-pressure testing. A simpler observational check is to print the extracted 'fingerprint' images and see whether ridge structure is visually resolvable at 64×64 resolution.","supporting_citations":[{"cited_title":"Seeing red: Ppg biometrics using smartphone cameras,","cited_arxiv_id":null,"evidence_quote":"The See Red camera-based PPG system is the primary baseline the proposed method improves upon (86.6% ACC / 13.9% EER re-evaluated)."},{"cited_title":"Evaluation of the time stability and uniqueness in ppg-based biometric system,","cited_arxiv_id":null,"evidence_quote":"Supplies the IR-device PPG benchmark, the CNN-LSTM PPG baseline in the ablation, and the data augmentation method applied to beats."},{"cited_title":"On the parameterization and initialization of diagonal state space models,","cited_arxiv_id":null,"evidence_quote":"The S4D diagonal state-space model used as the core of the homogeneous encoders for both modalities."},{"cited_title":"Contrast-limited adaptive histogram equalization: speed and effective- ness,","cited_arxiv_id":null,"evidence_quote":"The CLAHE contrast-enhancement method used in the fingerprint extraction pipeline."},{"cited_title":"Canny edge detection enhancement by scale multiplication,","cited_arxiv_id":null,"evidence_quote":"The Canny edge detector used to extract ridge edges from the fingertip frames."},{"cited_title":"Non-fiducial ppg-based authentication for healthcare application,","cited_arxiv_id":null,"evidence_quote":"An IR-device non-fiducial PPG baseline whose performance the single-session comparison is measured against."},{"cited_title":"Biometric authentication using the ppg: A long-term feasibility study,","cited_arxiv_id":null,"evidence_quote":"An IR-device long-term PPG feasibility baseline used in the two-session comparison."}],"review_version":1}