{"id":"e32103d3-e0cd-48ea-9524-1fa769016533","arxiv_id":"2508.11716","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Patch-based training on anonymized ID fragments detects physical and synthetic fake IDs across unseen datasets without sharing full identity documents.","lead":"Researchers built a privacy-preserving fake-ID detector that trains on anonymized patches of official Spanish IDs instead of full documents, and they released a 900K-patch benchmark database. It detects print, screen, and composite forgeries, reaching 8.9% error on one unseen dataset and 13.8% on another.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-database protocol uses only Spanish IDs as bona fide, so the reported 8.90%/13.84% EER may measure 'non-Spanish rejection', not fake-ID detection.","rationale":"The reader's weakest assumption flagged the small size/representativeness of the bona fide class in the cross-database protocol. My concern is more fundamental: the protocol never presents any unseen bona fide sample at all, so the reported cross-database EERs conflate template/domain discrimination with fake detection. This directly undermines the strongest claim (that FakeIDet2 'detects unseen physical and synthetic fake IDs') because the model could achieve the reported EERs by learning a Spanish-vs-non-Spanish decision rule, irrespective of document authenticity. The internal experiments (Sec. 6.1–6.5) and the release of FakeIDet2-db are valuable and largely unaffected, so the paper is not fatally flawed, but the headline cross-database numbers should be treated as preliminary and re-evaluated with a protocol that includes unseen bona fide documents. This is a concrete, testable methodological issue, not a matter of external consensus.","tokens_in":23292,"tokens_out":4694,"duration_ms":56962,"concrete_test":"Re-run the Sec. 6.6 protocol with an augmented bona fide set that includes (i) the evaluation split of FakeIDet2-db and (ii) the 'real'/bona fide samples from DLC-2021 and KID34K (or any unseen non-Spanish official IDs). Keep the attack sets unchanged. If the overall EER increases substantially (e.g., >2×), the original numbers are an artifact of the asymmetric protocol. Alternatively, compute the EER of a trivial baseline that uses DINOv2 embeddings to measure Euclidean distance to the mean Spanish-real embedding; if the baseline matches FakeIDet2's EER, the model has not learned authenticity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline out-of-distribution results in Sec. 6.6 (Table 9) are computed under a protocol that treats every sample in DLC-2021, KID34K, and Benalcazar as attacks and only the evaluation split of FakeIDet2-db (official Spanish IDs) as bona fide. Consequently, the classifier can achieve a low EER simply by learning the decision rule 'looks like a Spanish ID template → real; otherwise → fake'. Because the model never sees a genuine ID from another country or template, the benchmark does not measure the ability to detect fakeness; it measures the ability to reject non-Spanish documents. This confound is especially severe given the extreme asymmetry: the bona fide class is small (≈50 images from a handful of Spanish IDs acquired with the same 3 devices/lighting protocol used in training), while the attack class is large and diverse. The claim 'detects unseen physical and synthetic fake IDs' is therefore not established by Table 9.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a patch-based, privacy-aware methodology for fake ID detection, presents a new public database (FakeIDet2-db) of official Spanish ID images and patches with physical attacks (print, screen, composite), introduces a detection method (FakeIDet2) based on DINOv2 embeddings and a learned attention-fusion module, and reports results on internal and cross-database benchmarks. The headline claims are: 2.01% EER on the internal non-anonymized evaluation set (Table 5), 8.64% EER for the pseudo-anonymized configuration (Table 6), and 8.90%/13.84% EER on DLC-2021/KID34K in the cross-database evaluation (Table 9). The paper also releases a reproducible benchmark and the database, which are positioned as community resources for advancing ID presentation-attack detection.","tokens_in":23615,"tokens_out":8471,"duration_ms":93107,"significance":"If the results hold, this is a valuable contribution to the ID-PAD field. The release of a public database with official real IDs and physical composite attacks is a genuine novelty, and the reproducible benchmark addresses a recognized gap in the literature. The internal experiments (Tables 4–6) convincingly demonstrate the benefit of patch-level training over whole-ID fine-tuning and show that the proposed dynamic-class-weight scheme improves handling of the heavily imbalanced composite-attack class. The authors are also transparent about the poor synthetic-detection result (39.41% EER on Benalcazar). However, the cross-database generalization claim is weakened by a protocol confound, and the privacy-aware claim is asserted rather than measured; both points need attention before the central conclusions can be accepted.","major_comments":[{"comment":"The cross-database EERs are computed with the FakeIDet2-db evaluation split as the only bona fide class (a small set of official Spanish IDs, acquired with the same three devices and lighting/height protocol used in training) and every sample from DLC-2021, KID34K, and Benalcazar treated as an attack. Because the model is trained exclusively on Spanish ID patches, a low EER on KID34K (Korean) and Benalcazar (Chilean) can be achieved by rejecting non-Spanish document statistics without detecting fakeness. The abstract's claim that the method 'detects very realistic fake IDs under unseen type of attacks' is therefore not established by these numbers. Please add a control with genuine non-Spanish bona fide IDs (e.g., the 'real' samples from KID34K, despite being lab-created) and/or explicitly reframe the result as a domain-shift evaluation relative to a Spanish-ID reference, stating this co","section":"Sec. 6.6, Table 9"},{"comment":"The paper's central motivation is privacy: individual patches are claimed to contain 'much less sensitive information than the whole ID' and the framework is described as 'privacy compliant (GDPR, EU Data Act)'. However, no privacy property is defined or measured. There is no reconstruction-resistance test, membership-inference evaluation, or information-theoretic quantification, and the residual leakage of the pseudo-anonymization (black rectangles) is not analyzed. Because 'privacy-aware' is in the title and is a claimed contribution, this gap is load-bearing. Please either add a small empirical privacy evaluation (e.g., attempt to reconstruct the ID from patches, measure OCR leakage, or run a membership test) or explicitly scope the contribution as 'designed to reduce exposure', acknowledging that quantitative privacy guarantees are not provided.","section":"Sec. 3.3, Sec. 4, Sec. 7"}],"minor_comments":[{"comment":"The APCER equation appears to have a typo: the denominator and summation index use NBF, the number of bona fide samples, which is inconsistent with the definition of APCER over attack presentations. Please correct to use the number of attack presentations.","section":"Eq. (12)"},{"comment":"The composite-attack patch counts are ~15x lower than the other classes (e.g., 3,810 vs 56,017 in the 128x128 Non-Anon row). Please explain the cause (e.g., the 80% black-patch filtering) and discuss how this extreme imbalance interacts with the dynamic class-weight design.","section":"Table 3"},{"comment":"All EERs are point estimates with no confidence intervals or repeated trials. This is especially important for the cross-database results, where the bona fide set is very small. Please add intervals, bootstrap estimates, or at least a discussion of variance.","section":"Tables 4–9"},{"comment":"Sec. 2.1 describes DLC-2021 as built from the MIDV-family documents, but Sec. 6.6 states that 'DLC-2021 contains Spanish IDs'. Please clarify the template countries/versions in DLC-2021 and how this affects the interpretation of the cross-database protocol.","section":"Sec. 2.1 vs Sec. 6.6"},{"comment":"The AdaFace margin m=0.4 and scaling factor s=64 are described as 'the optimal experimental values' without an ablation or citation justifying them for this protocol. Please provide supporting experiments or a reference for these choices.","section":"Sec. 5.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. It delivers two things this field actually lacks: a public database with official Spanish IDs and physical composite attacks, and a reproducible benchmark that combines their data with DLC-2021, KID34K, and Benalcazar's synthetic set. That alone justifies peer review and citation. The method side is less novel—the patch-anonymization idea comes from their own FakeIDet—but the additions are real: learnable MHSA fusion over DINOv2/AdaFace patch embeddings and dynamic class weights that help with the composite-class imbalance. Internal experiments are consistent, and the leave-one-attack-out and leave-one-sensor-out analyses are genuinely useful stress tests.\n\nThe main problem is the cross-database protocol in Sec. 6.6. Every external sample is treated as an attack, and the only bona fide samples come from the evaluation split of FakeIDet2-db—roughly 50 official Spanish IDs acquired with three phones under one lighting protocol. With that setup, a model can reach low EER by learning 'Spanish-looking template means real, everything else means fake.' The claim that FakeIDet2 detects unseen physical and synthetic fake IDs is not established by Table 9. Those numbers are better described as out-of-distribution rejection of non-Spanish documents. This is a real confound, not a cosmetic issue, and it should be stated as such.\n\nA few smaller soft spots: composite patch counts are about 15x lower than other classes in Table 3, with no explanation; all EERs are point estimates with no error bars, so small gaps like 1.23% vs. 2.09% may not be meaningful; and the privacy claim is asserted, not measured—no reconstruction attempt or re-identification test is reported. Self-citation is present but not abusive: the central evaluation uses external datasets, and the benchmark is new.\n\nWho this is for: researchers working on ID presentation attack detection, especially anyone who needs official-ID data or a common evaluation protocol. The data contribution alone deserves a serious referee, provided the authors reframe the cross-database results and report uncertainty. My recommendation: send it out, and push reviewers to focus on the cross-database protocol and the composite sampling.","headline":"A real public database and benchmark for fake-ID detection, but the cross-database numbers partly measure 'non-Spanish rejection'; the data contribution is solid enough to warrant peer review.","tokens_in":24035,"tokens_out":1885,"would_cite":true,"duration_ms":22769,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fake IDs can be detected from anonymous ID patches, not whole documents","keywords":["fake ID detection","presentation attack detection","privacy-preserving patches","pseudo-anonymization","identity document","DINOv2","attention fusion","benchmark"],"falsifier":"Take the public FakeIDet2 checkpoint and evaluate it with a bona fide set of official IDs from a different country and a different capture device set than the iPhone, Xiaomi, and Redmi used here; if EER degrades substantially relative to the 8.90% and 13.84% cross-database numbers while attack data are held fixed, the claimed generalization tied to the privacy-aware patch training is not portable across real-ID distributions. A simpler check: compute the per-subject EER spread over the 47 Spanish IDs; if a few IDs drive the low EER, the small bona fide subset is not representative.","tokens_in":23281,"feed_emoji":"🪪","tokens_out":1492,"duration_ms":18375,"temperature":0.7,"pith_summary":"This paper argues that fake identity documents can be reliably detected using only small, privacy-preserving patches of the document, rather than the full ID image, which is normally too sensitive to share for research. The authors introduce FakeIDet2-db, a public dataset of over 900,000 real and fake ID patches from official Spanish IDs, including physical print, screen, and composite attacks. They also propose FakeIDet2, a detector that learns patch-level embeddings and fuses them with attention. Trained only on pseudo-anonymized patches, it achieves 8.90% and 13.84% EER on two challenging out-of-distribution databases, DLC-2021 and KID34K. The broader claim is that privacy and performance are not irreconcilable: ID holders can share anonymized patches instead of whole documents, opening a practical path for collaboration between data owners and AI researchers.","feed_headline":"Fake IDs detected from anonymous patches alone","feed_subtitle":"A patch-based detector trained on shuffled, anonymized ID fragments hits 8.9% error on unseen fake-ID data, easing privacy.","key_machinery":"The method FakeIDet2 comprises three modules: a Privacy-Aware Patch Extractor that divides an ID into non-overlapping 64x64 or 128x128 patches with optional anonymization; a Patch Embedding Extractor built on a frozen DINOv2 backbone plus an AdaFace margin loss with dynamic class weights that decay from inverse-frequency weights to uniform weights over training; and a Patch Embedding Fusion module using two stacked Multi-Head Self-Attention layers with residual connections followed by attention pooling to fuse patch embeddings into a single document-level score. The dynamic class weights address the severe imbalance of composite-attack patches, and the fusion module is the key mechanism for","core_discovery":"The central claim is that fake ID detection can be performed from small, anonymized image patches instead of complete ID scans, preserving privacy while remaining competitive with, and often better than, whole-ID training. The paper demonstrates that 64x64 patches drawn from pseudo-anonymized IDs retain enough discriminative information to detect unseen physical attacks, and that a learnable patch-embedding fusion based on multi-head self-attention handles composite attacks (where only part of the document is fake) much better than simple score averaging, reducing composite-attack EER from 54.08% to 2.09% on the paper's own evaluation set. On the cross-database benchmark, the privacy-aware c","pith_inferences":["The reported cross-database numbers may underestimate or overestimate deployment performance depending on how representative the 50 official Spanish IDs used as bona fide are of the target population of real IDs; the paper does not measure that generalization gap directly.","Because synthetic attacks (Benalcazar et al.) yield 39.41% EER, a plausible extension is to train or adapt the patch-level extractor on synthetic-to-physical style transfers, or to add a separate synthetic-attack detector, rather than treating physical and synthetic fakes as one class.","The attention-pooling fusion could plausibly be used for tamper localization: the per-patch attention weights already single out anomalous regions, so a lightly modified version of FakeIDet2 might output a localization map as a by-product, which the paper leaves as future work."],"forward_implications":["If valid, ID holders such as governments, banks, and police can contribute official real-ID data to train detectors without disclosing full documents, since only shuffled, partially anonymized patches are shared.","Remote identity-verification systems could deploy detectors trained on pseudo-anonymized patches, with only a modest performance drop (3.99% to 8.64% EER in the authors' intra-database comparison).","The released benchmark allows direct comparison of future fake-ID detectors against a standard protocol spanning physical and synthetic attacks across European, Asian, and South American ID templates.","Prior 'real' datasets that were actually laboratory-printed cards may have inflated apparent performance; the paper's cross-database protocol quantifies generalization to official IDs and unseen attacks."],"supporting_citations":[{"why":"The prior FakeIDet work supplies the baseline patch-based method and initial privacy-aware scenario that FakeIDet2 extends with learnable fusion modules.","marker":"[25]"},{"why":"DINOv2 provides the frozen foundation-model backbone used to extract patch embeddings in the Patch Embedding Extractor.","marker":"[27]"},{"why":"AdaFace supplies the quality-adaptive margin loss that, with the paper's dynamic class weights, defines the patch embedding learning objective.","marker":"[17]"},{"why":"Attention pooling following the multi-head self-attention layers is adapted from this work to compress the patch-embedding sequence into a single document embedding.","marker":"[4]"},{"why":"DLC-2021 serves as one of the two out-of-distribution physical-attack evaluation databases in the cross-database benchmark.","marker":"[30]"},{"why":"KID34K serves as the other out-of-distribution physical-attack evaluation database, with twelve acquisition devices and Korean ID templates.","marker":"[28]"},{"why":"The Benalcazar et al. database provides the synthetic-attack evaluation set generated with GANs from Chilean ID samples.","marker":"[5]"}],"fun_headline_variants":["Anonymized ID patches catch fake documents","Privacy-safe fake ID detection from tiny patches","Patch-based fake ID detector beats whole-image methods","Detect fake IDs without full document scans","Composite attack error drops from 54% to 2%"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The out-of-distribution results depend on the assumption that the 50 official Spanish IDs in the FakeIDet2-db evaluation split represent the distribution of real IDs a deployed system will encounter, in terms of templates, devices, users, and capture conditions.","fun_headline_variants_meta":{"raw":{"variants":["Anonymized ID patches catch fake documents","Privacy-safe fake ID detection from tiny patches","Patch-based fake ID detector beats whole-image methods","Detect fake IDs without full document scans","Composite attack error drops from 54% to 2%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000284,"raw_usage":{"total_tokens":1569,"prompt_tokens":858,"completion_tokens":711,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":602,"completion_tokens_details":{"reasoning_tokens":640}},"tokens_in":602,"tokens_out":711,"duration_ms":7834,"temperature":1.0,"reasoning_tokens":640,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:13:58.275092+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the public FakeIDet2 checkpoint and evaluate it with a bona fide set of official IDs from a different country and a different capture device set than the iPhone, Xiaomi, and Redmi used here; if EER degrades substantially relative to the 8.90% and 13.84% cross-database numbers while attack data are held fixed, the claimed generalization tied to the privacy-aware patch training is not portable across real-ID distributions. A simpler check: compute the per-subject EER spread over the 47 Spanish IDs; if a few IDs drive the low EER, the small bona fide subset is not representative.","supporting_citations":[{"cited_title":"DINOv2: Learning Robust Visual Features Without Supervision","cited_arxiv_id":null,"evidence_quote":"DINOv2 provides the frozen foundation-model backbone used to extract patch embeddings in the Patch Embedding Extractor."},{"cited_title":"AdaFace: Quality adaptive margin for face recognition, in: Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp","cited_arxiv_id":null,"evidence_quote":"AdaFace supplies the quality-adaptive margin loss that, with the paper's dynamic class weights, defines the patch embedding learning objective."},{"cited_title":"Document Live- ness Challenge Dataset (DLC-2021)","cited_arxiv_id":null,"evidence_quote":"DLC-2021 serves as one of the two out-of-distribution physical-attack evaluation databases in the cross-database benchmark."},{"cited_title":"KID34K: A Dataset for Online Identity Card Fraud Detection, in: Proc","cited_arxiv_id":null,"evidence_quote":"KID34K serves as the other out-of-distribution physical-attack evaluation database, with twelve acquisition devices and Korean ID templates."},{"cited_title":"Synthetic ID Card Image Generation for Improving Presentation Attack Detection","cited_arxiv_id":null,"evidence_quote":"The Benalcazar et al. database provides the synthetic-attack evaluation set generated with GANs from Chilean ID samples."}],"review_version":1}