{"id":"1c46aa85-adde-43a1-a632-8c9f1fd18e60","arxiv_id":"2608.09633","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Lightweight LoRA fine-tuning of 32 foundation models reaches below 2% error within each face anti-spoofing dataset but 20-43% error across datasets, showing that LoRA alone does not fix cross-dataset generalization.","lead":"The paper tests 32 pretrained vision models on face spoof detection, both as zero-shot and with light low-rank fine-tuning. It finds that light fine-tuning works within one dataset but fails to transfer across datasets, so low-rank adaptation alone cannot solve the generalization problem.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Cross-dataset conclusion rests on a single untested LoRA configuration; a rank/placement sweep is needed before 'LoRA alone is not enough' can be generalized.","rationale":"The reader's weakest_assumption identifies exactly this concern: the conclusion is conditioned on a single fixed LoRA recipe. I agree that this is the most load-bearing weakness. The other issues raised by the reader—NF4 quantization in the zero-shot VLM comparison, missing variance estimates, and absent code hashes—are secondary; none of them would falsify the headline claim if fixed. The single-recipe issue is different: if a higher rank, a different adapter site, or per-backbone tuning closes most of the 20–43% gap, then the paper's central statement that LoRA-based adaptation alone is not enough would be contradicted by a LoRA-based adaptation alone. This is not an internal inconsistency, and the paper is transparent about the restriction in Sec. 5, but the title, abstract, and Sec. 4.6 currently generalize beyond the evidence. A targeted sweep over rank, adapter placement, and learning rate on a representative subset would settle the question directly. If the band persists, the paper's claim is robust; if it collapses, the claim should be narrowed. Because the reader already made the verdict CONDITIONAL on essentially this point, my stress-test does not move the verdict.","tokens_in":20204,"tokens_out":5746,"duration_ms":57349,"concrete_test":"On a stratified subset of six backbones spanning the CD-ACER range—CLIP ViT-B/32, LLaVA-NeXT-7B-ViT, DINOv2-R-giant, InternViT-6B, Qwen3-VL-32B-ViT, and ViT-MAE-base—rerun the Sec. 4.6 protocol with (a) LoRA rank r in {2, 8, 32, 64} with alpha scaled proportionally, (b) adapter placement in {Q,V}, {Q,K,V}, and {Q,K,V plus MLP}, and (c) learning rate in {3e-5, 1e-4, 3e-4}, keeping all other settings identical. If no configuration reduces mean cross-dataset ACER below 20% for any backbone, or if the 20-43% band is reproduced across all sites and ranks, the generalization holds. If one or more configurations drops mean CD ACER below roughly 15%, the conclusion should be revised to 'one LoRA recipe is insufficient' rather than 'LoRA-based adaptation alone is not enough.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central negative claim—that LoRA adaptation cannot close the cross-dataset gap—is measured with exactly one configuration: Sec. 3.1 fixes rank r=8, alpha=16, dropout 0.05, adapters on WQ and WV only, lr=1e-4, batch size 32, and early stopping. No ablation varies rank, adapter placement, learning rate, or per-backbone tuning. Because LoRA's capacity and inductive bias are controlled by these choices, the 20–43% cross-dataset band in Table 4 could reflect under-parameterization (rank too low) or misplaced adapters (missing K, V, or MLP projections) rather than an inherent limit of low-rank adaptation. The paper's own Sec. 5 limitation acknowledges that \"lightweight adaptation is restricted to LoRA rather than a broader comparison of parameter-efficient fine-tuning methods,\" but Sec. 4.6 and the abstract state a stronger conclusion: \"lightweight adaptation does not resolve cross-dataset transfer.\" The measurement itself is internally sound; the under-supported step is generalizing from one fixed recipe to \"LoRA-based adaptation alone is not enough.\" The comparison against fully fine-tuned specialist baselines (FLIP, FSFM-FAS, DeepPixBiS) does not close this gap, because those baselines are not the same 32 backbones under full fine-tuning.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper asks whether parameter-efficient low-rank adaptation of foundation-model vision encoders is sufficient for cross-dataset face presentation attack detection. On the four MCIO datasets, the authors train a single linear head and LoRA adapters (r=8, alpha=16, dropout 0.05, query/value projections only) on 32 vision encoders and also prompt nine VLMs zero-shot. Under a single-source protocol they report intra-dataset ACER mostly below 2%, cross-dataset mean ACER between 19.8% and 42.6%, and a new G score that geometrically combines intra- and cross-dataset skill. They find that zero-shot prompting is near chance, that pretraining objective and training-dataset choice matter more than scale, and that adapted embeddings remain organized by acquisition domain, leading to the conclusion that LoRA adaptation alone does not close the cross-dataset gap.","tokens_in":20536,"tokens_out":7151,"duration_ms":64007,"significance":"The main negative result—that none of the 32 LoRA-adapted encoders escapes the 20–43% mean cross-dataset ACER band under a fixed protocol—is directly supported by Table 4 and is a useful contribution to the PAD literature, which has mostly focused on CLIP-based models. The breadth of backbones (contrastive, self-distillation, masked image modeling, VLM towers), the unified protocol, the public project page, and the geometric G score are strengths. The paper also supports a practically relevant observation that contrastive and DINO-family encoders transfer better than masked-image-modeling encoders. The primary significance is limited, however, by the fact that the negative conclusion is tied to a single LoRA configuration and a single zero-shot prompt; until robustness to these choices is demonstrated, the headline claim is a statement about this benchmark protocol rather than about low-rank adaptation in general.","major_comments":[{"comment":"Section 3.1 fixes rank r=8, scaling alpha=16, dropout 0.05, adapters on W_Q and W_V only, learning rate 1e-4, batch size 32, and early stopping, and Section 4.6 then generalizes from this single configuration to “lightweight adaptation does not resolve cross-dataset transfer” and the title claim. A rank/placement/learning-rate ablation on at least a few representative backbones (e.g., CLIP ViT-B/32, DINOv2-R-giant, and LLaVA-NeXT-7B-ViT) is needed to distinguish an inherent limit of low-rank adaptation from a property of this particular recipe; without it, the conclusion should be explicitly restricted to the tested configuration. The Sec. 5 limitation narrows the claim to LoRA among PEFT methods, but it does not address within-LoRA variation, so the present wording overstates the evidence.","section":"Sec. 3.1 and Sec. 4.6"},{"comment":"Table 2 and Sec. 4.3 report near-chance zero-shot performance using a single hand-written prompt and NF4-quantized weights with bf16/fp16 matmuls. Because the abstract and Sec. 4.3 state the result as a model-family-level negative finding, the protocol should include at least one alternative prompt and a full-precision or less aggressive quantization check; otherwise the near-chance outcome could reflect prompt sensitivity or quantization rather than the visual representations themselves. The Sec. 5 limitation mentions prompt sensitivity, but only after the broader claim has already been made.","section":"Sec. 4.3 and Table 2"}],"minor_comments":[{"comment":"The sentence “cross-dataset ACER remains between 20% and 43%” refers to mean CD ACER averaged over twelve transfer pairs; the paper should state this explicitly and ideally include the per-pair 4x4 transfer matrices, since individual transfer directions can be much worse than the mean and are the quantities most relevant to deployment.","section":"Sec. 4.6 and Table 4"},{"comment":"The claim that web-scale pretraining coverage is “necessary” for cross-dataset transfer is a correlational observation across models whose pretraining scale is confounded with architecture family and pretraining objective; it should be phrased as a hypothesis supported by the benchmark rather than as a causal necessity.","section":"Sec. 4.7 and Fig. 3"},{"comment":"The generalization score G is called “bias free,” but no statistical bias is defined or demonstrated; I suggest either providing a formal justification for the name or renaming it to “generalization score” to avoid overclaiming.","section":"Eq. (3)"},{"comment":"The specialist baselines (DeepPixBiS, FSFM-FAS, FLIP) are fully fine-tuned on the same datasets but not on the same backbones; a sentence acknowledging that these are reference points rather than controlled LoRA-versus-full-fine-tuning comparisons would clarify the interpretation.","section":"Sec. 3.3 and Table 4"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a solid benchmark paper, and the central measurement is credible. Running 32 vision backbones through one LoRA protocol on the four MCIO datasets, plus nine zero-shot VLMs, under a common evaluation, is genuinely broader than prior work like FoundPAD, FLIP, or Feng et al. The main quantitative result — that every backbone lands in a 20–43% cross-dataset ACER band after LoRA, regardless of pretraining objective, scale, or intra-dataset accuracy — reads directly off the tables and is internally consistent. The G score is a clean definition, not a fitted quantity, and the paper is honest about several of its own limiting conditions in the Limitations section. That part deserves to be cited. The soft spots are mostly about scope, not about the numbers being wrong. The most important is the one fixed LoRA recipe: rank 8, alpha 16, dropout 0.05, adapters on query and value only, lr 1e-4. The abstract and title say 'LoRA-based adaptation alone is not enough,' but the experiments only show that this particular configuration does not close the cross-dataset gap. A slightly different rank, adapter placement on K or MLP projections, or per-backbone hyperparameter tuning could plausibly shift the band. Since the paper's conclusion is explicitly about LoRA as a method, not about one hyperparameter set, this is a real gap — though the paper's own limitation text does acknowledge that only LoRA is compared. A small ablation varying rank and adapter site would materially strengthen it. The zero-shot VLM section has a second, more minor issue: all VLMs are loaded in NF4 quantized precision and tested with a single prompt. NF4 can degrade vision features, and prompt sensitivity is unexplored. That likely moves the 'near chance' conclusion somewhat, but it is unlikely to turn a 35–50% ACER into something viable. Still, a full-precision check on one or two models would be cheap and would make the claim cleaner. Variance estimates are missing across the board. ACER values are reported as point estimates with no standard deviation over runs or folds. In a benchmark whose headline is 'these models all plateau at 20–43%,' a reader needs to know whether the difference between 19.8% and 21.8% is meaningful. This is a minor fix — report repeats or at least cite the protocol's standard practice — but it should be done. Who is this for? People working on face anti-spoofing, and more broadly anyone using lightweight adaptation of foundation models for small cross-domain tasks. It is a useful reference point. It deserves a serious referee; I would recommend acceptance with revisions, asking for a LoRA ablation, a prompt/precision sensitivity check, variance reporting, and released code or commit hashes. The core benchmark will stand even if the title needs softening.","headline":"A genuinely broad and useful benchmark for face PAD, but the headline conclusion about LoRA generalizes further than the single fixed recipe actually supports.","tokens_in":752,"tokens_out":3325,"would_cite":true,"duration_ms":37819,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Low-rank adaptation alone leaves cross-dataset face spoof detection errors between 20% and 43% across all 32 tested foundation models.","keywords":["face presentation attack detection","low-rank adaptation","LoRA","foundation models","cross-dataset generalization","zero-shot prompting","vision encoders","average classification error rate"],"falsifier":"Repeat the same four-dataset, single-source protocol with LoRA variants of higher rank (e.g., 32), adapters on MLP or convolutional layers, per-backbone learning rates, and an early-stopping criterion, and record mean cross-dataset ACER and the source-dataset separability of the adapted embeddings. A configuration that clearly beats the 20-43% range, or that makes adapted features no longer separable by acquisition domain, would falsify the paper's general conclusion that LoRA alone cannot resolve cross-dataset transfer.","tokens_in":19950,"feed_emoji":"🎭","tokens_out":8721,"duration_ms":70075,"temperature":0.7,"pith_summary":"Face presentation attack detection (PAD) works well when the test camera matches the training data, but error rises sharply when the sensor or lighting changes. This paper asks whether low-rank adaptation (LoRA) of large pretrained vision encoders, updating fewer than 1% of the weights, is enough to close that cross-dataset gap. The authors evaluate 32 vision backbones and nine vision-language models on four standard PAD datasets under one identical protocol. LoRA brings most backbones below 2% intra-dataset average classification error, yet every model lands between roughly 20% and 43% cross-dataset error, and embeddings remain organized by source dataset rather than by spoof label. The paper concludes that lightweight adaptation alone does not overcome dataset bias; the backbone and the training dataset dominate cross-dataset transfer.","feed_headline":"Low-rank tuning alone can't fix cross-dataset face spoofing","feed_subtitle":"Across 32 foundation models, cross-dataset error stays at 20-43% even when same-dataset error is below 2%.","key_machinery":"The load-bearing mechanism is a fixed low-rank adaptation recipe applied across a deliberately diverse set of pretrained encoders. LoRA reparameterizes each frozen projection as $W' = W_0 + (\\alpha/r)BA$ with rank $r=8$, scaling $\\alpha=16$, dropout $0.05$, and $B$ initialized to zero, attached only to the query and value projections of every attention block. This same recipe is applied to 32 backbones spanning contrastive image-text, self-distillation, masked-image modeling, and multimodal-language pretraining, and each is scored by ACER on the same four datasets under a single-source protocol. The proposed generalization score $G = \\sqrt{(C-\\mathrm{ACER}_{\\mathrm{ID}})^+(C-\\mathrm{ACER}_{\\mathrm{CD}})^+}/C$ with $C=50$ summarizes joint intra- and cross-dataset skill, but the empirical core is the $4\\times4$ transfer matrix and its means over intra-dataset, cross-dataset, and all sixteen pairs.","core_discovery":"Under the single-source protocol on MSU-MFSD, CASIA-FASD, Replay-Attack, and OULU-NPU, the paper adapts 32 frozen vision encoders with the same LoRA recipe (rank 8, alpha 16, dropout 0.05, adapters on query and value projections, fewer than 1% trainable weights) plus a linear head. Most adapted backbones fall below 2% intra-dataset ACER, while cross-dataset ACER across the twelve transfer pairs lies between 19.8% and 42.6%; the best models are LLaVA-NeXT-7B-ViT at 19.8%, DINOv2-R-giant at 20.4%, and CLIP ViT-B/32 at 21.8%. The central discovery is that LoRA's gain is dataset-conditional: it sharpens the decision boundary within the training acquisition pipeline but leaves the feature geometry structured by acquisition domain rather than by attack class, so thresholds calibrated in one domain do not transfer. The paper states the primary limitation shifts from representation quality to the choice of backbone and the training dataset for the deployment domain.","pith_inferences":["Beyond the paper, a direct test of its scope is to vary the LoRA recipe itself: higher rank, adapters on MLP or convolutional layers, or per-backbone learning rates could push cross-dataset ACER clearly below the 20-43% band, which would narrow the conclusion to 'this LoRA recipe is not enough.'","The embedding analysis suggests a quantitative diagnostic the authors do not run: train a classifier to predict the source dataset from adapted features; a configuration that lowers domain-separability accuracy while keeping intra-dataset ACER low would be a promising next candidate for cross-dataset PAD.","The pattern that transfer tracks proximity to the original contrastive initialization in VLM towers implies that later multimodal co-training may erode transferable spatial structure, which could motivate freezing earlier contrastive layers during multimodal instruction tuning.","If the result generalizes beyond the MCIO benchmark, the bottleneck may be the single-source training protocol itself, making multi-source or synthetic-data training necessary for robust PAD rather than optional."],"forward_implications":["A practitioner can expect cross-dataset ACER between about 20% and 43% for any of these backbones under this LoRA recipe, so a deployment on a new sensor should not rely on lightweight adaptation alone.","Intra-dataset accuracy below 2% should be read as an upper bound under matched conditions, not as expected operational accuracy under domain shift.","Pretraining objective and curation outweigh scale: ImageNet-only backbones sit in the bottom third on the all-pairs aggregate, while web-scale contrastive and self-distillation encoders lead.","The training dataset choice is a dominant factor: CLIP ViT-B/32 adapted on Replay-Attack averages about 10% cross-dataset ACER, whereas the same backbone adapted on MSU-MFSD averages about 33%.","Zero-shot prompting of nine VLMs stays near chance (average ACER 35-50%), so semantic language alignment does not substitute for task-aligned adaptation of the vision tower."],"supporting_citations":[{"why":"Supplies the low-rank reparameterization $W' = W_0 + (\\alpha/r)BA$ that defines the adaptation method under test.","marker":"[32]"},{"why":"Provides the CLIP contrastive vision encoders that dominate the top of the cross-dataset ranking.","marker":"[55]"},{"why":"Provides the DINOv2 self-distillation encoders and the pretraining objective compared against contrastive and masked-image families.","marker":"[50]"},{"why":"Defines the register-token variant DINOv2-R that the paper tracks as a top cross-dataset model.","marker":"[17]"},{"why":"Is the LoRA-based PAD approach that this benchmark extends from CLIP to 32 backbones.","marker":"[52]"},{"why":"Is the vision-language PAD baseline (FLIP variants) fully fine-tuned under the same protocol for comparison.","marker":"[61]"},{"why":"Is the prior benchmarking study of vision foundation models for domain-generalizable face anti-spoofing that motivates the model zoo.","marker":"[25]"},{"why":"Is the classical CNN specialist baseline (DeepPixBiS) used as a pre-foundation-model reference.","marker":"[28]"},{"why":"Is the face-pretrained ViT baseline (FSFM-FAS) compared under the same cross-dataset protocol.","marker":"[65]"}],"fun_headline_variants":["LoRA adapters fail to transfer across face-spoof datasets","Same-dataset wins, cross-dataset loses: LoRA in PAD","LoRA sharpens boundaries but only within one dataset","Cross-dataset face spoofing: LoRA isn't the fix","LoRA tuning alone won't solve PAD generalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the single fixed LoRA configuration used for all 32 backbones, rank 8, alpha 16, dropout 0.05, adapters only on query/value projections, and learning rate 1e-4, represents low-rank adaptation generally; if another low-rank recipe, adapter placement, or per-backbone tuning substantially closes the 20-43% cross-dataset gap, the paper's conclusion would be overstated.","fun_headline_variants_meta":{"raw":{"variants":["LoRA adapters fail to transfer across face-spoof datasets","Same-dataset wins, cross-dataset loses: LoRA in PAD","LoRA sharpens boundaries but only within one dataset","Cross-dataset face spoofing: LoRA isn't the fix","LoRA tuning alone won't solve PAD generalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00041,"raw_usage":{"total_tokens":2170,"prompt_tokens":1033,"completion_tokens":1137,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":1050}},"tokens_in":649,"tokens_out":1137,"duration_ms":9658,"temperature":1.0,"reasoning_tokens":1050,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:14:02.878436+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the same four-dataset, single-source protocol with LoRA variants of higher rank (e.g., 32), adapters on MLP or convolutional layers, per-backbone learning rates, and an early-stopping criterion, and record mean cross-dataset ACER and the source-dataset separability of the adapted embeddings. A configuration that clearly beats the 20-43% range, or that makes adapted features no longer separable by acquisition domain, would falsify the paper's general conclusion that LoRA alone cannot resolve cross-dataset transfer.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is the vision-language PAD baseline (FLIP variants) fully fine-tuned under the same protocol for comparison."},{"cited_title":"In: 2019 international conference on biometrics (ICB)","cited_arxiv_id":null,"evidence_quote":"Is the classical CNN specialist baseline (DeepPixBiS) used as a pre-foundation-model reference."},{"cited_title":"In: Proceedings of the Computer Vision and Pattern Recognition Conference","cited_arxiv_id":null,"evidence_quote":"Is the face-pretrained ViT baseline (FSFM-FAS) compared under the same cross-dataset protocol."}],"review_version":2}