Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Multimodal LLMs can perform differential morphing attack detection zero-shot, with ChatGPT-4o outperforming Gemini on real biometric data.

desk verdict First D-MAD benchmark with multimodal LLMs on real data; the qualitative failure analysis is worth reading, but the reported accuracy numbers rest on an unspecified fusion rule and should be treated as illustrative, not measured. read the letter →

arxiv 2505.15332 v1 pith:Z7FRZABK submitted 2025-05-21 cs.CV

classification cs.CV
keywords differentialmorphingattackdetectionmultimodallargelanguagemodelszero-shotchain-of-thoughtpromptingChatGPT-4oGeminifaceexplainablebiometrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that multimodal large language models, prompted as forensic experts to reason step-by-step, can perform differential morphing attack detection (D-MAD) in a zero-shot manner on real biometric data. It presents the first such application, comparing ChatGPT-4o and Gemini on passport-style image pairs from 54 consented subjects. The central claim is that these models show promising detection accuracy—particularly ChatGPT-4o against GAN-based morphs, with a 0% half total error rate for MIPGAN2—while also supplying natural-language explanations, although both models remain vulnerable under difficult conditions. The paper argues that Chain-of-Thought prompting reduces refusal rates and improves reasoning transparency, making LLM-based D-MAD a plausible complement to conventional detectors.

What carries the argument

The key machinery is a deliberately engineered Chain-of-Thought (CoT) prompt that assigns the LLM the role of an image forensic expert, guides it through five visual checks (facial geometry, artifacts, identity consistency, natural capture, hairlines and marks), and demands a forced binary answer to two questions plus a 0–100 probability score and an explanation. Around this prompt, the framework uses three independent inference attempts per image pair, averages the probability scores for distribution analysis, and applies a logical OR across attempts as the binary decision rule for computing MACER and BPCER. This combination converts a general-purpose multimodal LLM into a structured, explainable D-MAD decision-maker.

What would settle it

Compute calibration error between each model's 0–100 confidence scores and ground-truth morph labels on a larger labeled dataset, or compare the model's score ranking against a panel of forensic experts ranking the same image pairs by morphing likelihood; if the scores are not well-calibrated or do not rank-correlate with expert judgment, the claimed zero-shot detection performance cannot be trusted.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a multimodal LLM can act as a zero-shot differential morphing attack detector when given a pair of facial images—a trusted capture and a possibly morphed reference—and a prompt that forces a binary decision, a 0–100 confidence score, and a step-by-step forensic rationale. On 150 bona fide–morphed pairs and 50 bona fide–bona fide pairs from 54 real subjects, ChatGPT-4o achieves half total error rates of 21.5% (landmark morphing), 3.5% (diffusion-based PIPE), and 0% (GAN-based MIPGAN2), while Gemini shows higher error rates and more overlap between bona fide and morph score distributions. The paper also finds that repeated inference with a logical OR decision rule and Chain-of-Thought prompting improves reliability, but that both models exhibit failures-to-answer, contradictory outputs, and hedging.

Load-bearing premise

The load-bearing premise is that the confidence scores the models give themselves reflect actual morphing likelihood; if these numbers are arbitrary or biased, the reported detection errors do not measure real detection performance.

Editorial extensions

If this is right

  • If the claim holds, zero-shot LLM-based D-MAD can be deployed without training data, offering immediate detection baselines for newly invented morphing techniques.
  • The explainable outputs could give border-control operators natural-language rationales for alarms, not just a score.
  • The logical OR fusion across repeated attempts suggests a cheap inference-time strategy to reduce LLM failures and hallucination-driven misses.
  • The strong ChatGPT-4o performance on GAN morphs indicates that some general-purpose vision-language models already encode cues that specialized detectors need to be trained to find.
  • The observed vulnerability of both models to landmark-based morphs defines a concrete gap for future prompt or model improvements.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A calibrated version of these LLM confidence scores could be plugged into existing D-MAD pipelines as an additional feature, rather than as a standalone decision.
  • The failure-to-answer and contradiction patterns suggest that ensembling multiple LLMs or fusing with a classical detector would likely improve robustness more than further prompt tuning alone.
  • Since the data is limited to 54 subjects and passport-style captures, the striking MIPGAN2 result should be re-tested on large public morph benchmarks before being taken as a general capability.
  • The conflicting 'same identity yet morphed' outputs highlight that identity verification and morph detection are not consistently aligned in LLMs; a testable fix is to prompt for joint reasoning that explicitly conditions morph decisions on identity match.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript presents the first application of multimodal large language models (LLMs) to differential morphing attack detection (D-MAD) on real biometric data. ChatGPT-4o and Gemini are given two facial images with a Chain-of-Thought (CoT) prompt that asks for an identity-verification decision, a morph-detection decision, a 0-100 probability score, and a natural-language explanation. The evaluation uses 50 bona fide-bona fide pairs and 50 bona fide-morphed pairs for each of three morphing methods (LMA, MIPGAN2, PIPE), with three inference attempts per pair fused by logical OR. Tables I and II report MACER, BPCER, and HTER by morph type and model, and the qualitative section documents failure-to-answer cases and conflicting responses. The central claimed result is that ChatGPT-4o outperforms Gemini, especially on GAN-based morphs.

Significance. If the quantitative comparison were robust, this would be a useful and timely contribution: it demonstrates that a frozen multimodal LLM can perform zero-shot D-MAD while supplying human-readable explanations, which is a practical advantage over black-box detectors. The qualitative failure analysis (Tables III-V) is a genuine strength, as are the use of real subjects and the explicit reporting of failure modes. However, the evaluation protocol currently leaves the decision rule underspecified and the sample sizes are small, so the headline numbers in Tables I-II should be treated as preliminary. The significance of the paper therefore depends on the proposed revision.

major comments (4)
  1. [Section IV-A; Tables I-II] The decision rule behind Tables I-II is not fully specified. The text states that the binary decision uses a logical OR across three inference attempts, but it never states the threshold on the 0-100 probability score that turns one attempt into a flag, and it does not define how failure-to-answer responses are coded. This matters because ChatGPT-4o frequently fails to answer (Tables III and V) while Gemini rarely does. Under a discard-failures coding, the effective number of attempts differs per pair; under a non-flag coding, a high-failure model is penalized; under a flag coding, a single spurious response creates a false alarm. The OR rule can therefore inflate Gemini's BPCER while selectively rescuing ChatGPT-4o's MACER. Please specify the threshold and failure coding, and recompute Tables I-II under single-attempt, majority-vote, and OR policies with a fixed threshold. As written, the reported 0% HTER for ChatGPT-4o on MIPGAN2 is not a well-defined measurement.
  2. [Section IV-A] The sample size of 50 pairs per cell is too small to support the claimed differences without uncertainty quantification. A reported MACER of 0% for MIPGAN2 corresponds to 0 errors out of 50, whose 95% Clopper-Pearson interval extends to about 7%; the ChatGPT-4o versus Gemini difference (0% vs 13% MACER) is not formally established. Report exact binomial or bootstrap confidence intervals for MACER, BPCER, and HTER, and use a paired test if the same subjects contribute multiple pairs. Without intervals, the reader cannot distinguish a genuine model difference from sampling noise.
  3. [Section III] The zero-shot claim is weakened by the prompt-development procedure. The paper says the prompt was 'iteratively refined through trial and error,' but it does not state whether the same evaluation pairs were used during that refinement. If they were, the prompt has effectively been tuned on the test set, so the evaluation measures the performance of a hand-adapted prompt rather than a zero-shot system. Please either specify that a separate development set was used, or split the data and report results on an untouched held-out subset.
  4. [Section V-A; Tables I-II] The paper provides no comparison with existing D-MAD methods. Even if the LLM comparison is internally consistent, the claim that LLM-based D-MAD is 'promising' requires a reference point: for example, a handcrafted-feature/SVM baseline or a deep Siamese method from the cited literature, evaluated under the same protocol and threshold. Add at least one conventional baseline so that the absolute HTER values and the trade-off between MACER and BPCER can be interpreted.
minor comments (6)
  1. [Section I; Figure 1] The word 'Diffrential' in the Figure 1 caption and the word 'quantitaive' in Section VI-A are typos.
  2. [Section III] The prompt contains the spacing errors 'ALW AYS' in two places; these should read 'ALWAYS'.
  3. [Section I] The introduction's roadmap says Section II presents the framework and Section IV the database, but Section III (prompt design) is omitted from the roadmap.
  4. [Section V-B; Tables III-IV] The tables of round-wise responses do not identify which image pair or morph type each example came from, making the qualitative claims hard to verify. Add image identifiers or morph-type labels.
  5. [References] Reference [9] is malformed ('F. landmark based face morphing. Open CV.') and should be completed or replaced with a proper citation.
  6. [Overall] For reproducibility, state the exact model versions, API access dates, and decoding parameters (e.g., temperature, max tokens) for both ChatGPT-4o and Gemini; LLM responses are stochastic and these details materially affect the results.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: this is an empirical benchmark against external pre-trained LLMs, with no fitted parameter or self-citation chain that forces the reported results.

full rationale

The paper contains no derivation chain in which a predicted quantity is defined in terms of its own outcome. The central results are measured outputs of two external, pre-trained multimodal LLMs (ChatGPT-4o and Gemini) on fixed image pairs, so the HTER/MACER/BPCER values are not constructed from the inputs by definition. Self-citations (e.g., [16], [30], [32], [33]) supply the morph-generation protocols, dataset construction guidance, and a prior S-MAD study; none of these is used to justify the D-MAD detection numbers themselves, and none is a uniqueness theorem that forbids alternatives. The iterative prompt refinement described in Section III is a test-set-contamination / zero-shot framing concern rather than a circular reduction, because the prompt is not a parameter fitted to the labels and then renamed a prediction. The logical-OR fusion rule and the unspecified probability threshold in Section IV-A are protocol ambiguities that affect interpretability and reproducibility, but they do not make the decision equal to the input score by construction. No step meets the required bar of quoting a specific equation or fitted parameter that reduces to the target result.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on domain assumptions about LLM score calibration, prompt effectiveness, and dataset representativeness. The free parameters are design choices (number of attempts, fusion rule, prompt content) that materially affect the reported error rates.

free parameters (3)
  • number_of_inference_attempts = 3
    Three independent inference attempts per image pair; scores are averaged and binary decisions are OR-ed. No sensitivity analysis is provided.
  • binary_fusion_rule = logical OR across attempts
    If any of the three attempts flags a pair as morphed, the final decision is 'morphed'. This rule reduces MACER at the cost of BPCER and is not a standard MAD evaluation protocol.
  • prompt_design = hand-crafted CoT prompt, iteratively refined
    The prompt was refined by trial and error on the same evaluation data (Section III), which risks overfitting to the test set.
assumptions (5)
  • domain assumption LLM probability scores are treated as calibrated vulnerability scores
    The quantitative analysis in Section V-A uses the model's self-reported probability for Q1/Q2 as a detection score without calibration.
  • domain assumption CoT prompting improves answer reliability and reasoning
    The method relies on Chain-of-Thought prompting (Section III) to reduce failure-to-answer rates, but no ablation compares CoT to simpler prompts.
  • domain assumption The first image in each pair is a known bona fide image
    The prompt instructs the model that the first image is known bona fide (Section III), which simplifies the D-MAD task relative to real border-control scenarios.
  • domain assumption LMA, MIPGAN2, and PIPE morphs are representative of realistic attacks
    The dataset uses three morphing techniques selected for high attack potential (Section IV), but no independent validation of this representativeness is given.
  • domain assumption The collected smartphone images approximate passport enrollment conditions
    Participants captured images under controlled settings (Section IV), but the equivalence to actual passport capture pipelines is asserted, not verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models." pith.science (2026). https://pith.science/paper/Z7FRZABK

@misc{pith2026250515332,
  author       = {Pith},
  title        = {Pith review of: Towards Zero-Shot Differential Morphing Attack Detection with Multimodal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Z7FRZABK}},
  note         = {Machine review of arXiv:2505.15332}
}
read the original abstract

Leveraging the power of multimodal large language models (LLMs) offers a promising approach to enhancing the accuracy and interpretability of morphing attack detection (MAD), especially in real-world biometric applications. This work introduces the use of LLMs for differential morphing attack detection (D-MAD). To the best of our knowledge, this is the first study to employ multimodal LLMs to D-MAD using real biometric data. To effectively utilize these models, we design Chain-of-Thought (CoT)-based prompts to reduce failure-to-answer rates and enhance the reasoning behind decisions. Our contributions include: (1) the first application of multimodal LLMs for D-MAD using real data subjects, (2) CoT-based prompt engineering to improve response reliability and explainability, (3) comprehensive qualitative and quantitative benchmarking of LLM performance using data from 54 individuals captured in passport enrollment scenarios, and (4) comparative analysis of two multimodal LLMs: ChatGPT-4o and Gemini providing insights into their morphing attack detection accuracy and decision transparency. Experimental results show that ChatGPT-4o outperforms Gemini in detection accuracy, especially against GAN-based morphs, though both models struggle under challenging conditions. While Gemini offers more consistent explanations, ChatGPT-4o is more resilient but prone to a higher failure-to-answer rate.

Figures

Figures reproduced from arXiv: 2505.15332 by the authors.

Figure 1
Figure 1. In a typical D-MAD scenario, facial images captured [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Block diagram of the proposed D-MAD framework using multimodal Large Language Models (LLMs). The model [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Example facial images corresponding to bona fide [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Kernel Density Estimate (KDE) plots illustrating the distribution of vulnerability scores for bona fide and morphed [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DifFoundMAD: Foundation Models meet Differential Morphing Attack Detection

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    DifFoundMAD improves differential morphing attack detection by replacing traditional embeddings with those from vision foundation models and applying class-balanced lightweight fine-tuning, cutting high-security error...

  2. Benchmarking Foundation Models for Zero-Shot Biometric Tasks

    cs.CV 2025-05 conditional novelty 5.0 of 10

    A benchmark of 41 foundation models shows CLIP/OpenCLIP/BLIP2 embeddings reach near-90% zero-shot face verification and DINO reaches 97.55% on IITD-R iris without fine-tuning.

Reference graph

Works this paper leans on

34 extracted references · 32 canonical work pages · cited by 2 Pith papers

  1. [1]

    Aghdaie, B

    P. Aghdaie, B. Chaudhary, S. Soleymani, J. Dawson, and N. Nasrabadi. Attention aware wavelet-based detection of morphed face images. In 2021 IEEE International Joint Conference on Biometrics (IJCB), pages 1–8, 2021

  2. [2]

    Borghi, E

    G. Borghi, E. Pancisi, M. Ferrara, and D. Maltoni. A double siamese framework for differential morphing attack detection.Sensors, 21(10), 2021

  3. [3]

    Caldeira, G

    E. Caldeira, G. Ozgur, T. Chettaoui, M. Ivanovska, P. Peer, F. Boutros, V . Struc, and N. Damer. Madation: Face morphing attack detection with foundation models, 2025

  4. [4]

    Clemens, S.Wojciech, A

    S. Clemens, S.Wojciech, A. Hilsmann, and P. Eisert. Accurate and robust neural networks for face morphing attack detection.Journal of Information Security and Applications, 53:102526, 2020

  5. [5]

    Damer, J

    N. Damer, J. H. Grebe, S. Zienert, F. Kirchbuchner, and A. Kuijper. On the generalization of detecting face morphing attacks as anomalies: Novelty vs. outlier detection. In2019 IEEE 10th International Conference on Biometrics Theory, Applications and Systems (BTAS), pages 1–5, 2019

  6. [6]

    Ferrara, A

    M. Ferrara, A. Franco, and D. Maltoni. Face demorphing in the presence of facial appearance variations. InProc. of the 26th European Signal Processing Conf. (EUSIPCO). IEEE, September 2018

  7. [7]

    Information Technology - Biometric presentation attack detection - Part 3: Testing and Reporting

    ISO/IEC JTC1 SC37 Biometrics.ISO/IEC 30107-3. Information Technology - Biometric presentation attack detection - Part 3: Testing and Reporting. International Organization for Standardization, 2017

  8. [8]

    Kraetzer, A

    C. Kraetzer, A. Makrushina, T. Neubert, M. Hildebrandt, and J. Dittmann. Modeling attacks on photo-ID documents and applying media forensics for the detection of facial morphing. InProc. Workshop on Information Hiding and Multimedia Security, pages 21– 32, 2017

Show all 34 references
  1. [9]

    landmark based face morphing

    F. landmark based face morphing. Open CV.https://www.learnopencv.com/ face-morph-using-opencv-cpp-python/

  2. [10]

    FRVT morph web site.https://pages.nist.gov/ frvt/html/frvt_morph.html

    NIST. FRVT morph web site.https://pages.nist.gov/ frvt/html/frvt_morph.html

  3. [11]

    Chatgpt.https://chat.openai.com/, 2024

    OpenAI. Chatgpt.https://chat.openai.com/, 2024. [Ac- cessed: Jan. 11, 2024]

  4. [12]

    Ortega-Delcampo, C

    D. Ortega-Delcampo, C. Conde, D. Palacios-Alonso, and E. Cabello. Border control morphing attack detection with a convolutional neural network de-morphing approach.IEEE Access, 8:92301–92313, 2020

  5. [13]

    Patwardhan, R

    S. Patwardhan, R. Ramachandra, and S. Venkatesh. Empowering morphing attack detection using interpretable image-text foundation model. InComputer Vision and Image Processing, pages 163–169, Singapore, 2024. Springer Singapore

  6. [14]

    Raghavendra, K

    R. Raghavendra, K. Raja, and C. Busch. Detecting morphed face images. In2016 IEEE 8th Intl. Conf. on Biometrics: Theory, Applica- tions and Systems (BTAS). 8th IEEE Intl. Conf. on Biometrics: Theory, Applications and Systems (BTAS-2016), IEEE, September 2016

  7. [15]

    Raghavendra, K

    R. Raghavendra, K. Raja, and C. Busch. Algorithmic fairness in face morphing attack detection. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 410–418, 2022

  8. [16]

    Raghavendra, K

    R. Raghavendra, K. Raja, S. Venkatesh, and C. Busch. Face morphing versus face averaging: Vulnerability and detection. InProc. Intl. Joint Conf. on Biometrics (IJCB), 2017

  9. [17]

    Raghavendra, K

    R. Raghavendra, K. Raja, S. Venkatesh, and C. Busch. Transferable deep-CNN features for detecting digital and print-scanned morphed face images. InIEEE Conf. on Computer Vision and Pattern Recog- nition Workshops (CVPRW), pages 1822–1830, 2017

  10. [18]

    Raghavendra, S

    R. Raghavendra, S. Venkatesh, K. Raja, and C. Busch. Towards making morphing attack detection robust using hybrid scale-space colour texture features. InIEEE 5th Intl. Conf. on Identity, Security, and Behavior Analysis (ISBA). IEEE, January 2019

  11. [19]

    Ramachandra and G

    R. Ramachandra and G. Li. Multimodality for reliable single image based face morphing attack detection.IEEE Access, 10:82418–82433, 2022

  12. [20]

    Ramachandra and G

    R. Ramachandra and G. Li. Residual colour scale-space gradients for reference-based face morphing attack detection. In2022 25th International Conference on Information Fusion (FUSION), pages 1–

  13. [21]

    Ramachandra, S

    R. Ramachandra, S. Venkatesh, G. Li, and K. Raja. Differential newborn face morphing attack detection using wavelet scatter network. In2023 5th International Conference on Bio-engineering for Smart Technologies (BioSMART), pages 1–4. IEEE, 2023

  14. [22]

    Scherhag, A

    U. Scherhag, A. Nautsch, C. Rathgeb, M. Gomez-Barrero, R. N. J. Veldhuis, L. Spreeuwers, M. Schils, D. Maltoni, P. Grother, S. Marcel, R. Breithaupt, R. Ramachandra, and C. Busch. Biometric systems under morphing attacks: Assessment of morphing techniques and vulnerability rep...

  15. [23]

    Scherhag, C

    U. Scherhag, C. Rathgeb, J. Merkle, and C. Busch. Deep face representations for differential morphing attack detection.IEEE Trans. on Information Forensics and Security, 2020

  16. [24]

    J. M. Singh and R. Ramachandra. Fusion of deep features for differential face morphing attack detection at automatic border control gates. In2022 10th European Workshop on Visual Information Processing (EUVIP), pages 1–5. IEEE, 2022

  17. [25]

    J. M. Singh and R. Ramachandra. Reliable face morphing attack detection in on-the-fly border control scenario with variation in image resolution and capture distance. In2022 IEEE International Joint Conference on Biometrics (IJCB), pages 1–10. IEEE, 2022

  18. [26]

    Soleymani, B

    S. Soleymani, B. Chaudhary, A. Dabouei, J. Dawson, and N. Nasrabadi. Differential morphed face detection using deep siamese networks. InInternational Conference on Pattern Recognition, pages 560–572. Springer, 2021

  19. [27]

    G. Team. Gemini: A family of highly capable multimodal models, 2024

  20. [28]

    Venkatesh, R

    S. Venkatesh, R. Ramachandra, K. Raja, and C. Busch. Face morphing attack generation and detection: A comprehensive survey.IEEE Transactions on Technology and Society, 2(3):128–145, Sept. 2021

  21. [29]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

  22. [30]

    Zhang, R

    H. Zhang, R. Ramachandra, K. Raja, and C. Busch. Morph-pipe: Plugging in identity prior to enhance face morphing attack based on diffusion model. InNorsk IKT-konferanse for forskning og utdanning (NISK), volume 3, pages 1–6, 2023

  23. [31]

    Zhang, R

    H. Zhang, R. Ramachandra, K. Raja, and C. Busch. Generalized single-image-based morphing attack detection using deep represen- tations from vision transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1510– 1518, 2024

  24. [32]

    Zhang, R

    H. Zhang, R. Ramachandra, K. Raja, and C. Busch. Chatgpt encoun- ters morphing attack detection: Zero-shot mad with multi-modal large language models and general vision models, 2025

  25. [33]

    Zhang, S

    H. Zhang, S. Venkatesh, R. Raghavendra, K. Raja, N. Damer, and C. Busch. MIPGAN—Generating strong and high quality morphing attacks using identity prior driven GAN.IEEE Transactions on Biometrics, Behavior, and Identity Science, 3(3):365–383, 2021

  26. [34]

    Zhang, X

    N. Zhang, X. Liu, X. Li, and G.-J. Qi. Morphganformer: Transformer- based face morphing and de-morphing, 2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.