Pith. sign in

REVIEW 3 major objections 5 minor 43 references

In medical visual question answering, top lexical answer scores do not reliably translate into faithful and complete clinical explanations, so evaluation must look beyond leaderboards.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 23:44 UTC pith:G4I2KCVO

load-bearing objection A well-hedged retrospective of a MedVQA challenge; the central descriptive claims are plausible, but the trustworthiness rankings rest entirely on an unvalidated LLM judge and no analysis artifacts are shipped. the 3 major comments →

arxiv 2607.15241 v1 pith:G4I2KCVO submitted 2026-07-16 cs.CL cs.CV

Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA

Sushant Gautam , Vajira Thambawita , Michael A. Riegler , P{\aa}l Halvorsen , Steven A. Hicks This is my paper
classification cs.CL cs.CV
keywords multimodal VQAmedical visual question answeringexplainabilityfaithfulnessrobust evaluationparameter-efficient fine-tuninggastrointestinal endoscopysemantic adjudication
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper retrospectively analyzes nine systems that participated in a gastrointestinal endoscopy visual question answering (VQA) challenge, comparing how well they answer questions and how well they explain those answers. It argues that parameter-efficiently adapted models can top lexical leaderboards without producing clinically trustworthy reasoning: explanation quality varies widely on faithfulness, clinical relevance, and completeness, while fluency stays high. Systems that enforce structured reasoning and explicit visual grounding show more reliable explanation behavior, but the evidence is correlational, not causal. The authors conclude that medical VQA evaluation needs semantic correctness checks, evidence-linked explanation standards, leakage-aware data splits, and lightweight robustness and calibration tests.

Core claim

The paper's central discovery is a decoupling: a system can rank at or near the top on standard lexical answer metrics while producing explanations that score notably lower on faithfulness and completeness, and vice versa. In the nine-system comparison, the largest score ranges across systems were in faithfulness, clinical relevance, and completeness, not in clarity, indicating that fluent language is easier to achieve than grounded justification. The paper also documents that difficulty is non-monotonic with question complexity and that fine-grained spatial/color classes are the persistent weak points. Its main claim follows from these patterns: because answer-level gains do not reliably tr

What carries the argument

The central machinery is a cross-system comparison: nine independently built systems, most adapting pretrained vision-language backbones with parameter-efficient fine-tuning, were scored on two connected tasks (answer generation and explanation generation). Explanation quality was measured by a rubric-based large-language-model adjudicator on five dimensions (correctness, faithfulness, clinical relevance, clarity, completeness). The analysis then maps design choices—self-probing pipelines that generate auxiliary clinical sub-questions before the final answer, multi-task grounded learning with vision-language grounding supervision, unified answer-plus-explanation heads, and answer-focused bas

Load-bearing premise

The whole comparison of explanation quality rests on the assumption that a large language model's rubric scores are a valid proxy for clinically meaningful explanation quality; the paper acknowledges the judge was not validated by clinicians and may be biased if submitted systems share its model family.

What would settle it

Collect answers and explanations from the nine systems and have both a panel of gastrointestinal clinicians and the rubric-based LLM judge score them. If the clinicians rank systems differently from the judge, or if an answer-optimized system with no grounding is judged as faithful as a structured-grounding system, then the central claim that answer accuracy decouples from reasoning trustworthiness loses its evidentiary base.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Benchmark rankings based on lexical overlap should be supplemented with semantic adjudication, exact-match counts, and normalized yes/no checks before any deployment decision is made.
  • Explanation outputs should be standardized with evidence links, such as visual regions and confidence values, so faithfulness and completeness can be audited.
  • Medical VQA data splits should be image-disjoint and accompanied by explicit overlap audits, since QA-level splitting creates a structural leakage risk.
  • Systems should be required to report lightweight robustness checks, such as corruption and transformation tests, along with calibration metadata.
  • Question classes requiring fine spatial or color discrimination should be reported separately, since they drive persistent failure even in otherwise strong systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The correlational link between structured reasoning or grounding and better explanation scores would be much stronger if tested by ablation on a single model; a controlled comparison turning these design elements on and off is the natural next experiment the paper does not run.
  • Because the explanation judge is a large language model that may favor fluent text, the observed clarity-versus-faithfulness gap could partly be judge bias; having clinicians score a sample of the same explanations would test whether the ranking survives human review.
  • The answer-reasoning decoupling likely generalizes to other medical imaging VQA domains, such as radiology or pathology, where spatial and color language is similarly central and lexical benchmarks are known to be weak.
  • The paper's proposed standardized explanation schema could be turned into a clinical safety filter: any explanation lacking an evidence region or a confidence estimate would be automatically flagged, a testable product implication the paper leaves implicit.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper is a retrospective analysis of the MediaEval Medico 2025 challenge, comparing nine documented systems for GI endoscopy VQA and explainable reasoning. It combines official leaderboard scores, organizer-released semantic adjudication, and rubric-based LLM scores to examine how answer accuracy, explanation faithfulness/completeness, robustness to private-set shift, and design choices (PEFT, self-probing, grounding) relate. The main claim is that lexical answer gains do not necessarily translate into faithful and complete clinical reasoning, and that methods enforcing structured reasoning and explicit grounding show more reliable behavior, although the evidence is explicitly correlational rather than ablation-based. The paper concludes with five practical recommendations for medical VQA evaluation, including semantic adjudication, image-level splits, explanation schemas, and calibration metadata.

Significance. The paper's value lies in using a real shared task to produce concrete, transferable observations about evaluation practice. It is unusually candid about limitations: it explicitly labels evidence as correlational, judge-dependent, and non-ablation-based, and it proposes specific, actionable evaluation upgrades. If the central finding holds, it strengthens the case for moving beyond BLEU-style metrics in medical VQA and for standardizing explanation artifacts. However, the central finding currently rests on a single unvalidated LLM judge, and the design-axis comparisons have very small per-cell sample sizes. These issues do not invalidate the contribution, but they limit the strength of the conclusions that can be drawn without additional validation.

major comments (3)
  1. [Section V.D and Section VII.D] The central RQ4 finding—that fluent language is easier to optimize than faithful, clinically grounded justification—depends entirely on Qwen3-30B-A3B rubric scores. The paper itself admits this judge 'lacked clinician validation at scale here and may introduce alignment bias if submitted systems use related model families' (Section V.D). No sensitivity analysis, second judge, inter-judge agreement, or human spot-check is reported. Because the same judge also produces the Subtask 1 semantic adjudication and the official Subtask 2 ranking, a systematic preference for fluent text could generate the observed clarity-faithfulness gap and the team ordering without reflecting clinically meaningful explanation quality. The manuscript's caveat that these are 'structured proxies rather than substitutes for clinician-led assessment' is appropriate but does not resolve the load-bearing nature of the
  2. [Section VII.D and Table I] The design-axis comparison is based on one or two teams per family. Within the self-probing family, Team Nepal has the highest faithfulness (0.74) while IReL@IIT(BHU) has the lowest (0.27), a spread larger than most between-family differences. The paper acknowledges this is exploratory, but Section VIII still concludes that 'explicit reasoning structure, grounding constraints, and calibrated confidence reporting are consistently associated with stronger trust signals.' With this sample, 'consistently' is unsupported. Please either present per-family aggregates with variance and a formal descriptive comparison, or limit the conclusion to the specific teams observed rather than to design families.
  3. [Section IV and Section VII.C] The leakage risk is noted but not resolved. The paper states that no complete per-team overlap audit was available, and that Lama4Vision reported substantial image-level overlap. Despite this, Test→Private BLEU drift is used as a robustness measure and teams are compared by it (e.g., CVG-IBA −0.048 vs EndoVision −0.005). If overlap differs across teams, the private-set comparison conflates generalization with contamination. The caveat that drift alone is insufficient is helpful, but the analysis in Section VII.C should either incorporate overlap information as a covariate or be explicitly demoted to a hypothesis-generating observation rather than a team-level robustness ranking.
minor comments (5)
  1. [Table II(a) and Section VII.A] Table II(a) reports only the top four teams, but the text claims behavior 'across all nine teams with both splits.' Please provide the full table (or an appendix) to support the reported mean drift and top-quartile statistics.
  2. [Section VII.B] The non-monotonic complexity profile (BLEU 0.356 at L1, 0.323 at L2, 0.416 at L3) is reported only for the test split. Clarify whether the private set shows the same pattern, and if not, discuss the difference.
  3. [Section VII.B vs Section VII.D] finding_presence is listed as a strongest class in Subtask 1 (0.929, semantic adjudication) but as the hardest in Subtask 2 (correctness 0.099, rubric score). Label the subtask and metric in both places to avoid an apparent contradiction.
  4. [Fig. 2(b)] The caption says the plot is 'from Team Nepal.' Clarify whether these are Team Nepal's outputs or aggregate scores across teams, and what the y-axis scale represents.
  5. [Section V.B] The replacement of the originally proposed expert evaluation with LLM adjudication is a significant protocol change. Consider discussing its implications more prominently, since it directly affects how readers interpret the absence of clinician validation.

Circularity Check

0 steps flagged

No significant circularity: the analysis is externally grounded in official challenge outputs and openly hedges its judge-based evidence.

full rationale

The paper's derivation chain is descriptive rather than definitional or fitted. Its central claims—that answer-level gains do not consistently translate into faithful/complete clinical reasoning and that structured reasoning/grounding are associated with more reliable behavior—are drawn from official MediaEval Medico 2025 leaderboards, organizer-released post-challenge diagnostics, and team working notes. These are external inputs produced by independent participating teams, not quantities derived from the paper's own assumptions. The authors are indeed the challenge organizers and dataset creators, and they cite their own overview papers ([1], [2], [11], [21]), but these citations establish the benchmark and scoring protocol, not the conclusions. No parameter is fitted and then renamed as a prediction; no equation in the paper reduces to its inputs by construction. The strongest potential concern is that Subtask 2 trustworthiness scores come entirely from the Qwen3-30B-A3B judge, which could make the 'faithfulness over fluency' lesson judge-dependent. However, the paper explicitly flags this in Section V.D: 'Qwen3-30B-A3B lacked clinician validation at scale here and may introduce alignment bias if submitted systems use related model families.' It also repeatedly labels the evidence as correlational and exploratory ('although the evidence is correlational rather than ablation-based'; 'family-vs-outcome results should be read as exploratory descriptive evidence rather than causal or inferential proof'). Acknowledged validity threats are not circularity: the claims could be wrong under a different judge, but they are not true by construction. The self-citations are present but not load-bearing in a circular sense; the central empirical content comes from independent challenge submissions and official rankings.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 0 invented entities

The paper performs no mathematical derivation and introduces no fitted parameters or new physical/clinical entities. Its inferential load is carried by evaluation-tool assumptions, dataset-label validity, and self-reported system descriptions.

axioms (5)
  • domain assumption Qwen3-30B-A3B rubric scores faithfully measure explanation trustworthiness (faithfulness, completeness, clinical relevance, clarity).
    Used for all Subtask 2 rankings and diagnostics (Section V.B). The paper notes in V.D that the judge lacked clinician validation and may have alignment bias, so this is a load-bearing assumption.
  • domain assumption The Kvasir-VQA-x1 complexity levels and question-class annotations are valid proxies for reasoning difficulty.
    All complexity-by-category analyses in Section VII use these labels as ground truth (Section IV). If the labels are noisy, the non-monotonic trajectories are less interpretable.
  • domain assumption Team working notes accurately describe the systems, training procedures, and interventions used by the nine teams.
    The taxonomy in Table I and the qualitative interventional claims rely on self-reports from working notes (Sections III and VIII), which the paper explicitly flags as non-uniform.
  • domain assumption The private ImageCLEF MEDVQA 2025 set provides a leakage-free generalization test.
    Section IV introduces the private set as cleaner than the QA-level split, but no complete per-team overlap audit was available, so the private set's freedom from leakage is assumed rather than proven.
  • domain assumption LLM-based semantic adjudication for Subtask 1 class-wise correctness is a valid measure of answer correctness.
    The class-wise diagnoses in Section VII.B are aggregated from Qwen3-30B-A3B binary aspect labels (Section V.A), making judge reliability a precondition for those patterns.

pith-pipeline@v1.3.0-alltime-deepseek · 10115 in / 9432 out tokens · 82201 ms · 2026-08-01T23:44:51.574891+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA." pith.science (2026). https://pith.science/paper/G4I2KCVO

@misc{pith2026260715241,
  author       = {Pith},
  title        = {Pith review of: Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G4I2KCVO}},
  note         = {Machine review of arXiv:2607.15241}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Healthcare multimodal AI must combine visual and textual evidence while remaining reliable and interpretable. Using MediaEval Medico 2025 as a retrospective GI endoscopy case study, we analyze design choices across nine documented systems for question answering and explanation quality. Parameter-efficient adaptation of pretrained backbones provides strong challenge performance, but answer-level gains do not consistently translate into faithful and complete clinical reasoning. Methods enforcing structured reasoning and explicit grounding show more reliable behavior across heterogeneous question types, although the evidence is correlational rather than ablation-based. These results motivate evaluation beyond lexical overlap, standardized evidence-linked explanations, leakage-aware data governance, and lightweight robustness and calibration checks. The findings support trustworthy multimodal healthcare AI based on data fusion, explainability, and resilient evaluation.

Figures

Figures reproduced from arXiv: 2607.15241 by Michael A. Riegler, P{\aa}l Halvorsen, Steven A. Hicks, Sushant Gautam, Vajira Thambawita.

Figure 1
Figure 1. Figure 1: Example of Task 1 (visual question answering) followed by probing [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Diagnostics plots for Subtask 1 and Subtask 2 (from Team Nepal). [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 1 canonical work pages

  1. [1]

    Medico 2025: Visual Question Answering for Gas- trointestinal Imaging,

    S. Gautamet al., “Medico 2025: Visual Question Answering for Gas- trointestinal Imaging,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025

    2025

  2. [2]

    Kvasir- VQA-x1:A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy,

    Gautam, Sushant and Riegler, Michael and Halvorsen, P ˚al, “Kvasir- VQA-x1:A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy,” inData Engineering in Medical Imaging, MICCAI. Springer, Oct. 2025, pp. 53–63, DOI: https://doi.org/10.1007/978-3-032-08009-7 6

    work page doi:10.1007/978-3-032-08009-7 2025

  3. [3]

    LoRA: Low-rank adaptation of large language models,

    E. J. Huet al., “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022. [Online]. Available: https://openreview.net/forum?id=nZeVKeeFYf9

    2022

  4. [4]

    Qlora: efficient finetuning of quantized llms,

    T. Dettmerset al., “Qlora: efficient finetuning of quantized llms,” in Proceedings of the 37th International Conference on Neural Information Processing Systems, ser. NIPS ’23. Red Hook, NY , USA: Curran Associates Inc., 2023

    2023

  5. [5]

    Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment,

    L. Xuet al., “Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment,”ArXiv e-prints, Dec. 2023

    2023

  6. [6]

    A Survey on Medical Large Language Models: Tech- nology, Application, Trustworthiness, and Future Directions,

    L. Liuet al., “A Survey on Medical Large Language Models: Tech- nology, Application, Trustworthiness, and Future Directions,”ArXiv e- prints, Jun. 2024

    2024

  7. [7]

    VQA-Med: Overview of the medical visual ques- tion answering task at imageclef 2019,

    A. Ben Abachaet al., “VQA-Med: Overview of the medical visual ques- tion answering task at imageclef 2019,” inCLEF2019 Working Notes, ser. CEUR Workshop Proceedings, Lugano, Switzerland, September 09- 12 2019

    2019

  8. [8]

    Medical visual question answering at imageclef-vqa med,

    M. Bansalet al., “Medical visual question answering at imageclef-vqa med,” inWorking Notes of CLEF 2019 – Conference and Labs of the Evaluation Forum, Sep. 2019, pp. –. [Online]. Available: https://ceur-ws.org/V ol-2380/paper 147.pdf

    2019

  9. [9]

    Medical visual question answering: A survey,

    Z. Linet al., “Medical visual question answering: A survey,”Artificial Intelligence in Medicine, vol. 143, p. 102611, Sep. 2023

    2023

  10. [10]

    Overview of ImageCLEFmedical 2025– Visual Question Answering and Synthetic Image Generation for Gastrointestinal Tract,

    S. Gautamet al., “Overview of ImageCLEFmedical 2025– Visual Question Answering and Synthetic Image Generation for Gastrointestinal Tract,”Working Notes of the Conference and Labs of the Evaluation Forum (CLEF 2025), 2025. [Online]. Available: https://ceur-ws.org/V ol-4038/paper 171.pdf

    2025

  11. [11]

    Kvasir-VQA: A Text-Image Pair GI Tract Dataset,

    Gautam, Sushant and Stor ˚as, Andrea M. and Midoglu, Cise and others, “Kvasir-VQA: A Text-Image Pair GI Tract Dataset,” inACM Confer- ences. New York, NY , USA: Association for Computing Machinery, Oct. 2024, pp. 3–12, DOI: https://doi.org/10.1145/3689096.3689458

    arXiv 2024

  12. [12]

    Exploring Vision-Language Models for Medical VQA on Gastrointestinal Images: A LoRA Fine-Tuning Study,

    V . Subramoniamet al., “Exploring Vision-Language Models for Medical VQA on Gastrointestinal Images: A LoRA Fine-Tuning Study,” in Working Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper30.pdf

    2025

  13. [13]

    LoRA-Enhanced PaliGemma for Efficient Visual Question Answering in Gastrointestinal Imaging,

    P. K. Jhaet al., “LoRA-Enhanced PaliGemma for Efficient Visual Question Answering in Gastrointestinal Imaging,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper44.pdf

    2025

  14. [14]

    Multimodal Explanations: Justifying Decisions and Pointing to the Evidence,

    D. H. Parket al., “Multimodal Explanations: Justifying Decisions and Pointing to the Evidence,” inIEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018, pp. 18–23

    2018

  15. [15]

    Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?

    A. Jacoviet al., “Towards faithfully interpretable NLP systems: How should we define and evaluate faithfulness?” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafskyet al., Eds. Online: Association for Computational Linguistics, Jul. 2020, pp. 4198–4205

    2020

  16. [16]

    Towards Faithful Model Explanation in NLP: A Survey,

    Q. Lyuet al., “Towards Faithful Model Explanation in NLP: A Survey,” Computational Linguistics, vol. 50, no. 2, pp. 657–723, Jun. 2024

    2024

  17. [17]

    LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods,

    H. Liet al., “LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods,”ArXiv e-prints, Dec. 2024

    2024

  18. [18]

    From Answers to Explanations: Self-Probing Efficiently Fine-Tuned Vision-Language Models for Medical VQA at Medico 2025,

    S. Gaihreet al., “From Answers to Explanations: Self-Probing Efficiently Fine-Tuned Vision-Language Models for Medical VQA at Medico 2025,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/ paper15.pdf

    2025

  19. [19]

    Curriculum-Guided Fine-Tuning for Multimodal VQA in GI Endoscopy (Team Lama4Vision),

    M. Azmoodeh-Kalatiet al., “Curriculum-Guided Fine-Tuning for Multimodal VQA in GI Endoscopy (Team Lama4Vision),” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper28.pdf

    2025

  20. [20]

    Qwen3 Technical Report,

    A. Yanget al., “Qwen3 Technical Report,” 2025. [Online]. Available: https://arxiv.org/abs/2505.09388

    Pith/arXiv arXiv 2025

  21. [21]

    Medico 2025: Visual Question Answering for Gastrointestinal Imaging,

    S. Gautamet al., “Medico 2025: Visual Question Answering for Gastrointestinal Imaging,” Overview presentation by Medico Challenge organizers at MediaEval 2025, Oct. 2025, available: https://doi.org/10.13140/RG.2.2.34380.88967

    arXiv 2025

  22. [22]

    ROUGE: A Package for Automatic Evaluation of Sum- maries,

    C.-Y . Lin, “ROUGE: A Package for Automatic Evaluation of Sum- maries,” inProceedings of the ACL Workshop on Text Summarization Branches Out, 2004, pp. 74–81

    2004

  23. [23]

    METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments,

    S. Banerjeeet al., “METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments,” inProceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005, pp. 65–72

    2005

  24. [24]

    chrF++: words helping character n-grams,

    M. Popovi ´c, “chrF++: words helping character n-grams,” inProceedings of the Second Conference on Machine Translation (WMT), 2017, pp. 612–618

    2017

  25. [25]

    BLEU: a Method for Automatic Evaluation of Machine Translation,

    K. Papineniet al., “BLEU: a Method for Automatic Evaluation of Machine Translation,” inProceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), 2002, pp. 311–318

    2002

  26. [26]

    BERTScore: Evaluating Text Generation with BERT,

    T. Zhanget al., “BERTScore: Evaluating Text Generation with BERT,” inInternational Conference on Learning Representations (ICLR), 2020

    2020

  27. [27]

    Evaluate: A library for easily evaluating machine learn- ing models and datasets,

    Hugging Face, “Evaluate: A library for easily evaluating machine learn- ing models and datasets,” https://huggingface.co/docs/evaluate, 2022, accessed: 2026-02-20

    2022

  28. [28]

    Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA,

    I. Safwanet al., “Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper36.pdf

    2025

  29. [29]

    Medico 2025: Visual Question Answering (with Multimodal Explanations) for Gastrointestinal Imaging,

    S. Chandran K. R.et al., “Medico 2025: Visual Question Answering (with Multimodal Explanations) for Gastrointestinal Imaging,” in Working Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper56.pdf

    2025

  30. [30]

    Enhancing Encoder-Decoder Architecture to Visual Question Answering Task for Gastrointestinal Images,

    M.-T. Buiet al., “Enhancing Encoder-Decoder Architecture to Visual Question Answering Task for Gastrointestinal Images,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper45.pdf

    2025

  31. [31]

    BLIP-2-based Visual Question Answering with Multimodal Explanations for Gastrointestinal Imaging,

    P. S. Lakshmiet al., “BLIP-2-based Visual Question Answering with Multimodal Explanations for Gastrointestinal Imaging,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper14.pdf

    2025

  32. [32]

    X-VQA for GI Diagnostics: Multimodal Visual Question Answering with Confidence-Aware Explanations,

    K. Tewariet al., “X-VQA for GI Diagnostics: Multimodal Visual Question Answering with Confidence-Aware Explanations,” inWorking Notes Proceedings of the MediaEval 2025 Workshop, 2025. [Online]. Available: https://2025.multimediaeval.com/paper38.pdf

    2025

  33. [33]

    Florence-2: Advancing a unified representation for a variety of vision tasks,

    B. Xiaoet al., “Florence-2: Advancing a unified representation for a variety of vision tasks,” 2023. [Online]. Available: https: //arxiv.org/abs/2311.06242

    Pith/arXiv arXiv 2023

  34. [34]

    Paligemma: A versatile 3b vlm for transfer,

    L. Beyeret al., “Paligemma: A versatile 3b vlm for transfer,” 2024. [Online]. Available: https://arxiv.org/abs/2407.07726

    Pith/arXiv arXiv 2024

  35. [35]

    Paligemma 2: A family of versatile vlms for transfer,

    A. Steineret al., “Paligemma 2: A family of versatile vlms for transfer,” 2024. [Online]. Available: https://arxiv.org/abs/2412.03555

    Pith/arXiv arXiv 2024

  36. [36]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,

    J. Liet al., “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” 2023. [Online]. Available: https://arxiv.org/abs/2301.12597

    Pith/arXiv arXiv 2023

  37. [37]

    Instructblip: Towards general-purpose vision-language models with instruction tuning,

    W. Daiet al., “Instructblip: Towards general-purpose vision-language models with instruction tuning,” 2023. [Online]. Available: https: //arxiv.org/abs/2305.06500

    Pith/arXiv arXiv 2023

  38. [38]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P. Wanget al., “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,” 2024. [Online]. Available: https://arxiv.org/abs/2409.12191

    Pith/arXiv arXiv 2024

  39. [39]

    Image Segmentation Using Text and Image Prompts,

    T. L ¨uddeckeet al., “Image Segmentation Using Text and Image Prompts,”ArXiv e-prints, Dec. 2021

    2021

  40. [40]

    Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,

    R. R. Selvarajuet al., “Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization,” inIEEE International Con- ference on Computer Vision (ICCV). IEEE, 2017, pp. 22–29

    2017

  41. [41]

    Quantifying attention flow in transformers,

    S. Abnaret al., “Quantifying attention flow in transformers,” inProceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, D. Jurafskyet al., Eds. Online: Association for Computa- tional Linguistics, Jul. 2020, pp. 4190–4197

    2020

  42. [42]

    Curriculum Learning: A Survey,

    P. Sovianyet al., “Curriculum Learning: A Survey,”ArXiv e-prints, Jan. 2021

    2021

  43. [43]

    LLM-FP4: 4-bit floating-point quantized transformers,

    S.-y. Liuet al., “LLM-FP4: 4-bit floating-point quantized transformers,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, H. Bouamoret al., Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 592–605

    2023