Pith. sign in

REVIEW 4 major objections 6 minor 25 references

Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs

T0 review · 4 major / 6 minor · reviewed 2026-07-30 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Multi-task fine-tuning with cheap grounding and description tasks makes small GI endoscopy VLMs both more accurate and more visually aligned.

desk verdict Solid applied recipe paper: multi-task grounding on small GI VLMs works directionally, with real ablations and OOD probes, but mask quality and judge protocol still need tightening before you lean on the numbers. read the letter →

arxiv 2607.27122 v1 pith:MX3VK5JG submitted 2026-07-29 cs.CV

Itbaan Safwan , Ramail Khan , Muhammad Annas Shaikh , Muhammad Atif Tahir This is my paper · ORCID
classification cs.CV
keywords VisionLanguageModelsWeakSupervisionMulti-TaskLearningMedicalVQAVisualGroundingParameter-EfficientFine-TuningGIendoscopyGrad-CAM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Clinical GI image tools are moving from single-label classifiers to visual question answering, but a correct answer is not enough if the model is not looking at the right tissue. This paper argues that small vision-language models can be made both more accurate and more trustworthy by a simple multi-task fine-tuning recipe that needs almost no new expert annotation. It reuses existing polyp masks, builds weak region masks for other findings with a GI-pretrained classifier and Grad-CAM, and adds terminology-free visual descriptions of the relevant region. Across three sub-2B backbones trained with low-rank adapters, the multi-task recipe beats matched VQA-only training, with the largest gains on appearance and spatial questions and on harder multi-aspect questions. A separate analysis of internal token-to-patch similarity shows stronger alignment between answer words and the true image region, including on an out-of-distribution polyp dataset the models never trained on.

What carries the argument

The multi-task recipe: expert polyp masks where available, Grad-CAM weak masks from a GastroNet-initialized classifier for other findings, and Gemma-generated terminology-free visual descriptions, all used as auxiliary targets under matched QLoRA fine-tuning against a VQA-only baseline.

What would settle it

A human spot-check of the Grad-CAM masks against expert outlines for non-polyp findings, or a controlled re-run that replaces those weak masks with true expert masks or random masks and checks whether VQA gains and token-to-patch alignment collapse.

Watch

Extended reading notes

Core claim

Jointly training small VLMs on GI VQA plus phrase-grounded localization and terminology-free visual description yields consistent answer-accuracy gains over VQA-only fine-tuning and improves implicit cross-modal alignment between answer tokens and the clinically relevant image region, on both in-distribution and out-of-distribution data.

Load-bearing premise

The weak Grad-CAM masks for non-polyp findings are good enough localization targets that the measured gains really come from teaching the model to look at the right tissue, not from noisy or merely class-correlated heatmaps.

Editorial extensions

If this is right

  • Small, single-GPU VLMs can be preferred for grounded GI VQA without sacrificing answer quality relative to VQA-only fine-tuning.
  • Appearance and spatial clinical questions, and multi-aspect questions, should improve most when localization and description auxiliaries are added.
  • Implicit answer-token to image-patch alignment can serve as an audit signal for trustworthiness even when the model is not asked to output a box or mask at inference.
  • Segmentation and description auxiliaries play complementary roles: description drives in-distribution concentration; localization generalizes better to unseen image sources.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If weak-mask quality is the main risk, the same recipe should transfer cleanly to any finding class that already has a reliable classifier but no pixel labels.
  • Architecture-sensitive gains (stronger on Florence-2 and InternVL than Qwen) suggest spatial token design, not just task mix, may gate how much grounding supervision sticks.
  • The same cheap auxiliary construction could be a practical regularizer for other medical VQA settings where full grounded conversation datasets are too expensive to collect.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript proposes a multi-task QLoRA fine-tuning recipe for GI endoscopy VQA on small (<2B) VLMs. Auxiliary tasks are constructed with minimal new annotation: expert Kvasir-SEG polyp masks are reused directly, Grad-CAM masks from a frozen GastroNet-5M ResNet-50 classifier provide weak localization supervision for abnormalities and landmarks, and Gemma-27B generates terminology-free visual descriptions as a third task. Three backbones (Florence-2 0.2B, Qwen3.5 0.8B, InternVL3.5 1.1B) are trained under matched VQA-only and multi-task recipes on a 50k subset of Kvasir-VQA-x1 and evaluated on a held-out 5k set with a DeepSeek-V4 rubric judge. The paper reports consistent overall accuracy gains (+1.37 to +2.74pp), concentrated in appearance/spatial question types and increasing with question complexity for two of three backbones, plus an implicit grounding analysis (decoder answer-token to encoder patch cosine similarity; pointing-game and concentration-ratio metrics) showing large improvements over VQA-only training on in-distribution Kvasir-SEG polyps and out-of-distribution CVC-ClinicDB, with a Wilcoxon signed-rank test (p=1.47e-36) on the OOD concentration ratio.

Significance. If the results hold, the work is a useful, honest contribution to medical VQA: it targets small, deployable VLMs rather than 4B+ models; the recipe is cheap (single RTX 3090 Ti, one epoch, no new pixel annotation); the matched VQA-only vs MT comparison across three backbones with per-complexity and per-question-type breakdowns is methodologically cleaner than much of the medical VQA literature; and the inclusion of an out-of-distribution grounding evaluation (CVC-ClinicDB) with a paired significance test is a genuinely falsifiable check that the grounding improvements are not merely in-distribution memorization. The Vis/Des ablation, showing complementary behavior of segmentation vs description supervision on discrete vs diffuse findings, is an informative result in its own right. The scope is workshop/conference-paper scale rather than a definitive clinical study, but within that scope the experimental design is reasonable and the claims are mostly calibrated.

major comments (4)
  1. [§4.1, Tables 1–3] All accuracy numbers appear to come from a single training run per backbone/recipe, with no seeds, confidence intervals, or per-class sample sizes. The headline gains are small (+1.37pp for Qwen3.5, +1.53pp for InternVL3.5), Qwen3.5 C1 actually regresses (87.0→86.6, Table 1), and the largest single claim (polyp size +12.1pp for Florence-2, Table 3) is on a class whose n is not reported (landmark color with n=11 was excluded, implying other classes may also be small). LoRA fine-tuning variance across seeds can easily exceed 1–2pp. The 'consistent gains' claim needs either multi-seed runs (≥3) or, at minimum, paired bootstrap confidence intervals on the 5k eval set and per-class n in Table 3.
  2. [§3 (evaluation protocol)] All reported accuracy depends on DeepSeek-V4 as a text-only rubric judge, with no validation of the judge itself. Two specific risks: (i) the partial-credit rubric (0.5 for partial multi-item coverage, 0.5 for off-by-one counts) is exactly where judge reliability matters most; (ii) MT training teaches the model to mimic Gemma-27B description phrasing, which could systematically shift answer style in ways a text-only judge rewards or penalizes independently of correctness. A small human-vs-judge agreement audit on a few hundred scored answers (reporting Cohen's kappa or accuracy of the judge against human labels) would substantially de-risk the paper's primary quantitative claim.
  3. [§4.2, Eq. (2)] The grounding metric's representations are under-specified. It is not stated whether e_ti and e_vj are static input embeddings or contextual hidden states, nor from which layer. If e_ti is a static vocabulary embedding, cosine similarity to encoder patch embeddings is only meaningful if the two modalities share a representation space, which is architecture-dependent (plausible for Florence-2, less obvious generally); if they are hidden states, the layer choice matters and should be justified. Relatedly, the in-distribution PG/CR evaluation in Table 4 uses the same Kvasir-SEG polyp masks that supply the MT segmentation supervision, so the in-distribution grounding improvement is partly by construction; the OOD CVC-ClinicDB result is the load-bearing evidence and the text should say so explicitly.
  4. [§2.1 (Grad-CAM mask pipeline)] The recipe's novelty rests on Grad-CAM masks from a frozen-backbone classifier being faithful localization targets for non-polyp findings, yet mask quality is explicitly deferred to future work (Conclusion). A cheap, in-scope sanity check exists: the classifier also predicts polyp, so the authors can compute IoU of the polyp Grad-CAM masks (thresholded by the same pipeline) against Kvasir-SEG expert masks and report it, even though those masks are unused in training. Without some such audit, the attribution of MT gains on abnormality/landmark classes to 'grounding supervision' rather than noisy class-correlated heatmaps remains an assumption.
minor comments (6)
  1. [§4.2, Table 4] The claim that 'Full MT achieves the best or near-best result in every condition' is strained by the OOD pointing-game numbers: MT scores 0.645 vs 0.727 for Vis-only, an 8.2pp gap. Please soften the wording and discuss why adding the description task degrades OOD PG relative to Vis alone.
  2. [§4.2] Only one significance test is reported (Wilcoxon on CVC CR). PG differences and the ablation comparisons are untested, and no multiple-comparison consideration is given across the four conditions × two datasets × two metrics. At minimum, report paired tests for the headline PG contrast.
  3. [§3] State the per-class sample counts in Table 3 and the exclusion threshold used (landmark color, n=11). Without n, readers cannot judge which per-class deltas are meaningful.
  4. [§2.3 / reproducibility] Hyperparameters are given, which is commendable, but no code or mask-release statement appears. Given the single-epoch, 50k-subset, single-GPU setup, releasing the training/eval code and the generated Grad-CAM masks would materially strengthen the paper.
  5. [§4.3, Fig. 2] The inflammation rows are assessed purely by visual inspection with no ground truth, which the authors acknowledge; please also note explicitly that the 'less noisy' judgment is qualitative and consider reporting a quantitative proxy (e.g., CR against the Grad-CAM masks used in training, with the caveat that this is partially circular).
  6. [General] There are spacing/typographical artifacts in the PDF (e.g., 'Grad-CAM-basedweaklocalizationandaterminology-free' in the contributions list, 'T able 1/2/3/4' captions). Please proofread the final camera copy.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: multi-task gains and grounding metrics are independently measured, not forced by construction from the training inputs.

full rationale

The paper’s central claims are empirical comparisons of matched VQA-only vs multi-task QLoRA fine-tuning on three small VLMs. Auxiliary targets (Kvasir-SEG expert polyp masks, Grad-CAM weak masks, Gemma-27B terminology-free descriptions) are training signals only; reported VQA accuracy is scored by an external text judge (DeepSeek-V4) on a held-out 5k split under a fixed rubric, and implicit grounding is a post-hoc decoder-token–to–encoder-patch cosine probe (pointing-game / concentration ratio) against expert masks, including fully OOD CVC-ClinicDB never used in training. These metrics are not algebraically or definitionally identical to the multi-task losses, and ablations (Vis-only, Des-only, full MT) further separate the signals. The self-citation to the authors’ prior Medico/arXiv Florence-2 multi-task work is background motivation and is not invoked as a uniqueness theorem or as the sole support for the present results. Weak-mask fidelity and LLM-generated description quality are assumption/correctness risks, not circular reductions. The derivation chain is self-contained experimental comparison against external benchmarks.

Assumptions & free parameters 5 free parameters · 6 assumptions · 2 invented entities

Empirical ML paper; load-bearing commitments are domain and pipeline assumptions plus hand-chosen thresholds, not physical free parameters. Central claim rests on faithfulness of weak masks, adequacy of one-epoch LoRA on a 50k subset, LLM-as-judge scoring without images, and cosine-similarity probes as a proxy for clinical trustworthiness. No new particles or forces; invented entities are methodological constructs (task mixture, metrics).

free parameters (5)
  • Grad-CAM keep threshold (classifier prob) = 0.8
    Images retained only if predicted class probability exceeds 0.8 before mask extraction; directly controls which weak labels enter training.
  • Minimum mask area and morphology kernel = 100 px; kernel 5
    Masks <100 pixels discarded; opening/closing kernel size 5—hand-set post-processing that changes supervision support.
  • LoRA rank/alpha and learning rates per backbone family = rank 128/8; lr 5e-5 / 1e-4
    Florence-2 rank 128 α=256 lr 5e-5; Qwen/InternVL rank 8 α=32 lr 1e-4—chosen training knobs that can move the reported deltas.
  • Training subset size and epoch count = 50k samples; 1 epoch
    50k of ~140k pairs, one epoch, for resource limits; absolute accuracies and MT margins may shift under full data/multi-epoch.
  • Top-10% patch centroid for pointing game = top-10% patches
    PG adapts excitation-backprop-style pointing using centroid of top-10% similarity patches—metric hyperparameter affecting grounding scores.
assumptions (6)
  • domain assumption Cross-entropy / standard VLM next-token training on mixed VQA, grounding, and description prompts improves task performance and representation alignment.
    Implicit throughout §2.3–4; standard multi-task VLM assumption, not proved here.
  • ad hoc to paper Grad-CAM on a GI-pretrained ResNet-50 classifier yields region masks semantically tied to finding labels well enough to serve as grounding supervision for non-polyp classes.
    Core of §2.1 abnormalities/landmarks track; authors flag validation as future work.
  • ad hoc to paper Gemma-27B terminology-free region descriptions are adequate targets for teaching visual-attribute grounding without clinical language leakage.
    §2.2 visual description construction; quality of synthetic targets not human-validated in-paper.
  • domain assumption DeepSeek-V4 text-only rubric scores against gold answer strings are a valid proxy for VQA correctness, including multi-item partial credit.
    §3 experimental setup; judge never sees the image.
  • domain assumption Decoder-token to encoder-patch cosine similarity (PG/CR) measures clinically meaningful implicit grounding / trustworthiness.
    §4.2 grounding analysis; proxy accepted from broader VQA interpretability literature (e.g., pointing game).
  • domain assumption Kvasir-SEG and CVC-ClinicDB expert polyp masks are correct evaluation ground truth for localization probes.
    Standard dataset trust; §3 and Table 4.
invented entities (2)
  • Multi-task GI VQA recipe (expert masks + Grad-CAM weak masks + terminology-free description)
    purpose: Package auxiliary supervision for small VLM fine-tuning without new pixel annotation campaigns.
    Methodological bundle claimed as contribution 1; composed of known parts, not a physical entity.
  • Token-to-patch concentration ratio (CR) as used here
    purpose: Summarize whether answer-token similarity mass concentrates inside GT masks vs outside.
    Defined in Eq. (3) §4.2 as an analysis metric; related to prior pointing/attention metrics but paper-specific formulation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs." pith.science (2026). https://pith.science/paper/MX3VK5JG

@misc{pith2026260727122,
  author       = {Pith},
  title        = {Pith review of: Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MX3VK5JG}},
  note         = {Machine review of arXiv:2607.27122}
}
read the original abstract

Gastrointestinal (GI) endoscopic image analysis has shifted from single-label classification toward visual question answering (VQA), where a model must answer free-form clinical questions about an image. While recent vision-language models (VLMs) achieve promising answer accuracy on this task, clinical adoption also requires the model's internal representations to reflect the visual evidence behind its answers. We propose a simple multi-task fine-tuning recipe that constructs auxiliary grounding and description tasks from an existing VQA dataset with minimal additional annotation: expert-annotated polyp masks are reused directly, while a GI-domain pretrained classifier with Grad-CAM localization provides weak supervision for finding categories that lack ground-truth masks. Three small VLM backbones are fine-tuned with low-rank adaptation under matched VQA-only and multi-task recipes on Kvasir-VQA-x1, and we show consistent accuracy gains together with improved implicit alignment between answer tokens and the relevant image region, evaluated on both in-distribution and out-of-distribution data.

Figures

Figures reproduced from arXiv: 2607.27122 by the authors.

Figure 1
Figure 1. Methodology overview. Phase 1 constructs auxiliary datasets via two tracks: GradCAM masks from a GastroNet-5M classifier (abnormalities/landmarks), Kvasir￾SEG expert masks (polyps), and Gemma-27B terminology-free visual descriptions. Phase 2 fine-tunes each backbone jointly on three tasks via QLoRA; only VQA is used at inference. 4 Results 4.1 VQA Accuracy Multi-task training consistently improves overall accuracy a… view at source ↗
Figure 2
Figure 2. Token-to-patch similarity maps (columns: original, VQA-only, Vis-only, Des￾only, MT ours). Row 1: polyp token; rows 2–3: inflammation token. Des localizes the polyp tightly; Vis produces the most structured response for diffuse inflammation; MT is the least noisy overall. No ground-truth mask exists for inflammation categories. For the polyp, all three auxiliary task conditions improve over VQA-only, and notably pro… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 2 canonical work pages

  1. [1]

    Das, A., Agrawal, H., Zitnick, L., Parikh, D., Batra, D.: Human attention in visual question answering: Do humans and deep networks look at the same regions? In: Proceedings of EMNLP 2016, pp. 932–937. ACL, Austin (2016).https://doi. org/10.18653/v1/D16-1092

    work page doi:10.18653/v1/d16-1092 2016

  2. [2]

    In: The Fourteenth International Conference on Learning Representations (ICLR) (2026)

    Liu, G., Yu, T., Ebrahimkhani, S., Shawn, L.Z.Z., Ng, K.P., Cheung, N.-M.: How do medical MLLMs fail? A study on visual grounding in medical images. In: The Fourteenth International Conference on Learning Representations (ICLR) (2026). https://openreview.net/forum?id=dXshexyFKx

    2026

  3. [3]

    In: MICCAI Workshop on Data Engineering in Medical Imaging, pp

    Gautam, S., Riegler, M., Halvorsen, P.: Kvasir-VQA-x1: A multimodal dataset for medical reasoning and robust MedVQA in gastrointestinal endoscopy. In: MICCAI Workshop on Data Engineering in Medical Imaging, pp. 53–63. Springer (2025)

    2025

  4. [4]

    In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp

    Lu, J., Goswami, V., Rohrbach, M., Parikh, D., Lee, S.: 12-in-1: Multi-task vi- sion and language representation learning. In: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), pp. 10437–10446 (2020)

    2020

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Deitke, M., Clark, C., Lee, S., et al.: Molmo and PixMo: Open weights and open data for state-of-the-art vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 91–104 (2025)

    2025

  6. [6]

    In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp

    Rui, S., Chen, L., Tang, Z., Wang, L., Liu, M., Zhang, S., Wang, X.: Multi-modal vision pre-training for medical image analysis. In: Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5164–5174 (2025)

    2025

  7. [7]

    arXiv:2511.04384 (2025)

    Safwan, I., Shaikh, M.A., Haaris, M., Khan, R., Tahir, M.A.: Multi-task learning for visually grounded reasoning in gastrointestinal VQA. arXiv:2511.04384 (2025)

    arXiv 2025

  8. [8]

    arXiv:2508.10869 (2025)

    Gautam, S., Thambawita, V., Riegler, M., Halvorsen, P., Hicks, S.: Medico 2025: Visual question answering for gastrointestinal imaging. arXiv:2508.10869 (2025)

    arXiv 2025

Show all 25 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp

    Lüddecke, T., Ecker, A.: Image segmentation using text and image prompts. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 7086–7096 (2022)

    2022

  2. [10]

    Gastroen- terology170(1), 174–187 (2026).https://doi.org/10.1053/j.gastro.2025.07

    Jong, M.R., Boers, T.G.W., Fockens, K.N., et al.: GastroNet-5M: A multicenter dataset for developing foundation models in gastrointestinal endoscopy. Gastroen- terology170(1), 174–187 (2026).https://doi.org/10.1053/j.gastro.2025.07. 030 10 I. Safwan et al

    2026 doi

  3. [11]

    arXiv:2503.19786 (2025)

    Gemma Team: Gemma 3 technical report. arXiv:2503.19786 (2025)

    2025 arXiv

  4. [12]

    Advances in Neural Information Processing Systems 36, 10088–10115 (2023)

    Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: QLoRA: Efficient fine- tuning of quantized LLMs. Advances in Neural Information Processing Systems 36, 10088–10115 (2023)

    2023

  5. [13]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Zhao, Y., Huang, J., Hu, J., et al.: SWIFT: A scalable lightweight infrastructure for fine-tuning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 28, pp. 29733–29735 (2025)

    2025

  6. [14]

    International Journal of Computer Vision128(2), 336–359 (2019).https://doi

    Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- CAM: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision128(2), 336–359 (2019).https://doi. org/10.1007/s11263-019-01228-7

    2019 doi

  7. [15]

    In: International Conference on Multimedia Modeling, pp

    Jha, D., Smedsrud, P.H., Riegler, M.A., Halvorsen, P., de Lange, T., Johansen, D., Johansen, H.D.: Kvasir-SEG: A segmented polyp dataset. In: International Conference on Multimedia Modeling, pp. 451–462. Springer (2020)

    2020

  8. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Xiao, B., Wu, H., Xu, W., Dai, X., Hu, H., Lu, Y., Zeng, M., Liu, C., Yuan, L.: Florence-2: Advancing a unified representation for a variety of vision tasks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4818–4829 (2024)

    2024

  9. [17]

    Qwen Team: Qwen3.5: Accelerating productivity with native multimodal agents (2026).https://qwen.ai/blog?id=qwen3.5

    2026

  10. [18]

    arXiv:2508.18265 (2025)

    Wang, W., Gao, Z., Gu, L., et al.: InternVL3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv:2508.18265 (2025)

    2025 arXiv

  11. [19]

    Pattern Recognition45(9), 3166–3182 (2012).https: //doi.org/10.1016/j.patcog.2012.03.002

    Bernal, J., Sánchez, J., Vilariño, F.: Towards automatic polyp detection with a polyp appearance model. Pattern Recognition45(9), 3166–3182 (2012).https: //doi.org/10.1016/j.patcog.2012.03.002

    2012 doi

  12. [20]

    In: Proceedings of the 8th ACM on Multimedia Systems Conference (MMSys’17), pp

    Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.-T., Lux, M., Schmidt, P.T., Riegler, M., Halvorsen, P.: KVASIR: A multi-class image dataset for computer aided gastroin- testinal disease detection. In: Procee...

    2017

  13. [21]

    In: MICCAI (2025, early accept, spotlight)

    Khanal, B., Pokhrel, S., Bhandari, S., Rana, R., Shrestha, N., Gurung, R.B., Linte, C., Watson, A., Shrestha, Y.R., Bhattarai, B.: Hallucination-aware multimodal benchmark for gastrointestinal image analysis with large vision-language models. In: MICCAI (2025, early accept, spotlight)

    2025

  14. [22]

    DeepSeek-AI: DeepSeek-V4: Towards highly efficient million-token context intelli- gence (2026)

    2026

  15. [23]

    International Journal of Computer Vision 126(10), 1084–1102 (2018)

    Zhang, J., Bargal, S.A., Lin, Z., Brandt, J., Shen, X., Sclaroff, S.: Top-down neu- ral attention by excitation backprop. International Journal of Computer Vision 126(10), 1084–1102 (2018)

    2018

  16. [24]

    Medical Image Analysis, 103789 (2025)

    Wang, G., Bai, L., Wang, J., et al.: EndoChat: Grounded multimodal large lan- guage model for endoscopic surgery. Medical Image Analysis, 103789 (2025)

    2025

  17. [25]

    arXiv:2305.11692 (2023)

    Bai, L., Islam, M., Seenivasan, L., Ren, H.: Surgical-VQLA: Transformer with gated vision-language embedding for visual question localized-answering in robotic surgery. arXiv:2305.11692 (2023)

    2023 arXiv

Pith tools

Reviewed July 30, 2026 · model on record in the stance chip above.