Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Computed Tomography Visual Question Answering with Cross-modal Feature Graphing

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A cross-modal graph that links CT slices to question tokens improves medical visual question answering on the M3D-VQA benchmark when used as a soft prompt for a multimodal LLM.

desk verdict A clean, honest incremental paper whose small consistent gains are real but whose central graph mechanism is under-tested because no ablation manipulates the edge set. read the letter →

arxiv 2507.04333 v1 pith:E27ZACUA submitted 2025-07-06 cs.CV cs.CL

Yuanhe Tian , Chen Su , Junwen Duan , Yan Song This is my paper
classification cs.CVcs.CL
keywords visualquestionansweringcomputedtomographycross-modalfeaturegraphattentiveconvolutionalnetworklargelanguagemodelmedicalimageanalysisM3D-VQAsoftprompting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that computed tomography visual question answering improves when the model is given an explicit graph of how CT slices relate to each other and to the words in the question. The authors argue that standard multimodal LLMs treat a CT volume as a bag of unrelated 2D images and therefore miss the spatial continuity that matters for clinical questions. Their framework builds a cross-modal graph whose nodes are slices and question tokens, fuses the nodes with an attentive graph convolutional network, and feeds the result to an LLM as a soft prompt. On the M3D-VQA benchmark, they report consistent gains over a vanilla LLM, LLM plus GCN, LLM plus GAT, and prior CT-VQA systems across BLEU, ROUGE, METEOR, and BERT-Score. If correct, the gain shows that explicitly modeling inter-slice structure is a useful complement to LLM-based medical image reasoning.

What carries the argument

Cross-modal feature graphing with an attentive graph convolutional network (A-GCN). The graph has $N$ slice nodes and $M$ token nodes, with an adjacency matrix that connects neighboring slices and connects every question token to every slice; the A-GCN replaces the binary adjacency entries with attention weights $w_{j,k}$ that depend on the representations of the two nodes, so that each layer learns which slice-token and slice-slice relations matter for the question. The final node representations are stacked and projected into the LLM's embedding space to serve as a soft prompt alongside the question. This is the component the paper credits for capturing inter-slice continuity and for focusing on question-relevant slices.

What would settle it

Run the same model with the slice order shuffled before graph construction, breaking the neighbor-slice edges while keeping all slice content and question tokens identical; if performance on M3D-VQA does not drop, the inter-slice adjacency is not doing the claimed work. A second check: replace the A-GCN attention weights with random fixed weights of the same scale; if BLEU and BERT-Score stay unchanged, the learned node attention is not the source of the gains.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck for CT VQA is not missing visual or textual capacity but missing structural fusion between slices and question tokens. The paper argues that treating the CT volume as a graph—each slice a node, each question token a node, edges between neighboring slices and between every token and every slice—lets an attentive GCN weight which slice-token relations matter, and that the resulting aggregated representation acts as a soft prompt that makes the LLM answer more accurately and more interpretably. On M3D-VQA they report that this approach beats the vanilla multimodal LLM and graph baselines on all four metrics and all five question types, with the largest relative improvements on abnormality and location questions. The attention weights also let a user trace which slices drove the answer, which the paper presents as evidence that the model identifies clinically relevant slices rather than averaging over all of them.

Load-bearing premise

The load-bearing premise is that the vanilla LLM baseline treats CT slices as unrelated 2D images and therefore misses inter-slice correlations; the paper does not measure how much inter-slice reasoning the LLM already performs internally, so part of the observed gain could come from added parameters or extra supervision rather than from the graph structure itself.

Editorial extensions

If this is right

  • If the claim holds, adding a small cross-modal graph module to a multimodal LLM can improve CT VQA without changing the LLM's decoding procedure; the graph output is simply a soft prompt.
  • Attention weights on slices provide a built-in explanation: clinicians or developers could trace which slices contributed to an answer, which the paper argues improves trustworthiness.
  • The reported gains on abnormality and location question types suggest graph fusion helps most where the answer depends on finding the right slice, not on global pattern recognition.
  • The approach transfers across LLM backbones and training strategies; the paper reports gains with full fine-tuning and LoRA and with different LLM sizes.
  • If the graph's inter-slice edges are what carry the improvement, then CT VQA systems should treat volumes as structured sequences rather than independent images.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate whether the gain comes from the graph structure itself or from the extra trainable parameters and attention machinery; an ablation that matches parameter count or applies A-GCN to randomly rewired graphs would settle this.
  • A stronger test of the inter-slice hypothesis would shuffle slice order before graph construction; if answers do not degrade, then the adjacency among neighboring slices is not the active ingredient.
  • The same graphing recipe could apply to other volumetric medical modalities such as MRI or ultrasound, and to non-medical 3D VQA, provided the node semantics of slice and token transfer.
  • One could test the soft-prompt design against hard prompting, for instance feeding the top-ranked slices by attention weight to the LLM as image input, to see whether the graph's value is selection or fusion.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an LLM-based CT VQA framework in which CT slices and question tokens are represented as nodes of a cross-modal graph; adjacent CT slices are connected to each other and every question token is connected to every slice. An attentive graph convolutional network (A-GCN) encodes this graph, and the resulting node representations are projected and used as a soft prompt for answer generation by Qwen2-VL. Experiments on the M3D-VQA benchmark compare the proposed approach against a vanilla LLM, LLM+GCN, and LLM+GAT, and report consistent improvements across BLEU, ROUGE, METEOR, and BERT-Score, along with additional experiments varying the LLM backbone, training strategy, and prompt content.

Significance. If the reported gains are robust, the paper offers a simple, modular mechanism for injecting volumetric inter-slice structure into multimodal LLMs for CT VQA, with a useful interpretability byproduct in the form of slice-level attention. The use of a public benchmark, the release of code, the comparison across multiple LLM backbones, and the inclusion of several baselines are strengths. However, the headline improvement over the strongest baseline is small, and the causal role of the specific graph edge construction is not directly tested, so the significance of the contribution currently rests on ablations that do not manipulate the graph structure itself.

major comments (3)
  1. [2.2.3, Eq. (7)] The attention weights w_jk are normalized by a denominator that sums over all M+N nodes rather than only over graph neighbors. Since w_jk = a_jk * exp(...) / sum_{k'=1}^{M+N} exp(...), every non-neighbor contributes to the denominator even though a_jk = 0 sets the numerator to zero. As a result, the weights assigned to neighbors do not sum to one, and the total message passed to a node is scaled by attention scores of nodes outside its neighborhood. This is a technical error in the attentive GCN formulation and should be corrected (the normalization should run over k with a_jk = 1) and the experiments rerun, since it may affect the comparison with LLM+GAT.
  2. [2.2.1; Tables 2, 4, 5, 6] The central claim that modeling inter-slice continuity drives the gains is not isolated. The graph construction in Section 2.2.1 includes adjacent-slice and full token-slice edges, but no experiment varies this edge set: Table 6 changes only which node embeddings are projected into the LLM, Tables 4 and 5 change the backbone or tuning strategy, and the LLM+GCN/LLM+GAT baselines use the same adjacency structure. A random-edge or edge-removal control is needed to attribute the observed improvement to the inter-slice edges rather than to added parameters, the soft-prompt pathway, or the token-slice edges.
  3. [Tables 2-6] All reported numbers appear to come from single runs with greedy decoding and no error bars or significance tests. The mean BLEU gain over LLM+GAT is 0.39 and over the vanilla LLM is 0.84; without uncertainty quantification it is difficult to judge whether the consistent pattern across metrics reflects a real advantage. I ask for repeated runs with different seeds or a bootstrap over the 13,791 test questions to support the 'consistently outperforms' claim.
minor comments (6)
  1. [Table 1] The reported average slices per CT volume do not match the counts: train 8,042,725/92,170 is approximately 87.3, not 69.3, and dev 224,891/2,000 is approximately 112.4, not 122.4. Please correct the table or clarify the definition of the reported average.
  2. [4.1] The sentence that GAT 'lacks the question related information in guidance' is hard to reconcile with Section 3.2, where the LLM+GAT baseline also models relationships among CT slices and question tokens; please clarify what specific information is missing in the GAT baseline.
  3. [Eq. (8)] Please use distinct symbols for the projection matrix and bias in Eq. (8) versus the layer-wise W^(l) and b^(l) in Eq. (6), because the current notation overloads W and b.
  4. [2.1] The text says column vectors represent patches in Eq. (2), while Eq. (5) treats node representations as vectors and Eq. (8) stacks representations as columns; please confirm and state the orientation convention consistently.
  5. [Figure 3] The caption should state explicitly that the red bars are the last-layer attention weights and should define the color scale in the right panel; it would also help to clarify whether the displayed weights are w_jk averaged over token nodes or slice-slice attention.
  6. [4.1] The claim that the M3D model 'benefits from training on a larger dataset' is unclear because both models are evaluated on M3D-VQA; please specify what additional data or pre-training M3D uses.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the method is trained and evaluated on an external benchmark, and the self-citations in related work are not load-bearing for the reported results.

full rationale

I examined the derivation chain from the overall formulation in Eq. (1) through graph construction (Eq. 5), attentive GCN encoding (Eqs. 6-7), projection to the LLM embedding space (Eq. 8), and answer generation (Eq. 9). The graph nodes are defined purely from CT slice embeddings and question token embeddings, and the adjacency is defined by spatial slice adjacency and complete token-slice connectivity; neither the graph nor the attention weights depend on the gold answers or on the evaluation metrics. The A-GCN update is a learned aggregation with attention, and the final soft prompt is fed to the LLM, whose output is compared against gold answers only during training. Test performance is measured on the held-out M3D-VQA split, so there is no fitted parameter that is renamed as a prediction and no term that is defined in terms of the target answer. The self-citations, such as [32] and [41], support general statements that graph models are effective and describe the attentive GCN mechanism, but they are not the evidence for the paper's empirical claims; that evidence is the external benchmark comparison. The paper's real limitations are experimental rather than circular: there is no ablation that removes or randomizes the inter-slice edges, all results are single runs without significance tests, and the contrast with the vanilla LLM baseline may conflate added parameters with graph structure. These are correctness and rigor concerns, not circular reasoning.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on standard supervised learning assumptions and a series of hand-picked hyperparameters; no new physical or conceptual entities are introduced.

free parameters (4)
  • learning_rate = 5e-5
    Chosen by hand for training; affects convergence and final performance.
  • batch_size = 16
    Chosen by hand; affects gradient estimates and memory usage.
  • num_epochs = 3
    Chosen by hand; limits training duration and may affect final performance.
  • num_gcn_layers_L = not specified
    Number of layers in the attentive GCN is stated as L but never given a value, a critical architectural choice.
assumptions (3)
  • domain assumption Pretrained Qwen2-VL and LLaVA-med provide useful visual and textual representations for CT slices and clinical questions.
    Section 3.3 relies on pretrained encoders without demonstrating their validity for CT imagery.
  • domain assumption The M3D-VQA dataset's annotations and the chosen evaluation metrics (BLEU, ROUGE, METEOR, BERT-Score) measure answer quality appropriately.
    Section 3.1 and 3.3 accept the benchmark and metrics as ground truth.
  • domain assumption Graph convolutional networks and attention mechanisms can model inter-slice and cross-modal relationships effectively.
    Section 2.2 motivates graph modeling by citing prior GCN work; it is assumed this inductive bias helps.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Computed Tomography Visual Question Answering with Cross-modal Feature Graphing." pith.science (2026). https://pith.science/paper/E27ZACUA

@misc{pith2026250704333,
  author       = {Pith},
  title        = {Pith review of: Computed Tomography Visual Question Answering with Cross-modal Feature Graphing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E27ZACUA}},
  note         = {Machine review of arXiv:2507.04333}
}
read the original abstract

Visual question answering (VQA) in medical imaging aims to support clinical diagnosis by automatically interpreting complex imaging data in response to natural language queries. Existing studies typically rely on distinct visual and textual encoders to independently extract features from medical images and clinical questions, which are subsequently combined to generate answers. Specifically, in computed tomography (CT), such approaches are similar to the conventional practices in medical image analysis. However, these approaches pay less attention to the spatial continuity and inter-slice correlations in the volumetric CT data, leading to fragmented and imprecise responses. In this paper, we propose a novel large language model (LLM)-based framework enhanced by a graph representation of salient features. Different from conventional multimodal encoding strategies, our approach constructs a cross-modal graph integrating both visual and textual features, treating individual CT slices and question tokens as nodes within the graph. We further leverage an attentive graph convolutional network to dynamically fuse information within this structure. The resulting aggregated graph features then serve as a soft prompt to guide a large language model in generating accurate answers. Extensive experiments on the M3D-VQA benchmark demonstrate that our approach consistently outperforms baselines across multiple evaluation metrics, offering more robust reasoning capabilities.

Figures

Figures reproduced from arXiv: 2507.04333 by the authors.

Figure 1
Figure 1. Comparison between CT slices and X-ray radio [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our approach for CT VQA. The bottom part illustrates the input CT slices with the question, the vision and text encoder, the cross-modal feature graphing, and the LLM to generate the answer. The top part presents the graph built over the CT slices and question tokens, as well as the attention mechanism used in the cross-modal feature graphing for distinguishing important slices for answering the quest… view at source ↗
Figure 3
Figure 3. A case study for CT VQA. The left part presents the attention weights (in red bars) for different CT slices, where [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ChiMed 2.0: Advancing Chinese Medical Dataset in Facilitating Large Language Modeling

    cs.CL 2025-07 conditional novelty 5.0 of 10

    ChiMed 2.0 is a 204.4M-character Chinese medical dataset spanning pretraining, SFT, and preference data that yields small gains on CMMLU and CEval medical subsets.

Reference graph

Works this paper leans on

53 extracted references · 35 canonical work pages · cited by 1 Pith paper

  1. [1]

    Visual question answering in the medical domain based on deep learning approaches: A comprehensive study,

    A. Al-Sadi, M. Al-Ayyoub, Y. Jararweh, and F. Costen, “Visual question answering in the medical domain based on deep learning approaches: A comprehensive study,” Pattern Recognition Letters , vol. 150, pp. 57–75, 2021

    work page 2021

  2. [2]

    Vqa: Visual question answering,

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, and D. Parikh, “Vqa: Visual question answering,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , 2015, pp. 2425–2433

    work page 2015

  3. [3]

    M3d: Advancing 3d medical image analysis with multi-modal large language models,

    F. Bai, Y. Du, T. Huang, M. Q.-H. Meng, and B. Zhao, “M3d: Advancing 3d medical image analysis with multi-modal large language models,” arXiv preprint arXiv:2404.00578, 2024

    arXiv 2024

  4. [4]

    Vqa-med: Overview of the medical visual question answering task at imageclef 2019,

    A. Ben Abacha, Y. Zhang, M. Mwangi, J. Shi, S. A. Hasan, Y. Liu, and D. Demner-Fushman, “Vqa-med: Overview of the medical visual question answering task at imageclef 2019,” in CEUR Work- shop Proceedings, vol. 2380, 2019, pp. 1–13

    work page 2019

  5. [5]

    Simvqa: Exploring simulated environments for visual question answering,

    P . Cascante-Bonilla, H. Wu, L. Wang, R. S. Feris, and V . Ordonez, “Simvqa: Exploring simulated environments for visual question answering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 5056–5066

    work page 2022

  6. [6]

    Miss: A generative pre-training and fine-tuning approach for med-vqa,

    J. Chen, D. Yang, Y. Jiang, Y. Lei, and L. Zhang, “Miss: A generative pre-training and fine-tuning approach for med-vqa,” in Interna- tional Conference on Artificial Neural Networks . Springer, 2024, pp. 299–313

    work page 2024

  7. [7]

    Vit-v-net: Vision trans- former for unsupervised volumetric medical image registration,

    J. Chen, Y. He, E. C. Frey, Y. Li, and Y. Du, “Vit-v-net: Vision trans- former for unsupervised volumetric medical image registration,” arXiv preprint arXiv:2104.06468, 2021

    arXiv 2021

  8. [8]

    Counterfactual samples synthesizing for robust visual question answering,

    L. Chen, X. Yan, J. Xiao, H. Zhang, S. Pu, and Y. Zhuang, “Counterfactual samples synthesizing for robust visual question answering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10 800–10 809

    work page 2020

Show all 53 references
  1. [9]

    Generative bias for robust visual question answering,

    J. W. Cho, D.-J. Kim, H. Ryu, and I. S. Kweon, “Generative bias for robust visual question answering,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 11 681–11 690

    2023

  2. [10]

    BERT: Pre- training of Deep Bidirectional Transformers for Language Un- derstanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre- training of Deep Bidirectional Transformers for Language Un- derstanding,” in NAACL, Minneapolis, Minnesota, Jun. 2019, pp. 4171–4186

    2019

  3. [11]

    Mukea: Mul- timodal knowledge extraction and accumulation for knowledge- based visual question answering,

    Y. Ding, J. Yu, B. Liu, Y. Hu, M. Cui, and Q. Wu, “Mukea: Mul- timodal knowledge extraction and accumulation for knowledge- based visual question answering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5089– 5098

    2022

  4. [12]

    Cross-modal self- attention with multi-task pre-training for medical visual question answering,

    H. Gong, G. Chen, S. Liu, Y. Yu, and G. Li, “Cross-modal self- attention with multi-task pre-training for medical visual question answering,” in Proceedings of the 2021 international conference on multimedia retrieval, 2021, pp. 456–460

    2021

  5. [13]

    Swapmix: Diagnosing and regularizing the over-reliance on vi- sual context in visual question answering,

    V . Gupta, Z. Li, A. Kortylewski, C. Zhang, Y. Li, and A. Yuille, “Swapmix: Diagnosing and regularizing the over-reliance on vi- sual context in visual question answering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5078–5088

    2022

  6. [14]

    LoRA: Low-rank Adaptation of Large Language Models,

    E. J. Hu, Y. Shen, P . Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank Adaptation of Large Language Models,” arXiv preprint arXiv:2106.09685, 2021. 9

    2021 arXiv

  7. [15]

    Expert knowledge-aware image dif- ference graph representation learning for difference-aware med- ical visual question answering,

    X. Hu, L. Gu, Q. An, M. Zhang, L. Liu, K. Kobayashi, T. Harada, R. M. Summers, and Y. Zhu, “Expert knowledge-aware image dif- ference graph representation learning for difference-aware med- ical visual question answering,” in Proceedings of the 29th ACM SIGKDD Conference on Kn...

    2023

  8. [16]

    Interpretable medical image visual question answering via multi-modal relationship graph learning,

    X. Hu, L. Gu, K. Kobayashi, L. Liu, M. Zhang, T. Harada, R. M. Summers, and Y. Zhu, “Interpretable medical image visual question answering via multi-modal relationship graph learning,” Medical Image Analysis, vol. 97, p. 103279, 2024

    2024

  9. [17]

    Medical knowledge-based network for patient-oriented visual question answering,

    J. Huang, Y. Chen, Y. Li, Z. Yang, X. Gong, F. L. Wang, X. Xu, and W. Liu, “Medical knowledge-based network for patient-oriented visual question answering,” Information Processing & Management , vol. 60, no. 2, p. 103241, 2023

    2023

  10. [18]

    Mistral 7b,

    A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M.-A. Lachaux, P . Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed, “Mistral 7b,” arXiv preprint arXiv:2310.06...

    arXiv

  11. [19]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” arXiv preprint arXiv:1609.02907 , 2016

    2016 arXiv

  12. [20]

    Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation,

    C. Lee, S. Park, C.-I. Shin, W. H. Choi, H. J. Park, J. E. Lee, and J. C. Ye, “Read like a radiologist: Efficient vision-language model for 3d medical imaging interpretation,” arXiv preprint arXiv:2412.13558 , 2024

    2024 arXiv

  13. [21]

    Llava-Med: Training a large language-and-vision assistant for biomedicine in one day,

    C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao, “Llava-Med: Training a large language-and-vision assistant for biomedicine in one day,” NeurIPS, vol. 36, 2024

    2024

  14. [22]

    Dynamic graph enhanced contrastive learning for chest x-ray report genera- tion,

    M. Li, B. Lin, Z. Chen, H. Lin, X. Liang, and X. Chang, “Dynamic graph enhanced contrastive learning for chest x-ray report genera- tion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3334–3343

    2023

  15. [23]

    Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering,

    P . Li, G. Liu, J. He, Z. Zhao, and S. Zhong, “Masked vision and language pre-training with unimodal and multimodal contrastive losses for medical visual question answering,” in International Conference on Medical Image Computing and Computer-Assisted In- tervention. Springer,...

    2023

  16. [24]

    Oscar: Object-semantics aligned pre-training for vision-language tasks,

    X. Li, X. Yin, C. Li, P . Zhang, X. Hu, L. Zhang, L. Wang, H. Hu, L. Dong, F. Weiet al., “Oscar: Object-semantics aligned pre-training for vision-language tasks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16....

    2020

  17. [25]

    Revive: Regional visual representation matters in knowledge-based visual question answering,

    Y. Lin, Y. Xie, D. Chen, Y. Xu, C. Zhu, and L. Yuan, “Revive: Regional visual representation matters in knowledge-based visual question answering,” Advances in Neural Information Processing Systems, vol. 35, pp. 10 560–10 571, 2022

    2022

  18. [26]

    Medical visual question answering: A survey,

    Z. Lin, D. Zhang, Q. Tao, D. Shi, G. Haffari, Q. Wu, M. He, and Z. Ge, “Medical visual question answering: A survey,” Artificial Intelligence in Medicine, vol. 143, p. 102611, 2023

    2023

  19. [27]

    Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering,

    B. Liu, L.-M. Zhan, L. Xu, L. Ma, Y. Yang, and X.-M. Wu, “Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering,” pp. 1650–1654, 2021

    2021

  20. [28]

    Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering,

    ——, “Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering,” in 2021 IEEE 18th Inter- national Symposium on Biomedical Imaging (ISBI) . IEEE, 2021, pp. 1650–1654

    2021

  21. [29]

    Efficient estimation of word representations in vector space,

    T. Mikolov, “Efficient estimation of word representations in vector space,” arXiv preprint arXiv:1301.3781, vol. 3781, 2013

    2013 arXiv

  22. [30]

    Beyond the hype: A dis- passionate look at vision-language models in medical scenario,

    Y. Nan, H. Zhou, X. Xing, and G. Yang, “Beyond the hype: A dis- passionate look at vision-language models in medical scenario,” arXiv preprint arXiv:2408.08704, 2024

    2024 arXiv

  23. [31]

    K-pathvqa: Knowledge-aware multimodal representation for pathology visual question answering,

    U. Naseem, M. Khushi, A. G. Dunn, and J. Kim, “K-pathvqa: Knowledge-aware multimodal representation for pathology visual question answering,” IEEE Journal of Biomedical and Health Infor- matics, 2023

    2023

  24. [32]

    Relation Extraction with Word Graphs from N-grams,

    H. Qin, Y. Tian, and Y. Song, “Relation Extraction with Word Graphs from N-grams,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, Nov. 2021, pp. 2860–2868

    2021

  25. [33]

    An improved attention for visual question answering,

    T. Rahman, S.-H. Chou, L. Sigal, and G. Carenini, “An improved attention for visual question answering,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 1653–1662

    2021

  26. [34]

    Learning Word Representations with Regularization from Prior Knowledge,

    Y. Song, C.-J. Lee, and F. Xia, “Learning Word Representations with Regularization from Prior Knowledge,” in Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), Aug. 2017, pp. 143–152

    2017

  27. [35]

    Complementary Learning of Word Em- beddings,

    Y. Song and S. Shi, “Complementary Learning of Word Em- beddings,” in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18, 7 2018, pp. 4368–4374

    2018

  28. [36]

    ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders,

    Y. Song, T. Zhang, Y. Wang, and K.-F. Lee, “ZEN 2.0: Continue Training and Adaption for N-gram Enhanced Text Encoders,” arXiv preprint arXiv:2105.01279, 2021

    2021 arXiv

  29. [37]

    VL- BERT: Pre-training of generic visual-linguistic representations,

    W. Su, X. Zhu, Y. Cao, B. Li, L. Lu, F. Wei, and J. Dai, “VL- BERT: Pre-training of generic visual-linguistic representations,” arXiv preprint arXiv:1908.08530, 2019

    1908 arXiv

  30. [38]

    ChiMed-GPT: A Chinese medical large language model with full training regime and better alignment to human preferences,

    Y. Tian, R. Gan, Y. Song, J. Zhang, and Y. Zhang, “ChiMed-GPT: A Chinese medical large language model with full training regime and better alignment to human preferences,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thai...

    2024

  31. [39]

    Chimed: A chinese medical corpus for question answering,

    Y. Tian, W. Ma, F. Xia, and Y. Song, “Chimed: A chinese medical corpus for question answering,” in Proceedings of the 18th BioNLP Workshop and Shared Task, 2019, pp. 250–260

    2019

  32. [40]

    Recurrent visual feature extraction and stereo attentions for ct report generation,

    Y. Tian, L. Mao, and Y. Song, “Recurrent visual feature extraction and stereo attentions for ct report generation,” arXiv preprint arXiv:2506.19665, 2025

    2025 arXiv

  33. [41]

    Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks,

    Y. Tian, Y. Song, and F. Xia, “Supertagging Combinatory Categorial Grammar with Attentive Graph Convolutional Networks,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), Nov. 2020, pp. 6037–6044

    2020

  34. [42]

    Dialogue Summarization with Mix- ture of Experts based on Large Language Models,

    Y. Tian, F. Xia, and Y. Song, “Dialogue Summarization with Mix- ture of Experts based on Large Language Models,” in Proceedings of the 62nd Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers) . Bangkok, Thailand: Association for Computatio...

    2024

  35. [43]

    Diffusion networks with task-specific noise control for radiology report generation,

    ——, “Diffusion networks with task-specific noise control for radiology report generation,” in Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 1771–1780

    2024

  36. [44]

    Learning multimodal contrast with cross-modal memory and reinforced contrast recognition,

    ——, “Learning multimodal contrast with cross-modal memory and reinforced contrast recognition,” in Findings of the Association for Computational Linguistics ACL 2024, 2024, pp. 6561–6573

    2024

  37. [45]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P . Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P . Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  38. [46]

    Open-ended medical visual question answering through prefix tuning of language models,

    T. Van Sonsbeek, M. M. Derakhshani, I. Najdenkoska, C. G. Snoek, and M. Worring, “Open-ended medical visual question answering through prefix tuning of language models,” in International Confer- ence on Medical Image Computing and Computer-Assisted Intervention. Springer, 2023...

    2023

  39. [47]

    Graph attention networks,

    P . Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P . Lio, and Y. Bengio, “Graph attention networks,” arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  40. [48]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,

    P . Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge et al. , “Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution,” arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  41. [49]

    MedCLIP: Contrastive Learning from Unpaired Medical Images and Text,

    Z. Wang, Z. Wu, D. Agarwal, and J. Sun, “MedCLIP: Contrastive Learning from Unpaired Medical Images and Text,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, Abu Dhabi, United Arab Emirates, Dec. 2022, pp. 3876– 3887

    2022

  42. [50]

    To- wards generalist foundation model for radiology,

    C. Wu, X. Zhang, Y. Zhang, Y. Wang, and W. Xie, “To- wards generalist foundation model for radiology,” arXiv preprint arXiv:2308.02463, 2023

    2023 arXiv

  43. [51]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “Bertscore: Evaluating text generation with bert,” arXiv preprint arXiv:1904.09675, 2019

    1904 arXiv

  44. [52]

    Pmc-vqa: Visual instruction tuning for medical visual question answering,

    X. Zhang, C. Wu, Z. Zhao, W. Lin, Y. Zhang, Y. Wang, and W. Xie, “Pmc-vqa: Visual instruction tuning for medical visual question answering,” arXiv preprint arXiv:2305.10415, 2023

    2023 arXiv

  45. [53]

    When ra- diology report generation meets knowledge graph,

    Y. Zhang, X. Wang, Z. Xu, Q. Yu, A. Yuille, and D. Xu, “When ra- diology report generation meets knowledge graph,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 07, 2020, pp. 12 910–12 917

    2020

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.