RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gaze

Honghan Wu; Jinge Wu; Yunsoo Kim

REVIEW 4 major objections 5 minor 30 references

RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gaze

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Showing a vision-language model a video of a radiologist's gaze improves chest X-ray report generation by up to 24.6% and lets a general model surpass task-specific medical models.

desk verdict A genuinely new prompting idea with a useful three-way ablation, but the headline numbers are overstated and the cross-model comparison is contaminated by train/test leakage from MIMIC-CXR. read the letter →

arxiv 2507.09097 v1 pith:PNO4XJDH submitted 2025-07-12 cs.CV

Yunsoo Kim , Jinge Wu , Honghan Wu This is my paper · ORCID

classification cs.CV

keywords ChestX-rayLargeVision-LanguageModelEyeGazeHuman-ComputerInteractionRadiologyreportgenerationVideopromptingEye-trackingMedicalimageanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a radiologist's eye-fixation sequence, encoded as a short video, is a more effective prompt for general-purpose large vision-language models than static gaze heatmaps or fixation-coordinate text. The authors report that the video prompt, which shows a red dot moving across the chest X-ray with timing proportional to fixation duration, improved diagnosis scores in all three video-capable LVLMs tested and raised average scaled performance by 15.2% across report generation and diagnosis, with up to 24.6% improvement in report generation. The central result is that LLaVA-OneVision, an open-domain model, prompted this way surpasses the task-specific medical LVLMs MAIRA-2 and CheXagent without medical fine-tuning. If that holds, expert eye-tracking data becomes a cheap human-centered way to inject domain knowledge into general models.

What carries the argument

The load-bearing object is the gaze-video prompt, constructed from fixation tuples $g_i=(x_i,y_i,t_i)$ with a red dot drawn at $(x_i,y_i)$ on the CXR and held for a number of frames proportional to $t_i$ at 10 frames per second. With a total frame count $F_{total}=\sum_i t_i\times 10$, a uniform sampler picks $k$ frames so the prompt fits the LVLM's fixed video window. This preserves both the spatial location and the sequence of fixations, which is the information the paper argues a heatmap or a duration-ordered coordinate list discards. It is the temporal ordering of expert attention, carried by the video representation, that the paper identifies as the active ingredient.

What would settle it

Compute the set overlap between the image IDs in the alpha and beta evaluation splits and the MIMIC-CXR training images used by CheXagent and MAIRA-2; if any exact image appears in both, rerun the comparison on images that are provably absent from the baselines' training data. The central claim would be falsified if LLaVA-OneVision with RadEyeVideo no longer outperforms those baselines on such a disjoint set.

Watch

Extended reading notes

Core claim

The core discovery is that the temporal structure of radiologist gaze, not just its spatial distribution, is usable by LVLMs when rendered as video frames. For each image, the paper builds a video from fixations $g_i=(x_i,y_i,t_i)$ by placing a red dot at each fixation coordinate and holding it for $t_i \times 10$ frames, then uniformly samples the frame sequence to the LVLM's fixed input length. In a benchmark on MIMIC-CXR-derived images with radiologist eye tracking, this RadEyeVideo prompt gave LLaVA-OneVision an overall scaled score of 154.6 against 100.0 for CheXagent and 152.2 for MAIRA-2, while also improving diagnosis in all three general-domain models. The authors interpret this as evidence that the order in which a radiologist scans an X-ray encodes diagnostic priority and that a sufficiently flexible general-domain LVLM can exploit that order from a video prompt alone.

Load-bearing premise

The headline comparison assumes the specialized medical models were not trained on the very chest X-ray images used for evaluation; if any of those images appear in their training data, the apparent superiority could be memorization rather than transfer.

Editorial extensions

If this is right

If the effect is real, general-domain LVLMs can be pushed toward medical competence purely by prompt design, avoiding the cost and data requirements of medical pretraining.
Static gaze visualizations leave useful signal unused: in this benchmark the video prompt produced diagnostic gains in all three models, whereas heatmap and fixation-text prompts did not consistently help.
The released code and dataset-derivation protocol let others rebuild the MIMIC-Eye-Video set and apply the same video prompt to other backbones and other imaging modalities.
The result also suggests that the choice of backbone matters: only LLaVA-OneVision improved on both tasks, so gaze-video prompting is not automatically helpful for every LVLM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the mechanism would be to shuffle or reverse the sampled gaze frames while keeping the same fixations; if diagnostic performance is unchanged, the sequential order is not the active ingredient and the paper's proposed explanation needs revision.
Since only LLaVA-OneVision improved on both tasks while VideoLLaMA3 lost report-generation quality, a practical deployment would need a way to predict which LVLM backbones benefit from gaze-video prompts rather than being destabilized by them.
The paper's Limitation section notes the eye-tracking dataset is small; if the benefit comes from generic scan-path structure instead of image-specific expert attention, a few thousand gaze recordings might transfer to much larger image corpora, but that transfer remains untested.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things before you read it. The video-gaze prompting idea is genuinely new: representing radiologist fixations as a temporally ordered video prompt, rather than a heatmap or a text list, is a clean and sensible extension of the prior work, and the paper's systematic comparison of three gaze representations on three video-capable LVLMs is a useful benchmark. The ablation table alone would have been a reasonable short paper.

The problems start with the claims. The abstract says "up to 24.6%" and "on average 15.2%". Table 3 shows the 24.6% is LLaVA-OneVision's report gain on the alpha split only, and the 15.2% is that same model's overall gain. VideoLLaMA3 drops 20.2 overall points, LongVA gains 2.1. "On average" across the three models is negative. The abstract also promises that RadEyeVideo lets a general-domain model surpass MAIRA-2 and CheXagent. That comparison is not trustworthy. Section 3.1 says the evaluation pools "both training and test images" from MIMIC-CXR into alpha/beta subsets. Section 3.2 says CheXagent, MAIRA-2, and CXR-LLaVA were trained on the full MIMIC-CXR training split. So some fraction of the alpha/beta images are training images for those baselines; the alpha/beta split does not separate MIMIC-CXR train from test. The baselines may simply be recognizing images. The within-model comparison (RadEyeVideo vs. NoEye on the same model) is unaffected, and that is the part that could still be true.

There are smaller issues too. The overall score averages a report score that ranges roughly 200-375 with a diagnosis score that ranges roughly 44-68, so report generation dominates the ordering. No significance tests, raw scores, or error bars are reported. The paper says code and protocols are shared but no link is given, and the video data itself is not released. The limitation paragraph acknowledges the small dataset but not the leakage issue.

Who should read this: anyone working on human-in-the-loop prompting for medical LLMs, or on evaluation design for foundation models in medical imaging. The ablation is worth seeing. But the paper needs a corrected split, a per-task presentation of gains, and a toned-down abstract before the results mean what the authors claim. If it comes to a journal or conference, send it to review; the idea deserves scrutiny, and a serious referee can push the authors to fix the evaluation.

Referee Report

4 major / 5 minor

Summary. The paper proposes RadEyeVideo, a prompting method that converts radiologists' eye-fixation sequences into video overlays on chest X-ray images, and evaluates it on report generation and disease diagnosis using three general-domain video-capable LVLMs (VideoLLaMA3, LongVA, LLaVA-OneVision). The authors report performance gains over no-gaze prompting, claim improvements up to 24.6% in report generation and 15.2% on average across both tasks, and further claim that LLaVA-OneVision with RadEyeVideo surpasses task-specific medical LVLMs such as CheXagent, MAIRA-2, and CXR-LLaVA. The evaluation uses the Eye Gaze and REFLACX datasets, which are derived from MIMIC-CXR, and introduces a scaled metric based on CheXagent.

Significance. If the claims were correct, RadEyeVideo would be a lightweight, training-free method for injecting expert perceptual knowledge into general-domain LVLMs, with potential value for human-AI collaboration in medical imaging. The paper also contributes a dataset derivation pipeline for gaze videos and a systematic comparison of heatmap, text, and video gaze prompting. However, the evidence is substantially weakened by internal inconsistencies between the abstract and the results table, by contamination of the evaluation set for the medical baseline models, and by the absence of a control for the video input format. These issues affect the central claims of the paper, not just the presentation.

major comments (4)

[Abstract and §4, Table 3] The abstract's claim of "on average 15.2% for both tasks" is contradicted by Table 3: VideoLLaMA3's overall scaled score drops by 20.2 points with RadEyeVideo, LongVA improves by only 2.1 points, and LLaVA-OneVision improves by 15.2 points. The 15.2% therefore reflects only one model, not an average. Similarly, "up to 24.6% in the report generation task" does not correspond to any relative improvement in Table 3; the largest report-generation gain (LLaVA-OneVision alpha, +32.4 scaled points) is a 13.7% relative increase over its 237.1 baseline. The central quantitative claims are not supported by the presented results.
[§3.1–3.2] The evaluation uses both training and test images from MIMIC-CXR, pooled into alpha and beta subsets (§3.1), while CheXagent, CXR-LLaVA, and MAIRA-2 are described as trained on the entire MIMIC-CXR training split (§3.2). Consequently, those medical baselines are evaluated on images they were trained on, making the claim that LLaVA-OneVision with RadEyeVideo "surpasses" them confounded by possible memorization. The alpha/beta split is a random split of pooled training and test images and does not separate MIMIC-CXR training from test data. This undermines the headline cross-model comparison, even though the within-model comparison for LLaVA-OneVision (gaze video vs. no gaze) is not affected by this particular issue.
[§2.2–2.3] The video prompt provides the LVLM with multiple frames of the same CXR image with a moving red dot, while the NoEye baseline presents a single static image. There is no control condition—such as a video of the image without any gaze overlay or with a dot following a random or fixed trajectory—so the measured improvement cannot be attributed specifically to the gaze information. The observed gains could plausibly arise from the video format itself, the larger number of input frames, or the model's ability to perform temporal attention over repeated image content. This is load-bearing for the causal interpretation of the method.
[§4, Table 3] The claim that LLaVA-OneVision with RadEyeVideo surpasses MAIRA-2 rests on a small margin in the overall scaled score (154.6 vs. 152.2) that is driven primarily by the alpha split; on beta, MAIRA-2's report-generation score (251.1) exceeds LLaVA-OneVision's (232.9). No significance testing, confidence intervals, or error bars are reported, and with only three general-domain models and two evaluation subsets, the robustness of the reported improvements is unclear. The scaled metric in Eq. (6), which averages ratios relative to CheXagent, is also sensitive to the choice of reference model and may exaggerate improvements on metrics where CheXagent is weak.

minor comments (5)

[§2.5] There is a typo in the phrase "Impression geneartion task"; it should read "generation."
[§2.2, Eq. (2)] The sampling formula v_j = floor(j · F_total / k) for j = 1, ..., k produces indices from floor(F_total/k) to F_total, excluding the first frame and possibly biasing the sample toward the end of the video. Using floor((j-1) · F_total / k) + 1 would provide a more balanced uniform sample.
[§3.2, Table 2] The table does not explain the difference between "Trained" and "Supported" modalities or the meaning of the checkmark symbols; please clarify in the table caption or footnotes.
[§3.3, Eq. (6)] The text states that CheXagent is "known to perform the best among the LVLMs for these two tasks," yet Table 3 shows CXR-LLaVA with much higher report-generation scaled scores (374.7 and 311.3) than CheXagent's 100.0. This statement and the choice of CheXagent as the normalization reference need clarification, since the scaled scores make CheXagent appear weaker than other models on some metrics.
[§5, Limitations] The limitation section mentions small dataset size and limited diversity but does not address the training/test contamination issue for the medical baselines or the absence of a video control condition, both of which are central to the validity of the main claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the gaze-video prompting pipeline is a deterministic input transformation evaluated on external metrics; the main caveats are data-contamination and metric-selection concerns, not circular derivation.

full rationale

The claimed derivation chain is self-contained. RadEyeVideo is a fixed rendering of radiologist fixations (Section 2.2, Eqs. 1-2): frames are generated deterministically with fps=10, radius=5, and uniform k-frame sampling, and no parameter is fitted to the report or diagnosis targets. The within-model comparisons (same LVLM with and without gaze video) are evaluated with external metrics (CheXbert, RadGraph-XL, RaTEScore) against ground-truth reports and diagnoses, so the improvement is an intervention effect of adding the gaze video, not an equivalence between input and output. The central cross-model claim is weakened by evaluation design rather than circularity: Section 3.1 says 'we leverage both training and test images into two evaluation subsets, labeled alpha and beta', while Section 3.2 states CheXagent, CXR-LLaVA, and MAIRA-2 'have been trained with the entire MIMIC-CXR training split', so the baselines may be evaluated on images they were trained on; this is a contamination/soundness problem, not a derivation that reduces to its inputs. The 'up to 24.6%' figure is a selected maximum across models and splits, and the scaled metric in Eq. 6 uses CheXagent as denominator, but scaling is linear and does not force the reported ranking by construction. Self-citations [23,24,25] appear only as background support for LVLM capabilities and hallucination concerns and are not load-bearing. Therefore no significant circularity is identified.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new physical or conceptual entities are postulated. The method introduces a derived video representation, but this is a data transformation, not an invented entity. The main free parameters are hand-set video construction choices, and the core domain assumption is that gaze order carries diagnostic value.

free parameters (4)

frame rate fps = 10 frames per second
Equation (1) converts fixation duration to frame count using fps=10; no sensitivity analysis is provided.
gaze radius = 5 pixels
Stated in Section 2.2; the red dot radius is fixed without justification or ablation.
number of sampled frames k = not specified in text
Equation (2) samples k frames, but the paper never states the value used for k in the experiments; this affects what the model sees.
number of exemplars E = 3
Three exemplar reports are used as in-context learning (Section 2.5); no ablation on the number or choice of exemplars.

assumptions (3)

domain assumption Eye-gaze fixations, when rendered as a video, carry useful diagnostic signal not already present in the CXR image.
This is the foundational premise of the method, stated in Section 2.1; it is the hypothesis being tested rather than an independently established fact.
domain assumption The evaluation images from MIMIC-CXR can be split into alpha and beta sets to monitor contamination from medical baselines trained on MIMIC-CXR.
Section 3.1 claims this, but the split does not prove the baselines never saw these images during training.
ad hoc to paper Uniform frame sampling preserves the temporal order and duration information of fixations.
Equation (2) defines the sampling; no validation shows that the sampled frames retain the relevant gaze dynamics.

how reviews work

0 comments

Cite this review

Pith. "Pith review of RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gaze." pith.science (2026). https://pith.science/paper/PNO4XJDH

@misc{pith2026250709097,
  author       = {Pith},
  title        = {Pith review of: RadEyeVideo: Enhancing general-domain Large Vision Language Model for chest X-ray analysis with video representations of eye gaze},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PNO4XJDH}},
  note         = {Machine review of arXiv:2507.09097}
}

read the original abstract

Large Vision-Language Models (LVLMs) have demonstrated promising performance in chest X-ray (CXR) analysis. To enhance human-computer interaction, several studies have incorporated radiologists' eye gaze, typically through heatmaps or textual prompts. However, these methods often overlook the sequential order of eye movements, which could provide valuable insights by highlighting both the areas of interest and the order in which they are examined. In this work, we propose a novel approach called RadEyeVideo that integrates radiologists' eye-fixation data as a video sequence, capturing both the temporal and spatial dynamics of their gaze. We evaluate this method in CXR report generation and disease diagnosis using three general-domain, open-source LVLMs with video input capabilities. When prompted with eye-gaze videos, model performance improves by up to 24.6% in the report generation task and on average 15.2% for both tasks using scaled evaluation metrics. Notably, RadEyeVideo enhanced an open-domain LVLM model, LLaVA-OneVision, to surpass task-specific medical LVLMs such as MAIRA-2 and CheXagent, trained on large Chest X-ray data. This work highlights that domain expert's knowledge (eye-gaze information in this case), when effectively integrated with LVLMs, can significantly enhance general-domain models' capabilities in clinical tasks. RadEyeVideo is a step toward a scalable human-centered approach of utilizing LVLMs in medical image analytics.

Figures

Figures reproduced from arXiv: 2507.09097 by the authors.

**Figure 2.** Eye gaze textual and visual prompts. Texts are highlighted in different [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Sample response from LLaVA-OneVision. Green - correct statements about the CXR; Red - incorrect statements. To comply with the MIMIC-CXR data usage license, the CXR image is substituted with a Wikimedia image depicting the same disease, and the text report is paraphrased. Limitations of Heatmap and Fixation Text Prompts While RadEyeVideo shows strong results, both Heat Map prompting and Fixation Text prompting [PI… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 15 canonical work pages

[1]

Journal of Medical Internet Research26, e59505 (2024) 12 Kim et al

AlSaad, R., Abd-Alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.A., Damseh, R., Sheikh, J.: Multimodal large language models in health care: Applications, challenges, and future outlook. Journal of Medical Internet Research26, e59505 (2024) 12 Kim et al

work page 2024
[2]

arXiv preprint arXiv:2406.04449 (2024)

Bannur, S., Bouzid, K., Castro, D.C., Schwaighofer, A., Bond-Taylor, S., Ilse, M., Pérez-García, F., Salvatelli, V., Sharma, H., Meissen, F., et al.: Maira-2: Grounded radiology report generation. arXiv preprint arXiv:2406.04449 (2024)

arXiv 2024
[3]

Artificial Intelligence in Medicine 127, 102285 (2022)

Calisto, F.M., Santiago, C., Nunes, N., Nascimento, J.C.: Breastscreening-ai: Eval- uating medical intelligent agents for human-ai interactions. Artificial Intelligence in Medicine 127, 102285 (2022)

work page 2022
[4]

arXiv preprint arXiv:2406.10185 (2024)

Chen, J., Yang, D., Wu, T., Jiang, Y., Hou, X., Li, M., Wang, S., Xiao, D., Li, K., Zhang, L.: Detecting and evaluating medical hallucinations in large vision language models. arXiv preprint arXiv:2406.10185 (2024)

arXiv 2024
[5]

arXiv preprint arXiv:2401.12208 (2024)

Chen, Z., Varma, M., Delbrouck, J.B., Paschali, M., Blankemeier, L., Van Veen, D., Valanarasu, J.M.J., Youssef, A., Cohen, J.P., Reis, E.P., et al.: Chexagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:2401.12208 (2024)

arXiv 2024
[6]

In: Findings of the Association for Computational Linguistics ACL 2024

Delbrouck,J.B.,Chambon,P.,Chen,Z.,Varma,M.,Johnston,A.,Blankemeier,L., Van Veen, D., Bui, T., Truong, S., Langlotz, C.: Radgraph-xl: A large-scale expert- annotated dataset for entity and relation extraction from radiology reports. In: Findings of the Association for Computational Linguistics ACL 2024. pp. 12902– 12915 (2024)

work page 2024
[7]

arXiv preprint arXiv:2311.13668 (2023)

Hyland, S.L., Bannur, S., Bouzid, K., Castro, D.C., Ranjit, M., Schwaighofer, A., Pérez-García, F., Salvatelli, V., Srivastav, S., Thieme, A., et al.: Maira-1: A spe- cialised large multimodal model for radiology report generation. arXiv preprint arXiv:2311.13668 (2023)

arXiv 2023
[8]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Ji, C., Du, C., Zhang, Q., Wang, S., Ma, C., Xie, J., Zhou, Y., He, H., Shen, D.: Mammo-net: Integrating gaze supervision and interactive information in multi- view mammogram classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 68–78. Springer (2023)

work page 2023

Show all 30 references

[9]

Scientific data 6(1), 317 (2019)

Johnson, A.E., Pollard, T.J., Berkowitz, S.J., Greenbaum, N.R., Lungren, M.P., Deng, C.y., Mark, R.G., Horng, S.: Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 6(1), 317 (2019)

2019
[10]

PhysioNet https://doi

Karargyris, A., Kashyap, S., Lourentzou, I., Wu, J., Tong, M., Sharma, A., Abedin, S., Beymer, D., Mukherjee, V., Krupinski, E., et al.: Eye gaze data for chest x-rays. PhysioNet https://doi. org/10.13026/QFDZ-ZR67 (2020)

2020 doi
[11]

Lanfredi, R.B., Zhang, M., Auffermann, W., Chan, J., Duong, P.A., Srikumar, V., Drew, T., Schroeder, J., Tasdizen, T.: Reflacx: Reports and eye-tracking data for localization of abnormalities in chest x-rays (2021)

2021
[12]

arXiv preprint arXiv:2310.18341 (2023)

Lee, S., Youn, J., Kim, M., Yoon, S.H.: Cxr-llava: Multimodal large language model for interpreting chest x-ray images. arXiv preprint arXiv:2310.18341 (2023)

2023 arXiv
[13]

arXiv preprint arXiv:2408.03326 (2024)

Li, B., Zhang, Y., Guo, D., Zhang, R., Li, F., Zhang, H., Zhang, K., Zhang, P., Li, Y., Liu, Z., et al.: Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326 (2024)

2024 arXiv
[14]

Advances in Neural Information Processing Systems36 (2024)

Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., Gao, J.: Llava-med: Training a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems36 (2024)

2024
[15]

IEEE Transactions on Medical Imaging (2023)

Ma, C., Zhao, L., Chen, Y., Wang, S., Guo, L., Zhang, T., Shen, D., Jiang, X., Liu, T.: Eye-gaze-guided vision transformer for rectifying shortcut learning. IEEE Transactions on Medical Imaging (2023)

2023
[16]

npj Digital Medicine 2(1), 111 (2019)

Patel, B.N., Rosenberg, L., Willcox, G., Baltaxe, D., Lyons, M., Irvin, J., Ra- jpurkar, P., Amrhein, T., Gupta, R., Halabi, S., Langlotz, C., Lo, E., Mammarap- pallil, J., Mariano, A.J., Riley, G., Seekins, J., Shen, L., Zucker, E., Lungren, RadEyeVideo 13 M.P.: Human–machine...

2019 doi
[17]

arXiv preprint arXiv:2404.18416 (2024)

Saab, K., Tu, T., Weng, W.H., Tanno, R., Stutz, D., Wulczyn, E., Zhang, F., Strother, T., Park, C., Vedadi, E., et al.: Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416 (2024)

2024 arXiv
[18]

arXiv preprint arXiv:2004.09167 (2020)

Smit, A., Jain, S., Rajpurkar, P., Pareek, A., Ng, A.Y., Lungren, M.P.: Chexbert: combining automatic labelers and expert annotations for accurate radiology report labeling using bert. arXiv preprint arXiv:2004.09167 (2020)

2020 arXiv
[19]

Springer: Enhancing human-computer interaction in chest x-ray analysis using vi- sion and language model with eye gaze patterns (2024)

2024
[20]

Springer: Human-in-the-Loop Chest X-Ray Diagnosis: Enhancing Large Multi- modal Models with Eye Fixation Inputs (2024)

2024
[21]

In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

Wang, B., Pan, H., Aboah, A., Zhang, Z., Keles, E., Torigian, D., Turkbey, B., Krupinski, E., Udupa, J., Bagci, U.: Gazegnn: A gaze-guided graph neural network for chest x-ray classification. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision....

2024
[22]

IEEE Transactions on Medical Imaging41(7), 1688–1698 (2022)

Wang, S., Ouyang, X., Liu, T., Wang, Q., Shen, D.: Follow my eye: Using gaze to supervise computer-aided diagnosis. IEEE Transactions on Medical Imaging41(7), 1688–1698 (2022)

2022
[23]

arXiv preprint arXiv:2312.13103 (2023)

Wu, J., Kim, Y., Keller, E.C., Chow, J., Levine, A.P., Pontikos, N., Ibrahim, Z., Taylor, P., Williams, M.C., Wu, H.: Exploring multimodal large language models for radiology report error-checking. arXiv preprint arXiv:2312.13103 (2023)

2023 arXiv
[24]

arXiv preprint arXiv:2409.13321 (2024)

Wu,J.,Kim,Y.,Shi,D.,Cliffton,D.,Liu,F.,Wu,H.:Slava-cxr:Smalllanguageand vision assistant for chest x-ray report automation. arXiv preprint arXiv:2409.13321 (2024)

2024 arXiv
[25]

arXiv preprint arXiv:2401.05827 (2024)

Wu, J., Kim, Y., Wu, H.: Hallucination benchmark in medical visual question answering. arXiv preprint arXiv:2401.05827 (2024)

2024 arXiv
[26]

arXiv preprint arXiv:2405.08603 (2024)

Xiao, H., Zhou, F., Liu, X., Liu, T., Li, Z., Liu, X., Huang, X.: A comprehensive surveyoflargelanguagemodelsandmultimodallargelanguagemodelsinmedicine. arXiv preprint arXiv:2405.08603 (2024)

2024 arXiv
[27]

arXiv preprint arXiv:2501.13106 (2025)

Zhang, B., Li, K., Cheng, Z., Hu, Z., Yuan, Y., Chen, G., Leng, S., Jiang, Y., Zhang, H., Li, X., et al.: Videollama 3: Frontier multimodal foundation models for image and video understanding. arXiv preprint arXiv:2501.13106 (2025)

2025 arXiv
[28]

arXiv preprint arXiv:2406.16852 (2024)

Zhang, P., Zhang, K., Li, B., Zeng, G., Yang, J., Zhang, Y., Wang, Z., Tan, H., Li, C., Liu, Z.: Long context transfer from language to vision. arXiv preprint arXiv:2406.16852 (2024)

2024 arXiv
[29]

medRxiv pp

Zhao, W., Wu, C., Zhang, X., Zhang, Y., Wang, Y., Xie, W.: Ratescore: A metric for radiology report generation. medRxiv pp. 2024–06 (2024)

2024
[30]

In: Proceedings of the AAAI Conference on Artificial Intelligence

Zhao,Z.,Wang,S.,Wang,Q.,Shen,D.:Mininggazeforcontrastivelearningtoward computer-assisted diagnosis. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 38, pp. 7543–7551 (2024)

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Journal of Medical Internet Research26, e59505 (2024) 12 Kim et al

AlSaad, R., Abd-Alrazaq, A., Boughorbel, S., Ahmed, A., Renault, M.A., Damseh, R., Sheikh, J.: Multimodal large language models in health care: Applications, challenges, and future outlook. Journal of Medical Internet Research26, e59505 (2024) 12 Kim et al

work page 2024

[2] [2]

arXiv preprint arXiv:2406.04449 (2024)

Bannur, S., Bouzid, K., Castro, D.C., Schwaighofer, A., Bond-Taylor, S., Ilse, M., Pérez-García, F., Salvatelli, V., Sharma, H., Meissen, F., et al.: Maira-2: Grounded radiology report generation. arXiv preprint arXiv:2406.04449 (2024)

arXiv 2024

[3] [3]

Artificial Intelligence in Medicine 127, 102285 (2022)

Calisto, F.M., Santiago, C., Nunes, N., Nascimento, J.C.: Breastscreening-ai: Eval- uating medical intelligent agents for human-ai interactions. Artificial Intelligence in Medicine 127, 102285 (2022)

work page 2022

[4] [4]

arXiv preprint arXiv:2406.10185 (2024)

Chen, J., Yang, D., Wu, T., Jiang, Y., Hou, X., Li, M., Wang, S., Xiao, D., Li, K., Zhang, L.: Detecting and evaluating medical hallucinations in large vision language models. arXiv preprint arXiv:2406.10185 (2024)

arXiv 2024

[5] [5]

arXiv preprint arXiv:2401.12208 (2024)

Chen, Z., Varma, M., Delbrouck, J.B., Paschali, M., Blankemeier, L., Van Veen, D., Valanarasu, J.M.J., Youssef, A., Cohen, J.P., Reis, E.P., et al.: Chexagent: Towards a foundation model for chest x-ray interpretation. arXiv preprint arXiv:2401.12208 (2024)

arXiv 2024

[6] [6]

In: Findings of the Association for Computational Linguistics ACL 2024

Delbrouck,J.B.,Chambon,P.,Chen,Z.,Varma,M.,Johnston,A.,Blankemeier,L., Van Veen, D., Bui, T., Truong, S., Langlotz, C.: Radgraph-xl: A large-scale expert- annotated dataset for entity and relation extraction from radiology reports. In: Findings of the Association for Computational Linguistics ACL 2024. pp. 12902– 12915 (2024)

work page 2024

[7] [7]

arXiv preprint arXiv:2311.13668 (2023)

Hyland, S.L., Bannur, S., Bouzid, K., Castro, D.C., Ranjit, M., Schwaighofer, A., Pérez-García, F., Salvatelli, V., Srivastav, S., Thieme, A., et al.: Maira-1: A spe- cialised large multimodal model for radiology report generation. arXiv preprint arXiv:2311.13668 (2023)

arXiv 2023

[8] [8]

In: International Conference on Medical Image Computing and Computer-Assisted Intervention

Ji, C., Du, C., Zhang, Q., Wang, S., Ma, C., Xie, J., Zhou, Y., He, H., Shen, D.: Mammo-net: Integrating gaze supervision and interactive information in multi- view mammogram classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 68–78. Springer (2023)

work page 2023