Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

A new benchmark, SpatialMed, shows that current medical multimodal large language models cannot reliably perform 3D spatial reasoning on CT scans.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 18:10 UTC pith:TCQKABR3

load-bearing objection Useful first benchmark for 3D spatial reasoning in medical MLLMs, with a believable core result, but a wrong metric equation, inconsistent counts, and un-audited mask ground truth. the 4 major comments →

arxiv 2603.13800 v2 pith:TCQKABR3 submitted 2026-03-14 cs.CV

Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space

Quoc-Huy Trinh , Xi Ding , Yang Liu , Zhenyue Qin , Xingjian Li , Gorkem Durak , Halil Ertugrul Aktas , Andrea M. Bejar
show 2 more authors
Ulas Bagci Min Xu
This is my paper
classification cs.CV
keywords spatial reasoningmultimodal large language models3D CT imagingmedical visual question answeringbenchmarkvolume estimationdistance reasoninghallucination
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper seeks to establish that today's multimodal large language models, despite their strong medical recognition abilities, cannot do the quantitative 3D spatial reasoning radiologists rely on: estimating organ and tumor volume, comparing sizes, judging distances between structures, and locating anatomy in computed tomography. To test this, the authors built SpatialMed, a benchmark of 9,782 question-answer pairs derived from 2,375 CT scans and radiologist-validated, and evaluated 14 models on it. Across six spatial task families plus direct volume estimation, the strongest model reaches only 50.81% multiple-choice accuracy against a 25% random baseline, distance reasoning is a consistent weak point, volume outputs are frequently unstable or missing, and most reasoning chains are judged hallucinated. If the finding holds, it matters because these spatial judgments underpin staging, surgical planning, and measurement-based diagnosis.

Core claim

The central claim is that current MLLMs lack robust spatial reasoning for medical imaging, and SpatialMed is the instrument that exposes the deficit. SpatialMed contains 9,782 QA pairs from 2,375 CT scans, spanning 117 anatomical structures and multiple tumor types, organized into directional, distance, extent/size/shape, volume-magnitude, and comparative multiple-choice tasks plus direct volume estimation. On this benchmark the best of 14 evaluated models scores 50.81% overall multiple-choice accuracy (random is 25%), distance reasoning is the hardest task across model families, medical pretraining does not uniformly help, and a human failure analysis finds that numeric and relational error

What carries the argument

The load-bearing object is the SpatialMed benchmark and the agentic pipeline that constructs it. Three computational tools—a volume calculator that counts masked voxels scaled by voxel spacing, a 3D bounding-box extractor, and a distance calculator based on Euclidean distance between bounding-box centers—turn CT segmentation masks into numerical spatial metadata. A multi-agent generation system combines that metadata with retrieved medical knowledge to produce diverse spatial QA pairs; three specialist agents then discard questions answerable from text alone, so remaining items genuinely require image-grounding; and three board-certified radiologists review the survivors for clinical logic,

Load-bearing premise

The benchmark's ground-truth volumes, bounding boxes, and distances are computed directly from existing segmentation masks, so if any source mask is inaccurate, the reference answer and every model score inherit that error.

What would settle it

Independently re-segment a random sample of the 2,375 CT scans—or have radiologists manually re-measure organ volumes, tumor volumes, and inter-structure distances—and compare against SpatialMed's derived answers; substantial disagreement on a meaningful fraction of cases would show the reported model gaps are not a pure measure of MLLM spatial reasoning.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the finding is correct, even the strongest current MLLMs are not safe for measurement-style radiology tasks such as tumor volume tracking or surgical margin assessment.
  • Distance reasoning is the universal bottleneck, so progress on other spatial skills does not automatically transfer to geometric relation judgments.
  • Medical pretraining alone does not close the gap, meaning spatially-aware training objectives rather than more recognition data are needed.
  • Volume estimation is numerically unstable across many models, with frequent non-numeric outputs, so forcing models to output numbers in a fixed format and calibrating them are concrete next targets.
  • High rates of hallucinated reasoning chains mean accuracy alone overstates model capability; evaluation and training that checks whether the explanation supports the answer is necessary for trustworthy deployment.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same grounded generation pipeline could be turned from benchmark into training data: fine-tuning models on SpatialMed-style QA pairs may improve distance and volume reasoning, but gains would need verification on independent clinical measurements.
  • The benchmark's distance ground truth is center-of-bounding-box Euclidean distance, while clinical decisions often hinge on surface-to-surface or margin distances; a variant using minimal surface distance would show whether model deficits persist under the clinically relevant definition.
  • Because all reference answers inherit from source segmentation masks, per-case mask accuracy could be reported alongside model scores to separate errors in perception from errors in spatial reasoning.
  • The observed volume-bucket sensitivity suggests a testable extension: calibration methods or constrained decoding should improve small-lesion volume estimation more than simply scaling model size.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SpatialMed, a benchmark for 3D spatial reasoning in medical CT, generated via an agentic pipeline with volume/distance calculators, RAG-based question generation, and radiologist validation. It reports evaluations of 14 MLLMs (multi-view 2D and 3D) on six spatial tasks and volume estimation, and concludes that current medical MLLMs lack robust 3D spatial reasoning. The paper argues the field has focused on diagnosis/segmentation and that SpatialMed is the first benchmark for grounded, quantitative spatial reasoning in 3D medical imaging.

Significance. If the benchmark construction is valid, this is a timely resource: it addresses a real gap, involves a nontrivial human-in-the-loop generation pipeline, and the qualitative pattern of near-random performance on several tasks is credible from the exact-match MCA results. The failure taxonomy and faithfulness analysis are useful contributions. However, the printed MRA formula is inverted, the abstract/body counts conflict, and the mask-derived ground truth is not audited. These issues must be resolved before the quantitative conclusions can be accepted.

major comments (4)
  1. [§4.1, Eq. (5)] The MRA definition uses the indicator 1(|ŷ−y|/y < 1−θ) with θ in {0.01, ..., 0.1}. This is the inverse of the stated tolerance: for θ=0.01 it counts predictions with relative error below 0.99, and for θ=0.1 it counts predictions with relative error below 0.90. The intended condition should be |ŷ−y|/y < θ. As printed, the reported volume MRA values (e.g., Qwen3-VL 4B 34.14, InternVL3 9B 35.92) are inflated and do not measure the claimed strict relative accuracy. The formula, the volume column of Table 2, and Figure 4(c) need to be recomputed with the correct inequality.
  2. [Abstract vs. §1/§3.2/Table 2] The reviewing copy of the abstract states 31,253 QA pairs and 24 evaluated MLLMs, while the full text consistently reports 9,782 QA pairs and 14 evaluated MLLMs. The methods describe 30,799 generated samples reduced to 10,487 and then to 9,782 after radiologist filtering, so the body's counts are internally coherent; the abstract must be corrected. Separately, §4.1 says RadFM was evaluated among 3D models, but Table 2 contains no RadFM row. Either report its results or remove the mention.
  3. [§3.1/§3.2, Eqs. (1)–(4)] All ground-truth labels are computed directly from the source segmentation masks, and the 2D visual inputs are cropped from those same masks. No audit of mask accuracy is reported. The radiologist review checks clinical logicality, plausible numerical ranges, and option exclusivity, but does not re-segment or verify the underlying mask boundaries. If the source masks contain systematic under- or over-segmentation, a nontrivial fraction of the correct options could be wrong, and low MLLM accuracy would reflect label noise or mis-specified visual cropping rather than deficient spatial reasoning. The authors should report a mask-quality audit (e.g., independent re-segmentation on a random sample) or, at minimum, add a clear limitation statement and a sensitivity analysis.
  4. [§3.2, Figure 2] The triviality filter removes any question answered correctly by at least two of InternLM2, Qwen-3, and Llama-3 when the three models are given retrieved context but no image. This assumes that joint failure of these three models is a reliable proxy for the question requiring visual-spatial evidence. No validation of this criterion is provided. The filter could remove questions that are hard for the three LLMs due to format, ambiguity, or knowledge gaps unrelated to spatial reasoning, and could retain questions that are answerable from textual priors. A precision/recall analysis against human non-triviality judgments would strengthen the claim that retained items are genuinely image-grounded.
minor comments (6)
  1. [§3.3] The text says the volume task is mainly from TotalSegmentator (18,675) and BraTS (3,714), but the final dataset contains only 9,782 QA pairs total. This is inconsistent. Clarify whether these are pre-filter counts or whether a different unit is being reported.
  2. [References] Yang et al. 2025a, 2025b, and 2025c appear to be the same paper cited three times. If different papers are intended, the entries need identifying details; if not, citations should be consolidated.
  3. [§4.2/Table 2] The table reports no confidence intervals or significance tests. Many cell counts are small enough that differences such as Qwen3-VL 4B vs 8B may be within noise; bootstrap intervals or per-task sample sizes would make the comparisons more interpretable.
  4. [§3.2] Typographical and readability issues remain: 'out agentic pipeline' should be 'our agentic pipeline'; the sentence beginning 'The spatial reasoning, and the language to interpret...' is ungrammatical; the figure text 'Both CT and QA pairs Score: 0/1/2' is unclear.
  5. [§4.1/Table 2] RadFM is mentioned as an evaluated 3D model but is absent from the results table. In addition, the 'NaN' entries in Table 2 are not explained in the caption; state whether each model refused to produce a number, generated a non-numeric string, or produced a parse failure.
  6. [§4.2] The claim that medical pretraining does not uniformly improve spatial reasoning is based on a small and heterogeneous set of models with different architectures, backbones, and training recipes. The text should explicitly note this confounding so readers do not overgeneralize.

Circularity Check

0 steps flagged

No significant circularity: the benchmark's accuracy conclusion is measured from independent zero-shot model responses; the only same-author citation is a non-load-bearing related-work mention.

full rationale

The derivation chain is a dataset-construction and evaluation chain, not a fitted prediction. Ground truth answers are computed from source segmentation masks (Eq. 1 volume; Eq. 2 bounding boxes; Eqs. 3–4 center-to-center distance) and the QA generator receives that same voxel-derived metadata as textual context; the evaluated MLLMs see CT volumes or mask-cropped multi-view slices and must produce answers, with exact-match/MRA scoring. Nothing is fit to model outputs and then renamed a prediction, so the low scores are not forced by construction. The triviality filter removes text-answerable questions, which makes the benchmark difficult by design but does not inject the conclusion. The one overlapping-author citation (PRS-Med, Trinh et al., 2025, Section 2) only motivates extending positional reasoning to volume/distance; it is not used to justify the benchmark's ground truth or the negative capability claim. The un-audited reliance on source segmentation masks (Section 3.2; Appendix C acknowledges masks but reports no mask-accuracy audit) is a data-validity/correctness risk: if masks are wrong, the radiologist-approved answers may be wrong and the low scores would not support the clinical conclusion. That is a validity threat, not a circularity, because the label is still defined by the mask rather than by the model being evaluated. No self-definitional reduction, fitted-input-as-prediction, or author-imported uniqueness theorem is present.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The benchmark has no fitted physical parameters. The hand-chosen filter thresholds affect dataset difficulty and MRA scores, but the central conclusion does not depend on a fitted model. The main external inputs are masks and voxel spacing from source datasets; their correctness is assumed without re-audit. No new scientific entities are introduced—SpatialMed is a dataset, not a causal or theoretical object.

free parameters (3)
  • Triviality cutoff = 2 of 3 specialist LLMs correct
    Section 3.2: questions answerable by at least two of InternLM2/Qwen-3/Llama-3 with RAG-only context are discarded. Hand-chosen threshold; no sensitivity analysis; directly shapes dataset difficulty.
  • Radiologist inclusion threshold = average score > 1.0
    Section 3.2: final QA pairs require average radiologist score greater than 1 on a 0/1/2 scale. Hand-chosen; no inter-rater reliability reported.
  • MRA tolerance schedule = θ = {0.01, 0.02, ..., 0.1}
    Equation 5 defines the metric using a hand-chosen confidence set; no sensitivity analysis and no correction for the apparent 1−θ vs θ inconsistency.
axioms (6)
  • domain assumption Source segmentation masks are accurate enough to serve as spatial ground truth.
    All volumes, boxes, and distances in Section 3.2 are computed from masks taken from TotalSegmentator, AMOS, MSD, KiTS, and BraTS. No mask-accuracy audit is reported.
  • domain assumption Voxel count times voxel spacing gives the clinically correct volume.
    Equation 1 defines volume purely from foreground voxel count and NIfTI spacing; partial-volume effects and segmentation boundary errors are ignored.
  • domain assumption Euclidean distance between bounding-box centers is a meaningful inter-structure distance.
    Equations 3-4 define distance between structures using bounding-box centers. This is a design choice; clinical distances often use surfaces or centroids, and the paper provides no validation that center-to-center distance matches clinical judgments.
  • ad hoc to paper Three specialist LLMs answering correctly with RAG-only context reliably identifies 'trivial' questions.
    Section 3.2 uses a 2-of-3 correct rule to discard text-answerable questions. This heuristic is not externally calibrated against human-judged triviality.
  • domain assumption Radiologist average score > 1 is a valid quality filter.
    Section 3.2 relies on three board-certified radiologists, but no inter-annotator agreement, scoring rubric calibration, or quantitative audit is reported.
  • domain assumption PubMed retrieval via top-5 RAG documents supplies sufficient and correct medical context.
    Appendix B: Qwen3-8B embeddings and Faiss index over PubMed are used to retrieve context. Retrieval quality and its effect on generated QA plausibility are not evaluated.

pith-pipeline@v1.3.0-alltime-deepseek · 16460 in / 11738 out tokens · 117308 ms · 2026-08-02T18:10:49.472901+00:00 · methodology

0 comments
read the original abstract

Visual spatial intelligence is critical for medical image interpretation, yet remains largely unexplored in Multimodal Large Language Models (MLLMs) for 3D imaging. This gap persists due to a systemic lack of datasets featuring structured 3D spatial annotations beyond basic labels. In this study, we introduce an agentic pipeline that autonomously synthesizes spatial visual question-answering (VQA) data by orchestrating computational tools such as volume estimation and bounding boxes extraction with multi-agent collaboration and expert radiologist validation. We present SpatialMed, the first comprehensive benchmark for evaluating 3D spatial intelligence in medical MLLMs, comprising 31,253 question-answer pairs across multiple organs and tumor types. Our evaluations on 24 state-of-the-art MLLMs and extensive analyses reveal that current models lack robust spatial reasoning capabilities for medical imaging.

Figures

Figures reproduced from arXiv: 2603.13800 by Andrea M. Bejar, Gorkem Durak, Halil Ertugrul Aktas, Min Xu, Quoc-Huy Trinh, Ulas Bagci, Xi Ding, Xingjian Li, Yang Liu, Zhenyue Qin.

Figure 1
Figure 1. Figure 1: Task demonstrations in the SpatialMed, covering six spatial reasoning tasks, with corresponding 3D CT visualizations. et al., 2024; Hong et al., 2025; Qwen, 2025; Chen et al., 2024c) demonstrate effectiveness across a wide range of multimodal tasks, with Qwen3-VL and InternVL3 explicitly incorporating spatial reasoning and achieving strong results in embodied settings. In the medical domain, most MLLMs ope… view at source ↗
Figure 2
Figure 2. Figure 2: Overview of three stages from the SpatialMed dataset pipeline. (1) Question–Answer Pair Generation, where agents produce clinically grounded QA pairs using medical knowledge and spatial analysis tools; (2) Data Quality Validation, in which multiple specialist agents verify medical correctness and the necessity of visual–spatial evidence; and (3) Radiologists validation, where multiple radiologists review p… view at source ↗
Figure 3
Figure 3. Figure 3: Benchmark Statistics. Left: Distribution of annotated anatomical regions in the MCA task. Right: Dataset distribution across the volume task, where the y-axis is shown on a log2 scale. Expert-in-the-loop Quality Review. To ensure the bench￾mark’s clinical integrity and mitigate ”hallucination arti￾facts” common in LLM-generated content, we implemented a rigorous, multi-stage validation phase involving thre… view at source ↗
Figure 4
Figure 4. Figure 4: Fine-grained performance analysis across anatomical structures, tumor types, and volume scales. (a) Per-organ accuracy across models in the MCA task. (b) Tumor-wise accuracy across selected models. (c) Performance stratified by anatomical volume buckets using Mean Relative Accuracy. in some operators may come at the cost of others when the backbone changes, which is consistent with the unstable ranking pat… view at source ↗
Figure 5
Figure 5. Figure 5: Failure and faithfulness analysis of MLLM spatial reasoning. (a) Human-annotated taxonomy of reasoning errors, where numeric and relational errors dominate. (b) Faithfulness matrix categorizing predictions into faithful reasoning, decision errors, lucky guesses, and hallucination. tent with broader findings that instruction-tuned models can be miscalibrated, motivating explicit calibration objectives or co… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Rad-JEPA 3D: Radiology Joint-Embedding Predictive Model for 3D Computed Tomography

    cs.CV 2026-07 conditional novelty 5.0

    A 19M-parameter JEPA-style 3D-CT encoder with a routed Mamba+GQA hybrid and orthogonal hidden-state regularization gives a 4B total model the best mean accuracy on M3D-VQA closed-ended questions and the best average o...

  2. Towards Enhancing 3D Spatial Reasoning in Medical Multimodal Large Language Models

    cs.CV 2026-07 reject novelty 5.0

    A synthetic chain-of-thought dataset generated from CT reports lets a 2D-pretrained medical MLLM improve on 3D CT spatial-reasoning benchmarks.

Reference graph

Works this paper leans on

26 extracted references · 17 linked inside Pith · cited by 2 Pith papers

  1. [5]

    Radimagenet-vqa: A large-scale ct and mri dataset for radiologic visual question answering.arXiv preprint arXiv:2512.17396,

    Butsanets, L., Corbi `ere, C., Khlaut, J., Manceron, P., and Dancette, C. Radimagenet-vqa: A large-scale ct and mri dataset for radiologic visual question answering.arXiv preprint arXiv:2512.17396,

    arXiv

  2. [6]

    Internlm2 technical report.arXiv preprint arXiv:2403.17297,

    Cai, Z., Cao, M., Chen, H., Chen, K., Chen, K., Chen, X., Chen, X., Chen, Z., Chen, Z., Chu, P., et al. Internlm2 technical report.arXiv preprint arXiv:2403.17297,

    Pith/arXiv arXiv

  3. [7]

    and Weis, S

    Canese, K. and Weis, S. Pubmed: the bibliographic database. The NCBI handbook, 2(1):2013,

    2013

  4. [10]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  5. [11]

    E., Er, S., Almas, F., Simsek, A

    Hamamci, I. E., Er, S., Almas, F., Simsek, A. G., Esir- gun, S. N., Dogan, I., Dasdelen, M. F., Durugol, O. F., Wittmann, B., Amiranashvili, T., Simsar, E., Simsar, M., Erdemir, E. B., Alanbay, A., Sekuboyina, A., Lafci, B., Bluethgen, C., Ozdemir, M. K., and Menze, B. Devel- oping generalist foundation models from a multimodal dataset for 3d computed tom...

    arXiv 1904

  6. [13]

    P., Yeung, S., Shah, N., and Fries, J

    Huang, S.-C., Huo, Z., Steinberg, E., Chiang, C.-C., Lan- glotz, C., Lungren, M. P., Yeung, S., Shah, N., and Fries, J. A. Inspect: A multimodal dataset for pulmonary embolism diagnosis and prognosis.arXiv preprint arXiv:2311.10798,

    Pith/arXiv arXiv

  7. [14]

    Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models.arXiv preprint arXiv:2404.10237,

    Jiang, S., Zheng, T., Zhang, Y ., Jin, Y ., Yuan, L., and Liu, Z. Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models.arXiv preprint arXiv:2404.10237,

    Pith/arXiv arXiv

  8. [15]

    Menze, B

    URL https://llava-vl.github.io/blog/ 2024-01-30-llava-next/. Menze, B. H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y ., Porz, N., Slotboom, J., Wiest, R., et al. The multimodal brain tumor image segmentation benchmark (brats).IEEE transactions on medical imaging, 34(10):1993–2024,

    2024

  9. [16]

    B., Chen, C., Ouyang, C., and Rueckert, D

    Pan, J., Liu, C., Wu, J., Liu, F., Zhu, J., Li, H. B., Chen, C., Ouyang, C., and Rueckert, D. Medvlm-r1: Incen- tivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning.arXiv preprint arXiv:2502.19634,

    Pith/arXiv arXiv

  10. [17]

    Rajabi, N

    URL https:// arxiv.org/abs/2505.09388. Rajabi, N. and Kosecka, J. Gsr-bench: A benchmark for grounded spatial reasoning evaluation via multimodal llms.arXiv preprint arXiv:2406.13246,

    Pith/arXiv arXiv

  11. [19]

    Thermometer: Towards universal calibration for large language models.arXiv preprint arXiv:2403.08819,

    Shen, M., Das, S., Greenewald, K., Sattigeri, P., Wornell, G., and Ghosh, S. Thermometer: Towards universal calibration for large language models.arXiv preprint arXiv:2403.08819,

    Pith/arXiv arXiv

  12. [20]

    Med-2e3: A 2d-enhanced 3d medical multimodal large language model.arXiv preprint arXiv:2411.12783,

    Shi, Y ., Zhu, X., Wang, K., Hu, Y ., Guo, C., Li, M., and Wu, J. Med-2e3: A 2d-enhanced 3d medical multimodal large language model.arXiv preprint arXiv:2411.12783,

    arXiv

  13. [21]

    Prs-med: Position reasoning segmentation with vision- language model in medical imaging.arXiv preprint arXiv:2505.11872,

    Trinh, Q.-H., Nguyen, M.-V ., Zeng, J., Bagci, U., and Jha, D. Prs-med: Position reasoning segmentation with vision- language model in medical imaging.arXiv preprint arXiv:2505.11872,

    arXiv

  14. [22]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191,

    Pith/arXiv arXiv

  15. [23]

    W., Han, R., Fei-Fei, L., and Xie, S

    Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., and Xie, S. Thinking in space: How multimodal large lan- guage models see, remember, and recall spaces. InPro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 10632–10643, 2025a. Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., and Xie, S. Thinking in space: How multimo...

    Pith/arXiv arXiv

  16. [24]

    Yi, K., Gan, C., Li, Y ., Kohli, P., Wu, J., Torralba, A., and Tenenbaum, J. B. Clevrer: Collision events for video representation and reasoning.arXiv preprint arXiv:1910.01442,

    Pith/arXiv arXiv 1910

  17. [26]

    All of the embeddings are stored via the index storage of Faiss (Johnson et al., 2019)

    dataset. All of the embeddings are stored via the index storage of Faiss (Johnson et al., 2019). During the inference process of the agent, the top 5 most relevant records are chosen as the input context for the agent. C. Impact Statement This paper introduces a new benchmark and data generation pipeline to study spatial reasoning in medical vision langua...

    2019

  18. [2013]

    H., Wang, X., Zhang, R., Cai, Z., Ji, K., et al

    Chen, J., Gui, C., Ouyang, R., Gao, A., Chen, S., Chen, G. H., Wang, X., Zhang, R., Cai, Z., Ji, K., et al. Huatuogpt-vision, towards injecting medical vi- sual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024a. Chen, Q. and Hong, Y . Medblip: Bootstrapping language- image pre-training from 3d medical images and texts. In Proc...

    Pith/arXiv arXiv

  19. [2016]

    Medgemma technical report.arXiv preprint arXiv:2507.05201,

    10 Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space Sellergren, A., Kazemzadeh, S., Jaroensri, T., Kiraly, A., Traverse, M., Kohlberger, T., Xu, S., Jamil, F., Hughes, C., Lau, C., et al. Medgemma technical report.arXiv preprint arXiv:2507.05201,

    Pith/arXiv arXiv

  20. [2017]

    T., Berger, C., Ha, S

    Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R. T., Berger, C., Ha, S. M., Rozy- cki, M., et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the brats challenge.arXiv preprint arXiv:1811.02629,

    Pith/arXiv arXiv

  21. [2019]

    Development of a large-scale grounded vision language dataset for chest ct analysis

    Zhang, X., Wu, C., Zhao, Z., Lei, J., Tian, W., Zhang, Y ., Xie, W., and Wang, Y . Development of a large-scale grounded vision language dataset for chest ct analysis. Scientific Data, 12(1):1636, 2025a. Zhang, Y ., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., and Zhou, J. Qwen3 embedding: Advancing text ...

    Pith/arXiv arXiv

  22. [2020]

    Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006,

    Hong, W., Yu, W., Gu, X., Wang, G., Gan, G., Tang, H., Cheng, J., Qi, J., Ji, J., Pan, L., et al. Glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006,

    Pith/arXiv arXiv

  23. [2022]

    E., Zhang, X., Zhu, M., Alabbad, M

    Baharoon, M., Luo, L., Moritz, M., Kumar, A., Kim, S. E., Zhang, X., Zhu, M., Alabbad, M. H., Alhazmi, M. S., Mistry, N. P., et al. Rexgroundingct: A 3d chest ct dataset for segmentation of findings from free-text reports.arXiv preprint arXiv:2507.22030,

    arXiv

  24. [2023]

    3d- rad: A comprehensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks

    Gai, X., Liu, J., Li, Y ., Meng, Z., Wu, J., and Liu, Z. 3d- rad: A comprehensive 3d radiology med-vqa dataset with multi-temporal analysis and diverse diagnostic tasks. arXiv preprint arXiv:2506.11147,

    arXiv

  25. [2024]

    Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923,

    Pith/arXiv arXiv

  26. [2025]

    Q.-H., and Zhao, B

    Bai, F., Du, Y ., Huang, T., Meng, M. Q.-H., and Zhao, B. M3d: Advancing 3d medical image analysis with multi-modal large language models.arXiv preprint arXiv:2404.00578,

    Pith/arXiv arXiv