Pith. sign in

REVIEW 3 major objections 4 minor 51 references

Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generation

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read At a fixed vision-token budget, anatomy-guided cropping is the most consistent lever for 3D radiology report generation, improving clinical macro F1 in 19 of 20 configurations.

desk verdict A large, careful empirical map of token-budget choices for 3D radiology report generation; the ROI-cropping result is real inside the evaluation setup, but label alignment limits how general the recommendation can be. read the letter →

arxiv 2608.08713 v1 pith:FKZNRPQ2 submitted 2026-08-09 cs.CV cs.AI

Jonathan Suprijadi , Raphael Stock , Moritz Langenberg , David Zimmerer , Kim-Celine Kahl , Stefan Denner , Yannick Kirchhoff , Karol Gotkowski
show 5 more authors
Maximilian Rokuss Jeremias Traub Tassilo Wald Constantin Ulrich Klaus Maier-Hein
This is my paper · ORCID
classification cs.CVcs.AI
keywords 3Dradiologyreportgenerationvision-languagemodeltokencompressionvision-to-languageprojectoranatomy-guidedROIcroppingclinicalmacroF1CT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generating radiology reports from whole 3D CT scans is expensive because a single scan can produce tens of thousands of visual tokens for the language model. This paper isolates one design question: given a fixed number of tokens, where should the budget be spent—on more of the body, on finer detail, or on better compression? Across two large CT-report datasets, four vision encoders, five token-reducing projectors, and five instruction-tuned language models, the answer is consistent: cropping to the anatomy of interest before encoding is the most reliable improvement, raising clinical macro F1 in 19 of 20 configurations by +3.7 points on average for the 3D ViT encoder and +1.1 for the 2D ViT encoder. Pushing input resolution beyond that only helps when the projector can turn many fine tokens into the same output without losing detail; the best configurations reach 49.5 macro F1 on CT-RATE and 49.0 on Merlin.

What carries the argument

The central object is the per-volume vision-token budget, held fixed across all comparisons, paired with three input regimes (whole volume, ROI crop, higher resolution) and projector families that reduce tokens either by local grouping or by learned global queries. The budget-matching design is what lets the paper separate 'how much context from where' from 'how many tokens the LLM sees.' The load-bearing mechanisms are the anatomy-guided crop (from segmentation masks) and the aggregation depth of query-based projectors, especially multiple Perceiver layers; the paper's empirical finding is that only projectors that can compress many fine tokens via iterative attention benefit from higher-resolution input.

What would settle it

Retrain the best CT-RATE configuration with a whole-body or foreground ROI covering all organs and evaluate it with a classifier whose label set includes extrapulmonary findings (liver, kidney, bone); if the lung-crop configuration no longer beats the whole-volume baseline, the ROI gain was an artifact of label–crop alignment.

Watch

Extended reading notes

Core claim

The paper's central claim is that at a matched vision-token budget, the allocation of that budget dominates projector and language-model choices for 3D radiology report generation. Anatomy-guided ROI cropping—cropping the volume to a segmentation-derived bounding box of the relevant anatomy before encoding—is the most consistent intervention, improving clinical macro F1 in 19 of 20 vision-encoder-projector-dataset settings, with average gains of +3.7 for Primus and +1.1 for Curia. Higher-resolution inputs only pay off when combined with projectors that can aggregate many encoder tokens under strong compression: the PerceiverResampler and, for Primus on Merlin, TokenPacker; grid- and pooling-based projectors often degrade. The vision encoder is the largest performance axis (over 15 F1 points between best and weakest), while language-model choice matters less (2–5 F1 points once retuned). At low compression, an uncompressed MLP projector remains a strong reference; among token-reducing projectors, the PerceiverResampler stands out, matching or exceeding the MLP baseline for Curia at high compression. The best reported configurations achieve 49.5 clinical macro F1 on CT-RATE and 49.0 on Merlin, exceeding the strongest published comparison systems on those benchmarks.

Load-bearing premise

The headline cropping benefit assumes the clinical metric scores exactly the anatomy kept in the crop: if CT-RATE were scored on extrapulmonary findings too, the lung crop would remove structures the reference reports describe.

Editorial extensions

If this is right

  • For any 3D radiology VLM with a fixed LLM context limit, replacing whole-volume resizing with a segmentation-based anatomy crop is likely the cheapest reliable win, worth roughly 3–5 macro-F1 points on chest CT and about 1 point on slice-based encoders, without changing downstream token count.
  • When compression is unavoidable, PerceiverResampler and TokenPacker are the projector families to prefer; ConcatAdapter and AveragePool degrade sharply at high compression ratios.
  • The dominant design choice is the vision encoder, not the language model; spending effort on encoder selection or pretraining should precede LLM swaps.
  • If higher-resolution inputs are desired, they must be paired with a projector able to aggregate the extra tokens; otherwise the larger input hurts performance.
  • The best configurations set new state-of-the-art clinical F1 on CT-RATE (49.5) and Merlin (49.0), showing that token-compressed pipelines can match or beat uncompressed ones when the projector is chosen correctly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the CT-RATE label set were expanded beyond the lung and cardiovascular classes, the crop advantage could shrink or reverse; the paper's own limitation note identifies this risk.
  • Editorial extension: the same token-budget control could be applied to other 3D vision-language tasks, suggesting that ROI selection and projector aggregation depth should be co-designed rather than chosen independently.
  • Editorial extension: because only 1.7B–4B LLMs were tested, the finding that the vision encoder dominates may not hold at larger scales, where the LLM can extract more per token; re-running the matrix at 7B+ would settle it.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a systematic empirical study of how to allocate the vision-token budget in 3D radiology report generation. It compares three input regimes (whole-volume encoding, anatomy-guided ROI cropping, and higher-resolution encoding), four frozen vision encoders (Primus, Curia, CT-FM, Merlin), five token-reducing projectors plus an uncompressed MLP baseline, and five instruction-tuned LLMs on CT-RATE and Merlin. The central finding is that ROI cropping improves clinical macro F1 in 19 of 20 VE-projector-dataset configurations at matched LLM token budgets, and that further resolution gains are projector-dependent, benefiting mainly TokenPacker and PerceiverResampler. The paper reports state-of-the-art test-set clinical macro F1 of 49.5 on CT-RATE and 49.0 on Merlin, and it releases code, models, and a ModernBERT classifier for Merlin.

Significance. This is a carefully controlled, large-scale study (three seeds, roughly 66,000 GPU-hours) that provides useful evidence on a design question rarely isolated in the 3D VLM literature: where the fixed vision-token budget should be spent. The internal evidence for the ROI-cropping effect is strong and clearly tabulated (Table 4), with standard deviations and per-cell comparisons. The paper also gives a clean characterization of projector behavior under compression (Table 5) and makes concrete recommendations, including the caution that most token-reducing projectors do not beat an MLP baseline. The public release of code, models, and the Merlin classifier is a concrete asset. The significance is tempered by two issues: the ROI crops coincide with the abnormality classes used for evaluation, which limits external validity of the 'most consistent strategy' claim, and the final SOTA comparison (Table 7) does not report uncertainty, so the 1.5-point lead over AdaRAG-CT may be within seed noise.

major comments (3)
  1. [§3.1 and §5] The headline ROI-cropping claim is confounded with label-set alignment. On CT-RATE the ROI is the bounding box of the five lung lobes (Section 3.1) and the clinical metric scores exactly 18 pulmonary and cardiovascular classes (Section 3.2.1); on Merlin the ROI is the whole-body foreground and the metric scores 30 abdominal classes. The paper itself states in Section 5 that a lung-crop 'removes anatomy the reference reports describe, which can encourage statements about structures the input no longer contains.' The F1 gain may therefore partly reflect the crop removing findings that the metric neither rewards nor penalizes, rather than a general improvement in report quality. To support the broad recommendation, please provide a concrete test on a setting where the crop is not aligned with the evaluation label set (for example, scoring extra-ROI findings such as liver lesions or bone metastases in chest CT, or evaluating on a full-body/open-set dataset), or substantially qualify the recommendation and the wording 'generally applicable' in the abstract and Section 5.
  2. [Table 7 and §4.4] The state-of-the-art comparison reports seed-averaged test-set macro F1 values without standard deviations or confidence intervals. The claimed lead over AdaRAG-CT is 1.5 points (49.5 vs. 48.0), while seed-to-seed standard deviations in the validation results of Table 4 are typically 0.5-1.5 F1 points for the same configurations. As reported, the SOTA claim is not statistically distinguishable from a tie. Please report test-set standard deviations/confidence intervals across seeds for both of the 'Ours' rows and for any recomputed baselines, or temper the SOTA claim to 'comparable or best in our evaluation setup.'
  3. [§3.4 and §4.3] The claims that the vision encoder is the dominant axis and that LLM choice has a smaller effect rely on configurations for which hyperparameters were selected only for Primus-1B on CT-RATE and then transferred to all other VEs, datasets, and LLMs. The paper acknowledges this in Section 5, but the limitation is load-bearing for the relative-importance conclusions in Section 4.3: the comparison is between a tuned development configuration and untuned transferred configurations. Please either retune the most important off-development settings (at least the LLM rows in Table 6) or present the VE/LLM relative-importance statement explicitly as a lower-bound comparison under shared hyperparameters.
minor comments (4)
  1. [Table 4] The '19 of 20' count treats the Curia ConcatAdapter cell on CT-RATE (48.0 to 48.0) as a non-improvement; please state explicitly that one cell is a tie rather than an improvement so readers can verify the count.
  2. [§3.2.1 and Table 7] The Merlin baseline rows in Table 7 are recomputed with the authors' ModernBERT/GPT-OSS evaluation framework; the caption should state whether the published Merlin and Jolia papers used the same label extraction procedure, since differences in label extraction could affect comparability.
  3. [§5] The Limitations paragraph already discloses the ROI-label alignment issue; consider moving a one-sentence version of this caveat to the abstract or Section 4.1 so readers do not encounter the strong recommendation without the associated scope restriction.
  4. [§4.2 / Table 5] For the MLP baseline rows, the token count is much larger than the compressed rows (for example, 8,192 vs. 1,024 for Curia-8S); the table caption should remind readers that MLP rows are not at the same token budget as the compressed rows, even though they appear in the same table.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claims are held-out empirical comparisons; ROI/metric alignment is a generalization caveat, not a circular step.

full rationale

The paper's central claims—ROI cropping improves clinical macro F1 at matched token budgets, projector behavior under compression, and encoder/LLM influence—are empirical findings obtained by training on train splits, selecting checkpoints by validation macro F1, and reporting on held-out test sets (Sections 3.3–3.5, Tables 4–6). Nothing is derived analytically from the metric or from a fitted parameter; the ROI-crop comparison is a measured difference, not a construction. The lung/foreground ROI is indeed aligned with the 18/30 classes scored by the clinical classifiers (Sections 3.1 and 3.2.1), and the paper itself concedes that a lung-crop 'removes anatomy the reference reports describe, which can encourage statements about structures the input no longer contains' (Section 5). That is a threat to external validity of the design recommendation, not a circular derivation: the model is not optimized on test labels, and the crop is not defined as the set of metric-positive regions by construction. Self-citations (Primus [20], the OpenMind warm-up [48]) supply components and training recipes, but no load-bearing claim is justified solely by those citations; the relevant comparisons are run in this paper. The Merlin classifier is a proxy teacher, explicitly flagged as 'fidelity to a flawed teacher,' which affects interpretability of the F1 numbers but does not make the headline result equivalent to its inputs. Score 0.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new conceptual entities such as particles, forces, or dimensions are introduced. All pipeline components are existing architectures or combinations of existing components. The main load-bearing assumptions are segmentation accuracy, proxy-label validity, token-budget comparability, frozen-encoder transferability, and split representativeness.

free parameters (6)
  • LLM vision-token budget M = 1728 (Primus), 2048 (Curia)
    Hand-chosen matched budgets define the entire comparison; all FOV, resolution, and projector conclusions are conditioned on these fixed token counts.
  • ROI expansion margin = 4% of side length per axis
    Chosen by hand without ablation; this margin controls how much context remains around the target anatomy and therefore affects the magnitude of ROI gains.
  • Compression ratios evaluated = 8x/27x/64x Primus; 4x/16x/64x Curia; 8x/27x CT-FM; 8x Merlin
    The set of ratios is hand-selected to span the design space, and projector rankings change across ratios, so this choice shapes the conclusions.
  • PerceiverResampler projector-only warm-up = 25% of total steps instead of 6.25%
    Ad hoc extension introduced because preliminary experiments showed the standard warm-up was insufficient; this changes optimization and therefore results.
  • Curia axial feature slices = 8 (baseline) or 64 (high-res)
    This hand-chosen count defines the two resolution regimes for Curia and is the core of the resolution intervention.
  • Lung-volume exclusion threshold = 1 L
    Excludes 829 CT-RATE scans from the training set; if excluded scans are non-random, this can affect reported F1.
assumptions (5)
  • domain assumption TotalSegmentator segmentation is accurate enough that the union-of-foreground bounding box preserves all clinically relevant anatomy for report generation.
    Invoked in Sections 2.3.1 and 3.1 for both CT-RATE and Merlin ROIs; if segmentation fails, ROI cropping may remove or misalign findings.
  • domain assumption Clinical macro F1 computed by RadBERT/ModernBERT classifiers on LLM-extracted labels is a valid proxy for clinical report quality.
    Used as the primary metric throughout Sections 3.2 and 4; the authors admit it measures fidelity to a flawed teacher in Section 5.
  • domain assumption Token-budget matching isolates input regime from sequence length; projectors with the same output token count M are comparable even though their parameter counts and compute differ.
    Core comparison logic in Section 2.2.4; the ConcatAdapter parameter count grows with compression, so matched tokens do not mean matched capacity or compute.
  • domain assumption Frozen encoders plus LoRA-adapted LLMs is a sufficiently representative adaptation setting for conclusions about VE and projector design.
    Stated in Sections 2.1 and 5; all results are specific to frozen VEs, LoRA, and 1.7B-4B LLMs, and other adaptation regimes may rank components differently.
  • domain assumption The CT-RATE official validation split and the internally created validation split are representative of test performance; no official test split exists.
    Section 3.1; the paper evaluates final models on the official validation split as the test set, which could differ from an unseen test distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generation." pith.science (2026). https://pith.science/paper/FKZNRPQ2

@misc{pith2026260808713,
  author       = {Pith},
  title        = {Pith review of: Resolution Meets Reduction: Efficient Visual Context for 3D Radiology Report Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FKZNRPQ2}},
  note         = {Machine review of arXiv:2608.08713}
}
read the original abstract

Vision-language models offer a promising path toward automating radiology report generation, but applying them to full 3D CT volumes poses substantial computational challenges. Modern foundation vision encoders (VEs) can produce tens of thousands of vision tokens per scan, making the visual sequence passed to the large language model (LLM) a primary computational bottleneck. Vision-to-language projectors can compress this sequence to reduce computation, but may discard clinically relevant detail; conversely, effective compression can accommodate higher-resolution inputs while keeping the downstream token count fixed. How this vision-token budget should be allocated across input field of view, spatial resolution, and vision-to-language projection therefore remains an open design question. We systematically evaluate four heterogeneous VEs (CNN- and ViT-based), five token-reducing projectors at up to 64x compression alongside a non-reducing MLP projector baseline, and five instruction-tuned LLMs (1.7B--4B) on two large-scale CT report datasets (CT-RATE and Merlin). At matched LLM token budgets, anatomy-guided region of interest cropping is the most consistent strategy, improving clinical macro F1 in 19 of 20 settings by +3.7 points on average for the 3D ViT Primus encoder and +1.1 for the slice-based 2D ViT Curia encoder. Increasing input resolution further is strongly projector-dependent: the PerceiverResampler, paired with higher-resolution Curia features, yields the strongest configuration in the resolution study on both datasets. Our best configurations achieve state-of-the-art clinical macro F1 on the test sets, reaching 49.5 on CT-RATE and 49.0 on Merlin. Code and models will be published upon publication.

Figures

Figures reproduced from arXiv: 2608.08713 by the authors.

Figure 1
Figure 1. Overview of our VLM pipeline for 3D radiology report gen￾eration. Each 3D CT volume is first preprocessed according to one of three in￾put regimes: whole-volume encoding, anatomy-guided ROI cropping, or higher￾resolution encoding, before a frozen vision encoder extracts vision tokens. The projector compresses these tokens before they are passed to the LLM for report generation. This work systematically investigates … view at source ↗
Figure 2
Figure 2. 3.2 Evaluation Metrics We use clinical macro F1 as the primary evaluation metric throughout all model￾selection, hyperparameter-tuning, and ablation experiments. Lexical metrics are reported only for the final comparison of our best-performing configurations with existing methods and do not inform any development decisions. Commonly used lexical metrics primarily measure surface-level similarity to reference reports… view at source ↗
Figure 2
Figure 2. Image dimension distributions before and after ROI crop￾ping. Distributions along the X, Y, and Z axes are shown for the raw scans (dark blue) and cropped ROIs (orange). CT-RATE uses a lung ROI, while Mer￾lin uses a foreground ROI. do not account for the clinical relevance or correctness of the generated content, making them unreliable indicators of medical report quality [37, 38]. 3.2.1 Clinical Evaluation For CT-R… view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Class-wise and qualitative results. (left) Macro F1 (first row) and class-wise F1 scores [%] for Curia-64S PerceiverResampler in dark blue (cf [PITH_FULL_IMAGE:figures/full_fig_p013_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

51 extracted references · 29 canonical work pages

  1. [1]

    Radiologists’ variation of time to read across different procedure types,

    D. Forsberg, B. Rosipko, and J. L. Sunshine, “Radiologists’ variation of time to read across different procedure types,”Journal of digital imaging, vol. 30, no. 1, pp. 86–94, 2017

    work page 2017

  2. [2]

    Maira-2: Grounded radiology report generation,

    S. Bannur, K. Bouzid, D. C. Castro, A. Schwaighofer, A. Thieme, S. Bond- Taylor, M. Ilse, F. P´ erez-Garc ´ ıa, V. Salvatelli, H. Sharmaet al., “Maira-2: Grounded radiology report generation,”arXiv preprint arXiv:2406.04449, 2024

    arXiv 2024

  3. [3]

    Scaling medical imaging re- port generation with multimodal reinforcement learning,

    Q. Liu, S. Zhang, G. Qin, Y. Gu, Y. Jin, S. Preston, Y. Xu, S. Ki- blawi, W.-w. Yim, T. Ossowskiet al., “Scaling medical imaging re- port generation with multimodal reinforcement learning,”arXiv preprint arXiv:2601.17151, 2026. 19

    arXiv 2026

  4. [4]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,”Advances in neural information processing systems, vol. 36, pp. 34 892–34 916, 2023

    work page 2023

  5. [5]

    Compre- hensive language-image pre-training for 3d medical image understanding,

    T. Wald, I. E. Hamamci, Y. Gao, S. Bond-Taylor, H. Sharma, M. Ilse, C. Lo, O. Melnichenko, A. Schwaighofer, N. C. Codellaet al., “Compre- hensive language-image pre-training for 3d medical image understanding,” arXiv preprint arXiv:2510.15042, 2025

    arXiv

  6. [6]

    M3d: Advancing 3d medical image analysis with multi-modal large language models,

    F. Bai, Y. Du, T. Huang, M. Q.-H. Meng, and B. Zhao, “M3d: Advancing 3d medical image analysis with multi-modal large language models,”arXiv preprint arXiv:2404.00578, 2024

    arXiv 2024

  7. [7]

    Generalist foundation models from a multimodal dataset for 3D computed tomography,

    I. E. Hamamci, S. Er, C. Wang, F. Almas, A. G. Simsek, S. N. Esirgun, I. Dogan, O. F. Durugol, B. Hou, S. Shit, W. Dai, M. Xu, H. Reynaud, M. F. Dasdelen, B. Wittmann, T. Amiranashvili, E. Simsar, M. Simsar, E. B. Erdemir, A. Alanbay, A. Sekuboyina, B. Lafci, A. Kaplan, Z. Lu, M. Polacin, B. Kainz, C. Bluethgen, K. Batmanghelich, M. K. Ozdemir, and B. Men...

    2026

  8. [8]

    Merlin: A computed tomography vision-language foundation model and dataset,

    L. Blankemeier, A. Kumar, J. P. Cohen, J. Liu, L. Liu, D. Van Veenet al., “Merlin: A computed tomography vision-language foundation model and dataset,”Nature, vol. 652, no. 8112, pp. 1318–1328, 2026

    work page 2026

Show all 51 references
  1. [9]

    Large language model with region- guided referring and grounding for ct report generation,

    Z. Chen, Y. Bie, H. Jin, and H. Chen, “Large language model with region- guided referring and grounding for ct report generation,”IEEE transactions on Medical Imaging, vol. 44, no. 8, pp. 3139–3150, 2025

    2025

  2. [10]

    CT- GRAPH: Hierarchical graph attention network for anatomy-guided ct re- port generation,

    H. Kalisch, F. H¨ orst, J. Kleesiek, K. Herrmann, and C. Seibold, “CT- GRAPH: Hierarchical graph attention network for anatomy-guided ct re- port generation,” inProceedings of the IEEE/CVF International Confer- ence on Computer Vision (ICCV) Workshops, October 2025, pp. 6834– 6843

    2025

  3. [11]

    MedGemma 1.5 technical report,

    A. Sellergrenet al., “MedGemma 1.5 technical report,”arXiv preprint arXiv:2604.05081, 2026

    2026 arXiv

  4. [12]

    The effect of image resolution on deep learning in radiography,

    C. F. Sabottke and B. M. Spieler, “The effect of image resolution on deep learning in radiography,”Radiology: Artificial Intelligence, vol. 2, no. 1, p. e190015, 2020

    2020

  5. [13]

    Better tokens for better 3d: Advanc- ing vision-language modeling in 3d medical imaging,

    I. E. Hamamci, S. Er, S. Shit, H. Reynaud, D. Yang, P. Guo, M. Edgar, D. Xu, B. Kainz, and B. Menze, “Better tokens for better 3d: Advanc- ing vision-language modeling in 3d medical imaging,”Advances in Neural Information Processing Systems, vol. 38, pp. 135 074–135 102, 2026

    2026

  6. [14]

    Jolia: Concept- level vision-language alignment for 3d ct contrastive learning,

    J. Khlaut, C. Corbi` ere, B. Callard, A. Prat, L. Butsanets, A. Saporta, T. Danielou, L. Machado, K. L. Floch, T. Boekenet al., “Jolia: Concept- level vision-language alignment for 3d ct contrastive learning,”arXiv preprint arXiv:2606.24570, 2026. 20

    2026 arXiv

  7. [15]

    SegVol: Universal and interactive volumetric medical image segmentation,

    Y. Du, F. Bai, T. Huang, and B. Zhao, “SegVol: Universal and interactive volumetric medical image segmentation,” inAdvances in Neural Informa- tion Processing Systems, vol. 37, 2024, pp. 110 746–110 783

    2024

  8. [16]

    Revisiting 2d founda- tion models for scalable 3d medical image classification,

    H. Liu, B. Georgescu, Y. Zhang, Y. Yoo, M. Baumgartner, R. Gao, J. Wang, G. Zhao, E. Gibson, D. Comaniciu, and S. Grbic, “Revisiting 2d founda- tion models for scalable 3d medical image classification,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2026

  9. [17]

    Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data,

    C. Wu, X. Zhang, Y. Zhang, H. Hui, Y. Wang, and W. Xie, “Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data,”Nature Communications, vol. 16, no. 1, p. 7866, 2025

    2025

  10. [18]

    Exploring the design space of 3d mllms for ct report generation,

    M. Baharoon, J. Ma, C. Fang, A. Toma, and B. Wang, “Exploring the design space of 3d mllms for ct report generation,” inInternational Con- ference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2025, pp. 237–246

    2025

  11. [19]

    Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation,

    Y. Ji, H. Bai, C. Ge, J. Yang, Y. Zhu, R. Zhang, Z. Li, L. Zhanng, W. Ma, X. Wanet al., “Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation,”Advances in Neural Information Processing Systems, vol. 35, pp. 36 722–36 732, 2022

    2022

  12. [20]

    Primus: Enforcing attention usage for 3d medical image segmentation,

    T. Wald, S. Roy, F. Isensee, C. Ulrich, S. Ziegler, D. Trofimova, R. Stock, M. Baumgartner, G. Koehler, and K. Maier-Hein, “Primus: Enforcing attention usage for 3d medical image segmentation,”Transactions on Machine Learning Research, 2026. [Online]. Available: https: //openr...

    2026

  13. [21]

    Curia: A multi- modal foundation model for radiology,

    C. Dancette, J. Khlaut, A. Saporta, H. Philippe, E. Ferreres, B. Callard, T. Danielou, L. Alberge, L. Machado, D. Tordjmanet al., “Curia: A multi- modal foundation model for radiology,”arXiv preprint arXiv:2509.06830, 2025

    2025 arXiv

  14. [22]

    Smarter, bet- ter, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,

    B. Warner, A. Chaffin, B. Clavi´ e, O. Weller, O. Hallstr¨ om, S. Taghadouini, A. Gallagher, R. Biswas, F. Ladhak, T. Aarsenet al., “Smarter, bet- ter, faster, longer: A modern bidirectional encoder for fast, memory efficient, and long context finetuning and inference,”arXiv p...

    2024 arXiv

  15. [23]

    gpt-oss-120b & gpt-oss-20b model card,

    S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Baoet al., “gpt-oss-120b & gpt-oss-20b model card,”arXiv preprint arXiv:2508.10925, 2025

    2025 arXiv

  16. [24]

    Beyond the embedding bottleneck: Adaptive retrieval-augmented 3d ct report generation,

    R. Liang, Y. Ma, Y. Xing, Z. Fan, J. Pan, C. Sun, L. Li, K. Gong, and J. Xu, “Beyond the embedding bottleneck: Adaptive retrieval-augmented 3d ct report generation,”arXiv preprint arXiv:2603.15822, 2026. 21

    2026

  17. [25]

    Qwen3-vl technical report,

    S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Geet al., “Qwen3-vl technical report,”arXiv preprint arXiv:2511.21631, 2025

    2025 arXiv

  18. [26]

    Tokenpacker: Efficient visual projector for multimodal llm,

    W. Li, Y. Yuan, J. Liu, D. Tang, S. Wang, J. Qin, J. Zhu, and L. Zhang, “Tokenpacker: Efficient visual projector for multimodal llm,”International Journal of Computer Vision, vol. 133, no. 10, pp. 6794–6812, 2025

    2025

  19. [27]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynoldset al., “Flamingo: a visual language model for few-shot learning,”Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022

    2022

  20. [28]

    Totalsegmentator: robust segmentation of 104 anatomic structures in ct images,

    J. Wasserthal, H.-C. Breit, M. T. Meyer, M. Pradella, D. Hinck, A. W. Sauter, T. Heye, D. T. Boll, J. Cyriac, S. Yanget al., “Totalsegmentator: robust segmentation of 104 anatomic structures in ct images,”Radiology: Artificial Intelligence, vol. 5, no. 5, p. e230024, 2023

    2023

  21. [29]

    Vision foundation models for computed tomogra- phy,

    S. Pai, I. Hadzic, D. Bontempi, K. Bressem, B. H. Kann, A. Fedorov, R. H. Mak, and H. J. Aerts, “Vision foundation models for computed tomogra- phy,”arXiv preprint arXiv:2501.09001, 2025

    2025 arXiv

  22. [30]

    Masked au- toencoders are scalable vision learners,

    K. He, X. Chen, S. Xie, Y. Li, P. Doll´ ar, and R. Girshick, “Masked au- toencoders are scalable vision learners,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16 000– 16 009

    2022

  23. [31]

    DINOv2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y. Huang, S.-W. Li, I. Misra, M. Rabbat, V. Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. J...

    2024

  24. [32]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clarket al., “Learning transferable visual models from natural language supervision,” inInternational confer- ence on machine learning. PMLR, 2021, pp. 8748–8763

    2021

  25. [33]

    A simple framework for contrastive learning of visual representations,

    T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” inInternational conference on machine learning. PMLR, 2020, pp. 1597–1607

    2020

  26. [34]

    Qwen3 technical report,

    A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lvet al., “Qwen3 technical report,”arXiv preprint arXiv:2505.09388, 2025. 22

    2025 arXiv

  27. [35]

    Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of- loras,

    A. Abouelenin, A. Ashfaq, A. Atkinson, H. Awadalla, N. Bach, J. Bao, A. Benhaim, M. Cai, V. Chaudhary, C. Chenet al., “Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of- loras,”arXiv preprint arXiv:2503.01743, 2025

    2025 arXiv

  28. [36]

    Gemma 3 technical report,

    G. Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Per- rin, T. Matejovicova, A. Ram´ e, M. Rivi` ereet al., “Gemma 3 technical report,”arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  29. [37]

    Reevalmed: Rethinking medical report evaluation by aligning metrics with real-world clinical judgment,

    R. Li, J. Li, B. Jian, K. Yuan, and Y. Zhu, “Reevalmed: Rethinking medical report evaluation by aligning metrics with real-world clinical judgment,” inProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 11 823–11 837

    2025

  30. [38]

    Green: Generative radiology report evaluation and error notation,

    S. Ostmeier, J. Xu, Z. Chen, M. Varma, L. Blankemeier, C. Bluethgen, A. E. M. Md, M. Moseley, C. Langlotz, A. S. Chaudhariet al., “Green: Generative radiology report evaluation and error notation,” inFindings of the association for computational linguistics: EMNLP 2024, 2024, ...

    2024

  31. [39]

    Radbert: adapting transformer-based language models to radiology,

    A. Yan, J. McAuley, X. Lu, J. Du, E. Y. Chang, A. Gentili, and C.-N. Hsu, “Radbert: adapting transformer-based language models to radiology,” Radiology: Artificial Intelligence, vol. 4, no. 4, p. e210258, 2022

    2022

  32. [40]

    Eval- uating the evaluators: On the reliability of automated label extraction for radiology reports,

    R. Stock, M. Langenberg, D. Zimmerer, K. Dvornikovich, J. C. Holzschuh, J. Suprijadi, C. Ulrich, A. Rastogi, K. Schlamp, P. Vollmuthet al., “Eval- uating the evaluators: On the reliability of automated label extraction for radiology reports,” inMedical Imaging with Deep Learni...

    2026

  33. [41]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” inProceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318

    2002

  34. [42]

    Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “Meteor: An automatic metric for mt evaluation with improved correlation with human judgments,” inProceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, 2005, pp. 65–72

    2005

  35. [43]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out, 2004, pp. 74–81

    2004

  36. [44]

    Microsoft coco captions: Data collection and evaluation server,

    X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Doll´ ar, and C. L. Zitnick, “Microsoft coco captions: Data collection and evaluation server,” arXiv preprint arXiv:1504.00325, 2015

    2015 arXiv

  37. [45]

    pycocoevalcap: Microsoft coco caption evaluation,

    Salaniz, “pycocoevalcap: Microsoft coco caption evaluation,” https:// github.com/salaniz/pycocoevalcap, 2020. 23

    2020

  38. [46]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022

    2022

  39. [47]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  40. [48]

    An openmind for 3d medical vision self- supervised learning,

    T. Wald, C. Ulrich, J. Suprijadi, S. Ziegler, M. Nohel, R. Peretzke, G. Kohler, and K. Maier-Hein, “An openmind for 3d medical vision self- supervised learning,” inProceedings of the IEEE/CVF International Con- ference on Computer Vision, 2025, pp. 23 839–23 879

    2025

  41. [49]

    Curriculum-driven 3d ct report generation via language-free visual graft- ing and zone-constrained compression,

    V. Bumgardner, M. A. Klusty, M. S. Gokmen, and E. W. Damron, “Curriculum-driven 3d ct report generation via language-free visual graft- ing and zone-constrained compression,”arXiv preprint arXiv:2603.23308, 2026

    2026

  42. [50]

    Ct-agent: A multimodal-llm agent for 3d ct radiology question answering,

    Y. Mao, W. Xu, Y. Qin, and Y. Gao, “Ct-agent: A multimodal-llm agent for 3d ct radiology question answering,”Science China Information Sci- ences, vol. 69, no. 5, p. 150107, 2026

    2026

  43. [51]

    U-vlm: Hi- erarchical vision language modeling for report generation,

    P. Shi, M. Zhang, K. Song, J. Liu, Y. Gu, and X. Zhang, “U-vlm: Hi- erarchical vision language modeling for report generation,”arXiv preprint arXiv:2603.00479, 2026. 24

    2026

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.