Pith. sign in

REVIEW 4 major objections 5 minor 37 references

NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that routing MRI evidence through a patient-specific anatomical graph, rather than pooling it into generic tokens, is what makes multimodal glioma reasoning accurate, grounded, and calibrated.

desk verdict A carefully engineered systems paper with a thoughtful evaluation protocol, but the central mechanism claim rests on an ablation without any confidence interval and the matched-baseline premise can't be audited — conditional, not verified. read the letter →

arxiv 2608.03187 v1 pith:DXMGI5FL submitted 2026-08-04 cs.NE

Yantong Liu , Zheyu Zhang , Runpeng Liu , Mu Xitang , Seong-Yoon Shin , Hyun-Ae Lee This is my paper
classification cs.NE
keywords multimodallargelanguagemodelgliomaMRIanatomy-indexedroutingmolecularpredictionevidencegroundingcalibrationselectiverisk
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NeuroMosaic is a 3D multimodal language model that claims to make glioma diagnosis from brain MRI and clinical notes both more accurate and more traceable by routing evidence through a patient-specific anatomical graph. Instead of compressing a whole volumetric scan into generic visual tokens, it converts each MRI sequence into regional tokens tied to atlas parcels and tumor compartments, then sparsely selects the nodes relevant to the task before the language model reads them. On four glioma cohorts the model reports higher integrated-subtype macro-F1 than matched baselines, and on the primary external cohort the gain over the strongest baseline is 3.6 percentage points (95% CI 1.8 to 5.4, adjusted p = 0.0018). The authors argue the ablation pattern—losing 0.030 macro-F1 and 0.072 pointing accuracy when the anatomical graph is removed—shows the gain comes from anatomy-indexed routing, not added capacity. A sympathetic reader would care because the same mechanism produces a readable audit trail: each diagnostic clause is linked to specific regions and sequences, and deleting those regions degrades predictions far more than deleting random regions.

What carries the argument

The load-bearing mechanism is the task-conditioned sparse router over a patient-specific anatomical graph. Graph nodes are atlas parcels and tumor compartments carrying position, scale, sequence provenance, tumor overlap, uncertainty, and pooled image features; edges encode physical adjacency, tract-level neighborhood, hemispheric symmetry, and containment. The router computes a relevance logit for each node given a task query, selects a differentiable top-k subgraph, passes messages along the graph, and feeds only the selected nodes into cross-attention with the language model. This mechanism preserves small-lesion evidence that global pooling would wash out, keeps sequence and spatial prov

What would settle it

Rerun the primary UPenn-GBM comparison with the stated matched configuration—same MRI encoder, language backbone, loss weights, training schedule, token budget, and inference budget—with only the anatomical graph router toggled, and publish seeds and hyperparameters; if the 0.036 macro-F1 difference disappears or reverses, the mechanism claim is refuted. A second decisive check is the paper's own interaction prediction: stratify routing benefit by tumor volume, compartment entropy, and available sequences; the claim predicts the gain concentrates in small, heterogeneous, incomplete-evidence ca

Watch

Extended reading notes

Core claim

The paper's central claim is that anatomy-indexed sparse routing is a measurable mechanism for accurate, grounded, and calibrated volumetric medical-language reasoning. In NeuroMosaic, a hierarchical 3D tokenizer emits fine, intermediate, and coarse tokens; atlas parcels, tumor compartments, and habitat clusters form a patient-specific graph; a task-conditioned differentiable top-k router selects a sparse set of nodes; and a molecular concept memory constrains output language to WHO-aligned relations. The authors report integrated-subtype macro-F1 of 0.827 internal and 0.784, 0.761, and 0.742 on three external cohorts, IDH, 1p/19q, and MGMT AUROCs of 0.918, 0.861, and 0.781 on UPenn-GBM, evi

Load-bearing premise

The load-bearing premise is that the comparison baseline is genuinely matched in every component except the routing mechanism—identical MRI encoders, language backbones, loss weights, training schedules, token budgets, and inference budgets—so that the 3.6-point gain is attributable to anatomy-indexed routing rather than to an under-trained or differently budgeted baseline.

Editorial extensions

If this is right

  • If the mechanism claim holds, anatomy-indexed sparse routing should generalize to other volumetric medical tasks where the discriminative signal is spatially sparse, giving the same classification gain and region-level audit trail.
  • The controlled evidence-deletion results imply that a prediction whose supporting regions can be removed without changing the answer is not actually grounded, making deletion faithfulness a practical test-time check.
  • Selective risk control at fixed coverage—risk 0.122 versus 0.168 for Retrieval MLLM and 0.194 for Generic MLLM at 80 percent coverage—means the model can abstain when evidence is incomplete rather than forcing a confident answer.
  • Missing-modality robustness numbers (macro-F1 changes of -0.018 for missing FLAIR, -0.026 for missing T1ce, and -0.055 for missing two sequences) set a concrete benchmark for how a grounded volumetric multimodal language model should degrade under realistic acquisition gaps.
  • The paper's matched-input comparison, if reproducible, establishes that the gain is attributable to routing and memory rather than to more parameters or longer context.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive extension the paper motivates but does not fully execute is stratifying routing benefit by tumor volume, compartment entropy, and available sequences; a monotonic interaction between gain and evidence sparsity would directly confirm the mechanism, while a flat interaction would suggest generic regularization.
  • If the mechanism is real, it suggests a general design principle for medical multimodal language models: give the model a sparse, anatomically typed evidence graph and supervise evidence deletion as a faithfulness objective, rather than simply adding more visual tokens.
  • The same evidence-routing architecture could be carried to other spatially structured modalities, such as whole-slide pathology or multimodal stroke imaging, where diagnostic language must be tied to regions rather than to global features.
  • For clinical deployment, the paper's claim that routing entropy, unsupported-concept rate, and calibration drift can serve as early shift monitors remains untested; that monitoring hypothesis is a natural prospective study.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents NeuroMosaic, a 3D multimodal large language model for glioma diagnosis that converts multi-sequence MRI into anatomy-indexed regional tokens, routes them through a patient-specific neuroanatomical graph, couples them with a molecular concept memory, and generates structured evidence-linked outputs. The central empirical claim is that anatomy-indexed sparse routing yields measurable gains in classification, grounding, calibration, and selective risk. On the UPenn-GBM external cohort, NeuroMosaic is reported to improve subtype macro-F1 by 3.6 percentage points over a matched Retrieval MLLM baseline (95% CI 0.018–0.054, adjusted p=0.0018), and ablations removing the anatomical graph are claimed to reduce macro-F1 by 0.030 and pointing accuracy by 0.072. The paper also reports internal macro-F1 of 0.827, external macro-F1 values of 0.784/0.761/0.742, IDH/1p/19q/MGMT AUROCs of 0.918/0.861/0.781, pointing accuracy of 0.703, and improved ECE. The manuscript includes a detailed evaluation protocol with patient-level split locking, Holm correction, DeLong tests, missing-modality robustness, and a shortcut audit. However, the central mechanism attribution is not statistically secured, and the matched-baseline comparison is not verifiable from the submitted material.

Significance. If the empirical claims held, the explicit anatomical routing and the broad reliability evaluation would represent a worthwhile step for volumetric medical vision-language reasoning. The study design has notable strengths: prespecified confirmatory endpoints, patient-level split governance, Holm-adjusted multiple comparisons, missing-sequence robustness tests, and a falsifiable interaction prediction between routing benefit and evidence sparsity. These are more careful than typical in the systems literature. However, the current evidence does not establish the central claim that anatomy-indexed routing is the cause of the reported gains. The key ablation is reported without uncertainty quantification, the baseline matching is not substantiated by released code or hyperparameter details, and the planned mechanism test (interaction with tumor volume/completeness) is not reported. These are load-bearing gaps, not presentational issues. The paper would be a useful contribution after these are addressed, but in its current form the quantitative support for the mechanism is incomplete.

major comments (4)
  1. [§5.3, §6.3, Table 5, Appendix Table A4] The central mechanism claim relies on the graph-removal ablation, but the reported -0.030 macro-F1 and -0.072 pointing-accuracy changes are single point estimates with no confidence interval, no p-value, and no n. Section 5.3 states that all inferential comparisons use patient-level stratified bootstrap with 2,000 resamples; these ablation rows are excluded from that standard. Given that the headline gain over Retrieval MLLM is only 0.036 (95% CI 0.018 to 0.054), a -0.030 effect could easily be within sampling noise across 520 patients. The same issue applies to the concept-memory factual-error change of +0.036. Without uncertainty intervals or a multiseed analysis, the statement in Section 7 that the findings 'directly attribute these gains to the proposed evidence-routing mechanism' is not statistically supported.
  2. [§5.2, §8, Reproducibility section] The paper claims that the Retrieval MLLM baseline is 'matched' in MRI encoder, language backbone, task heads, input sequences, and inference budget, isolating the anatomical graph router and concept memory as the only differing components. However, no implementation details, hyperparameter settings, token budgets, training schedules, or code are provided. The Reproducibility section says artifacts 'will be versioned' and 'will include' future deliverables, but nothing is shipped with the submission. Without verifiable matching, the 3.6-point gain over Retrieval MLLM could arise from an undertrained baseline, a different number of visual tokens, or a longer/shorter inference budget. This is a load-bearing premise for the paper's central attribution and must be substantiated with a detailed configuration table and, ideally, released checkpoints and evaluation scripts.
  3. [§1, §6.3] The Introduction previews a falsifiable mechanism test: 'Stratifying the primary endpoint by tumor volume, compartment entropy, and modality availability therefore becomes a mechanism test. A monotonic interaction between routing benefit and evidence sparsity would directly support the proposed representation.' No such stratified interaction analysis is reported in §6.3. The only subgroup statement is that differences remained within the 0.05 safety boundary, which does not address the predicted monotonic interaction. The paper's own confirmation standard is left unmet; this weakens the causal interpretation of the router's contribution.
  4. [§5.2, §6.2] The grounding claims rest on 'pointing accuracy' and 'region IoU', but the manuscript never defines how ground-truth evidence regions were established for the test cohorts. Section 3 lists possible sources (voxel masks, atlas regions, tumor compartments, report alignments), but §5.2 and §6.2 do not specify which source was used, whether expert annotation was performed, how inter-rater reliability was assessed, or how partial credit was computed. Without this operational definition, the reported pointing accuracy of 0.703 and region IoU of 0.426 are uninterpretable as evidence of anatomical grounding.
minor comments (5)
  1. [Appendix Table A1] The data-integrity table itself is a red flag: only 40 of 123 rows have confidence intervals, 21 have adjusted p-values, and 30 contain placeholder keys. The text claims every numeric statement maps to an export row, but the key ablation rows in Table A4 are among the missing-uncertainty rows. Please clarify whether this appendix is a data statement or part of the intended output; as written, it undermines the paper's statistical credibility.
  2. [Figures A1–A10] Appendix Figures A1–A10 are captioned 'Extended data-derived diagnostic 1' through '10' but contain no visible image or axes. These appear to be placeholder figures. They should be replaced with actual plots or removed.
  3. [Appendix Table A5] The 'Narrative placeholder reconciliation' table lists items such as 'B_BOOT bootstrap resamples 2,000' and 'N_EXT_A patients 520'. This suggests the main text may still contain placeholder macros. Please verify that all placeholders are resolved and that all numbers in the text (including CIs and p-values) appear explicitly and consistently.
  4. [References] Several bibliography entries have placeholder source types, e.g., 'conference-paper' for refs [10] and [34], and some entries lack journal or arXiv identifiers. These should be consistently formatted.
  5. [Discussion] The paper lacks a dedicated Limitations section. Given the reliance on public cohorts, the single held-out institution sample size (N=184), and the self-referential nature of the deletion-faithfulness metric, a candid limitations paragraph would aid the reader's assessment.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; central claims rest on external cohorts, expert-annotated evidence, and architectural ablations.

full rationale

The paper is an empirical systems paper, and its central claims are supported by comparisons to external cohorts (UPenn-GBM, UCSF-PDGM, held-out institution), expert-annotated evidence pointing accuracy, and architectural ablations (removing the anatomical graph or concept memory). These are not derived from the model's own outputs by construction: the external macro-F1 and pointing accuracy use held-out labels and expert evidence annotations, and the ablations change the model architecture rather than re-fitting the target metric. The deletion faithfulness test removes the model's own top-ranked evidence nodes, but it is presented as a self-consistency check alongside externally grounded pointing accuracy, not as the sole evidence for grounding. No load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations appear. The paper does not fit a parameter to the primary endpoint and then call the result a prediction. The main weaknesses are statistical (e.g., missing confidence intervals for the graph-removal ablation) and reproducibility (no code or hyperparameter details for the matched baselines), but these are validity concerns, not circularity. Therefore the derivation chain is self-contained and non-circular.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The paper's central claims rest on a large set of engineering choices that are either fitted on validation data or chosen by the authors: loss weights, router sparsity k, patch resolutions, calibration parameters, and the details of the WHO concept memory. The most consequential assumption is that the matched baselines are truly matched, which is stated but not evidenced. No code, data, or model artifacts are shipped, so none of these free parameters are pinned down, and the external-cohort numbers cannot be independently checked.

free parameters (4)
  • Router top-k (sparsity k) = chosen on validation
    The router sparsity k is selected through a risk-performance curve on validation data and frozen before external evaluation (Section 4.3). This is a free parameter that directly controls how much evidence the model uses.
  • Loss weights lambda_pre, lambda_seg, lambda_cls, lambda_gen, lambda_ground, lambda_cal = selected on internal validation cohort
    The joint objective L (Section 3) has six loss terms whose weights are chosen on the internal validation cohort and frozen before external evaluation. These weights are not derived from theory and can absorb task-priority preferences.
  • Patch resolution for fine/middle/coarse streams = not stated numerically
    The multi-resolution tokenizer uses three spatial resolutions, but the actual voxel sizes are not reported. This is a free architectural choice that affects how small enhancing foci survive.
  • Calibration temperature and conformal/selective-risk parameters = not stated
    Stage IV fits temperature, conformal, and selective-risk components on a dedicated calibration set. These are standard fitted parameters, but they are not reported, so the claimed ECE improvement cannot be separated from the temperature fit.
assumptions (4)
  • domain assumption Patient-level split locking is correctly implemented
    The entire external validation strategy assumes that no external-cohort information leaks into model or threshold selection. The paper states this is done, but provides no certificate or audit trail beyond prose.
  • domain assumption The expert rules used to align evidence regions with report phrases are valid ground truth
    Evidence supervision and pointing accuracy both rely on 'expert rules' aligning voxel masks to report phrases (Section 3). These rules are not described in detail, and they also generate the training labels, so their reliability is an unstated assumption.
  • domain assumption The matched baselines (3D discriminative, late fusion, generic MLLM, retrieval MLLM) are implemented with equivalent capacity, training budget, and tuning effort
    The 3.6-point gain over Retrieval MLLM is the headline result, but the paper does not specify the baselines' parameter counts, training epochs, or tuning procedure beyond a table of 'controlled axes'. This is an assumption that cannot be verified from the text.
  • domain assumption The WHO/EANO concept memory is 'frozen' and does not encode the authors' preferred outputs
    The concept memory is derived from 'a frozen version of WHO and guideline terminology' (Section 4.4). This is a reasonable source, but the exact version and the admissibility relations are not released, so the memory could be a hidden prior that biases outputs toward certain diagnoses.
invented entities (2)
  • Anatomy-indexed sparse routing (patient-specific graph)
    purpose: Selects sparse subsets of anatomical nodes for cross-attention, giving the model a structural prior that connects diagnoses to MRI regions.
    This is the paper's central architectural contribution, but its benefit is only measured through the model's own ablations and deletion tests, not through an external benchmark. It is a new internal mechanism, not an independently measurable entity.
  • Molecular concept memory
    purpose: Stores versioned WHO-aligned diagnostic concepts and admissible relations, binding imaging evidence to molecular semantics.
    The memory is described as a component, but its content and version are not released. Its effect is measured only through the factual-error-rate delta of 0.036, which is an internal comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives." pith.science (2026). https://pith.science/paper/DXMGI5FL

@misc{pith2026260803187,
  author       = {Pith},
  title        = {Pith review of: NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DXMGI5FL}},
  note         = {Machine review of arXiv:2608.03187}
}
read the original abstract

Multimodal medical large language models remain structurally weak for neuro-oncology because volumetric evidence is compressed into generic visual tokens and diagnostic conclusions often lack an auditable link to MRI regions. We present NeuroMosaic, a 3D multimodal language model that converts multi-sequence brain MRI into anatomy-indexed regional tokens, aligns them with clinical narrative and molecular concepts, and generates evidence-linked outputs. The architecture combines a multi-resolution volumetric tokenizer, a neuroanatomical graph router, a molecular concept memory, and selective risk control. Across four glioma cohorts, NeuroMosaic achieved an internal subtype macro-F1 of 0.827 and external macro-F1 values of 0.784, 0.761, and 0.742. On UPenn-GBM, it improved over the strongest matched-input baseline by 3.6 percentage points (95% CI: 1.8 to 5.4, adjusted p = 0.0018), with IDH, 1p/19q, and MGMT AUROCs of 0.918, 0.861, and 0.781. Evidence pointing accuracy reached 0.703, and targeted evidence deletion reduced correct-answer probability by 0.187, compared with 0.046 for random deletion. These results establish anatomy-indexed routing as a measurable mechanism for accurate, grounded, and calibrated volumetric medical-language reasoning.

Figures

Figures reproduced from arXiv: 2608.03187 by the authors.

Figure 1
Figure 1. NeuroMosaic system overview. Multi-sequence MRI and clinical text are encoded separately, routed through a patient-specific anatomical graph, coupled to a molecular concept memory, and decoded into structured, evidence-linked outputs. region, mask the supporting sequence, randomize the anatom￾ical edges, or replace the retrieved concept. A contribution survives review only when the observed performance change follow… view at source ↗
Figure 2
Figure 2. Anatomical token router. Multi-scale MRI patches are pooled into atlas and tumor-compartment nodes, connected through a neuroanatomical graph, and sparsely selected by task queries. 4.4 4.4 Clinical text encoder and molecular con￾cept memory Clinical text is segmented into indication, prior treatment, symptoms, prior imaging comparison, pathology, and molecu￾lar evidence. Each span is encoded with temporal and prove… view at source ↗
Figure 3
Figure 3. Four-stage training curriculum and loss stack. The final calibration stage is isolated from representation learning to preserve an auditable risk-control boundary. selected node set and attention mass are exported with the answer. Router sparsity k is chosen on validation data through a risk-performance curve, then frozen. The model therefore cannot increase its test-time evidence budget when a case is difficult. Co… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Evaluation protocol. Split locking precedes task evaluation, shift testing, blinded review, and paired statistical analysis [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Patient-level subtype macro-F1 across the internal and three external cohorts. Error bars denote 95% confidence intervals [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Paired ablation, missing-sequence, and subgroup effects. Error bars denote 95% confidence intervals where available. Controlled graph and deletion tests directly attribute these gains to the proposed evidence-routing mechanism. Author Contributions Conceptualization, Y…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 13 canonical work pages

  1. [1]

    Krumholz, Jure Leskovec, Eric J

    Michael Moor, Oishi Banerjee, Zahra Shakeri Hossein Abad, Harlan M. Krumholz, Jure Leskovec, Eric J. Topol, et al.. Foundation models for generalist medical artificial in- telligence. Nature, 2023. https://doi.org/10.1038/ s41586-023-05881-4

    2023

  2. [2]

    Falcone, Pranav Rajpurkar, Eric J

    Julián Acosta, Guido J. Falcone, Pranav Rajpurkar, Eric J. Topol. Multimodal biomedical AI. Nature Medicine, 2022. https://doi.org/10.1038/ s41591-022-01981-2

    work page 2022

  3. [3]

    Towards Generalist Biomedical AI

    Tao Tu, Shekoofeh Azizi, Danny Driess, Mike Schaekermann, Mohamed Amin, Pi-Chuan Chang, et al.. Towards Generalist Biomedical AI. NEJM AI, 2024. https://doi.org/10. 1056/aioa2300138

    work page 2024

  4. [4]

    LLaV A-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, et al.. LLaV A-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day. arXiv (Cornell University), 2023.https://doi.org/ 10.48550/arxiv.2306.00890

    arXiv doi:10.48550/arxiv.2306.00890 2023

  5. [5]

    Med-Flamingo: a Multimodal Medical Few-shot Learner

    Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Cyril Zakka, Yash Dalmia, et al.. Med-Flamingo: a Multimodal Medical Few-shot Learner. arXiv (Cornell University), 2023. https://doi.org/10.48550/arxiv.2307.15189

    arXiv doi:10.48550/arxiv.2307.15189 2023

  6. [6]

    BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Xu, Hanwen, Jaspreet Bagga, Robert Tinn, et al.. BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv (Cornell University), 2023. https://doi.org/10.48550/arxiv.2303.00915

    arXiv doi:10.48550/arxiv.2303.00915 2023

  7. [7]

    Castro, Anton Schwaighofer, Stephanie L

    Benedikt Boecking, Naoto Usuyama, Shruthi Bannur, Daniel C. Castro, Anton Schwaighofer, Stephanie L. Hyland, et al.. Making the Most of Text Semantics to Improve Biomedical Vision-Language Processing. Lecture notes in computer science, 2022. https://doi.org/10.1007/ 978-3-031-20059-5_1

    work page 2022

  8. [8]

    Manning, Curtis P

    Yuhao Zhang, Hang Jiang, Yasuhide Miura, Christopher D. Manning, Curtis P. Langlotz. Contrastive Learning of Med- ical Visual Representations from Paired Images and Text. arXiv (Cornell University), 2020. https://doi.org/10. 48550/arxiv.2010.00747

    arXiv doi:10.48550/arxiv.2010.00747 2020

Show all 37 references
  1. [9]

    Lungren, Ser- ena Yeung

    Shih-Cheng Huang, Liyue Shen, Matthew P. Lungren, Ser- ena Yeung. GLoRIA: A Multimodal Global-Local Represen- tation Learning Framework for Label-efficient Medical Im- age Recognition. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 2021. https://doi.org/10. ...

    2021

  2. [10]

    Agarwal, Jimeng Sun

    Zifeng Wang, Zhenbang Wu, D.C. Agarwal, Jimeng Sun. Med- CLIP: Contrastive Learning from Unpaired Medical Images and Text. conference-paper, 2022. https://doi.org/10. 18653/v1/2022.emnlp-main.256

    2022

  3. [11]

    Learning Transfer- able Visual Models From Natural Language Supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, et al.. Learning Transfer- able Visual Models From Natural Language Supervision. arXiv (Cornell University), 2021. https://doi.org/10. 48550/arxiv.2103.00020

    2021 doi arXiv

  4. [12]

    Flamingo: a Visual Language Model for Few-Shot Learning

    Alayrac, Jean-Baptiste, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, et al.. Flamingo: a Visual Language Model for Few-Shot Learning. arXiv (Cornell Uni- versity), 2022. https://doi.org/10.48550/arxiv. 2204.14198

    2022 doi arXiv

  5. [13]

    Junnan Li, Dongxu Li, Silvio Savarese, Steven C. H. Hoi. BLIP- 2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv (Cor- nell University), 2023. https://doi.org/10.48550/ arxiv.2301.12597

    2023 doi arXiv

  6. [14]

    Visual Instruction Tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, Yong Jae Lee. Visual Instruction Tuning. arXiv (Cornell University), 2023. https: //doi.org/10.48550/arxiv.2304.08485

    2023 doi arXiv

  7. [15]

    Gomez, et al

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, et al.. Attention Is All You Need. Advances in Neural Information Processing Systems, 2017. https://arxiv.org/abs/1706.03762

    2017 arXiv

  8. [16]

    The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)

    Bjoern Menze, András Jakab, Stefan Bauer, Jayashree Kalpathy-Cramer, Keyvan Farahani, Justin Kirby, et al.. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Transactions on Medical Imaging, 2014. https://doi.org/10.1109/tmi.2014.2377694

    2014

  9. [17]

    Advancing The Cancer Genome Atlas glioma MRI collections with expert seg- mentation labels and radiomic features

    Spyridon Bakas, Hamed Akbari, Aristeidis Sotiras, Michel Bilello, Martin Rozycki, Justin Kirby, et al.. Advancing The Cancer Genome Atlas glioma MRI collections with expert seg- mentation labels and radiomic features. Scientific Data, 2017. https://doi.org/10.1038/sdata.2017.117

    2017 doi

  10. [18]

    The Uni- versity of Pennsylvania glioblastoma (UPenn-GBM) co- hort: advanced MRI, clinical, genomics, & radiomics

    Spyridon Bakas, Chiharu Sako, Hamed Akbari, Michel Bilello, Aristeidis Sotiras, Gaurav Shukla, et al.. The Uni- versity of Pennsylvania glioblastoma (UPenn-GBM) co- hort: advanced MRI, clinical, genomics, & radiomics. Scientific Data, 2022. https://doi.org/10.1038/ s41597-022-01560-7

    2022

  11. [19]

    Rudie, Andreas M

    Evan Calabrese, Javier Villanueva-Meyer, Jeffrey D. Rudie, Andreas M. Rauschecker, Ujjwal Baid, Spyridon Bakas, et al.. The University of California San Francisco Preoperative Diffuse Glioma MRI (UCSF-PDGM) Dataset. arXiv (Cor- nell University), 2021. https://doi.org/10.48550/...

    2021 doi arXiv

  12. [20]

    Louis, Arie Perry, Pieter Wesseling, Daniel J

    David N. Louis, Arie Perry, Pieter Wesseling, Daniel J. Brat, Ian A. Cree, Dominique Figarella-Branger, et al.. The 2021 WHO Classification of Tumors of the Central Nervous System: a summary. Neuro-Oncology, 2021. https://doi.org/ 10.1093/neuonc/noab106

    2021 doi

  13. [21]

    van den Bent, Matthias Preusser, Émilie Le Rhun, Jörg C

    Michael Weller, Martin J. van den Bent, Matthias Preusser, Émilie Le Rhun, Jörg C. Tonn, Giuseppe Minniti, et al.. EANO guidelines on the diagnosis and treatment of diffuse gliomas of adulthood. Nature Reviews Clinical Oncology, 2020. https: //doi.org/10.1038/s41571-020-00447-z

    2020 doi

  14. [22]

    TransBTS: Multimodal Brain Tu- mor Segmentation Using Transformer

    Wenxuan Wang, Chen Chen, Meng Ding, Hong Yu, Sen Zha, Jiangyun Li. TransBTS: Multimodal Brain Tu- mor Segmentation Using Transformer. Lecture notes in computer science, 2021. https://doi.org/10.1007/ 978-3-030-87193-2_11

    2021

  15. [23]

    Roth, Daguang Xu

    Ali Hatamizadeh, Vishwesh Nath, Yucheng Tang, Dong Yang, Holger R. Roth, Daguang Xu. Swin UNETR: Swin Trans- formers for Semantic Segmentation of Brain Tumors in MRI Images. Lecture notes in computer science, 2022. https: //doi.org/10.1007/978-3-031-08999-2_22

    2022 doi

  16. [24]

    Jaeger, Simon A

    Fabian Isensee, Paul F. Jaeger, Simon A. A. Kohl, Jens Pe- tersen, Klaus H. Maier-Hein. nnU-Net: a self-configuring method for deep learning-based biomedical image segmen- tation. Nature Methods, 2020. https://doi.org/10. 1038/s41592-020-01008-z

    2020

  17. [25]

    Segment anything in medical images

    Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, Bo Wang. Segment anything in medical images. Nature Communications, 2024. https://doi.org/10.1038/ s41467-024-44824-z

    2024

  18. [26]

    Weinberger

    Chuan Guo, Geoff Pleiss, Yu Sun, Kilian Q. Weinberger. On Calibration of Modern Neural Networks. arXiv (Cornell Uni- versity), 2017. https://doi.org/10.48550/arxiv. 1706.04599

    2017 doi arXiv

  19. [27]

    Detecting hallucinations in large language models using seman- tic entropy

    Sebastian Farquhar, Jannik Kossen, Lorenz Kuhn, Yarin Gal. Detecting hallucinations in large language models using seman- tic entropy. Nature, 2024. https://doi.org/10.1038/ s41586-024-07421-0

    2024

  20. [28]

    Confor- mal prediction: A unified review of theory and new chal- lenges

    Matteo Fontana, Gianluca Zeni, Simone Vantini. Confor- mal prediction: A unified review of theory and new chal- lenges. Bernoulli, 2022. https://doi.org/10.3150/ 21-bej1447

    2022

  21. [29]

    John Mongan, Linda Moy, Charles E. Kahn. Checklist for Arti- ficial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. Radiology Artificial Intelligence, 2020. https://doi.org/10.1148/ryai.2020200029

    2020 doi

  22. [30]

    Collins, Karel G.M

    Professor Gary S. Collins, Karel G.M. Moons, Paula Dhiman, Richard D Riley, Andrew L. Beam, Ben Van Calster, et al.. TRIPOD+AI statement: updated guidance for reporting clin- ical prediction models that use regression or machine learn- ing methods. BMJ, 2024. https://doi.org/1...

    2024

  23. [31]

    Chen, Judy J

    Richard J. Chen, Judy J. Wang, Drew F. K. Williamson, Tiffany Chen, Jana Lipková, Ming Y . Lu, et al.. Algorithmic fair- ness in artificial intelligence for medicine and healthcare. Na- ture Biomedical Engineering, 2023. https://doi.org/ 10.1038/s41551-023-01056-8

    2023 doi

  24. [32]

    Evaluation and mitigation of the limitations of large language models in clinical decision-making

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, et al.. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine, 2024. https: //doi.org/10.1038/s41591-024-03097-1

    2024 doi

  25. [33]

    Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, et al.. Retrieval- Augmented Generation for Knowledge-Intensive NLP Tasks. UCL Discovery (University College London), 2020. https: //discovery.ucl.ac.uk/id/eprint/10100504/

    2020

  26. [34]

    Gen- erating Radiology Reports via Memory-driven Transformer

    Zhihong Chen, Yan Song, Tsung-Hui Chang, Xiang Wan. Gen- erating Radiology Reports via Memory-driven Transformer. conference-paper, 2020. https://doi.org/10.18653/ v1/2020.emnlp-main.112

    2020

  27. [35]

    Alistair E. W. Johnson, Tom Pollard, Seth J. Berkowitz, Nathaniel R. Greenbaum, Matthew P. Lungren, Chih-Ying Deng, et al.. MIMIC-CXR: A large publicly available database of labeled chest radiographs. arXiv (Cornell University), 2019. https://arxiv.org/pdf/1901.07042.pdf

    2019 arXiv

  28. [36]

    RadGraph: Extract- ing Clinical Entities and Relations from Radiology Reports

    Saahil Jain, Ashwin Agrawal, Adriel Saporta, Steven Qh Truong, Du Nguyen Duong, Tan Bui, et al.. RadGraph: Extract- ing Clinical Entities and Relations from Radiology Reports. arXiv (Cornell University), 2021. https://doi.org/10. 48550/arxiv.2106.14463

    2021 doi arXiv

  29. [37]

    Ng, Matthew P

    Akshay Smit, Saahil Jain, Pranav Rajpurkar, Anuj Pareek, An- drew Y . Ng, Matthew P. Lungren. CheXbert: Combining Auto- matic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERT. arXiv (Cornell University), 2020. https://doi.org/10.48550/arxiv.200...

    2020 doi arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.