Pith. sign in

REVIEW 3 major objections 5 minor 31 references

ORCA compresses 3D CT visual tokens 64x and writes each region's centroid back into the token, preserving anatomical information better than grid averaging or token pruning at the same budget.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:39 UTC pith:BY4FQ3SN

load-bearing objection ORCA is a solid, well-ablated training-free 3D CT token compressor; the main caveat is that its largest low-budget location gains may partly reflect shared organ-mask signal between method and evaluation. the 3 major comments →

arxiv 2608.00345 v1 pith:BY4FQ3SN submitted 2026-07-31 cs.CV cs.AI

ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression

Renjie Liang , Zijian Xu , Jinqian Pan , Chengkun Sun , Zhengkang Fan , Shawn Li , You Qin , Mei Liu
show 1 more author
Jie Xu
This is my paper
classification cs.CV cs.AI
keywords token compression3D CTvision-language modelstraining-freeorgan guidancecentroid encodingattribute probingvolumetric medical imaging
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that token compression for 3D CT should be judged by how much anatomical evidence survives for the downstream reader, not by reconstruction fidelity. It presents ORCA, a training-free compressor that merges spatially connected, feature-similar token regions under soft organ-mask guidance, then appends a sinusoidal encoding of each region's 3D centroid to every output token. Across two datasets (chest and abdomen CT) and five encoders with different pretraining objectives, ORCA beats grid averaging, pruning, and feature-merging baselines at matched token budgets, with the largest gains on location and density. At 64x compression it stays close to the uncompressed token reference, while shrinking the language-model context 64x, its KV-cache 50x, and prefill latency 31x.

Core claim

At matched token budgets, ORCA preserves more attribute information than grid-average pooling, slice pooling, diversity pruning, attention-based pruning, and similarity-based merging, consistently across five encoders and two CT datasets. The effect is most striking for location attributes, which jump from near-chance (R2 about 0.25 for pooling baselines) to about 0.68 once each region's centroid is written into the token value; ORCA reaches 0.677 at the 216-token budget where the best pooled baseline scores 0.271. The paper's ablations show that centroid encoding helps every baseline on location, while ORCA's organ-guided aggregation adds a further, smaller gain on density and texture; toge

What carries the argument

Ward-linkage connected-region aggregation: ORCA starts with one region per token and repeatedly merges the adjacent pair with the smallest increase in within-region distortion, where the merge feature is the visual embedding concatenated with a variance-scaled organ-coverage vector. The organ mask thus biases which neighbors merge without dictating boundaries. After merging, each output token is the mean of its region's original visual embeddings plus a sinusoidal (Fourier) encoding of the region's normalized 3D centroid, restoring spatial layout that the non-grid token order no longer carries.

Load-bearing premise

The location, size, and density labels are computed from organ-segmentation masks, and ORCA's merging is guided by organ masks from the same source; the evaluation never shows these two uses of the masks are independent, and the baselines do not receive the same organ features.

What would settle it

Run ORCA and the baselines on attribute labels that come from independent radiologist annotations rather than from segmentation-mask measurements (e.g., manually measured aortic diameter or heart position), and check whether the location and size margins over a grid-average-plus-centroid baseline persist. A secondary check: disable organ guidance entirely and evaluate on a mask-free dataset.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • ORCA can replace grid-average pooling as a drop-in, training-free compressor at any encoder-output interface, with an adjustable token budget and no model surgery or text query.
  • Because the LLM cost scales with the number of visual tokens, the 64x context reduction directly cuts prefill latency and KV-cache memory, enabling 8B-parameter models to run on 24GB GPUs.
  • The encoder/compressor split implies that a strong encoder's information can be squandered by an anatomy-blind compressor, so compression deserves to be a first-class variable in VLM evaluation.
  • The position-encoding result indicates that spatial coordinates should be re-injected by any volumetric token compressor, since the paper shows this benefits every tested baseline.
  • On reconstruction-trained encoders, organ-guided aggregation recovers substantial disease-signal headroom (a 0.097 AUROC lift), suggesting the biggest compressor gains are where the encoder was never language-supervised.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The largest reported gains concentrate on attributes whose labels are computed from the same kind of organ-segmentation masks that supply ORCA's guidance; a fair test with independently annotated labels could shrink those margins, especially against a strong 'grid average + centroid' baseline.
  • The paper's lambda sweep (flat from zero mask weight to the organ-dominated limit) implies the mask is replaceable by any coarse grouping signal, such as an atlas or unsupervised patch clustering, which would make ORCA usable where organ segmentation is unavailable.
  • A practical takeaway not stated in the paper: since centroid encoding nearly closes the location gap by itself at 216 tokens, future work on compression should focus on content-adaptive aggregation for density and texture, where the remaining gap lies.
  • The 'encoder ceiling, compressor survival' framing suggests a model-selection rule: spend a better compressor on reconstruction-trained encoders, and a better encoder on language-supervised ones.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ORCA, a training-free visual-token compressor for 3D CT volumes. ORCA starts from a dense encoder token grid, repeatedly merges spatially adjacent regions using Ward linkage with a soft organ-mask term, and appends a sinusoidal encoding of each region’s 3D centroid to the aggregated visual feature. The token budget B is a free parameter, and the method is designed as a drop-in replacement for grid-average pooling at the encoder–LLM interface. The authors evaluate ORCA on CT-RATE and Merlin across five encoders, using attribute-probing (disease, size, density, location, texture), measurement VQA, and report generation. At matched budgets they report consistent gains over grid average, ToMe, DivPrune, MedPruner-DINS, slice pooling, and MedRegion-CT pooling, with especially large gains on the location family. Ablations show the centroid encoding and feature-similarity aggregation each contribute, and that the organ-guidance weight is robust over a wide range.

Significance. If the reported results hold, ORCA is a genuinely useful contribution: it is training-free, budget-adjustable, requires no model surgery, and appears to preserve substantially more localized anatomical information than grid-average or pruning baselines at 8–64x token reduction. The paper is also commendably thorough: it includes component ablations, a centroid-equalization analysis, budget sweeps, held-out validation, per-attribute tables, and released code. The central mechanism is simple and well explained. The main reservation is evaluation confound: the same TotalSegmentator masks that inform ORCA’s merge cost are also used to derive the location/size/density/texture probing labels, so the headline gains on location may partly reflect input/label proximity rather than superior compression. Because the missing control is an additional experiment rather than a conceptual error, the central claim is defensible and the work warrants revision.

major comments (3)
  1. [§A.2, Table 14; §Method: Organ guidance] The evaluation labels for size, density, location, and texture are computed directly from TotalSegmentator masks and HU values (Table 14, Supp. A.2), while ORCA’s merge cost consumes the same TotalSegmentator coverage vectors (§Method: Organ guidance). This is a direct input–label overlap. The baselines do not receive mask features, so the comparison is not controlled for the information that the masks provide. The issue is most severe for the location family: Table 3 shows ORCA at 0.622/0.677 vs 0.247/0.271 for grid average at B=27/216, and the location targets in Table 14 are organ-centroid positions. Please add mask-free ORCA runs (λ=0 or 'aggregate+centroid') at B=27 and B=8, where the location advantage is largest; currently Fig. 5 and Table 7 report such runs only at B=216/256/512.
  2. [Tables 3, 10; §B.2] The claim that ORCA’s advantage 'comes from adaptive, organ-aligned aggregation rather than from position' is only partially supported. After the centroid encoding is equalized across baselines (Table 10), ORCA’s location lead at B=216 shrinks to 0.677 vs 0.668–0.674 for centroid-augmented grid average/DivPrune, which is within the reported seed noise (std ≈ 0.013). The large low-budget location lead at B=27 is exactly the regime where the organ-mask confound is unresolved, since no λ=0 aggregate+centroid row is reported at B=27. A compact control table with λ=0 and aggregate+centroid at B=27 and B=8, and ideally with masks provided to a grid-average baseline, would either close or substantiate the concern.
  3. [§A.3, Table 7] The 'uncompressed + centroid' reference rows in Table 7 are read out with a higher-capacity probe than the compressor rows, as stated in A.3. This means the claim that ORCA 'stays within a small margin of the uncompressed tokens' is not a head-to-head comparison on equal probe capacity. The direction of the bias favors the reference, so it does not invalidate ORCA’s advantage over other compressors, but the margin to the uncompressed ceiling should be re-quantified with a shared probe capacity, or the higher-capacity probe should be applied to all rows.
minor comments (5)
  1. [Abstract; §Efficiency, Table 8] The abstract’s '31x faster to process each volume' refers to prefill latency at B=216 (371.5 ms to 11.8 ms on B200), not end-to-end processing. Table 8 shows end-to-end report time changes only modestly. Please qualify the wording.
  2. [Fig. 2 and Fig. 5 captions] The budget axes are not linear and Fig. 2 omits grid average at B=125 without explanation in the caption. A note that the x-axis is a categorical budget grid would help avoid misinterpretation.
  3. [Eq. (5)] The centroid encoding formula is written compactly; please define k=0,…,F−1 explicitly and clarify that F=4 yields 24 dimensions for three axes (2F×3).
  4. [§A.4, MedRegion-CT] MedRegion-CT is reported at its natural count ̅N=549, not at the matched budgets; this is disclosed, but the abstract’s 'at matched token budgets' phrasing should be qualified to note that organ-pooling baseline is compared at its fixed operating point.
  5. [Table 6, Noise row] The noise-token control is a useful sanity check, but it is only shown for B=8. Reporting it at B=216 as well would strengthen the interpretation of the generation results.

Circularity Check

2 steps flagged

Location 'prediction' reads back ORCA's own centroid input, and the attribute benchmark labels are derived from the same TotalSegmentator masks ORCA consumes.

specific steps
  1. self definitional [Method — Centroid position encoding (z_j formula); Table 14 (location); Supp. A.2]
    "For each region R_j, it computes the normalized 3D centroid c_j = (1/|R_j|) sum_{i in R_j} q_i. It then appends a sinusoidal encoding of this centroid ... z_j = [mu_j; s*phi(c_j)]. ... Table 14: heart_x normalized left-right position of the heart centroid."

    With organ guidance, regions are steered to follow organs (Fig. 3: 'ORCA aggregates tokens into spatially contiguous regions that follow the organ contours'), so for a heart-aligned region c_j.x is approximately the heart centroid. Table 14 defines the location label heart_x as exactly that centroid; the output token z_j contains sin/cos of c_j.x. The probe can therefore decode the answer from ORCA's own auxiliary input. The paper concedes 'the location gain comes from recording where each region sits.' The centroid-equalized ablation narrows the gap but does not remove this read-back for organ-aligned regions, so the headline location margin (0.677 vs 0.247 at B=216) is substantially by construction.

  2. other [Supp. A.2 / A.1; Method — Organ guidance]
    "The benchmark comprises a set of image-derived anatomical attributes whose reference values are computed directly from each CT volume, its corresponding segmentation masks, and Hounsfield unit (HU) values. ... ORCA uses organ masks deciding which regions to merge. In our experiments, these masks are produced by TotalSegmentator. ... organ masks from TotalSegmentator."

    The same TotalSegmentator masks generate the gold attributes (organ volumes, organ centroids, HU within organ masks) and are fed to ORCA as m_i in its merge cost; baselines are not given this signal. The attribute-probing benchmark is a self-cited preprint by the first author (Liang 2026), and its complete construction is not reproduced here. The lambda=0 ablation (Fig. 5) is reported only at loose budgets (B=216/256), so the tight-budget gains where ORCA's largest location advantages appear are never tested without mask input, and no independent annotation replaces the mask-derived labels.

full rationale

The core compressor (connected Ward merging plus centroid re-injection) is a concrete, training-free algorithm and is not circular in itself; the content-family and report-generation results retain independent support, especially the centroid-equalized and lambda=0 ablations at looser budgets and the held-out test. However, the evaluation chain is partially circular. The location label is an organ centroid from TotalSegmentator masks, and ORCA appends a sinusoidal encoding of each region's centroid, with regions steered by the same masks, so the probe can read the answer back from ORCA's auxiliary input. More broadly, the size/density/location/texture benchmark derives its ground truth from the same masks ORCA consumes, so the baselines are measured on a harder task. These issues do not void the method but mean the 'consistent improvement' claim is partly confounded; the missing lambda=0 results at B=27/B=8 leave the tight-budget location advantage under-tested. Score 5 reflects partial, family-specific circularity rather than a fully forced derivation.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central method introduces no new physical or mathematical entities; it introduces two design parameters (lambda, F, s) and relies on an external segmentation model. The main ledger concern is the input/label overlap between organ-guidance masks and the mask-derived evaluation targets.

free parameters (3)
  • lambda (organ-guidance weight) = 0.5 (COLIPRI), 2 (Merlin)
    Scales organ-coverage features relative to visual features in the merge cost; chosen from the sweep in Fig. 5 and then fixed per dataset.
  • F (centroid encoding frequencies) = 4
    Number of sinusoidal frequencies for the centroid position encoding; robustness claimed in Appendix B.3.
  • s (centroid encoding scale) = 2
    Multiplier applied to the position vector before concatenation; fixed globally.
axioms (5)
  • standard math Ward linkage with 6-neighbor connectivity produces a valid hierarchical merge tree
    Relies on Ward Jr. (1963) agglomerative clustering; correct but not re-proven in the text.
  • domain assumption TotalSegmentator organ masks are accurate and available for every test volume
    Used to build the organ-coverage vector m_i (Method, 'Organ guidance'; A.1). Inaccurate or missing masks would corrupt the merge costs and the reported gains.
  • domain assumption Probing readouts faithfully measure information available to downstream generation tasks
    Core evaluation tool; the paper validates the probe on a held-out split but does not prove probe rankings transfer to every generation setting.
  • domain assumption Sinusoidal centroid encoding makes region position readable by the downstream model
    Empirical claim supported by the ablation (Table 7); no formal guarantee is provided.
  • ad hoc to paper The segmentation masks used to derive evaluation labels are independent of the mask signal ORCA receives
    Table 14 derives size/density/location/texture labels from segmentation masks, while ORCA's organ coverage comes from TotalSegmentator masks (A.1). The paper does not establish that these mask sets are disjoint or that baselines are not disadvantaged by lacking this input.

pith-pipeline@v1.3.0-alltime-deepseek · 26572 in / 12478 out tokens · 120359 ms · 2026-08-04T00:39:51.551006+00:00 · methodology

0 comments
read the original abstract

A 3D CT scan entering a vision-language model produces a long sequence of visual tokens, often thousands to tens of thousands per volume, and this sequence must be compressed before a language model can consume it. Token compression is well studied in general vision, but little of it targets 3D CT specifically. A common baseline is grid average, which pools regular grid cells and can blend distinct anatomy, lesion, and air into one token. We present \textbf{ORCA} (ORgan-Centroid Aggregation), a token compressor for 3D CT. It merges adjacent tokens with organ guidance and adds a sinusoidal encoding of each region's centroid to preserve spatial layout. This preserves the anatomical information a downstream model needs. ORCA is training-free and plug-and-play, producing an adjustable token set without any model change or text query. We evaluate it across two datasets (CT-RATE and Merlin) and five encoders. The evaluation spans two task types: attribute prediction over five families (size, density, location, texture, and disease) and text generation (visual question answering and report generation). At matched token budgets, ORCA improves consistently over existing compression methods. It shrinks the visual context $64\times$ and its KV-cache $50\times$, and is $31\times$ faster to process each volume. Code released at https://github.com/renjie-liang/ORCA-3DCT.

Figures

Figures reproduced from arXiv: 2608.00345 by Chengkun Sun, Jie Xu, Jinqian Pan, Mei Liu, Renjie Liang, Shawn Li, You Qin, Zhengkang Fan, Zijian Xu.

Figure 1
Figure 1. Figure 1: ORCA overview. A frozen 3D encoder turns a CT volume into dense visual tokens. Grid average pools them over fixed [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Budget curve on COLIPRI. Probe R2 versus token budget. Each measured budget is an equal slot; the axis is not linear in B. Bands are ±1 sd over three seeds. Grid average is ratio-based and is absent at the non-cubic budget 125 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Representative measurement VQA questions and [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Case study on one CT volume (COLIPRI, B=27) in three anatomical views. The first column is the CT scan with lung, heart, and spine contours; the remaining columns overlay each compressor’s token boundaries on the embedding-norm heatmap, and the bottom row recolors the same partitions so that each token receives one color. ORCA aggregates tokens into spatially contiguous regions that follow the organ contou… view at source ↗
Figure 6
Figure 6. Figure 6: VQA accuracy across token budgets on CT-RATE [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Probing budget curves on the Merlin encoders, [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-epoch VQA accuracy across the two training stages on CT-RATE/COLIPRI. Rows are token budgets, columns [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 16 linked inside Pith

  1. [4]

    Cao, W.; Zhang, J.; Shui, Z.; Wang, S.; Chen, Z.; Li, X.; Lu, L.; Ye, X.; Liang, T.; Zhang, Q.; and Zhang, L

    Curriculum-Driven 3D CT Report Generation via Language-Free Visual Graft- ing and Zone-Constrained Compression.arXiv preprint arXiv:2603.23308. Cao, W.; Zhang, J.; Shui, Z.; Wang, S.; Chen, Z.; Li, X.; Lu, L.; Ye, X.; Liang, T.; Zhang, Q.; and Zhang, L

    arXiv

  2. [6]

    Davenport, M

    An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision- Language Models.arXiv preprint arXiv:2403.06764. Davenport, M. S.; Fruscello, T.; Chatfield, M.; Weinstein, S.; Sensakovic, W. F.; and Larson, D. B

    Pith/arXiv arXiv

  3. [8]

    InInter- national Conference on Learning Representations (ICLR)

    Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models. InInter- national Conference on Learning Representations (ICLR). ArXiv:2603.25155. Hamamci, I. E.; Er, S.; and Menze, B. H

    arXiv

  4. [9]

    InMedical Image Computing and Computer As- sisted Intervention – MICCAI 2024, volume 15012 ofLec- ture Notes in Computer Science, 476–486

    CT2Rep: Automated Radiology Report Generation for 3D Medical Imaging. InMedical Image Computing and Computer As- sisted Intervention – MICCAI 2024, volume 15012 ofLec- ture Notes in Computer Science, 476–486. Springer Nature Switzerland. Hamamci, I. E.; Er, S.; Shit, S.; Reynaud, H.; Kainz, B.; and Menze, B. H. 2025a. CRG Score: A Distribution- Aware Clin...

    Pith/arXiv arXiv 2024

  5. [10]

    Hu,E.J.;Shen,Y.;Wallis,P.;Allen-Zhu,Z.;Li,Y.;Wang,S.; Wang,L.;andChen,W.2021

    Generalist Founda- tion Models from a Multimodal Dataset for 3D Computed Tomography.Nature Biomedical Engineering. Hu,E.J.;Shen,Y.;Wallis,P.;Allen-Zhu,Z.;Li,Y.;Wang,S.; Wang,L.;andChen,W.2021. LoRA:Low-RankAdaptation of Large Language Models. arXiv:2106.09685. Ilse,M.;Tomczak,J.;andWelling,M.2018.Attention-Based DeepMultipleInstanceLearning. InProceedings...

    Pith/arXiv arXiv 2021

  6. [11]

    Kyung,S.;Seo,J.;Lim,H.;Kim,D.;Park,H.;Sung,J.;Kim, J.; Jo, W.; Nam, Y.; and Kim, N

    CT-GRAPH: Hierarchical Graph Attention NetworkforAnatomy-GuidedCTReportGeneration.arXiv preprint arXiv:2508.05375. Kyung,S.;Seo,J.;Lim,H.;Kim,D.;Park,H.;Sung,J.;Kim, J.; Jo, W.; Nam, Y.; and Kim, N

    Pith/arXiv arXiv

  7. [12]

    arXivpreprintarXiv:2506.23102

    Region-Aware Mul- timodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation. arXivpreprintarXiv:2506.23102. AcceptedtoECCV2026. Li,W.;Yuan,Y.;Liu,J.;Tang,D.;Wang,S.;Qin,J.;Zhu,J.; andZhang,L.2024. TokenPacker:EfficientVisualProjector for Multimodal LLM.arXiv preprint arXiv:2407.02392. Li, W.; Yuille, A.; a...

    Pith/arXiv arXiv 2024

  8. [13]

    Liang,R.2026

    How Well Do Su- pervised 3D Models Transfer to Medical Imaging Tasks? InInternational Conference on Learning Representations (ICLR). Liang,R.2026. CheapProbesPredictExpensiveTrainingin 3D-CT Vision-Language Models. arXiv:2607.22771. Liao, C.; Wang, W.; Wen, Z.; Zheng, X.; Wang, Y.; He, H.; Lyu,Y.;Jiang,L.;Zou,X.;Fu,Y.;Ren,B.;Zhang,L.;andHu, X.2025. AreWeU...

    Pith/arXiv arXiv 2026

  9. [14]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y

    CT-GLIP: 3D Grounded Language- ImagePretrainingwithCTScansandRadiologyReportsfor Full-Body Scenarios.arXiv preprint arXiv:2404.15272. Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J

    arXiv

  10. [15]

    InAdvances in Neural Information Processing Systems, volume 36, 34892–34916

    Visual Instruc- tion Tuning. InAdvances in Neural Information Processing Systems, volume 36, 34892–34916. Liu,S.;Ye,Z.;Lin,Y.;Hu,C.;Geng,W.;Han,X.;Ibragimov, B.;Zheng,Y.;andYuan,Y.2026. MedPruner:Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models.arXiv preprint arXiv:2603.11625. Llama Team

    Pith/arXiv arXiv 2026

  11. [16]

    arXiv:2407.21783

    The Llama 3 Herd of Models. arXiv:2407.21783. Loshchilov,I.;andHutter,F.2019. DecoupledWeightDecay Regularization. InInternational Conference on Learning Representations. Ostmeier, S.; Xu, J.; Chen, Z.; Varma, M.; Blankemeier, L.; Bluethgen, C.; Michalson, A. E.; Moseley, M.; Langlotz, C.; Chaudhari, A. S.; and Delbrouck, J.-B

    Pith/arXiv arXiv 2019

  12. [17]

    arXiv:2405.03595

    GREEN: GenerativeRadiologyReportEvaluationandErrorNotation. arXiv:2405.03595. Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J

    Pith/arXiv arXiv

  13. [18]

    InIEEE/CVFInternationalCon- ference on Computer Vision (ICCV)

    LLaVA-PruMerge: Adaptive Token Reduction for Efficient LargeMultimodalModels. InIEEE/CVFInternationalCon- ference on Computer Vision (ICCV). ArXiv:2403.15388. Shao, K.; Tao, K.; Zhang, K.; Feng, S.; Cai, M.; Shang, Y.; You,H.;Qin,C.;Sui,Y.;andWang,H.2025.ASurveyofTo- ken Compression for Efficient Multimodal Large Language Models.arXiv preprint arXiv:2507....

    arXiv 2025

  14. [19]

    Tancik,M.;Srinivasan,P.P.;Mildenhall,B.;Fridovich-Keil, S.; Raghavan, N.; Singhal, U.; Ramamoorthi, R.; Barron, J.T.;andNg,R.2020

    Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding.arXiv preprint arXiv:2501.14548. Tancik,M.;Srinivasan,P.P.;Mildenhall,B.;Fridovich-Keil, S.; Raghavan, N.; Singhal, U.; Ramamoorthi, R.; Barron, J.T.;andNg,R.2020. FourierFeaturesLetNetworksLearn High Frequency Functions in Low Dimensional Domains. InAdvances in ...

    Pith/arXiv arXiv 2020

  15. [21]

    arXiv:2510.15042

    Comprehensive Language– Image Pre-training for 3D Medical Image Understanding. arXiv:2510.15042. Ward Jr., J. H

    arXiv

  16. [23]

    Yan, A.; McAuley, J.; Lu, X.; Du, J.; Chang, E

    Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysis.arXiv preprint arXiv:2503.20047. Yan, A.; McAuley, J.; Lu, X.; Du, J.; Chang, E. Y.; Gentili, A.;andHsu,C.-N.2022. RadBERT:AdaptingTransformer- based Language Models to Radiology.Radiology: Artificial Intelligence, 4(4): e210258. Yang,S.;Chen,Y.;Tian,Z.;Wang,C.;Li,J.;Yu,B.;andJia, J

    arXiv 2022

  17. [24]

    Zaheer, M.; Kottur, S.; Ravanbakhsh, S.; Póczos, B.; Salakhutdinov, R.; and Smola, A

    VisionZip: Longer is Better but Not Necessary in VisionLanguageModels.arXivpreprintarXiv:2412.04467. Zaheer, M.; Kottur, S.; Ravanbakhsh, S.; Póczos, B.; Salakhutdinov, R.; and Smola, A. J

    arXiv

  18. [25]

    JournaloftheAmericanCollegeofRadiology,23(6):1041–

    US Radiology Imaging and Workforce Volumes 2017–2024: An Analysis of 46.4 Mil- lion Imaging Examinations From 167 Radiology Facilities. JournaloftheAmericanCollegeofRadiology,23(6):1041–

    2017

  19. [27]

    Howwouldyoucharacterizethecardiothoracic(heart- to-lung) size ratio?

    Each scan in both datasets is paired with a radiology report. Ab- normalitylabelscomefromRadBERT(Yanetal.2022)and organmasksfromTotalSegmentator(Wasserthaletal.2023). CT-CLIPandBTB3Dtrainonthereconstruction-levelsplit, whereasCOLIPRIusesthevolume-levelsplit;Merlinusesa 20,364-scan subset. A.2 Measurement VQA benchmark Our probing and visual question answe...

    2022

  20. [29]

    is adapted as feature- similarity merging on the flattened 3D token sequence. We firstuniformlypoolthedensegridtoa1,024-tokencandidate size density location texture size density location texture COLIPRIB=27B=216 Grid average 0.676±0.0050.806±0.0050.266±0.0230.683±0.0080.681±0.0020.865±0.0040.247±0.0140.760±0.025 + centroid 0.685±0.0100.815±0.0060.489±0.02...

    2025

  21. [30]

    and downstream VQA (Table 11). Because a centroid can be computed for any retained or mergedtoken,thisequalizespositionacrossmethods,soany remaininggapreflectshowthetokensareformedratherthan whether they carry position. On Merlin, whose encoders are sensitivetoprobecapacity,wereadoutwithahigh-capacity probe so that no baseline is limited by the probe; COL...

    2026

  22. [31]

    Each cell is the held-out score with the held-out minus validation difference in grey

    27 0.842±0.001−0.0020.601±0.004−0.0120.783±0.012−0.0010.134±0.004−0.0370.625±0.010−0.019 216 0.846±0.001−0.0010.630±0.005−0.0110.835±0.006+0.0000.182±0.003−0.0350.674±0.019−0.051 ORCA 27 0.848±0.001−0.0020.689±0.002−0.0020.888±0.004−0.0040.573±0.019−0.0190.762±0.014−0.002 216 0.849±0.001−0.0020.713±0.007+0.0030.909±0.003−0.0030.648±0.015−0.0120.769±0.011−...

    2017

  23. [33]

    Computational Radiomics System to Decode the Radiographic Phenotype

    vanGriethuysen,J.J.M.;Fedorov,A.;Parmar,C.;Hosny,A.; Aucoin,N.;Narayan,V.;Beets-Tan,R.G.H.;Fillion-Robin, J.-C.;Pieper,S.;andAerts,H.J.W.L.2017. Computational Radiomics System to Decode the Radiographic Phenotype. Cancer Research, 77(21): e104–e107. Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, L.; and Polosukhin, I

    2017

  24. [1048]

    Supplementary Material ORCA: ORgan-Centroid Aggregation for Training-Free 3D CT Visual Token Compression A Experimental setup A.1 Datasets and encoders CT-RATE Merlin anatomy chest abdomen contrast non-contrast portal-venous patients 21,304 18,317 abnormality labels 18 30 CT volumes 25,692 25,494 train / valid 24,128 / 1,564 15,309 / 5,055 reconstructions...

    2026

  25. [1963]

    Wasserthal, J.; Breit, H.-C.; Meyer, M

    Hierarchical Grouping to Optimize an Objective Function.Journal of the American Statistical Association, 58(301): 236–244. Wasserthal, J.; Breit, H.-C.; Meyer, M. T.; Pradella, M.; Hinck, D.; Sauter, A. W.; Heye, T.; Boll, D. T.; Cyriac, J.; Yang,S.;Bach,M.;andSegeroth,M.2023. TotalSegmenta- tor:RobustSegmentationof104AnatomicStructuresinCT Images.Radiolo...

    Pith/arXiv arXiv 2023

  26. [2016]

    Alvar, S

    Understanding Intermedi- ate Layers Using Linear Classifier Probes.arXiv preprint arXiv:1610.01644. Alvar, S. R.; Singh, G.; Akbari, M.; and Zhang, Y

    Pith/arXiv arXiv

  27. [2018]

    followed by a small MLP, trained with AdamW (Loshchilov and Hutter 2019); every probing number is the mean over three seeds. Two compar- isonsinsteaduseahigher-capacityheadsothatnomethodis bottleneckedbytheprobe:theuncompressedreferencerows in Table 7, and the Merlin centroid-encoding comparison (Table12).Encoderfeaturesarenormalizedtotheencoder’s scale b...

    2019

  28. [2023]

    Fang, C.; Guo, H.; Jiang, Z.; He, C.; Li, X.; and Xu, M

    SegVol: Universal and Interactive Volumetric Medical Image Seg- mentation.arXiv preprint arXiv:2311.13385. Fang, C.; Guo, H.; Jiang, Z.; He, C.; Li, X.; and Xu, M

    Pith/arXiv arXiv

  29. [2024]

    Belinkov, Y

    M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models.arXiv preprint arXiv:2404.00578. Belinkov, Y

    Pith/arXiv arXiv

  30. [2025]

    Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B

    BoostingVisionSemanticDensitywithAnatomyNormality Modeling for Medical Vision-language Pre-training.arXiv preprint arXiv:2508.03742. Chen, L.; Zhao, H.; Liu, T.; Bai, S.; Lin, J.; Zhou, C.; and Chang, B

    Pith/arXiv arXiv

  31. [2026]

    Bolya, D.; Fu, C.-Y.; Dai, X.; Zhang, P.; Feichtenhofer, C.; andHoffman,J.2023

    Mer- lin:AComputedTomographyVision–LanguageFoundation Model and Dataset.Nature, 652: 1318–1328. Bolya, D.; Fu, C.-Y.; Dai, X.; Zhang, P.; Feichtenhofer, C.; andHoffman,J.2023. TokenMerging:YourViTButFaster. InInternational Conference on Learning Representations (ICLR). ArXiv:2210.09461. Bumgardner, V. K. C.; Klusty, M. A.; Gokmen, M. S.; and Damron, E. W

    Pith/arXiv arXiv 2023