Pith. sign in

REVIEW 3 major objections 4 minor 3 cited by

RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By training a student vision encoder to match all of its teachers at every input resolution, this paper eliminates a resolution-dependent "mode switch" in agglomerative models and produces encoders that retain accuracy from 256 to 1024…

desk verdict Solid, reproducible step forward for agglomerative vision encoders, but the multi-resolution causal story needs one more ablation to separate mosaic augmentation from the schedule. read the letter →

arxiv 2412.07679 v2 pith:ZVAJEP5Y submitted 2024-12-10 cs.CV cs.AI

Greg Heinrich , Mike Ranzinger , Hongxu (Danny) Yin , Yao Lu , Jan Kautz , Andrew Tao , Bryan Catanzaro , Pavlo Molchanov This is my paper · ORCID
classification cs.CVcs.AI
keywords agglomerativevisionfoundationmodelsmulti-teacherdistillationmulti-resolutiontrainingmodeswitchingmosaicaugmentationtokenmergingPHI-Sstandardizationvision-language
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that an agglomerative vision model, one student distilled from multiple teacher networks, can be made accurate across a wide range of input resolutions if it is trained to match every teacher at every resolution. It identifies a "mode switch" defect in the prior baseline: at low resolutions the student's features resemble DINO's, and at high resolutions they resemble SAM's, because the student only saw SAM at high resolution during training. The proposed fix is a three-stage multi-resolution schedule, aided by mosaic augmentation that lets SAM features supervise low-resolution student inputs without interpolating SAM's features. The result is a family of encoders whose zero-shot accuracy stays roughly flat from 256 to 1024 pixels, with improved segmentation, 3D probing, and vision-language reasoning scores. A reader should care because it suggests one encoder can serve classification, dense prediction, and VLM tasks without resolution-specific retraining.

What carries the argument

The machinery has three parts. First, multi-resolution training: a three-stage schedule (256², 432², then simultaneous 432² and 1024²) in which the student's backbone is supervised by all teachers at all resolutions; student features are spatially interpolated down to match low-resolution teachers, and the high-resolution SAM teacher is fed mosaics of $k \times k$ images whose features are cropped to supervise low-resolution students, avoiding interpolation of SAM features. Second, PHI-S (PCA–Hadamard isotropic standardization), a per-teacher rotation and scaling of feature activations to unit variance, which balances the distillation losses and is inverted by adjusting the adaptor's final projection. Third, output token merging, a one-shot bipartite soft matching of patch tokens at the encoder output, replacing pixel unshuffling, to keep high-resolution information in a fixed token budget for language models.

What would settle it

Measure SAM's features on a single low-resolution image (padded to 1024² and cropped, as in the paper) versus SAM's features for the same image cropped out of a mosaic: if the normalized distance between these two feature maps is comparable to SAM's own scale variance, the mosaic proxy is sound; if it is much larger, the proxy fails. A second end-to-end test: if the multi-resolution training claim is right, the zero-shot ImageNet accuracy of the released models should be approximately flat from 256² to 1024² inputs, so a reader can run the released weights and check whether accuracy still drops sharply above 512², as it did for the baseline.

Watch

Extended reading notes

Core claim

The central claim is that resolution mode switching in agglomerative distillation is a training artifact, not an inherent property of multi-teacher models. In the baseline, the student sees the DINO and CLIP teachers only at low resolution and the SAM teacher only at high resolution, so the student alternates between mimicking DINO (low-resolution, semantic) and mimicking SAM (high-resolution, contour-oriented) features. The paper shows that when the student is trained to match all teachers at all resolutions, downsampling student features where the teacher is low-resolution and using mosaic augmentation to keep SAM's high-resolution supervision available for low-resolution student inputs, the mode switch disappears, scale equivariance approaches that of the multi-resolution DINOv2 teacher, and downstream accuracy improves across dense tasks and VLM benchmarks. The paper also reports that PHI-S standardization of teacher feature distributions prevents SAM's large activation variance from dominating the loss, and that a one-shot token-merging step at the encoder output compresses visual tokens for VLMs better than pixel unshuffling, with the largest gains on its own encoder.

Load-bearing premise

Section 4.3 assumes that running the high-resolution SAM teacher on a mosaic of $k \times k$ small images and cropping out each sub-image's features gives a faithful surrogate for SAM's features on those images seen individually at low resolution, with no harmful cross-image interference; if that fails, low-resolution distillation against SAM breaks down and the multi-resolution fix loses its high-resolution teacher.

Editorial extensions

If this is right

  • If the claim holds, a single vision encoder can serve many downstream tasks at input resolutions from 256 to 1024 pixels without resolution-specific retraining or tiling.
  • SAM is a useful teacher when its features are not interpolated and its distribution is balanced: including it improves dense tasks such as ADE20k semantic segmentation and 3D probing over omitting it.
  • Token merging at the encoder output yields better VLM scores than pixel unshuffling at the same token count, and the improvement is larger for this encoder than for SigLIP, suggesting the learned feature space has many redundant tokens.
  • Feeding intermediate layer activations to non-linear heads such as DPT improves dense prediction, while linear probes and VLM use do not benefit from those activations.
  • Fewer training partitions, with all teachers in one batch, improves summarization-task quality when compute budget allows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If mosaic augmentation is a faithful proxy, SAM features are largely local and do not depend on global image context; this could be tested with images whose meaning changes with context, and the limits of the proxy would appear there.
  • The success of output token merging suggests the encoder's feature space is highly redundant, possibly inherited from SAM's contour-homogeneous features; a testable prediction is that token redundancy drops if SAM is removed from the teacher set.
  • Resolution robustness might reduce the need for tiling in high-resolution VLM inference, saving latency and token budget, though the paper's tiling experiments still show additional gains from tiling with this encoder.
  • The per-teacher whitening idea could generalize beyond distillation to any multi-modal or multi-task feature fusion where sources have mismatched scales.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents RADIOv2.5, an improved recipe for agglomerative vision foundation models trained by multi-teacher distillation from CLIP, SigLIP, DINOv2, and SAM. The proposed components are multi-resolution training to remove the resolution-dependent 'mode switch' observed in AM-RADIO, mosaic augmentation to obtain low-resolution SAM targets efficiently, PHI-S teacher balancing, a SigLIP teacher, larger backbones, and one-shot token merging at the ViT output for VLM integration. The authors evaluate the recipe with an incremental ablation in Table 1 and with extensive downstream benchmarks (ImageNet, ADE20k, COCO, NYUDv2, Pascal Context, VILA-based VLMs, and Probe3D), and they release multiple model scales, code, and pretrained weights.

Significance. If the central claims hold, this is a practically valuable baseline paper: it identifies a real resolution-robustness failure in AM-RADIO, offers a plausible fix, and provides open models at several scales with broadly positive dense and VLM results. The manuscript's strengths include a clean incremental ablation chain, release of code and weights, and a wider downstream evaluation than most comparable papers, including quantitative scale-equivariance measurements and a thorough VLM token-compression study. The main reason the paper is not immediately acceptable is that the causal attribution of the mode-switch fix depends on the faithfulness of mosaic augmentation as a proxy for low-resolution SAM features, and the paper does not yet quantify that faithfulness or separate it from the multi-resolution training effect.

major comments (3)
  1. [Section 4.3, Figures 6/A4/A5] The mosaic augmentation's validity as a proxy for low-resolution SAM features is asserted only qualitatively. Because SAM-H uses global attention, features computed on a k-by-k mosaic of unrelated images with padding may differ systematically from SAM features computed on a single low-resolution image, and the paper provides only PCA visualizations (Figures 6, A4, A5) as evidence of faithfulness. This is load-bearing for Finding 2 and Figure 2: the mode-switch fix is attributed to multi-resolution training, but Table 1 shows no configuration that applies multi-resolution training without mosaic augmentation, so the direct effect of multi-resolution training is confounded with the effect of the mosaic target. Please quantify the fidelity of the mosaic proxy (e.g., cosine similarity or MSE between SAM features on a mosaic crop and SAM features on the same image passed individually at the same resolution via the Section 4.2 padding/crop protocol), and ideally ablate config B with direct padded SAM features instead of mosaic features, so the causal story is supported.
  2. [Section 5.1 and Table A6] The semantic segmentation protocol trains probes at three resolutions and reports only the best result for each model. This systematically favors RADIOv2.5 because AM-RADIO's mIoU collapses above 512 px (e.g., AM-RADIO-H goes from 51.34 at 512 to 35.78 at 768 in Table A6, while RADIOv2.5-H stays flat or improves: 51.58 to 52.45). Consequently, the dense-task comparisons in Table 8 may reflect the probe-resolution selection rule rather than a fixed evaluation protocol, which undermines the claim of a universal dense-task improvement. Please report results at a single pre-specified probe resolution for every model, or report all three resolutions with confidence intervals and state whether the conclusions are robust to the choice.
  3. [Tables 1, 8, 9] The experiments appear to be single runs with no error bars, bootstrap confidence intervals, or multiple seeds, while several load-bearing claims rest on differences of only a few points (e.g., config B vs. A in Table 1: +2.8 ADE20k and +3.4 VLM average). Given the best-probe-resolution selection noted above, it is not yet clear that these differences exceed run-to-run and evaluation-protocol noise. Please add variance estimates for at least the smaller ablations, or otherwise justify that the observed differences are not within the noise of the evaluation pipeline.
minor comments (4)
  1. [Section 2.2 vs Section 4.2] The baseline training description in Section 2.2 says the two concurrent partitions use 432^2 and 1024^2 resolutions for 600k iterations, while Section 4.2 describes three successive stages (256, 432, and simultaneous 432+1024). Please reconcile these descriptions so the reader knows which procedure generated config A.
  2. [Table 3] The formatting of the fidelity table appears corrupted (e.g., entries such as '1.1432.411' and '7.799 8.377'), and the distinction between the Fidelity column and the F[X] columns is unclear. Please reformat and define each column explicitly, including the units of F[X].
  3. [Section 4.4 and Table 1] PHI-S is introduced as a key balancing method, but Table 1's ablation chain does not isolate PHI-S; Table 3 reports only fidelity statistics, not downstream benchmark results. A sentence or a small table showing the downstream effect of PHI-S alone would make Finding 4 testable.
  4. [Figure 2] The legend labels 'Multi-Stage Stage 2' and 'Multi-Stage Stage 3' are not defined in the main text, and it is not clear which of these corresponds to the three-stage schedule in Section 4.2. Please define these configurations or remove the labels.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RADIOv2.5's improvements are validated on external benchmarks and incremental ablations, and the paper does not derive its predictions from fitted inputs or self-citations.

full rationale

The paper's central claims are empirical: multi-resolution training, mosaic augmentation, PHI-S balancing, and token merging are each evaluated by ablations in Table 1 and by external benchmarks (ImageNet, ADE20k, COCO, VILA, Probe3D, NYUDv2, Pascal Context). Configuration B is compared against configuration A, configuration C against B, and so on, so the reported gains are measured differences rather than identities imposed by construction. The scale-equivariance metric (Appendix A.5) is a definition of a measure, not a fitted parameter later reported as a prediction. PHI-S is adopted from the authors' prior work [35], but its effect is independently ablated in Table 3 and it is not used to guarantee the paper's headline results. AM-RADIO [36] and PHI-S [35] self-citations describe baselines and tooling; they do not supply an unverified theorem that forces the paper's conclusions. The mosaic augmentation proxy concern raised in the skeptic summary is a correctness risk about whether cropped mosaic SAM features faithfully match single-image SAM features; the paper reports only qualitative visual evidence for this. However, that is not circularity: the student is trained against actual SAM outputs, and the downstream numbers are measured externally. No equation in the paper reduces a claimed prediction to its own input, and no fitted parameter is renamed as a prediction. The paper is self-contained against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its free parameters are training choices (teacher weights, merge ratios) that are not disclosed in full. The main axiomatic load comes from the knowledge-distillation paradigm and the assumption that mosaic/padded teacher features are faithful.

free parameters (2)
  • Teacher loss weighting coefficients (lambda_t) = not reported
    Used in the overall loss function in Eq. (6). The values are not given in the paper, making the training recipe incomplete.
  • Token merging ratio r = 2048, 3584, 2108 (per configuration)
    Chosen by hand to control the number of output tokens in VLM experiments (Tables 6, A3, A4). The choice affects benchmark results but is not fitted to a target accuracy.
assumptions (4)
  • domain assumption Knowledge agglomeration assumption: distilling from multiple vision foundation models into one student transfers their capabilities.
    Stated in Section 2.1 and inherited from AM-RADIO; the paper does not derive this, it relies on it.
  • domain assumption Cropped Position Embeddings (CPE) generalize to unseen resolutions.
    Invoked in Section 2.2 (Baseline Model); the paper relies on the prior work of Kim et al. for this property.
  • domain assumption PHI-S standardization preserves task-relevant information while balancing teacher feature scales.
    Adopted in Section 4.4 from Ranzinger et al. (2024); the paper uses it rather than proving it.
  • ad hoc to paper Mosaic/padded SAM features are valid training targets for low-resolution student inputs.
    Introduced in Section 4.3; the paper provides qualitative evidence but no formal justification that cropped mosaic features match single-image SAM features.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models." pith.science (2026). https://pith.science/paper/ZVAJEP5Y

@misc{pith2026241207679,
  author       = {Pith},
  title        = {Pith review of: RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZVAJEP5Y}},
  note         = {Machine review of arXiv:2412.07679}
}
read the original abstract

Agglomerative models have recently emerged as a powerful approach to training vision foundation models, leveraging multi-teacher distillation from existing models such as CLIP, DINO, and SAM. This strategy enables the efficient creation of robust models, combining the strengths of individual teachers while significantly reducing computational and resource demands. In this paper, we thoroughly analyze state-of-the-art agglomerative models, identifying critical challenges including resolution mode shifts, teacher imbalance, idiosyncratic teacher artifacts, and an excessive number of output tokens. To address these issues, we propose several novel solutions: multi-resolution training, mosaic augmentation, and improved balancing of teacher loss functions. Specifically, in the context of Vision Language Models, we introduce a token compression technique to maintain high-resolution information within a fixed token count. We release our top-performing variants at multiple scales (-B, -L, -H, and -g), along with inference code and pretrained weights

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics

    cs.CV 2026-02 conditional novelty 6.0 of 10

    Semantic encoders can harm relighting, and ALI—fusing dense visual features with latent intrinsics—improves relighting on glossy and specular materials.

  2. Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    Eagle2-9B matches or outperforms much larger vision-language models on many benchmarks through a carefully constructed post-training data strategy.

  3. Image Recognition with Vision and Language Embeddings of VLMs

    cs.CV 2025-09 conditional novelty 4.0 of 10

    A benchmark of dual-encoder VLMs finds text and image embeddings give complementary class accuracy, and a per-class precision fusion rule adds about 0.4% accuracy over either alone on ImageNet.

Reference graph

Works this paper leans on

61 extracted references · 51 canonical work pages · cited by 3 Pith papers

  1. [1]

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization, 2016

    2016

  2. [2]

    Alabdulmohsin, and Filip Pavetic

    Lucas Beyer, Pavel Izmailov, Alexander Kolesnikov, Mathilde Caron, Simon Kornblith, Xiaohua Zhai, Matthias Minderer, Michael Tschannen, Ibrahim M. Alabdulmohsin, and Filip Pavetic. Flexivit: One model for all patch sizes.2023 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 14496–14506, 2022

    work page 2023

  3. [3]

    Knowledge distillation: A good teacher is patient and consistent

    Lucas Beyer, Xiaohua Zhai, Amelie Royer, Larisa Markeeva, Rohan Anil, and Alexander Kolesnikov. Knowledge distillation: A good teacher is patient and consistent . In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 10915–10924, Los Alamitos, CA, USA, 2022. IEEE Computer Society

    work page 2022

  4. [4]

    Token merging for fast stable diffusion.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR W), pages 4599–4603, 2023

    Daniel Bolya and Judy Hoffman. Token merging for fast stable diffusion.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR W), pages 4599–4603, 2023

    work page 2023

  5. [5]

    Token merging: Your ViT but faster

    DanielBolya, Cheng-YangFu, XiaoliangDai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your ViT but faster. InInternational Conference on Learning Representations, 2023

    work page 2023

  6. [6]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv’e J’egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. 2021 IEEE/CVF International Confer- ence on Computer Vision (ICCV) , pages 9630–9640, 2021

    work page 2021

  7. [7]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiao wen Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. In European Conference on Computer Vision, 2023

    work page 2023

  8. [8]

    How far are we to gpt-4v? closing the gap to commercial multi- modal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multi- modal models with open-source suites.arXiv preprint arXiv:2404.16821, 2024

    arXiv 2024

Show all 61 references
  1. [9]

    MMSegmenta- tion: Openmmlab semantic segmentation toolbox and benchmark

    MMSegmentation Contributors. MMSegmenta- tion: Openmmlab semantic segmentation toolbox and benchmark. https://github.com/open-mmlab/ mmsegmentation, 2020

    2020

  2. [10]

    Vision transformers need registers

    Timothée Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In Proceedings of the 12th International Conference on Learning Representations (ICLR) , 2024

    2024

  3. [11]

    Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution

    Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim M Alabdulmohsin, et al. Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution. Advances in Neura...

    2024

  4. [12]

    An image is worth 16x16 words: Transformers for image recognition at scale, 2021

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszko- reit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  5. [13]

    Probing the 3D Awareness of Visual Foundation Models

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Mani- nis, Abhishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Probing the 3D Awareness of Visual Foundation Models. InCVPR, 2024

    2024

  6. [14]

    Mark Everingham, Luc Van Gool, Christopher K. I. Williams, John M. Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge.Int. J. Comput. Vis. , 88(2):303–338, 2010

    2010

  7. [15]

    Williams, J

    M.Everingham, S.M.A.Eslami, L.VanGool, C.K.I. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: A retrospective.Inter- national Journal of Computer Vision , 111(1):98–136, 2015

    2015

  8. [16]

    Data filtering networks, 2023

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. Data filtering networks, 2023

    2023

  9. [17]

    Data- comp: In search of the next generation of multimodal datasets, 2023

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bit- ton, Kalyani Marathe, Stephen Mussm...

    2023

  10. [18]

    Videosam: Open-world video segmen- tation, 2024

    Pinxue Guo, Zixu Zhao, Jianxiong Gao, Chongruo Wu, Tong He, Zheng Zhang, Tianjun Xiao, and Wen- qiang Zhang. Videosam: Open-world video segmen- tation, 2024. 12 RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

    2024

  11. [19]

    Gaussian error linear units (gelus), 2023

    Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (gelus), 2023

    2023

  12. [20]

    NAVI: Category-agnostic image col- lections with high-quality 3d shape and pose annota- tions

    Varun Jampani, Kevis-Kokitsi Maninis, Andreas En- gelhardt, Arjun Karpur, Karen Truong, Kyle Sar- gent, Stefan Popov, Andre Araujo, Ricardo Martin- Brualla, Kaushal Patel, Daniel Vlasic, Vittorio Fer- rari, Ameesh Makadia, Ce Liu, Yuanzhen Li, and Howard Zhou. NAVI: Category-a...

    2023

  13. [21]

    Brave: Broadening the visual encoding of vision-language models

    Oğuzhan Fatih Kar, Alessio Tonioni, Petra Pok- lukar, Achin Kulshrestha, Amir Zamir, and Federico Tombari. Brave: Broadening the visual encoding of vision-language models. In European Conference on Computer Vision, pages 113–132. Springer, 2024

    2024

  14. [22]

    Dahun Kim, Anelia Angelova, and Weicheng Kuo. Region-aware pretraining for open-vocabulary object detection with vision transformers.2023 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 11144–11154, 2023

    2023

  15. [23]

    Berg, Wan-Yen Lo, Piotr Dollár, and Ross B

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloé Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross B. Girshick. Segment anything. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pages...

    2023

  16. [24]

    Vila: On pre- training for visual language models.2024 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 26679–26689, 2023

    Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, and Song Han. Vila: On pre- training for visual language models.2024 IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pages 26679–26689, 2023

    2024

  17. [25]

    Llava- next: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuan- han Zhang, Sheng Shen, and Yong Jae Lee. Llava- next: Improved reasoning, ocr, and world knowledge, 2024

    2024

  18. [26]

    Efficientvit: Mem- ory efficient vision transformer with cascaded group attention

    Xinyu Liu, Houwen Peng, Ningxin Zheng, Yuqing Yang, Han Hu, and Yixuan Yuan. Efficientvit: Mem- ory efficient vision transformer with cascaded group attention. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 14420–14430, 2023

    2023

  19. [27]

    Ocrbench: On the hidden mystery of ocr in large multimodal models, 2024

    Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xucheng Yin, Cheng lin Liu, Lianwen Jin, and Xiang Bai. Ocrbench: On the hidden mystery of ocr in large multimodal models, 2024

    2024

  20. [28]

    Chartqa: A benchmark for question answering about charts with visual and logical reasoning, 2022

    Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning, 2022

    2022

  21. [29]

    V Jawahar

    Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, and C. V Jawahar. Infographicvqa, 2021

    2021

  22. [30]

    Minesh Mathew, Dimosthenis Karatzas, and C. V. Jawahar. Docvqa: A dataset for vqa on document images, 2021

    2021

  23. [31]

    The role of context for object detection and semantic segmentation in the wild

    Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam- Gyu Cho, Seong-Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. The role of context for object detection and semantic segmentation in the wild. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014

    2014

  24. [32]

    Indoor segmentation and support inference from rgbd images

    Pushmeet Kohli Nathan Silberman, Derek Hoiem and Rob Fergus. Indoor segmentation and support inference from rgbd images. InECCV, 2012

    2012

  25. [33]

    Learning transferable visual models from natural language su- pervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language su- pervision. In International...

    2021

  26. [34]

    Vision transformers for dense prediction

    René Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vision transformers for dense prediction. In Proceedings of the IEEE/CVF international con- ference on computer vision , pages 12179–12188, 2021

    2021

  27. [35]

    Phi- s: Distribution balancing for label-free multi-teacher distillation, 2024

    Mike Ranzinger, Jon Barker, Greg Heinrich, Pavlo Molchanov, Bryan Catanzaro, and Andrew Tao. Phi- s: Distribution balancing for label-free multi-teacher distillation, 2024

    2024

  28. [36]

    Am-radio: Agglomerative vision foundation model reduce all domains into one

    Mike Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12490–12500, 2024

    2024

  29. [37]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge, 2015

    2015

  30. [38]

    Unic: Universal classification models via multi- teacher distillation, 2024

    Mert Bulent Sariyildiz, Philippe Weinzaepfel, Thomas Lucas, Diane Larlus, and Yannis Kalan- tidis. Unic: Universal classification models via multi- teacher distillation, 2024

    2024

  31. [39]

    May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, and Laura Herlant

    Jinghuan Shang, Karl Schmeckpeper, Brandon B. May, Maria Vittoria Minniti, Tarik Kelestemur, David Watkins, and Laura Herlant. Theia: Distilling diverse vision foundation models for robot learning, 2024

    2024

  32. [40]

    Eagle: Exploring the design space for multimodal llms with mixture of encoders, 2024

    Min Shi, Fuxiao Liu, Shihao Wang, Shijia Liao, Sub- hashree Radhakrishnan, De-An Huang, Hongxu Yin, Karan Sapra, Yaser Yacoob, Humphrey Shi, Bryan Catanzaro, Andrew Tao, Jan Kautz, Zhiding Yu, and Guilin Liu. Eagle: Exploring the design space for multimodal llms with mixture o...

    2024

  33. [41]

    Towards vqa models that can read, 2019

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read, 2019

    2019

  34. [42]

    Llm pruning and distillation in practice: The minitron approach, 2024

    Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, and Pavlo Molchanov. Llm pruning and distillation in practice: The minitron approach, 2024

    2024

  35. [43]

    Does knowledge distillation really work? In Advances in Neural Information Processing Systems , 2021

    Samuel Don Stanton, Pavel Izmailov, Polina Kirichenko, Alexander A Alemi, and Andrew Gordon Wilson. Does knowledge distillation really work? In Advances in Neural Information Processing Systems , 2021

    2021

  36. [44]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024

    Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Iyer, Xichen Pan, Austin Wang, Rob Fergus, Yann Le- Cun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms, 2024

    2024

  37. [45]

    Sam-clip: Merging vi- sion foundation models towards semantic and spa- tial understanding

    Haoxiang Wang, Pavan Kumar Anasosalu Vasu, Far- tash Faghri, Raviteja Vemulapalli, Mehrdad Fara- jtabar, Sachin Mehta, Mohammad Rastegari, Oncel Tuzel, and Hadi Pouransari. Sam-clip: Merging vi- sion foundation models towards semantic and spa- tial understanding. 2024 IEEE/CVF...

    2024

  38. [46]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution, 2024

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhi- hao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024

  39. [47]

    Skil: Semantic keypoint imitation learning for generalizable data-efficient manipulation, 2025

    Shengjie Wang, Jiacheng You, Yihang Hu, Jiongye Li, and Yang Gao. Skil: Semantic keypoint imitation learning for generalizable data-efficient manipulation, 2025

    2025

  40. [48]

    Vision-flan: Scaling human-labeled tasks in visual instruction tuning

    Zhiyang Xu, Chao Feng, Rulin Shao, Trevor Ashby, Ying Shen, Di Jin, Yu Cheng, Qifan Wang, and Lifu Huang. Vision-flan: Scaling human-labeled tasks in visual instruction tuning. arXiv preprint arXiv:2402.11690, 2024

    2024 arXiv

  41. [49]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10371–10381, 2024

    2024

  42. [50]

    Multi-task dense prediction via mixture of low-rank experts

    Yuqi Yang, Peng-Tao Jiang, Qibin Hou, Hao Zhang, Jinwei Chen, and Bo Li. Multi-task dense prediction via mixture of low-rank experts. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024

    2024

  43. [51]

    Dense connector for mllms, 2024

    Huanjin Yao, Wenhao Wu, Taojiannan Yang, YuXin Song, Mengxi Zhang, Haocheng Feng, Yifan Sun, Zhi- heng Li, Wanli Ouyang, and Jingdong Wang. Dense connector for mllms, 2024

    2024

  44. [52]

    Sigmoid loss for language image pre- training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre- training. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pages 11941–11952, 2023

    2023

  45. [53]

    Semantic understanding of scenes through the ade20k dataset

    Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision , 127:302 – 321, 2016

    2016

  46. [54]

    Are they the same? exploring visual correspondence shortcomings of multimodal llms, 2025

    Yikang Zhou, Tao Zhang, Shilin Xu, Shihao Chen, Qianyu Zhou, Yunhai Tong, Shunping Ji, Jiangn- ing Zhang, Xiangtai Li, and Lu Qi. Are they the same? exploring visual correspondence shortcomings of multimodal llms, 2025

    2025

  47. [55]

    Block-wise

    Yi Zhu, Yanpeng Zhou, Chunwei Wang, Yang Cao, Jianhua Han, Lu Hou, and Hang Xu. Unit: Unifying image and text recognition in one vision encoder, 2024. 14 RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models A.0 Supplementary Material 1 RADIOv2.5: Improved B...

    2024

  48. [56]

    Let𝐹min denote the tensor with the smallest spatial dimensions(𝐻min,𝑊 min,𝐶 )

  49. [57]

    Compute the per-channel mean and variance of𝐹min: 𝜇𝑐 = 1 𝐻min𝑊min 𝐻min∑︁ ℎ=1 𝑊min∑︁ 𝑤=1 𝐹min(ℎ,𝑤,𝑐 ) (10) 𝜎2 𝑐 = 1 𝐻min𝑊min 𝐻min∑︁ ℎ=1 𝑊min∑︁ 𝑤=1 (𝐹min(ℎ,𝑤,𝑐 )−𝜇𝑐)2 (11) 5 RADIOv2.5: Improved Baselines for Agglomerative Vision Foundation Models

  50. [58]

    Normalize each tensor𝐹𝑖 using 𝜇𝑐 and 𝜎𝑐: ^𝐹𝑖(ℎ,𝑤,𝑐 ) = 𝐹𝑖(ℎ,𝑤,𝑐 )−𝜇𝑐 𝜎𝑐 (12)

  51. [59]

    Bilinearly interpolate each normalized tensor^𝐹𝑖 down to (𝐻min,𝑊 min,𝐶 ), resulting in tensors{ ˜𝐹𝑖}

  52. [60]

    Stack all resized tensors{ ˜𝐹𝑖} along a new dimension and compute variance along this new dimension: 𝜎2(ℎ,𝑤,𝑐 ) = Var({ ˜𝐹𝑖(ℎ,𝑤,𝑐 )}) (13)

  53. [61]

    keys" (we use the attention keys) or

    Finally, compute the average variance over the spatial dimensions: 𝜎2 scale = 1 𝐻min𝑊min 𝐻min∑︁ ℎ=1 𝑊min∑︁ 𝑤=1 𝜎2(ℎ,𝑤,𝑐 ) (14) A.5.1. Scale Variance Implementation 1 2 def s c a l e _ v a r i a n c e ( tensors : List , scale_up : bool ) : 3 """ Compute feature variance across ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.