{"id":"3ef90a82-e678-457a-b64a-371b39f956b3","arxiv_id":"2412.07679","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RADIOv2.5 adds multi-resolution training, mosaic augmentation, and one-shot token merging, producing vision encoders that stay accurate across resolutions and improve VLM and dense-task performance over earlier baselines.","lead":"This paper presents RADIOv2.5, a new generation of vision encoders trained by combining knowledge from CLIP, DINO, SAM, and SigLIP teachers. The models improve accuracy across image, dense, 3D, and vision-language benchmarks, and the authors released code and pretrained weights at four scales.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mosaic augmentation's faithfulness as a proxy for low-resolution SAM features is unquantified and load-bearing for the multi-resolution fix.","rationale":"The reader's weakest assumption correctly identifies the mosaic proxy as the most load-bearing step in the paper's central argument. The paper's own Figure 2 partially mitigates the concern by showing high SAM spatial fidelity for the multi-resolution model at low resolutions, but the fidelity computation is not described in enough detail to rule out that the same mosaic-derived targets are used in the evaluation. The manipulation is also entangled with the 'multi-resolution' label, since config B changes both the teacher schedule and the SAM low-resolution target generation. I do not regard this as a demonstration of error; rather, it is an unverified assumption that is both cheap to test directly and decisive for the causal attribution. The paper otherwise provides substantial empirical support through ablations, released weights, and code, which is why the verdict should remain unchanged rather than be lowered without the proposed test. My concern is therefore an 'agree' with the reader's weakest assumption, sharpened into a quantitative protocol, not a new objection.","tokens_in":24220,"tokens_out":8104,"duration_ms":74794,"concrete_test":"Compute SAM features for a held-out set of, say, 1,000 DataComp images under two protocols at the training resolutions (256^2 and 432^2): (a) the §4.2 protocol of padding a single image to 1024^2 and cropping the output features; (b) the §4.3 mosaic protocol with the same image as one sub-image among randomly chosen others. Measure per-patch cosine similarity and normalized MSE between (a) and (b). If mean per-patch cosine similarity is below 0.95, or if the difference exceeds the student-teacher fidelity gaps reported in Table 3, the mosaic proxy is not faithful, and the multi-resolution training is learning a biased SAM target. As a stronger but more expensive check, retrain config B using protocol (a) instead of mosaic and compare the mode-switch metrics in Figure 2; if the fix disappears, the effect belongs to mosaic augmentation rather than to multi-resolution training per se.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that multi-resolution training fixes mode switching (Finding 2, Fig. 2) rests on the student receiving a faithful SAM target at low resolution. Section 4.3 replaces direct low-resolution SAM features with features cropped from a k×k mosaic of unrelated images padded to 1024^2. Because SAM uses global attention, features for one sub-image can be influenced by the other k^2−1 sub-images and by the surrounding padding; the paper reports only qualitative 'cleaner features' (Fig. 6, A4/A5) and never quantifies how well these agree with features Sam would produce for a single image at the same resolution via the §4.2 padding/crop protocol. If the mosaic features are a biased proxy, then the low-resolution student is trained against systematically distorted SAM targets. The improvement from config A to config B (Table 1, +2.8 ADE20k, +3.4 VLM average) is then not cleanly attributable to 'multi-resolution training' alone, since config B also introduces mosaic augmentation. This would undercut the causal story of the mode-switch fix, even if the empirical gains are real.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents RADIOv2.5, an improved recipe for agglomerative vision foundation models trained by multi-teacher distillation from CLIP, SigLIP, DINOv2, and SAM. The proposed components are multi-resolution training to remove the resolution-dependent 'mode switch' observed in AM-RADIO, mosaic augmentation to obtain low-resolution SAM targets efficiently, PHI-S teacher balancing, a SigLIP teacher, larger backbones, and one-shot token merging at the ViT output for VLM integration. The authors evaluate the recipe with an incremental ablation in Table 1 and with extensive downstream benchmarks (ImageNet, ADE20k, COCO, NYUDv2, Pascal Context, VILA-based VLMs, and Probe3D), and they release multiple model scales, code, and pretrained weights.","tokens_in":24413,"tokens_out":4465,"duration_ms":43803,"significance":"If the central claims hold, this is a practically valuable baseline paper: it identifies a real resolution-robustness failure in AM-RADIO, offers a plausible fix, and provides open models at several scales with broadly positive dense and VLM results. The manuscript's strengths include a clean incremental ablation chain, release of code and weights, and a wider downstream evaluation than most comparable papers, including quantitative scale-equivariance measurements and a thorough VLM token-compression study. The main reason the paper is not immediately acceptable is that the causal attribution of the mode-switch fix depends on the faithfulness of mosaic augmentation as a proxy for low-resolution SAM features, and the paper does not yet quantify that faithfulness or separate it from the multi-resolution training effect.","major_comments":[{"comment":"The mosaic augmentation's validity as a proxy for low-resolution SAM features is asserted only qualitatively. Because SAM-H uses global attention, features computed on a k-by-k mosaic of unrelated images with padding may differ systematically from SAM features computed on a single low-resolution image, and the paper provides only PCA visualizations (Figures 6, A4, A5) as evidence of faithfulness. This is load-bearing for Finding 2 and Figure 2: the mode-switch fix is attributed to multi-resolution training, but Table 1 shows no configuration that applies multi-resolution training without mosaic augmentation, so the direct effect of multi-resolution training is confounded with the effect of the mosaic target. Please quantify the fidelity of the mosaic proxy (e.g., cosine similarity or MSE between SAM features on a mosaic crop and SAM features on the same image passed individually at the same resolution via the Section 4.2 padding/crop protocol), and ideally ablate config B with direct padded SAM features instead of mosaic features, so the causal story is supported.","section":"Section 4.3, Figures 6/A4/A5"},{"comment":"The semantic segmentation protocol trains probes at three resolutions and reports only the best result for each model. This systematically favors RADIOv2.5 because AM-RADIO's mIoU collapses above 512 px (e.g., AM-RADIO-H goes from 51.34 at 512 to 35.78 at 768 in Table A6, while RADIOv2.5-H stays flat or improves: 51.58 to 52.45). Consequently, the dense-task comparisons in Table 8 may reflect the probe-resolution selection rule rather than a fixed evaluation protocol, which undermines the claim of a universal dense-task improvement. Please report results at a single pre-specified probe resolution for every model, or report all three resolutions with confidence intervals and state whether the conclusions are robust to the choice.","section":"Section 5.1 and Table A6"},{"comment":"The experiments appear to be single runs with no error bars, bootstrap confidence intervals, or multiple seeds, while several load-bearing claims rest on differences of only a few points (e.g., config B vs. A in Table 1: +2.8 ADE20k and +3.4 VLM average). Given the best-probe-resolution selection noted above, it is not yet clear that these differences exceed run-to-run and evaluation-protocol noise. Please add variance estimates for at least the smaller ablations, or otherwise justify that the observed differences are not within the noise of the evaluation pipeline.","section":"Tables 1, 8, 9"}],"minor_comments":[{"comment":"The baseline training description in Section 2.2 says the two concurrent partitions use 432^2 and 1024^2 resolutions for 600k iterations, while Section 4.2 describes three successive stages (256, 432, and simultaneous 432+1024). Please reconcile these descriptions so the reader knows which procedure generated config A.","section":"Section 2.2 vs Section 4.2"},{"comment":"The formatting of the fidelity table appears corrupted (e.g., entries such as '1.1432.411' and '7.799 8.377'), and the distinction between the Fidelity column and the F[X] columns is unclear. Please reformat and define each column explicitly, including the units of F[X].","section":"Table 3"},{"comment":"PHI-S is introduced as a key balancing method, but Table 1's ablation chain does not isolate PHI-S; Table 3 reports only fidelity statistics, not downstream benchmark results. A sentence or a small table showing the downstream effect of PHI-S alone would make Finding 4 testable.","section":"Section 4.4 and Table 1"},{"comment":"The legend labels 'Multi-Stage Stage 2' and 'Multi-Stage Stage 3' are not defined in the main text, and it is not clear which of these corresponds to the three-stage schedule in Section 4.2. Please define these configurations or remove the labels.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The core empirical contribution appears solid: the released models are likely to be useful baselines, and the ablation chain is a good organizing structure. The main technical risk is the unquantified mosaic proxy for SAM features, which is directly connected to the paper's central causal claim about fixing the mode switch. I would be willing to accept after the authors supply a quantitative fidelity check and a cleaner segmentation reporting protocol. The paper reads as an industrial technical report; this is appropriate for the venue if the missing analyses are added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a solid, reproducible step forward for agglomerative vision encoders, and the released models will get adopted. But the headline causal story — that multi-resolution training alone fixes the mode switch — is less clean than it looks, because the multi-resolution config also introduces mosaic augmentation and the two are never ablated separately.\n\nWhat's genuinely new: the mode-switch analysis of AM-RADIO is concrete and useful, with a defined scale-equivariance metric; the three-stage multi-resolution training plus mosaic augmentation produces models that hold up from 256 to 1024 pixels; and one-shot output token merging for VLMs is a practical contribution that beats pixel unshuffling and benefits RADIOv2.5 more than SigLIP. The release of B/L/H/g weights and code is a real asset — people can verify and build on these. The evaluation is broad, and the appendix includes per-resolution segmentation numbers, so the 'best probe resolution' reporting is transparent even if it flatters the model.\n\nSoft spots, in proportion. The mosaic augmentation assumption is the main one. Section 4.3 replaces direct low-res SAM features with crops from a k×k mosaic of unrelated images, and SAM's global attention means features for one sub-image can be influenced by the others and by padding. The paper shows only qualitative 'cleaner features' and never quantifies agreement between mosaic-cropped SAM features and SAM features from a single padded image at the same resolution. Because config B bundles mosaic into multi-resolution training, the +2.8 ADE20k and +3.4 VLM gains are not cleanly attributable to 'matching all teachers at all resolutions' — mosaic might be acting as a regularizer or data augmentation. This is fixable: compare multi-res with padding/crop versus mosaic, and report a fidelity metric. It doesn't invalidate the empirical results, but it does weaken Finding 2's causal narrative. Also, there are no error bars anywhere, and most ablations are single runs, so the smaller effects (SAM inclusion, partitioning) should be read cautiously. Minor: the abstract lists teacher-loss balancing as a novel contribution, but PHI-S is from the authors' own prior work; acceptable to use, not to bill as new.\n\nWho this is for: anyone working on vision backbones for VLMs or dense prediction. The models are strong open baselines. This deserves a serious referee — the empirical work is solid, the releases are reproducible, and the main weakness can be addressed with one careful ablation. Send it to review with a request to add the mosaic ablation and, if possible, variance estimates.","headline":"Solid, reproducible step forward for agglomerative vision encoders, but the multi-resolution causal story needs one more ablation to separate mosaic augmentation from the schedule.","tokens_in":24978,"tokens_out":4869,"would_cite":true,"duration_ms":42449,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By training a student vision encoder to match all of its teachers at every input resolution, this paper eliminates a resolution-dependent \"mode switch\" in agglomerative models and produces encoders that retain accuracy from 256 to 1024…","keywords":["agglomerative vision foundation models","multi-teacher distillation","multi-resolution training","mode switching","mosaic augmentation","token merging","PHI-S standardization","vision-language models"],"falsifier":"Measure SAM's features on a single low-resolution image (padded to 1024² and cropped, as in the paper) versus SAM's features for the same image cropped out of a mosaic: if the normalized distance between these two feature maps is comparable to SAM's own scale variance, the mosaic proxy is sound; if it is much larger, the proxy fails. A second end-to-end test: if the multi-resolution training claim is right, the zero-shot ImageNet accuracy of the released models should be approximately flat from 256² to 1024² inputs, so a reader can run the released weights and check whether accuracy still drops sharply above 512², as it did for the baseline.","tokens_in":23999,"feed_emoji":"🖼️","tokens_out":8600,"duration_ms":70262,"temperature":0.7,"pith_summary":"The paper claims that an agglomerative vision model, one student distilled from multiple teacher networks, can be made accurate across a wide range of input resolutions if it is trained to match every teacher at every resolution. It identifies a \"mode switch\" defect in the prior baseline: at low resolutions the student's features resemble DINO's, and at high resolutions they resemble SAM's, because the student only saw SAM at high resolution during training. The proposed fix is a three-stage multi-resolution schedule, aided by mosaic augmentation that lets SAM features supervise low-resolution student inputs without interpolating SAM's features. The result is a family of encoders whose zero-shot accuracy stays roughly flat from 256 to 1024 pixels, with improved segmentation, 3D probing, and vision-language reasoning scores. A reader should care because it suggests one encoder can serve classification, dense prediction, and VLM tasks without resolution-specific retraining.","feed_headline":"Fix a vision model's mode switch by training at every resolution","feed_subtitle":"A student that matches all teachers at all scales stays accurate from 256 to 1024 pixel inputs, lifting dense and VLM scores","key_machinery":"The machinery has three parts. First, multi-resolution training: a three-stage schedule (256², 432², then simultaneous 432² and 1024²) in which the student's backbone is supervised by all teachers at all resolutions; student features are spatially interpolated down to match low-resolution teachers, and the high-resolution SAM teacher is fed mosaics of $k \\times k$ images whose features are cropped to supervise low-resolution students, avoiding interpolation of SAM features. Second, PHI-S (PCA–Hadamard isotropic standardization), a per-teacher rotation and scaling of feature activations to unit variance, which balances the distillation losses and is inverted by adjusting the adaptor's final projection. Third, output token merging, a one-shot bipartite soft matching of patch tokens at the encoder output, replacing pixel unshuffling, to keep high-resolution information in a fixed token budget for language models.","core_discovery":"The central claim is that resolution mode switching in agglomerative distillation is a training artifact, not an inherent property of multi-teacher models. In the baseline, the student sees the DINO and CLIP teachers only at low resolution and the SAM teacher only at high resolution, so the student alternates between mimicking DINO (low-resolution, semantic) and mimicking SAM (high-resolution, contour-oriented) features. The paper shows that when the student is trained to match all teachers at all resolutions, downsampling student features where the teacher is low-resolution and using mosaic augmentation to keep SAM's high-resolution supervision available for low-resolution student inputs, the mode switch disappears, scale equivariance approaches that of the multi-resolution DINOv2 teacher, and downstream accuracy improves across dense tasks and VLM benchmarks. The paper also reports that PHI-S standardization of teacher feature distributions prevents SAM's large activation variance from dominating the loss, and that a one-shot token-merging step at the encoder output compresses visual tokens for VLMs better than pixel unshuffling, with the largest gains on its own encoder.","pith_inferences":["If mosaic augmentation is a faithful proxy, SAM features are largely local and do not depend on global image context; this could be tested with images whose meaning changes with context, and the limits of the proxy would appear there.","The success of output token merging suggests the encoder's feature space is highly redundant, possibly inherited from SAM's contour-homogeneous features; a testable prediction is that token redundancy drops if SAM is removed from the teacher set.","Resolution robustness might reduce the need for tiling in high-resolution VLM inference, saving latency and token budget, though the paper's tiling experiments still show additional gains from tiling with this encoder.","The per-teacher whitening idea could generalize beyond distillation to any multi-modal or multi-task feature fusion where sources have mismatched scales."],"forward_implications":["If the claim holds, a single vision encoder can serve many downstream tasks at input resolutions from 256 to 1024 pixels without resolution-specific retraining or tiling.","SAM is a useful teacher when its features are not interpolated and its distribution is balanced: including it improves dense tasks such as ADE20k semantic segmentation and 3D probing over omitting it.","Token merging at the encoder output yields better VLM scores than pixel unshuffling at the same token count, and the improvement is larger for this encoder than for SigLIP, suggesting the learned feature space has many redundant tokens.","Feeding intermediate layer activations to non-linear heads such as DPT improves dense prediction, while linear probes and VLM use do not benefit from those activations.","Fewer training partitions, with all teachers in one batch, improves summarization-task quality when compute budget allows."],"supporting_citations":[{"why":"Supplies the baseline agglomerative model analyzed here; the multi-teacher distillation setup and the observed resolution mode-switching behavior come from this work.","marker":"[36]"},{"why":"Supplies PHI-S, the per-teacher PCA–Hadamard standardization used to balance teacher feature distributions and losses.","marker":"[35]"},{"why":"Supplies the bipartite soft matching (ToMeSD) algorithm that the paper adapts for one-shot token merging at the encoder output.","marker":"[4]"},{"why":"The SAM teacher, whose high-resolution features drive the mode-switch phenomenon and motivate the mosaic augmentation for low-resolution distillation.","marker":"[23]"},{"why":"The DINOv2-g-reg teacher, the only multi-resolution-capable teacher, provides the scale-equivariance comparison and one low-resolution supervision source.","marker":"[10]"},{"why":"Provides cropped position embeddings (CPE) that give the student backbone flexible multi-resolution support, required for the training schedule.","marker":"[22]"}],"fun_headline_variants":["Training at every resolution kills vision model mode switching","Multi-res training eliminates mode shift in agglomerative models","Token merging beats unshuffling for vision-language tokens","Mosaic augmentation keeps SAM supervision at low resolution"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Section 4.3 assumes that running the high-resolution SAM teacher on a mosaic of $k \\times k$ small images and cropping out each sub-image's features gives a faithful surrogate for SAM's features on those images seen individually at low resolution, with no harmful cross-image interference; if that fails, low-resolution distillation against SAM breaks down and the multi-resolution fix loses its high-resolution teacher.","fun_headline_variants_meta":{"raw":{"variants":["Training at every resolution kills vision model mode switching","Multi-res training eliminates mode shift in agglomerative models","Token merging beats unshuffling for vision-language tokens","Mosaic augmentation keeps SAM supervision at low resolution"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1385,"prompt_tokens":929,"completion_tokens":456,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":545,"completion_tokens_details":{"reasoning_tokens":393}},"tokens_in":545,"tokens_out":456,"duration_ms":4647,"temperature":1.0,"reasoning_tokens":393,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:36:19.009061+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure SAM's features on a single low-resolution image (padded to 1024² and cropped, as in the paper) versus SAM's features for the same image cropped out of a mosaic: if the normalized distance between these two feature maps is comparable to SAM's own scale variance, the mosaic proxy is sound; if it is much larger, the proxy fails. A second end-to-end test: if the multi-resolution training claim is right, the zero-shot ImageNet accuracy of the released models should be approximately flat from 256² to 1024² inputs, so a reader can run the released weights and check whether accuracy still drops sharply above 512², as it did for the baseline.","supporting_citations":[{"cited_title":"Am-radio: Agglomerative vision foundation model reduce all domains into one","cited_arxiv_id":null,"evidence_quote":"Supplies the baseline agglomerative model analyzed here; the multi-teacher distillation setup and the observed resolution mode-switching behavior come from this work."},{"cited_title":"Phi- s: Distribution balancing for label-free multi-teacher distillation, 2024","cited_arxiv_id":null,"evidence_quote":"Supplies PHI-S, the per-teacher PCA–Hadamard standardization used to balance teacher feature distributions and losses."},{"cited_title":"Token merging for fast stable diffusion.2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPR W), pages 4599–4603, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the bipartite soft matching (ToMeSD) algorithm that the paper adapts for one-shot token merging at the encoder output."},{"cited_title":"Berg, Wan-Yen Lo, Piotr Dollár, and Ross B","cited_arxiv_id":null,"evidence_quote":"The SAM teacher, whose high-resolution features drive the mode-switch phenomenon and motivate the mosaic augmentation for low-resolution distillation."},{"cited_title":"Vision transformers need registers","cited_arxiv_id":null,"evidence_quote":"The DINOv2-g-reg teacher, the only multi-resolution-capable teacher, provides the scale-equivariance comparison and one low-resolution supervision source."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides cropped position embeddings (CPE) that give the student backbone flexible multi-resolution support, required for the training schedule."}],"review_version":1}