{"id":"e3e3210f-60fc-4afd-84bc-b9cca820862b","arxiv_id":"2509.00509","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"ATGC selects the best input scale for a black-box open-vocabulary segmentation API, using DINOv2 attention entropy, improving one-hot-label distillation on Cityscapes and ACDC.","lead":"This paper introduces a black-box distillation method for semantic segmentation, where a local model learns from an API that returns only one-hot labels. It uses attention-map entropy to pick the best image zoom before querying the API, giving small but consistent improvements over naive transfer.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Entropy-based scale selection (Eq. 3) is not validated as a proxy for pseudo-label quality; without a quantitative correlation, ATGC's gains over random scaling are unexplained.","rationale":"The reader's weakest assumption identified exactly the load-bearing point: the entropy-based proxy in Eq. (3) is not established. Agreement is strong because the paper's own supplementary failure cases (Fig. 13, App. C.5) expose the proxy's unreliability, and the only justification is a speculative appeal to the Platonic Representation Hypothesis. The proposed concrete test directly measures the entropy–quality correlation, which would settle the concern. I also looked for other possible weak points: the improvements over Random are small (1–5 mIoU), no error bars are reported despite 'averaged over 3 random seeds', and the consistency threshold τ is tuned on the validation set (App. C.1). These reinforce the need for conditional acceptance, but they are not the primary mechanism-level risk. If the correlation test fails, ATGC would offer no clear advantage over random scale selection, making the paper's central claim unsupported. Therefore, the verdict should remain conditional pending this quantitative validation; the setting itself is novel and the experiments are honestly reported, so rejection is not warranted.","tokens_in":22013,"tokens_out":8035,"duration_ms":98323,"concrete_test":"On Cityscapes validation (ground truth available), sample ~500 crops. For each crop and each scale in S, compute the DINOv2 attention entropy (used in Eq. 3) and the API pseudo-label accuracy against ground truth for both SAN and CLIP-DINOiser. Compute the per-crop Spearman rank correlation between entropy and PL accuracy across the 13 scales; report the mean, the fraction of crops with positive correlation, and a paired permutation test against zero. If the mean correlation is not significantly positive (or the fraction is near 0.5), the entropy proxy fails. Additionally, compare ATGC's final mIoU with a baseline that samples scales from ATGC's empirical s* distribution instead of using entropy, to check whether the dynamic selection contributes beyond the scale marginal.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central contribution is ATGC, which selects the API query scale s* = argmin_j S(A_j) (Eq. 3) using DINOv2 attention entropy as a proxy for one-hot pseudo-label (PL) quality. The load-bearing assumption is that lower entropy correlates with higher PL accuracy. Evidence for this is only qualitative (Figs. 5, 12) plus an appeal to the Platonic Representation Hypothesis (Sec. 4.1); the paper itself documents counter-examples in Fig. 13 and App. C.5, where the entropy-selected scale yields worse PLs than the original crop (e.g., sharp discontinuities, class bleed). No quantitative correlation between entropy and PL accuracy is reported on labeled validation sets, so it is unknown whether the proxy carries any signal. Without that signal, the selection in Eq. 3 is a deterministic function of image content but not necessarily of PL quality, and the observed margins over Random (50.1 vs 48.6 mIoU for SAN; 37.9 vs 35.5 for CLIP-DINOiser on Cityscapes) could stem from the induced marginal distribution of selected scales rather than per-image optimality. The paper does not ablate this: replacing the entropy score with random selection from the same scale distribution would isolate whether the 'dynamic' ranking actually matters.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes Black-Box Distillation (B2D) for semantic segmentation, in which a local student model is trained using only one-hot pseudo-labels from a black-box open-vocabulary API teacher, without access to logits or internal representations. The authors identify a 'curse of resolution': different object classes are segmented best at different input scales. Their method, ATGC, uses a frozen DINOv2 student encoder to compute attention-map entropy at multiple scales and selects the scale with lowest entropy as the query scale for the API; pseudo-labels are then filtered by agreement with the student and used to train the student via cross-entropy. Experiments on Cityscapes and ACDC with SAN and CLIP-DINOiser as APIs show that ATGC outperforms Naive Transfer, Random, Average, and a reimplemented CoRTE baseline, while being below an Oracle that selects the best pseudo-labels using ground truth. The paper openly discusses limitations and shows failure cases.","tokens_in":22388,"tokens_out":4126,"duration_ms":52616,"significance":"If the core mechanism is validated, the paper makes a useful practical contribution: it addresses a realistic constrained setting (API returns only hard labels), provides a reproducible pipeline (code released), and introduces a scale-selection method that does not require any API-internal information. The comparison against an Oracle and the inclusion of two different API models are commendable, as is the candid reporting of failure cases in App. C.5. However, the main empirical gains are modest (1.3-2.4 mIoU over Random on Cityscapes), and the central proxy assumption that DINOv2 attention entropy ranks pseudo-label quality is supported only by qualitative examples and an appeal to the Platonic Representation Hypothesis, not by a quantitative correlation. The hyperparameters (tau and the scale grid S) are selected on the same validation sets used for evaluation, and no variance across seeds is reported. These issues currently leave the central claim under-supported, though the approach is plausible and the gaps are addressable.","major_comments":[{"comment":"The method's load-bearing assumption is that the scale minimizing attention entropy also minimizes pseudo-label error. This is not validated quantitatively. Figs. 5 and 12 show selected examples, but the paper itself documents counterexamples in Fig. 13 and App. C.5 where the entropy-selected scale yields worse pseudo-labels than the original crop. The appeal to the Platonic Representation Hypothesis is not a substitute for a direct measurement. Please report, on a labeled validation set (e.g., Cityscapes val), a per-image and per-scale correlation between attention entropy and API pseudo-label accuracy (pixel accuracy or mIoU), and the fraction of images where argmin entropy matches argmax accuracy. Additionally, to show that the ranking—not just the induced scale distribution—matters, ablate Eq. (3) by replacing it with a random selection from the same per-image scale distribution (i.e","section":"Sec. 4.1, Eq. (3)"},{"comment":"The filtering threshold tau=0.7 is chosen by an ablation on the same Cityscapes/CLIP-DINOiser validation set used for the main results, and the scale grid S is hand-specified with no sensitivity analysis. Moreover, although Tables 2-3 state results are averaged over 3 random seeds, no per-seed values, standard deviations, or confidence intervals are provided. With margins of only 1.3-2.4 mIoU over Random, the statistical significance of the improvement is unclear. Please provide per-seed results or error bars, and report sensitivity to tau and to the composition of S (e.g., a coarser or finer grid). If possible, select tau on a separate validation split or use a procedure that does not reuse the test/validation set for hyperparameter choice.","section":"Sec. 5.1 and App. C.1, Tables 2-3"},{"comment":"It is unclear whether the pixel-level consistency filter (IoU >= tau) is applied to all baselines or only to ATGC. Algorithm 2 describes the filtering as part of ATGC's training, but the definitions of Naive Transfer, Random, and Average in Sec. 5.1 do not explicitly state that they use the same filtering. If the baselines are trained without filtering while ATGC is trained with it, the comparison conflates scale selection with filtering. Please state explicitly whether each baseline uses the same tau and the same filtering procedure. In addition, provide an ablation isolating the contributions: ATGC with and without filtering, and Random with and without filtering. This is important because the student model used for the consistency filter is itself trained on API pseudo-labels, so the filter is not independent of the teacher.","section":"Sec. 4.2, Algorithm 2, and Sec. 5.1 baselines"}],"minor_comments":[{"comment":"In the sentence 'a phenomenon we examine in 3', the cross-reference is incomplete; it should point to the relevant subsection or appendix.","section":"Sec. 5.3"},{"comment":"The caption says 'The resolution with the highest spatially averaged attention score is selected', but the method (Sec. 4.1, Eq. 3) selects the scale with the lowest entropy. Please align the caption with the method.","section":"Fig. 5 caption"},{"comment":"The Oracle uses ground truth to select the pseudo-label with the highest pixel accuracy, while evaluation is mIoU. Pixel accuracy can favor larger classes; this mismatch should be noted so readers do not interpret the Oracle as an upper bound on mIoU.","section":"Sec. 5.1, Oracle baseline"},{"comment":"The tau ablation is reported only for CLIP-DINOiser. Reporting it for SAN as well would help confirm that tau=0.7 is not API-specific.","section":"App. C.1"},{"comment":"The paper would benefit from stating explicitly for each baseline whether the same scale grid S and the same prompt ensembling are used. Currently the experimental setup gives details for ATGC but is less explicit for baselines.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practical setting, and the authors are honest about limitations. My main concern is that the central mechanism—entropy-based scale selection—is not quantitatively validated as a proxy for pseudo-label quality; the documented failure cases make this more than a formality. The empirical margins are small, and the lack of variance reporting plus potential baseline asymmetries in filtering make the strength of the evidence hard to assess. A revision that adds a correlation analysis, a random-scale-distribution ablation, and per-seed results would substantially strengthen the paper. The scope and framing are appropriate for a computer vision venue, and I do not see a fundamental correctness error that would justify rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should read this paper if you care about distilling from closed APIs. The B2D setting – only one-hot predictions, open-vocabulary teacher – is a fair formalization of what commercial APIs actually give you, and it is a step beyond the “gray-box” assumption in CoRTE. The paper is honest about what it does and does not do.\n\nATGC uses DINOv2 attention entropy to pick a scale for the API query. The empirical gains are real but modest: 50.1 vs. 48.8 (Naive Transfer) and 48.6 (Random) on Cityscapes with SAN, with similar margins elsewhere. The paper compares against a reimplemented CoRTE that has access to logits, and ATGC beats it without logits. That is a solid result for the setting.\n\nWhere I agree with the skeptic: the load-bearing assumption that lower attention entropy means better pseudo-labels is not properly validated. You get qualitative examples and an appeal to the Platonic Representation Hypothesis, but no quantitative correlation on labeled validation data. The paper itself documents failure cases in Fig. 13 where the entropy-selected scale gives worse predictions. So the mechanism could be doing something simpler than per-image optimality – for example, just skewing the selected-scale distribution toward scales that are globally good. The fixed-scale grid in Fig. 6 suggests ATGC is not finding optimal scales for several classes, which is consistent with that concern. The paper deserves credit for including these failure cases and a limitation discussion, but the central claim that entropy is a reliable proxy is not yet earned.\n\nThe other soft spots are minor: no error bars despite three seeds, tau=0.7 tuned on the same validation sets, and the scale grid is hand-picked. None of these break the central result; they just weaken the quantitative claims.\n\nBottom line: this is a solid submission for the segmentation/distillation crowd. The setting is useful, the baseline comparisons are fair, and the writing is candid. It should go to peer review. I would ask the authors for a direct validation of the entropy–pseudo-label-quality correlation and an ablation that matches the marginal distribution of selected scales (e.g., random selection from the same scale frequencies). That would settle the load-bearing question.","headline":"A realistic new black-box distillation setting with a modest, honest, but incompletely validated scale-selection trick; worth refereeing, but the core entropy proxy needs quantitative support.","tokens_in":22840,"tokens_out":2321,"would_cite":true,"duration_ms":26654,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper introduces Black-Box Distillation for semantic segmentation, training a specialized local model from only one-hot outputs of an open-vocabulary API, and shows that picking the API’s input scale with DINOv2 attention-map entropy b","keywords":["Black-Box Distillation","semantic segmentation","open-vocabulary segmentation","pseudo-labels","resolution selection","DINOv2 attention maps","one-hot API outputs","knowledge distillation"],"falsifier":"Take a labeled set of target images; for each crop compute ATGC’s argmin-entropy scale and the oracle-best scale (the one whose API pseudo-labels have the highest pixel accuracy against ground truth). If the entropy-selected scale’s accuracy is not positively correlated with the oracle-best scale’s accuracy, or if students trained on entropy-selected pseudo-labels do not beat students trained on fixed scale-1 pseudo-labels on average, the central claim is falsified.","tokens_in":21977,"feed_emoji":"🎯","tokens_out":6695,"duration_ms":80558,"temperature":0.7,"pith_summary":"The paper proposes a new setting, Black-Box Distillation (B2D), in which a local semantic segmentation model is trained using nothing but one-hot segmentation maps returned by an open-vocabulary API—no weights, no logits, no internal representations. It argues that the central obstacle in this setting is the “curse of resolution”: different object classes are segmented best at very different input scales, so no single query resolution yields high-quality pseudo-labels for every pixel. To overcome this, the paper introduces ATGC, which runs a frozen DINOv2 encoder at several scales, scores the resulting attention maps by Shannon entropy, and sends the lowest-entropy crop to the API for pseudo-labelling. The contribution matters because it matches the constraints real commercial APIs impose, and the experiments show ATGC improving over naive fixed-scale transfer and over a reimplemented logit-requiring baseline—for example 50.1 vs 48.8 mIoU on Cityscapes with SAN—while staying within one-hot access.","feed_headline":"Entropy-guided zoom beats logit-based distillation from black-box APIs","feed_subtitle":"A new black-box distillation setting picks the best input scale with DINOv2 attention, beating fixed-scale and logit-based baselines.","key_machinery":"ATGC (ATtention-Guided sCaler) is a plug-and-play scale selector. For each crop it computes DINOv2 attention maps at N scales, treats each averaged [CLS]-patch attention map as a spatial probability distribution, and picks s* = argmin of its Shannon entropy; the API is queried at that scale and the pseudo-labels are rescaled to the original crop. The load-bearing logic is that a low-entropy, peaked attention map indicates a scale where the encoder recognizes objects, and—via the Platonic Representation Hypothesis—that recognition is treated as a proxy for the black-box API’s own segmentation quality.","core_discovery":"ATGC establishes that open-vocabulary black-box teachers are strongly scale-dependent in segmentation: per-class IoU peaks vary from low scales for large context classes like “road” to high scales for small distant classes like “traffic sign”. Given an unlabeled target image crop, ATGC rescales it to each candidate scale, extracts DINOv2’s [CLS]-to-patch attention maps averaged over heads, normalizes each map as a distribution, and selects the scale with minimal entropy. That scale is then used to query the API and produce one-hot pseudo-labels, which are rescaled back and used as hard supervision for a student network, with a pixel-level agreement filter between API and student predictions.","pith_inferences":["Editorial inference: because ATGC selects one scale per crop rather than per class, it will systematically favor whatever class dominates the attention map; a natural extension is class-level or region-level scale mixing, and the paper’s own failure cases point in this direction.","Editorial inference: if DINOv2 entropy is a reliable proxy for API labeling quality, the same selector should transfer to other dense-prediction tasks such as depth estimation, panoptic segmentation, or instance segmentation, and to other frozen self-supervised encoders.","Editorial inference: a direct correlation test—measuring DINOv2 attention entropy against API pseudo-label pixel accuracy across scales on a labeled validation set—would sharpen the Platonic Representation Hypothesis from a heuristic into a quantitative design rule; the paper gives qualitative examples but not a systematic correlation.","Editorial inference: the offline caching of attention maps means ATGC can be cost-aware in practice—only the selected scale is ever sent to the paid API—though the paper does not formally measure API-call budgets."],"forward_implications":["If B2D holds, segmentation APIs that return only one-hot outputs can still be distilled into deployable local models; no logit access is required.","Because the teacher is open-vocabulary, the same pipeline can specialize a student to any user-chosen class vocabulary, rather than a fixed pretraining label set.","Pseudo-label quality in black-box segmentation distillation can be improved more by choosing the query scale than by confidence thresholding; ATGC’s scale mining plus a simple consistency filter captures much of the gap to the Oracle bound in the paper’s experiments.","Attention maps can be computed once per dataset, independent of the API model, so the scale-selection step can be amortized offline rather than costing extra API queries during training.","The domain-generalization results—training on Cityscapes and evaluating on ACDC outperforming direct ACDC training—suggest that larger unlabeled pools can outweigh noisy pseudo-labels under adverse-condition shifts."],"supporting_citations":[{"why":"Defines the closest prior black-box segmentation adaptation and the logit-requiring baseline that ATGC is compared against and outperforms.","marker":"[18]"},{"why":"SAN, the open-vocabulary segmentation model used as the main black-box API teacher in the Cityscapes and ACDC experiments.","marker":"[64]"},{"why":"CLIP-DINOiser, a training-free open-vocabulary model used as a second, harder black-box teacher to test generality.","marker":"[62]"},{"why":"DINOv2 supplies the frozen student encoder and the attention maps that ATGC scores by entropy to choose scales.","marker":"[46]"},{"why":"The Platonic Representation Hypothesis is the stated justification for why student attention maps can proxy the black-box API’s behavior.","marker":"[30]"},{"why":"Prior work documenting that segmentation quality varies with input resolution, grounding the curse-of-resolution observation.","marker":"[29]"},{"why":"Related work naming the “resolution curse” for vision-language open-vocabulary segmentation, which the paper extends to black-box distillation.","marker":"[72]"}],"fun_headline_variants":["Attention entropy picks the zoom that unlocks black-box distillation","Beat logits: Scale selection via DINOv2 attention for one-hot APIs","Curse of resolution solved with DINOv2 attention maps for distillation","Choose the right scale: Distilling from open-vocabulary APIs with entropy","Rescale smart: ATGC uses attention entropy to boost black-box student"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method bets that DINOv2 attention entropy reliably predicts which input zoom will make the black-box API label a crop most accurately; when that correlation fails, the selected scale can produce worse pseudo-labels than the original crop, as the paper’s own failure figures show.","fun_headline_variants_meta":{"raw":{"variants":["Attention entropy picks the zoom that unlocks black-box distillation","Beat logits: Scale selection via DINOv2 attention for one-hot APIs","Curse of resolution solved with DINOv2 attention maps for distillation","Choose the right scale: Distilling from open-vocabulary APIs with entropy","Rescale smart: ATGC uses attention entropy to boost black-box student"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1393,"prompt_tokens":780,"completion_tokens":613,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":518}},"tokens_in":524,"tokens_out":613,"duration_ms":7797,"temperature":1.0,"reasoning_tokens":518,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:30:09.403393+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a labeled set of target images; for each crop compute ATGC’s argmin-entropy scale and the oracle-best scale (the one whose API pseudo-labels have the highest pixel accuracy against ground truth). If the entropy-selected scale’s accuracy is not positively correlated with the oracle-best scale’s accuracy, or if students trained on entropy-selected pseudo-labels do not beat students trained on fixed scale-1 pseudo-labels on average, the central claim is falsified.","supporting_citations":[{"cited_title":"Cross-domain transfer learning with corte: Consistent and reliable transfer from black-box to lightweight segmentation model","cited_arxiv_id":null,"evidence_quote":"Defines the closest prior black-box segmentation adaptation and the logit-requiring baseline that ATGC is compared against and outperforms."},{"cited_title":"Side adapter network for open- vocabulary semantic segmentation","cited_arxiv_id":null,"evidence_quote":"SAN, the open-vocabulary segmentation model used as the main black-box API teacher in the Cityscapes and ACDC experiments."},{"cited_title":"Clip-dinoiser: Teaching clip a few dino tricks for open-vocabulary semantic segmentation","cited_arxiv_id":null,"evidence_quote":"CLIP-DINOiser, a training-free open-vocabulary model used as a second, harder black-box teacher to test generality."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"DINOv2 supplies the frozen student encoder and the attention maps that ATGC scores by entropy to choose scales."},{"cited_title":"Hrda: Context-aware high-resolution domain-adaptive se- mantic segmentation","cited_arxiv_id":null,"evidence_quote":"Prior work documenting that segmentation quality varies with input resolution, grounding the curse-of-resolution observation."},{"cited_title":"MROVSeg: Breaking the Resolution Curse of Vision-Language Models in Open-Vocabulary Image Segmentation","cited_arxiv_id":"2408.14776","evidence_quote":"Related work naming the “resolution curse” for vision-language open-vocabulary segmentation, which the paper extends to black-box distillation."}],"review_version":1}