Pith. sign in

REVIEW 1 major objections 14 references

Adapting the SAM3 image encoder with a lightweight decoder and selective self-distillation reaches 69.73% composite mIoU on the GOOSE 2D test set.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Reports a 4th-place GOOSE 2D challenge entry adapting SAM3 with self-distillation on select classes and image-level multi-scale TTA, reaching 69.73% mIoU, with photometric distortion as the largest gain source.

T0 review reviewed 2026-06-26 challenge →

load-bearing objection A practical competition report on adapting SAM3 for the GOOSE challenge that hits 69.73% mIoU on the official test set, but the claim about photometric distortion as the top contributor lacks isolated ablations. the 1 major comments →

arxiv 2606.20130 v1 pith:BP6QRTL4 submitted 2026-06-18 cs.CV

SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation

Xuesong Wang This is my paper
classification cs.CV
keywords semantic segmentationself-distillationSAM3fine-grained segmentationtest-time augmentationphotometric distortionGOOSE challenge
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper describes a competition entry that starts from the image encoder of Segment Anything Model 3 and adds only a lightweight decoder to handle fine-grained semantic segmentation. It adds a self-distillation step that lets SAM3 itself serve as teacher, prompted with ground-truth boxes, but only on the classes where validation shows SAM3 already beats the student. An image-level multi-scale test-time augmentation is obtained simply by rescaling the input image rather than the model. The largest single gain comes from transplanting an aggressive photometric distortion pipeline from an earlier winning entry.

Core claim

A model built on the SAM3 image encoder plus lightweight decoder, selective self-distillation from SAM3 prompted with ground-truth boxes on outperforming classes, image-level multi-scale test-time augmentation, and transplanted photometric distortion achieves 69.73% composite mean Intersection-over-Union on the official 1,815-image GOOSE 2D test set, with the photometric distortion identified as the dominant source of improvement.

What carries the argument

The self-distillation scheme that re-uses SAM3, prompted with ground-truth boxes, as teacher only on the classes where it outperforms the student model.

Load-bearing premise

Classes where SAM3 outperforms the student can be identified reliably from validation performance, and distilling from SAM3 prompted with ground-truth boxes produces net positive transfer without introducing label noise or class-specific biases that lower overall mIoU.

What would settle it

Measure mIoU after removing the photometric distortion step; if the score falls substantially below 69.73% while every other component stays fixed, the claim that distortion is the single largest source is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Selective self-distillation from the same foundation model improves mIoU on the chosen classes without requiring a separate teacher network.
  • Rescaling the full image before feeding it to a fixed-input-size model restores multi-scale inference at test time.
  • Transplanting photometric distortion from a prior top entry produces a larger mIoU lift than the other added components combined.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same selective-distillation pattern could be tested on other foundation-model backbones for segmentation tasks where validation already shows uneven class performance.
  • If photometric distortion dominates, then systematic sweeps of color and brightness augmentations may be more cost-effective than architectural changes for this dataset.
  • The image-level multi-scale TTA method removes the need to change model input resolution, which may simplify deployment on hardware with fixed memory limits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 0 minor

Summary. The manuscript reports a 4th-place entry to the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge that achieves 69.73% composite mIoU on the official 1,815-image test set. The method adapts the SAM3 image encoder with a lightweight decoder, adds a class-selective self-distillation procedure that uses box-prompted SAM3 as teacher only on classes where it outperforms the student, introduces image-level multi-scale test-time augmentation, and presents the empirical finding that an aggressive photometric distortion transplanted from a prior winning entry is the single largest source of improvement.

Significance. The official test-set mIoU supplies direct, externally validated evidence of effectiveness on a fine-grained segmentation benchmark. If the attribution of gains to the transplanted photometric distortion is confirmed by isolated controls, the work supplies a concrete, reproducible demonstration that standard augmentation remains high-impact even when starting from a visual foundation model. The selective self-distillation scheme also illustrates a practical way to harvest teacher signals without introducing uniform label noise.

major comments (1)
  1. [Abstract] Abstract: the claim that the transplanted photometric distortion constitutes the single largest source of improvement is load-bearing for the stated contributions yet is unsupported by any ablation that adds or removes the distortion while freezing the SAM3 encoder adaptation, the validation-based class selection for self-distillation, and the image-level multi-scale TTA. Without such isolation, interactions between the augmentation and teacher prompting cannot be ruled out.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful review and the opportunity to clarify the evidential basis for our claims. We address the single major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that the transplanted photometric distortion constitutes the single largest source of improvement is load-bearing for the stated contributions yet is unsupported by any ablation that adds or removes the distortion while freezing the SAM3 encoder adaptation, the validation-based class selection for self-distillation, and the image-level multi-scale TTA. Without such isolation, interactions between the augmentation and teacher prompting cannot be ruled out.

    Authors: We agree that the abstract's attribution of the largest gain to photometric distortion would be strengthened by an ablation that adds or removes only that component while freezing the SAM3 encoder adaptation, the validation-driven class selection for self-distillation, and the image-level multi-scale TTA. Our existing experiments added components sequentially and observed the largest delta from the distortion, but they do not isolate it from potential interactions with the teacher-prompting pipeline. We will run the requested controlled ablation on the validation set, report the results, and revise the abstract (and any related discussion) to reflect the new evidence. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical leaderboard report with no derivations or self-referential chains

full rationale

The manuscript contains no equations, fitted parameters, derivations, or mathematical claims. All reported results are direct measurements of mIoU on an external challenge test set (1,815 images) and an empirical observation about the effect of a transplanted augmentation. The self-distillation scheme and multi-scale TTA are described as engineering choices whose performance impact is measured on the leaderboard rather than derived from prior results by the same authors. No self-citation load-bearing steps, uniqueness theorems, or ansatzes appear. The paper is therefore self-contained against the external benchmark and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical competition entry. It introduces no mathematical derivations, free parameters, axioms, or new postulated entities; all performance claims rest on experimental results obtained on the GOOSE 2D test set.

reviewed 2026-06-26 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation." pith.science (2026). https://pith.science/paper/BP6QRTL4

@misc{pith2026260620130,
  author       = {Pith},
  title        = {Pith review of: SAM3 Self-Distillation for Fine-Grained GOOSE 2D Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BP6QRTL4}},
  note         = {Machine review of arXiv:2606.20130}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We describe our 4th-place entry to the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge, which reached a composite mean Intersection-over-Union (mIoU) of 69.73% on the official 1,815-image test set. Our model adapts the image encoder of a recent visual foundation model, Segment Anything Model 3 (SAM3), with a lightweight decoder. Beyond this, we contribute two techniques and one empirical finding: (i) a self-distillation scheme that re-uses SAM3 itself, prompted with ground-truth boxes, as a teacher on the classes where it outperforms our own model; (ii) an image-level multi-scale test-time augmentation scheme that restores multi-scale inference for a fixed-input-size model by rescaling the image rather than the model input; and (iii) the finding that an aggressive photometric distortion from a winning 2025 GOOSE 2D entry, transplanted onto our pipeline, is its single largest source of improvement.

Figures

Figures reproduced from arXiv: 2606.20130 by Xuesong Wang.

Figure 1
Figure 1. Figure 1: Model architecture. The deployed model is the top row: a partially fine-tuned SAM3 image encoder produces a four-level feature pyramid that [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Qualitative results on the validation set. Each row shows the input image (left), the ground-truth labeling colored by class (center), and our model’s [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

  1. [1]

    The goose dataset for perception in unstructured environments,

    P. Mortimer, R. Hagmanns, M. Granero, T. Luettel, J. Petereit, and H.-J. Wuensche, “The goose dataset for perception in unstructured environments,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2024. [Online]. Available: https://arxiv.org/abs/2310.16788

    work page arXiv 2024

  2. [2]

    Excavating in the wild: The goose-ex dataset for semantic segmentation,

    R. Hagmanns, P. Mortimer, M. Granero, T. Luettel, and J. Petereit, “Excavating in the wild: The goose-ex dataset for semantic segmentation,” inProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2025. [Online]. Available: https://arxiv.org/abs/2409.18788

    work page arXiv 2025

  3. [3]

    SAM 3: Segment anything with concepts,

    N. Carion, L. Gustafson, Y .-T. Hu, S. Debnath, R. Hu, D. S. Coll-Vinent, C. Ryali, K. V . Alwala, H. Khedr, A. Huang, J. Lei, T. Ma, B. Guo, A. Kalla, M. Marks, J. Greer, M. Wang, P. Sun, R. R ¨adle, T. Afouras, E. Mavroudi, K. Xu, T.-H. Wu, Y . Zhou, L. Momeni, R. HAZRA, S. Ding, S. Vaze, F. Porcher, F. Li, S. Li, A. Kamath, H. K. Cheng, P. Dollar, N. R...

    2026

  4. [4]

    Technical report for icra 2025 goose 2d semantic segmentation challenge: Boosting off-road segmentation via photometric distortion and exponential moving average,

    W. Kim, L.-k. Lee, and S.-Y . An, “Technical report for icra 2025 goose 2d semantic segmentation challenge: Boosting off-road segmentation via photometric distortion and exponential moving average,”arXiv preprint arXiv:2505.11769, 2025

    work page arXiv 2025

  5. [5]

    Feature pyramid networks for object detection,

    T.-Y . Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125

    2017

  6. [6]

    Unified perceptual parsing for scene understanding,

    T. Xiao, Y . Liu, B. Zhou, Y . Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 418–434

    2018

  7. [7]

    Ssd: Single shot multibox detector,

    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y . Fu, and A. C. Berg, “Ssd: Single shot multibox detector,” inEuropean conference on computer vision. Springer, 2016, pp. 21–37

    2016

  8. [8]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inInternational Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7

    2019

  9. [9]

    DINOv2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V . V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. HAZIZA, F. Massa, A. El-Nouby, M. Assran, N. Ballas, W. Galuba, R. Howes, P.-Y . Huang, S.-W. Li, I. Misra, M. Rabbat, V . Sharma, G. Synnaeve, H. Xu, H. Jegou, J. Mairal, P. Labatut, A. Joulin, and P. Bojanowski, “DINOv2: Learning robust visual features without ...

    2024

  10. [10]

    Self-supervised learning from images with a joint-embedding predictive architecture,

    M. Assran, Q. Duval, I. Misra, P. Bojanowski, P. Vincent, M. Rabbat, Y . LeCun, and N. Ballas, “Self-supervised learning from images with a joint-embedding predictive architecture,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 15 619–15 629

    2023

  11. [11]

    Internimage: Exploring large-scale vision foundation models with deformable convolutions,

    W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T. Lu, L. Lu, H. Liet al., “Internimage: Exploring large-scale vision foundation models with deformable convolutions,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 14 408–14 419

    2023

  12. [12]

    Masked-attention mask transformer for universal image segmenta- tion,

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmenta- tion,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1290–1299

    2022

  13. [13]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,”Advances in neural information processing sys- tems, vol. 34, pp. 12 077–12 090, 2021

    2021

  14. [14]

    Balancing logit variation for long-tailed semantic segmentation,

    Y . Wang, J. Fei, H. Wang, W. Li, T. Bao, L. Wu, R. Zhao, and Y . Shen, “Balancing logit variation for long-tailed semantic segmentation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 19 561–19 573

    2023

This paper was first reviewed by grok-4.3 on June 26, 2026.