pith. sign in

arxiv: 2606.23113 · v1 · pith:A466GFF3new · submitted 2026-06-22 · 💻 cs.CV

Technical Report for the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge: Pretraining-Diverse Ensemble of Foundation Vision Encoders for Robust Outdoor Scene Understanding

Boyan Wang , Yongxi Huang , Wenjing Li , Tianrui Hui , Shaofei Huang , Nan Pu , Zhun Zhong This is my paper

Pith reviewed 2026-06-26 08:42 UTC · model grok-4.3

classification 💻 cs.CV
keywords semantic segmentationfoundation modelsensemble learningoutdoor scenesfine-grained categoriespretraining objectivesvision encodersMask2Former
0
0 comments X

The pith

Encoder pretraining recipe dominates accuracy over model size or decoder in outdoor fine-grained segmentation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The report shows that an ensemble of three foundation vision encoders pretrained with distinct objectives can parse unstructured outdoor scenes into 56 fine-grained categories with high accuracy. By weighting the encoders per class using validation IoU and pairing them with a standard decoder plus strong training and test-time augmentation, the method reaches 75.40 percent mIoU on the official test set. The authors conclude that pretraining differences drive performance more than parameter count or decoder architecture. This matters for building robust perception systems where choosing complementary pretraining sources can substitute for simply scaling models.

Core claim

Our submission pairs foundation vision encoders including DINOv3, SigLIP2, and InternImage with a Mask2Former decoder, trains them using long schedules, exponential moving average, larger crop size, and multi-scale plus flip test-time augmentation, and combines the three via per-class validation-IoU weighting to exploit complementary pretraining objectives, achieving 75.40 percent composite mIoU and second place while demonstrating that the encoder pretraining recipe is the dominant accuracy factor rather than parameter count or decoder design.

What carries the argument

Pretraining-diverse ensemble of foundation vision encoders (DINOv3, SigLIP2, InternImage) combined by per-class validation-IoU weighting

If this is right

  • Encoders with distinct pretraining objectives cover fine-grained outdoor categories more effectively than any one encoder alone.
  • Long training schedules, EMA, larger crops, and multi-scale flip TTA reliably improve the ensemble output.
  • Pretraining choice can be prioritized over increasing model parameters when accuracy on this benchmark is the goal.
  • Per-class IoU weighting produces a combination that generalizes from validation to the official test distribution.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The result suggests selecting future encoders by pretraining data diversity rather than architecture scale when facing unstructured outdoor scenes.
  • The same weighting approach could be tested on other segmentation benchmarks to check whether pretraining complementarity reduces the need for task-specific data.
  • If pretraining remains dominant, lighter decoders might suffice once the right encoder mix is chosen.

Load-bearing premise

The three encoders supply enough complementary information from their different pretraining objectives that per-class validation-IoU weighting will combine them without overfitting to the validation set.

What would settle it

A single encoder or a different weighting scheme matching or exceeding 75.40 percent mIoU on the unseen test set would show that the pretraining-diverse ensemble is not required.

Figures

Figures reproduced from arXiv: 2606.23113 by Boyan Wang, Nan Pu, Shaofei Huang, Tianrui Hui, Wenjing Li, Yongxi Huang, Zhun Zhong.

Figure 1
Figure 1. Figure 1: Overall architecture of our solution. Three encoders with delib [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
read the original abstract

This report presents our solution for the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge, which requires parsing unstructured outdoor scenes from four camera platforms into 56 fine-grained categories. Our approach pairs foundation vision encoders (including DINOv3, SigLIP2, and InternImage) with a Mask2Former decoder, and trains them with a strong recipe including long training schedules, exponential moving average, a larger crop size, and multi-scale plus flip test-time augmentation. The three encoders, chosen for their complementary pretraining objectives, are combined into a pretraining-diverse ensemble through per-class validation-IoU weighting. Evaluated on the official GOOSE test set, our submission achieves 75.40% composite mIoU and wins the second place of the challenge. Our study further shows that the encoder's pretraining recipe, rather than its parameter count or the decoder design, is the dominant factor for accuracy on this benchmark.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. This technical report describes a solution for the ICRA 2026 GOOSE 2D Fine-Grained Semantic Segmentation Challenge that ensembles three foundation encoders (DINOv3, SigLIP2, InternImage) paired with Mask2Former decoders. The encoders are selected for complementary pretraining objectives and combined via per-class validation-IoU weighting; the system is trained with long schedules, EMA, large crops, and multi-scale/flip TTA. The submission reports 75.40% composite mIoU on the official held-out test set and second place in the challenge. The authors further claim that encoder pretraining recipe, rather than parameter count or decoder design, is the dominant accuracy factor on this benchmark.

Significance. The reported test-set result is a concrete, externally validated performance point on a challenging 56-class outdoor segmentation task across multiple camera platforms. If the pretraining-dominance interpretation were supported by controlled comparisons, the work would usefully highlight the value of pretraining diversity in ensembles for robust scene understanding. As written, the empirical result stands but the interpretive claim does not add substantial new insight beyond the leaderboard placement.

major comments (2)
  1. [Abstract] Abstract: the central claim that 'the encoder's pretraining recipe, rather than its parameter count or the decoder design, is the dominant factor' is unsupported. The manuscript presents only a single ensemble configuration (all three encoders with Mask2Former under identical training/TTA) and its test mIoU; no tables or sections isolate pretraining effects while holding model size and decoder fixed, nor compare the three encoders individually under matched conditions.
  2. [Abstract / method description] The complementarity assumption underlying the ensemble (distinct pretraining objectives plus per-class IoU weighting) is stated but not tested for robustness. No ablation shows that removing any one encoder or altering the weighting scheme materially changes test performance, leaving open whether the reported gain is due to pretraining diversity or simply to ensembling three strong models.
minor comments (2)
  1. [Abstract] Abstract and results section: no error bars, run-to-run variance, or statistical tests accompany the 75.40% mIoU figure, making it difficult to assess whether the second-place margin is reliable.
  2. [method / experiments] The manuscript would benefit from a short table listing the three encoders' individual parameter counts, pretraining datasets/objectives, and (if available) their standalone validation mIoU under the shared training recipe.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on the strength of our interpretive claims. We agree that the manuscript does not contain the controlled experiments needed to support the stated conclusions about pretraining dominance and ensemble complementarity, and we will revise the text accordingly.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that 'the encoder's pretraining recipe, rather than its parameter count or the decoder design, is the dominant factor' is unsupported. The manuscript presents only a single ensemble configuration (all three encoders with Mask2Former under identical training/TTA) and its test mIoU; no tables or sections isolate pretraining effects while holding model size and decoder fixed, nor compare the three encoders individually under matched conditions.

    Authors: We agree that the central claim is unsupported by the experiments in the manuscript, which reports only the final ensemble result. We will revise the abstract to remove this interpretive statement and focus solely on the empirical test-set performance and the ensemble design choices. revision: yes

  2. Referee: [Abstract / method description] The complementarity assumption underlying the ensemble (distinct pretraining objectives plus per-class IoU weighting) is stated but not tested for robustness. No ablation shows that removing any one encoder or altering the weighting scheme materially changes test performance, leaving open whether the reported gain is due to pretraining diversity or simply to ensembling three strong models.

    Authors: We acknowledge that no ablations on encoder removal or weighting variations are included. We will revise the method description to present the encoder selection (based on distinct pretraining objectives) and per-class weighting as design decisions motivated by the challenge setting, without claiming untested robustness or complementarity. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical competition report with external test evaluation

full rationale

The paper is a technical report describing an ensemble method for a semantic segmentation challenge. It reports measured mIoU on the official held-out test set after training on provided data. No mathematical derivations, equations, or 'predictions' appear. The per-class validation-IoU weighting is a standard post-hoc combination step using separate validation data; it does not reduce any claimed result to itself by construction. No self-citation load-bearing steps, uniqueness theorems, or ansatzes are invoked to justify central claims. The dominance statement about pretraining is an empirical assertion based on the single reported system, but this is an evidence gap rather than circularity. The derivation chain is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that the three encoders' pretraining objectives are complementary and that validation-based weighting transfers to test data. No new entities or free parameters beyond standard ensemble weighting are introduced.

axioms (1)
  • domain assumption Foundation encoders pre-trained on different objectives will exhibit complementary errors on fine-grained outdoor segmentation.
    Invoked to justify the choice of DINOv3, SigLIP2, and InternImage and the ensemble strategy.

pith-pipeline@v0.9.1-grok · 5736 in / 1074 out tokens · 20221 ms · 2026-06-26T08:42:04.778342+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 3 canonical work pages · 2 internal anchors

  1. [1]

    Masked-attention mask transformer for universal image segmenta- tion,

    B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Girdhar, “Masked-attention mask transformer for universal image segmenta- tion,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 1290–1299

    2022

  2. [2]

    DINOv3

    O. Sim ´eoni, H. V . V o, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V . Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa,et al., “Dinov3,” arXiv preprint arXiv:2508.10104, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features

    M. Tschannen, A. Gritsenko, X. Wang, M. F. Naeem, I. Alabdul- mohsin, N. Parthasarathy, T. Evans, L. Beyer, Y . Xia, B. Mustafa,et al., “Siglip 2: Multilingual vision-language encoders with improved se- mantic understanding, localization, and dense features,”arXiv preprint arXiv:2502.14786, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  4. [4]

    Internimage: Exploring large-scale vision foundation models with deformable convolutions,

    W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T. Lu, L. Lu, H. Li,et al., “Internimage: Exploring large-scale vision foundation models with deformable convolutions,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 14 408–14 419

    2023

  5. [5]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 012–10 022

    2021

  6. [6]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11 976–11 986

    2022

  7. [7]

    Convnext v2: Co-designing and scaling convnets with masked au- toencoders,

    S. Woo, S. Debnath, R. Hu, X. Chen, Z. Liu, I. S. Kweon, and S. Xie, “Convnext v2: Co-designing and scaling convnets with masked au- toencoders,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 16 133–16 142

    2023

  8. [8]

    Modality adaptation via feature difference learning for depth human parsing,

    S. Huang, T. Hui, Y . Gong, F. Peng, Y . Fang, J. Wang, B. Ma, X. Wei, and J. Han, “Modality adaptation via feature difference learning for depth human parsing,”Computer Vision and Image Understanding, vol. 247, p. 104070, 2024

    2024

  9. [9]

    Transferring clip’s knowledge into zero-shot point cloud semantic segmentation,

    Y . Wang, S. Huang, Y . Gao, Z. Wang, R. Wang, K. Sheng, B. Zhang, and S. Liu, “Transferring clip’s knowledge into zero-shot point cloud semantic segmentation,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 3745–3754

    2023

  10. [10]

    Fully convolutional networks for semantic segmentation,

    J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431–3440

    2015

  11. [11]

    Ordnet: Capturing omni-range dependencies for scene parsing,

    S. Huang, S. Liu, T. Hui, J. Han, B. Li, J. Feng, and S. Yan, “Ordnet: Capturing omni-range dependencies for scene parsing,”IEEE Transactions on Image Processing, vol. 29, pp. 8251–8263, 2020

    2020

  12. [12]

    Unified perceptual parsing for scene understanding,

    T. Xiao, Y . Liu, B. Zhou, Y . Jiang, and J. Sun, “Unified perceptual parsing for scene understanding,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 418–434

    2018

  13. [13]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,” vol. 34, 2021, pp. 12 077–12 090

    2021

  14. [14]

    Revisiting audio-visual segmentation with vision-centric transformer,

    S. Huang, R. Ling, T. Hui, H. Li, X. Zhou, S. Zhang, S. Liu, R. Hong, and M. Wang, “Revisiting audio-visual segmentation with vision-centric transformer,” inProceedings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 8352–8361

    2025

  15. [15]

    C.-C. Hsu, I. Wu, W.-H. Tseng, C.-H. Cheng, M.-H. Wu, J.-H. Jiang, Y .-J. Hsiao,et al., “Technical report for icra 2025 goose 2d semantic segmentation challenge: Leveraging color shift correction, rope-swin backbone, and quantile-based label denoising strategy for robust outdoor scene understanding,”arXiv preprint arXiv:2505.06991, 2025

    work page arXiv 2025

  16. [16]

    Mmsegmentation: Openmmlab se- mantic segmentation toolbox and benchmark,

    MMSegmentation Contributors, “Mmsegmentation: Openmmlab se- mantic segmentation toolbox and benchmark,” https://github.com/ open-mmlab/mmsegmentation, 2020

    2020

  17. [17]

    Multimodal autoregressive pre-training of large vision encoders,

    E. Fini, M. Shukor, X. Li, P. Dufter, M. Klein, D. Haldimann, S. Aitharaju, V . G. T. da Costa, L. B´ethune, Z. Gan,et al., “Multimodal autoregressive pre-training of large vision encoders,” pp. 9641–9654, 2025

    2025

  18. [18]

    Demystifying clip data,

    H. Xu, S. Xie, X. Tan, P.-Y . Huang, R. Howes, V . Sharma, S.-W. Li, G. Ghosh, L. Zettlemoyer, and C. Feichtenhofer, “Demystifying clip data,” inInternational Conference on Learning Representations, vol. 2024, 2024, pp. 47 812–47 831

    2024

  19. [19]

    Eva-02: A visual representation for neon genesis,

    Y . Fang, Q. Sun, X. Wang, T. Huang, X. Wang, and Y . Cao, “Eva-02: A visual representation for neon genesis,” vol. 149. Elsevier, 2024, p. 105171

    2024

  20. [20]

    Swin transformer v2: Scaling up capacity and resolution,

    Z. Liu, H. Hu, Y . Lin, Z. Yao, Z. Xie, Y . Wei, J. Ning, Y . Cao, Z. Zhang, L. Dong,et al., “Swin transformer v2: Scaling up capacity and resolution,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 12 009–12 019

    2022

  21. [21]

    The goose dataset for perception in unstructured environments,

    P. Mortimer, R. Hagmanns, M. Granero, T. Luettel, J. Petereit, and H.-J. Wuensche, “The goose dataset for perception in unstructured environments,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 14 838–14 844

    2024