GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain

Jyothiraditya Lingam; Nikhileswara Rao Sulake; Sai Manikanta Eswar Machara

REVIEW 1 major objections 1 minor 16 references

GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain

T0 review · 1 major / 1 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GOOSE-M2F adapts Mask2Former with 200 queries, a refinement module, and auxiliary supervision to reach 70.08% mIoU on long-tailed 64-class outdoor segmentation.

desk verdict GOOSE-M2F reaches third on the GOOSE leaderboard via Mask2Former adaptations, but attribution of gains to the new modules is not supported by ablations. read the letter →

arxiv 2606.15937 v2 pith:IVCDWRRR submitted 2026-06-14 cs.CV

Jyothiraditya Lingam , Nikhileswara Rao Sulake , Sai Manikanta Eswar Machara This is my paper · ORCID

classification cs.CV

keywords Mask2Formersemanticsegmentationlong-taileddistributionfine-grainedoutdoorterrainobjectqueriesfeaturerefinement

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper demonstrates an adaptation of Mask2Former for the GOOSE 2D Fine-Grained Semantic Segmentation challenge involving 64 classes in unstructured outdoor terrain with severe long-tailed imbalance where rare classes occupy fewer than 50 pixels per image. It extends the baseline by raising object queries to 200, inserting a Feature Refinement Module that pairs ASPP-lite with CBAM dual-attention, and adding an Auxiliary Supervision Head for direct per-pixel gradients on rare classes. These elements combine with a multi-stage training regimen using distribution-balanced loss, rare-class copy-paste, dynamic re-weighting, and EMA, followed by sliding-window inference with Gaussian blending and 4-scale test-time augmentation. A sympathetic reader would care because the approach targets a practical robotics setting where accurate recognition of infrequent terrain elements matters for navigation safety.

What carries the argument

The Feature Refinement Module (FRM) that merges ASPP-lite with CBAM dual-attention, paired with 200 object queries and an auxiliary per-pixel supervision head, to improve representation and gradients for rare classes under long-tailed distributions.

What would settle it

An ablation that trains the plain Mask2Former baseline with the identical multi-stage recipe and inference engine and then checks whether composite mIoU stays near 70% or falls well below it on the GOOSE test set.

Watch

Extended reading notes

Core claim

Extending Swin-Large Mask2Former with 200 object queries to avoid saturation, a Feature Refinement Module combining ASPP-lite and CBAM, and an Auxiliary Supervision Head, together with multi-stage training and dense inference, produces 70.08% Official Composite mIoU (63.55% fine, 76.61% coarse) and third place on the GOOSE 2D FGSS leaderboard.

Load-bearing premise

The listed architectural changes and training steps, rather than hyperparameter search or the Swin-Large backbone alone, are the main sources of the mIoU improvement.

Editorial extensions

If this is right

Raising object queries to 200 removes representational saturation when modeling 64 classes.
The FRM supplies refined features that help distinguish fine-grained terrain details.
The auxiliary head supplies direct gradients that improve learning of classes with very few pixels.
Distribution-balanced loss combined with rare-class copy-paste reduces the impact of long-tailed imbalance.
Sliding-window inference with Gaussian blending and multi-scale TTA contributes an additional 10.57% to the final score.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same query increase and auxiliary head could be tested on other long-tailed segmentation datasets outside outdoor robotics.
The sliding-window blending technique may transfer to high-resolution tasks such as aerial or medical image analysis.
The FRM pattern could be swapped into other transformer-based segmentors to check for similar gains on imbalanced data.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

GOOSE-M2F reaches third on the GOOSE 2D FGSS leaderboard with 70.08% composite mIoU by extending Mask2Former. The authors increase object queries to 200, add a Feature Refinement Module with ASPP-lite and CBAM, insert an auxiliary supervision head, and apply a multi-stage training recipe that includes distribution-balanced loss, rare-class copy-paste, dynamic re-weighting, and EMA. At inference they use sliding-window processing with Gaussian blending and 4-scale TTA.

The paper does one thing clearly: it ships a working system for a 64-class long-tailed outdoor segmentation task and releases both code and trained models. That makes the result immediately usable for anyone who needs a strong baseline on the GOOSE benchmark or similar unstructured terrain data.

The soft spot is the missing evidence on what actually drove the score. The abstract and the stress-test note both indicate no ablation tables that isolate the three listed additions from the Swin-Large backbone, the loss functions, the augmentation choices, or the inference pipeline. In a severely long-tailed setting, it is possible the bulk of the lift comes from tuning or the base architecture rather than the targeted modules. Without those breakdowns the central claim about the three contributions stays under-supported.

This paper is for people already working on the GOOSE challenge or on fine-grained segmentation for robotics. A reader who wants the leaderboard number and the public implementation will find it useful. It does not reorganize the field or introduce new theory.

I would bring it to a reading group as maybe. I would not cite it in my own work in the next year unless I needed the exact baseline. It deserves peer review because the empirical result is verifiable with the released artifacts, even if the analysis of contributions needs tightening.

Referee Report

1 major / 1 minor

Summary. The paper presents GOOSE-M2F, a task-specific adaptation of Mask2Former (Swin-Large backbone) for the GOOSE 2D FGSS benchmark involving 64 fine-grained classes in long-tailed unstructured outdoor terrain. It introduces three targeted extensions—200 object queries, a Feature Refinement Module (FRM) combining ASPP-lite and CBAM, and an Auxiliary Supervision Head—paired with a multi-stage training recipe (Distribution-Balanced loss, Rare-Class Copy-Paste, dynamic re-weighting, EMA) and inference pipeline (sliding-window with Gaussian blending and 4-scale TTA). The model reports 70.08% Official Composite mIoU (63.55% fine, 76.61% coarse), placing 3rd on the leaderboard, with public code and models released.

Significance. If the reported gains hold and are attributable to the proposed modules, the work supplies a competitive, reproducible baseline for long-tailed fine-grained semantic segmentation in challenging outdoor settings. The public GitHub and Hugging Face releases constitute a clear strength for verification and extension by the community.

major comments (1)

[Abstract, §3] Abstract and §3 (contributions and method): The central claim attributes the 70.08% composite mIoU primarily to the three listed additions (200 queries, FRM, Auxiliary Supervision Head) plus the described training/inference pipeline. No ablation tables or quantitative isolation of these components from the Swin-Large backbone, Distribution-Balanced loss, Rare-Class Copy-Paste, or TTA are referenced, leaving open the possibility that gains derive mainly from hyper-parameter tuning or the base architecture rather than the targeted modules.

minor comments (1)

[Abstract] Abstract: The Hugging Face link contains the placeholder 'XYZ9843'; replace with the actual model identifier for reproducibility.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and for recognizing the significance and reproducibility of the work through the public code and model releases. We address the single major comment below and will incorporate the requested changes in the revised manuscript.

read point-by-point responses

Referee: [Abstract, §3] Abstract and §3 (contributions and method): The central claim attributes the 70.08% composite mIoU primarily to the three listed additions (200 queries, FRM, Auxiliary Supervision Head) plus the described training/inference pipeline. No ablation tables or quantitative isolation of these components from the Swin-Large backbone, Distribution-Balanced loss, Rare-Class Copy-Paste, or TTA are referenced, leaving open the possibility that gains derive mainly from hyper-parameter tuning or the base architecture rather than the targeted modules.

Authors: We agree that the current version of the manuscript does not provide ablation tables that isolate the contributions of the three proposed modules (200 queries, FRM, Auxiliary Supervision Head) from the Swin-Large Mask2Former baseline, the Distribution-Balanced loss, Rare-Class Copy-Paste augmentation, or the TTA/inference pipeline. This leaves the attribution of the reported gains open to the interpretation raised by the referee. In the revised manuscript we will add a dedicated ablation study section that reports incremental performance when each component is added in turn, together with controls that hold the training recipe and inference pipeline fixed. These tables will be referenced from both the abstract and §3. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: empirical leaderboard result with no internal derivations

full rationale

The paper reports an empirical mIoU score (70.08% composite) obtained by training an adapted Mask2Former on the external GOOSE benchmark and submitting to its public leaderboard. No equations, parameter fits, or self-citations are invoked to derive or predict this metric from internal quantities; the result is produced by standard supervised training and inference. The three listed contributions (200 queries, FRM, auxiliary head) plus training recipe are presented as engineering choices whose impact is measured externally rather than defined into the output. This matches the default case of a self-contained empirical paper with no load-bearing self-referential steps.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The work rests on the standard Mask2Former architecture and common computer-vision training practices; the only explicit free parameter highlighted is the choice of 200 queries.

free parameters (1)

number of object queries = 200
Set to 200 to avoid representational saturation on 64 classes; value chosen rather than derived.

assumptions (1)

domain assumption Swin-Large Mask2Former provides a suitable starting point for fine-grained outdoor segmentation
The paper extends this baseline without independent justification of its suitability for the long-tailed regime.

how reviews work

0 comments

Cite this review

Pith. "Pith review of GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain." pith.science (2026). https://pith.science/paper/IVCDWRRR

@misc{pith2026260615937,
  author       = {Pith},
  title        = {Pith review of: GOOSE-M2F: Adapting Mask2Former for High-Fidelity, Long-Tailed Fine-Grained Semantic Segmentation in Unstructured Outdoor Terrain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IVCDWRRR}},
  note         = {Machine review of arXiv:2606.15937}
}

read the original abstract

We present GOOSE-M2F, a task-specific adaptation of Mask2Former for the GOOSE 2D Fine-Grained Semantic Segmentation (FGSS) Challenge at ICRA 2026. The GOOSE benchmark spans 64 fine-grained classes across unstructured outdoor terrain with a severely long-tailed distribution, where rare classes occupy fewer than 50 pixels per image. We extend the Swin-Large Mask2Former baseline with three targeted contributions: (1) 200 object queries to eliminate representational saturation; (2) a Feature Refinement Module (FRM) combining ASPP-lite and CBAM dual-attention; and (3) an Auxiliary Supervision Head that delivers direct per-pixel gradients for rare classes. A multi-stage training strategy pairs Distribution-Balanced loss, Rare-Class Copy-Paste augmentation, dynamic IoU-aware re-weighting, and EMA. At inference, a dense sliding-window engine with 2D Gaussian kernel blending and 4-scale TTA adds +10.57%. GOOSE-M2F achieves 70.08% Official Composite mIoU (63.55% fine, 76.61% coarse), placing 3rd on the GOOSE 2D FGSS leaderboard. Code and trained models are publicly available at GitHub: https://github.com/Aditya-Lingam-9000/GOOSE-M2F and Hugging Face: https://huggingface.co/XYZ9843/GOOSE-M2F.

Figures

Figures reproduced from arXiv: 2606.15937 by the authors.

**Figure 2.** Qualitative segmentation results on GOOSE and GOOSE-Ex [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 2 canonical work pages

[1]

Masked- attention mask transformer for universal image segmentation,

B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Garg, “Masked- attention mask transformer for universal image segmentation,” inProc. CVPR, 2022, pp. 1290–1299

2022
[2]

Swin transformer: Hierarchical vision transformer using shifted windows,

Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProc. ICCV, 2021, pp. 10012–10022

2021
[3]

End-to-end object detection with transformers,

N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. ECCV, 2020, pp. 213–229

2020
[4]

Deformable DETR: Deformable transformers for end-to-end object detection,

X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: Deformable transformers for end-to-end object detection,” inProc. ICLR, 2021

2021
[5]

Rethinking Atrous Convolution for Semantic Image Segmentation

L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Re- thinking atrous convolution for semantic image segmentation,” arXiv:1706.05587, 2017

work page Pith review arXiv 2017
[6]

CBAM: Convolutional block attention module,

S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” inProc. ECCV, 2018, pp. 3–19

2018
[7]

Fully convolutional networks for semantic segmentation,

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” inProc. CVPR, 2015, pp. 3431–3440

2015
[8]

SegFormer: Simple and efficient design for semantic segmentation with transformers,

E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” inProc. NeurIPS, 2021, pp. 12077–12090

2021

Show all 16 references

[9]

The Cityscapes dataset for semantic urban scene understanding,

M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Be- nenson, U. Franke, S. Roth, and B. Schiele, “The Cityscapes dataset for semantic urban scene understanding,” inProc. CVPR, 2016, pp. 3213–3223

2016
[10]

Scene parsing through ADE20K dataset,

B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso, and A. Torralba, “Scene parsing through ADE20K dataset,” inProc. CVPR, 2017, pp. 633–641

2017
[11]

Distribution-balanced loss for multi-label classification in long-tailed datasets,

T. Wu, Q. Liu, A. Huang, Y . Zhou, and Y . Lin, “Distribution-balanced loss for multi-label classification in long-tailed datasets,” inProc. ECCV, 2020, pp. 162–178

2020
[12]

LVIS: A dataset for large vocabulary instance segmentation,

A. Gupta, P. Dollar, and R. Girshick, “LVIS: A dataset for large vocabulary instance segmentation,” inProc. CVPR, 2019, pp. 5356– 5364

2019
[13]

Simple copy-paste is a strong data augmentation method for instance segmentation,

G. Ghiasi, Y . Cui, A. Srinivas, R. Qian, T.-Y . Lin, E. D. Cubuk, Q. V . Le, and B. Zoph, “Simple copy-paste is a strong data augmentation method for instance segmentation,” inProc. CVPR, 2021, pp. 2918–2928

2021
[14]

Decoupled weight decay regularization,

I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” inProc. ICLR, 2019

2019
[15]

The GOOSE dataset for perception in unstructured environments,

P. Mortimer, R. Hagmanns, M. Granero, T. Luettel, J. Petereit, and H.-J. Wuensche, “The GOOSE dataset for perception in unstructured environments,” inProc. ICRA, 2024, pp. 14838–14844

2024
[16]

Excavating in the wild: The GOOSE-Ex dataset for semantic segmen- tation,

R. Hagmanns, P. Mortimer, M. Granero, T. Luettel, and J. Petereit, “Excavating in the wild: The GOOSE-Ex dataset for semantic segmen- tation,”arXiv:2409.18788, 2024

2024

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.

[1] [1]

Masked- attention mask transformer for universal image segmentation,

B. Cheng, I. Misra, A. G. Schwing, A. Kirillov, and R. Garg, “Masked- attention mask transformer for universal image segmentation,” inProc. CVPR, 2022, pp. 1290–1299

2022

[2] [2]

Swin transformer: Hierarchical vision transformer using shifted windows,

Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProc. ICCV, 2021, pp. 10012–10022

2021

[3] [3]

End-to-end object detection with transformers,

N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in Proc. ECCV, 2020, pp. 213–229

2020

[4] [4]

Deformable DETR: Deformable transformers for end-to-end object detection,

X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable DETR: Deformable transformers for end-to-end object detection,” inProc. ICLR, 2021

2021

[5] [5]

Rethinking Atrous Convolution for Semantic Image Segmentation

L.-C. Chen, G. Papandreou, F. Schroff, and H. Adam, “Re- thinking atrous convolution for semantic image segmentation,” arXiv:1706.05587, 2017

work page Pith review arXiv 2017

[6] [6]

CBAM: Convolutional block attention module,

S. Woo, J. Park, J.-Y . Lee, and I. S. Kweon, “CBAM: Convolutional block attention module,” inProc. ECCV, 2018, pp. 3–19

2018

[7] [7]

Fully convolutional networks for semantic segmentation,

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” inProc. CVPR, 2015, pp. 3431–3440

2015

[8] [8]

SegFormer: Simple and efficient design for semantic segmentation with transformers,

E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” inProc. NeurIPS, 2021, pp. 12077–12090

2021