{"id":"a5ce327a-c8b2-4a8d-8eb1-4fdc15663465","arxiv_id":"2505.06991","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"The authors achieve 84.8% mIoU on the GOOSE test set by combining MaskDINO, RoPE-ViT backbone, CSEC color correction, and quantile-based training data filtering.","lead":"A robotics team reports an 84.8% mean intersection-over-union score on the outdoor GOOSE segmentation benchmark using MaskDINO with rotary position embeddings, color correction, and a data filter that drops hard training images. The report shows which known techniques combine to win a challenge, and it also reveals a self-selection caveat in the filtering step.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The quantile-based denoising strategy is circular and internally inconsistent, so the claimed robustness to label noise rests on an unsupported and potentially biased step.","rationale":"The reader's weakest assumption identifies the circularity of using the model's own predictions to filter training samples, which I agree is the most load-bearing concern for the central claim of robustness to label noise. The paper's internal inconsistency between pixel-level downweighting and sample-level removal compounds the issue, making the method ambiguous and hard to verify. The validation ablation does not convincingly isolate the denoising effect because it lacks controls, and no test-set ablation is provided. These issues do not invalidate the reported test mIoU, which is likely an accurate challenge result, but they do weaken the paper's interpretative claim that the denoising strategy contributes to robustness. A conditional acceptance requiring clarification and a randomization control is the appropriate verdict, matching the reader's assessment. No change to the reader's verdict is needed.","tokens_in":5169,"tokens_out":4386,"duration_ms":42196,"concrete_test":"Run a controlled ablation on the GOOSE validation set: (a) train the full pipeline with the proposed denoising, (b) train with a randomly selected 2.5% of training samples removed, and (c) train without any denoising, each with multiple seeds to obtain error bars. If the proposed denoising does not significantly outperform the random-removal baseline, the reported gain is selection bias rather than successful label-noise handling. Additionally, implement both interpretations (per-pixel downweighting and per-sample removal) and report which one was actually used, reproducing the validation and test mIoU under each.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section II.C describes using the existing model to predict each training sample, computing pixel-wise error rates, and then removing samples above the 97.5th percentile. This creates circularity: the model's own errors define what counts as noise, conflating genuinely mislabeled pixels with valid but hard examples. Removing such samples biases the training distribution and can inflate validation scores without improving true robustness. The abstract and introduction claim downweighting the top 2.5% of high-error pixels, while the method removes whole samples; these are different operations, and the paper does not clarify which was implemented. Table II shows only a 0.24 mIoU improvement on the validation set from denoising, with no error bars, no randomization baseline, and no test-set ablation. Consequently, the contribution of denoising to the central claim of robustness to label noise is not established, and the reported test mIoU of 0.848 cannot be attributed to this component without further evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This technical report describes the ACVLAB submission to the ICRA 2025 GOOSE 2D Semantic Segmentation Challenge. The proposed method builds on MaskDINO and adds a Color Shift Estimation-and-Correction (CSEC) module, a RoPE-based Vision Transformer backbone, and a quantile-based label-denoising step that filters training samples. The authors report a test-set mIoU of 84.8% on the official GOOSE test set and provide validation-set ablations of the individual components.","tokens_in":5433,"tokens_out":4367,"duration_ms":41530,"significance":"The paper documents a competitive pipeline for semantic segmentation across heterogeneous outdoor robotic platforms, and the reported test result, if reliable, is of practical interest to the field-robotics community. The system-level design is coherent and the problem setting is clearly motivated. However, the paper's central robustness claims rest on the denoising component, whose empirical support is limited to a single validation-set comparison with a 0.24 mIoU gain and no error bars, and whose description is internally inconsistent. The absence of code, full training details, and test-set ablations further limits the value of the report beyond the competition outcome. The strengths are the concrete challenge result, the use of the official evaluation protocol, and the clear presentation of the architectural components.","major_comments":[{"comment":"The abstract and introduction describe the quantile-based strategy as 'downweighting the top 2.5% of highest-error pixels,' but Section II.C states that 'we remove samples that fall above the 97.5th percentile of the error rate distribution.' These operations are not equivalent: pixel-level downweighting preserves all training samples while reducing the loss contribution of selected pixels, whereas sample-level removal discards entire images. The paper must clarify which procedure was actually implemented and justify the discrepancy, because the claimed mechanism and the reported ablation depend on this distinction.","section":"Abstract and Section II.C"},{"comment":"The denoising step selects training samples based on the model's own per-image error rate. This is a self-referential selection procedure: the model determines which training examples are 'noisy,' so the filtered training distribution is not independent of the model. High-error samples may be hard but valid examples, such as rare classes or boundary cases, and their removal can bias the training distribution and inflate validation mIoU without improving true robustness. With an improvement of only 0.24 mIoU (88.89 to 89.13) in Table II, no error bars, no randomization baseline, and no test-set ablation, the evidence is too weak to establish the claimed robustness to label noise.","section":"Section II.C and Table II"},{"comment":"The central test-set result (mIoU 84.8%) is reported as a single number without confidence intervals, without a per-platform breakdown despite the evaluation weights (67% MuCAR-3, 24% ALICE, 6% Spot v2, 3% Spot v1), and without a comparison of the ablated configurations from Tables I and II on the test set. This makes it impossible to determine whether the denoising step contributes to the test result or whether the validation-set gain is statistically meaningful. The authors should provide test-set results for all ablated configurations, ideally with multiple runs or an estimate of variance.","section":"Section III"},{"comment":"The title and Figure 1 refer to a 'RoPE-Swin' backbone, but Section II.B states that the default Swin-L backbone is replaced by 'RoPE-ViT,' a Vision Transformer variant. This terminology inconsistency should be corrected, and the authors should clarify the exact architecture used, including whether the rotary positional embeddings are applied within a Swin-style or ViT-style model.","section":"Section II.B and Fig. 1"}],"minor_comments":[{"comment":"The experimental details are incomplete: the paper reports only the batch size and GPU type, and omits the number of training epochs, learning rate schedule, optimizer, data augmentation, and the specific point in training at which the denoising filter is applied.","section":"Section III"},{"comment":"In Equation (1), the summation index i and the domain of the feature maps x_i, weights w_i, and offsets Δp_i are not fully defined; please specify the ranges and shapes.","section":"Section II.A, Eq. (1)"},{"comment":"Tables I and II report validation mIoU values differing by 0.5 to 1.0 points, but no variance estimates or repeated runs are given; the authors should state whether these differences are stable across seeds.","section":"Tables I and II"},{"comment":"Reference [9] for RoPE and reference [2] for MaskDINO would benefit from complete venue information (e.g., ECCV or CVPR pages) to aid readers.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a competition technical report, and the main quantitative claim is a single test-set mIoU. The internal inconsistency between the abstract's 'downweight pixels' and Section II.C's 'remove samples' is a substantive flaw that the authors must fix, and the self-referential nature of the denoising procedure needs a rigorous control experiment. Given the format of the report, I would not reject outright, but the current evidence does not support the robustness claim attributed to the denoising component."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"It's a solid workshop entry that reports a real challenge result: 84.8% mIoU on the GOOSE test set. The specific combination of MaskDINO, RoPE, CSEC, and quantile filtering is new to the benchmark, and the validation ablations in Table I show each component gives a small, consistent gain. The paper is plain and honest about the validation/test split. That's the good part.\n\nThe soft spots are real. The denoising strategy is described inconsistently: the abstract says it downweights the top 2.5% high-error pixels, while Section II.C says it removes whole training samples above the 97.5th percentile of per-sample error rate. Those are different operations, and the paper never says which one was actually used. Table II shows only a +0.24 mIoU gain on validation, with no error bars, no randomization baseline, and no test-set ablation. The deeper problem is circularity: the filter is based on the model's own predictions on the training set. Hard but correctly labeled samples get dropped, which biases the training distribution and can inflate validation scores without improving true robustness. The paper doesn't address that at all.\n\nThat said, the denoising is a minor part of the pipeline. The main components are borrowed and well-established, and the central test number stands on its own as a reported result. The claim of state-of-the-art is unsupported because there's no comparison table with other methods on the test set, but for a challenge technical report that's not unusual.\n\nWho is this for? Readers who want to know what won on GOOSE or how CSEC and RoPE behave when dropped into MaskDINO. It's not a methodological breakthrough. As a workshop report it deserves a serious referee: the issues are fixable and the authors should be asked to clarify the denoising implementation, show the test-set ablation if they have it, and discuss the selection bias. I would accept it for a workshop with those revisions, and would not send it to a top-tier venue in its current form.","headline":"A genuine challenge win on GOOSE with a plausible pipeline, but the denoising step is described two ways and relies on the model's own errors, so the robustness claim for that component is under-supported.","tokens_in":5883,"tokens_out":2551,"would_cite":false,"duration_ms":26778,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A MaskDINO-based segmentation pipeline that adds color-shift correction, rotary position embeddings, and quantile-based label denoising reaches 0.848 mean IoU on the GOOSE outdoor test set.","keywords":["semantic segmentation","GOOSE dataset","outdoor scene understanding","color shift correction","rotary position embedding","quantile-based label denoising","MaskDINO","cross-platform generalization"],"falsifier":"Re-annotate the training samples the quantile filter removes above the 97.5th percentile error threshold, treating a human labeler as the reference; if most removed samples have human labels matching the original ground truth rather than the model's predictions, the denoising gain is an artifact of dropping hard examples rather than removing noise.","tokens_in":4970,"feed_emoji":"🌄","tokens_out":9200,"duration_ms":80660,"temperature":0.7,"pith_summary":"This paper tries to establish that three targeted modifications to the MaskDINO segmentation architecture make it more reliable in uncontrolled outdoor settings: a color-shift estimation-and-correction preprocessing step, a Vision Transformer backbone with rotary position embeddings, and a training-time filter that drops the 2.5% of samples with the highest pixel-wise prediction error. The authors report that each change helps on the GOOSE validation set, and that the full pipeline reaches 0.848 mean Intersection over Union on the official GOOSE test set, which aggregates nine semantic classes across four robotic platforms. If the claim is right, illumination normalization, relative position encoding, and error-based denoising can be combined without architectural overhauls to improve cross-platform outdoor scene understanding.","feed_headline":"Three tweaks push outdoor segmentation to 0.848 mIoU","feed_subtitle":"Color correction, rotary position embeddings, and quantile denoising each add accuracy on the GOOSE benchmark.","key_machinery":"The load-bearing pieces are three modules grafted onto MaskDINO. CSEC first estimates a color-shift offset through a convolution with learned spatial offsets, then a color modulation module maps the image back to balanced colors by fusing self-correlation matrices of the original, darkened, and brightened feature maps with learned weights. RoPE-ViT replaces the default backbone by applying per-dimension rotations to query and key vectors in self-attention, with frequencies $\\omega_i = 10000^{-2i/d}$, so relative patch positions enter the attention inner product directly. The quantile denoiser uses the current model to compute each training sample's pixel-wise error against its ground-truth mask and removes everything above the 97.5th percentile before continuing training.","core_discovery":"On the official GOOSE test set, the paper reports a mean IoU of 0.848 for the full system: MaskDINO with a RoPE-ViT backbone, CSEC color correction, and quantile-based sample filtering. On the validation set the ablations are 87.92 without both RoPE and CSEC, 88.18 without RoPE, 88.72 without CSEC, 88.89 with both, and 89.13 after adding denoising. The authors read these numbers as evidence that color correction handles illumination inconsistencies, rotary position embeddings improve spatial generalization across camera resolutions and viewpoints, and removing the top 2.5% of high-error training samples stabilizes learning under label noise.","pith_inferences":["Editorial inference: the quantile denoiser relies on the model's own training-set predictions, so high-error samples may be hard-but-valid examples; a manual re-annotation audit of the removed samples would settle whether the gain is real or a distribution-shift artifact.","Editorial inference: the gap between 89.13 validation mIoU and 84.8 test mIoU points to residual domain shift between splits; per-platform reporting would show where color correction and rotary embeddings help most.","Editorial inference: a soft-weighting version of the denoiser that downweights high-error pixels instead of deleting them is a natural testable extension that could preserve borderline training signals."],"forward_implications":["MaskDINO users can adopt CSEC as a preprocessing stage and the RoPE backbone as a drop-in replacement without changing the segmentation head.","On the GOOSE validation split, removing either RoPE or CSEC lowers mIoU, so the two modules contribute independently rather than only in combination.","The quantile filter's validation gain from 88.89 to 89.13 suggests that even a simple hard sample-removal rule can improve training stability when annotation noise is present.","Because the reported test mIoU is a weighted average (67% MuCAR-3, 24% ALICE, 6% Spot v2, 3% Spot v1), the headline number is driven mainly by two of the four platforms."],"supporting_citations":[{"why":"Supplies the GOOSE dataset, its nine semantic categories, and the platform-weighted mIoU evaluation protocol that defines the challenge.","marker":"[1]"},{"why":"Provides MaskDINO, the transformer-based segmentation architecture onto which the three proposed components are added.","marker":"[2]"},{"why":"Supplies the Color Shift Estimation-and-Correction (CSEC) module used for illumination normalization during preprocessing.","marker":"[8]"},{"why":"Supplies the RoPE-ViT backbone with rotary position embeddings that replaces the default Swin backbone.","marker":"[9]"},{"why":"Defines the Swin Transformer whose hierarchical design and shifted-window attention form the base of the backbone used.","marker":"[10]"},{"why":"Provides the sinusoidal encoding logic that RoPE's per-dimension rotation frequencies are adapted from.","marker":"[11]"}],"fun_headline_variants":["Three tweaks, one benchmark: 0.848 mIoU on GOOSE","Color, RoPE, denoising each boost GOOSE segmentation","Ablation confirms each trick helps: 0.848 mIoU","Swin+RoPE and color shift deliver 0.848 mIoU","Quantile denoising pushes GOOSE mIoU to 0.848"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The denoising step assumes that pixels the model itself predicts most incorrectly are mislabeled or anomalous, not valid hard examples, so discarding the top 2.5% cannot skew the training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Three tweaks, one benchmark: 0.848 mIoU on GOOSE","Color, RoPE, denoising each boost GOOSE segmentation","Ablation confirms each trick helps: 0.848 mIoU","Swin+RoPE and color shift deliver 0.848 mIoU","Quantile denoising pushes GOOSE mIoU to 0.848"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000371,"raw_usage":{"total_tokens":1952,"prompt_tokens":875,"completion_tokens":1077,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":974}},"tokens_in":491,"tokens_out":1077,"duration_ms":9894,"temperature":1.0,"reasoning_tokens":974,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:26:55.263475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the training samples the quantile filter removes above the 97.5th percentile error threshold, treating a human labeler as the reference; if most removed samples have human labels matching the original ground truth rather than the model's predictions, the denoising gain is an artifact of dropping hard examples rather than removing noise.","supporting_citations":[{"cited_title":"The goose dataset for perception in unstructured environments,","cited_arxiv_id":null,"evidence_quote":"Supplies the GOOSE dataset, its nine semantic categories, and the platform-weighted mIoU evaluation protocol that defines the challenge."},{"cited_title":"Color shift estimation-and- correction for image enhancement,","cited_arxiv_id":null,"evidence_quote":"Supplies the Color Shift Estimation-and-Correction (CSEC) module used for illumination normalization during preprocessing."},{"cited_title":"Rotary position embedding for vision transformer,","cited_arxiv_id":null,"evidence_quote":"Supplies the RoPE-ViT backbone with rotary position embeddings that replaces the default Swin backbone."}],"review_version":1}