{"id":"c969209b-3676-4b43-9cab-7138a6b9df49","arxiv_id":"2412.10589","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"PanSR, an object-centric mask transformer, achieves +3.4 PQ over prior state of the art on LaRS panoptic segmentation and 67.2 PQ on Cityscapes.","lead":"A new mask-transformer architecture for panoptic segmentation uses object-center proposals and bounding-box-constrained masks to improve small-object detection and reduce instance merging. It reports a new state of the art on the maritime LaRS benchmark and matches top methods on Cityscapes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set selection bias likely inflates the +3.4 PQ claim: all ablations and model selection run on the LaRS test set, with no held-out validation, so the reported margin may be optimistic.","rationale":"The reader's weakest assumption identifies precisely the most load-bearing risk: the reported +3.4 PQ gain is the primary evidence that PanSR's architecture is superior, and that number is produced from test-set-driven model selection. This is not an internal inconsistency; the method is plausible, and the qualitative analyses show real qualitative differences. But the absence of any held-out validation means the exact magnitude of the claimed improvement is not trustworthy. A re-evaluation with ablations on the LaRS validation split, followed by a single test evaluation, would settle the issue. Until then, a conditional acceptance is appropriate: the architecture deserves credit for a well-motivated design and consistent internal ablations, but the central quantitative claim lacks the statistical grounding required for full acceptance. I agree with the reader's assessment and see no additional concern that changes the verdict.","tokens_in":11866,"tokens_out":10732,"duration_ms":93024,"concrete_test":"Train the full PanSR and the strongest baseline (Mask DINO 2S, Swin-L) with identical training recipes on the LaRS training split; perform all ablations and hyperparameter selection on the LaRS validation split, using the test set exactly once at the very end. If the PanSR-to-Mask DINO PQ gap on the test set remains above 2 points (directionally consistent with +3.4), the concern is mitigated. If the gap collapses to ~0 or reverses, the reported gain is substantially attributable to test-set selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim, that PanSR outperforms state-of-the-art by +3.4 PQ on LaRS, depends on a model whose design was selected using the LaRS test set itself. Section 4.4 explicitly states: 'We ablate the major contributions of PanSR on the LaRS test set.' All component choices (learnable vs. proposal queries, mask constraint type, matching scheme, conditioned-query variant, and associated thresholds such as θ, ϵw, ϵh and NMS parameters) are evaluated on the test set, even though LaRS provides a legitimate validation split (Section 4.2). This creates a selection bias: the reported +3.4 PQ is the maximum over the tried configurations, not an unbiased measure of the final architecture. On a small dataset with roughly 4,000 images and 21,000 thing instances, test-set selection can easily move PQ by a few points. The small reported differences—e.g., 0.3 PQ from mask-conditioned queries, 0.9 PQ from proposal-aware matching—are exactly the scale of noise that such selection can produce. The Cityscapes val result (67.2 PQ) does provide evidence of generalization, but it does not validate the magnitude of the LaRS-specific improvement. Without an unbiased evaluation of a single pre-registered model on the test set, the headline advantage over Mask DINO remains unsubstantiated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PanSR, an object-centric mask transformer for panoptic segmentation. The method introduces three main components: an Object-Centric Proposal (OCP) module that extracts instance queries from object-center predictions rather than pixel-level top-k selection, a proposal-aware matching scheme that allows multiple queries to match one ground-truth instance while removing low-overlap matches, and an object-centric mask prediction head that zeros mask logits outside a dilated predicted bounding box for thing classes. The authors also add mask-conditioned training queries to improve robustness to proposal noise. The central empirical claim is a +3.4 PQ improvement over Mask DINO on the LaRS test set, with state-of-the-art performance on Cityscapes val (67.2 PQ), and ablation results in Table 3 attribute the gains to the OCP module, relaxed bounding-box mask constraints, and proposal-aware matching.","tokens_in":12160,"tokens_out":3902,"duration_ms":36787,"significance":"If the reported results are unbiased, PanSR would be a meaningful advance for panoptic segmentation in maritime and crowded scenes, where small objects and instance merging are known failure modes. The OCP module is a plausible remedy for the scale bias of top-k query selection, and the bounding-box-constrained mask prediction directly targets the merging problem. The Cityscapes result provides some evidence that the architecture does not simply overfit LaRS. The paper also includes a clear ablation structure and will release code and models, which would help reproducibility. However, the evaluation protocol substantially weakens confidence in the headline margin, because all ablations and model-selection decisions are made on the LaRS test set rather than a held-out validation split, and no repeated runs or error bars are reported.","major_comments":[{"comment":"The ablation study and model selection are performed on the LaRS test set. Section 4.4 states explicitly: 'We ablate the major contributions of PanSR on the LaRS test set,' even though LaRS has a validation split (Section 4.2). This means the reported +3.4 PQ gain over Mask DINO is the result of selecting components and thresholds on the test set itself. The margins of some ablations (e.g., 0.3 PQ for mask-conditioned queries, 0.9 PQ for proposal-aware matching) are exactly the scale at which selection noise can act. Please rerun the ablations on the validation split, freeze a single architecture and hyperparameter configuration, and then report the test-set result once for that configuration. Reporting standard deviations or seed sensitivity would also strengthen the claim.","section":"§4.4 (Table 3)"},{"comment":"The comparison with Mask DINO is under-specified. The text says 'We apply the same training schedule to baseline methods,' but it is not clear whether Mask DINO was retrained under PanSR's exact protocol, including the copy-paste augmentation, the same optimizer settings, and the same number of iterations, or whether published numbers were quoted. This matters because Table 1 shows PanSR with ResNet-50 matching Mask DINO with Swin-L. Please document the exact retraining protocol for every baseline, including data augmentation, compute, and any modifications to the original implementation, and make the configuration files available.","section":"§4.2 (Table 1)"},{"comment":"The claimed reduction in instance merging is partly guaranteed by construction. Equation (6) sets the mask probability to zero outside the dilated predicted bounding box, so spatially well-separated instances cannot produce a merged mask regardless of learned feature quality. This is a legitimate inductive bias, but the paper should frame the improvement accordingly and not present it as evidence that the network has learned better instance separation. The Table 3 comparison between 'Global' and 'Relaxed BBox' changes the output space, so part of the +2.2 PQ gain is expected from the constraint itself. Please discuss what is learned versus what is enforced, and consider reporting merging rates conditioned on predicted box overlap to quantify the effect.","section":"§3.2 (Eq. 6)"},{"comment":"No repeated runs, error bars, or statistical significance measures are reported anywhere. All central comparisons—including the +3.4 PQ difference in Table 1 and the individual ablations in Table 3—are single runs. On a dataset with roughly 4,000 images, PQ differences of 0.3–0.9 points can be within run-to-run variation. At minimum, report multiple seeds with mean and standard deviation for the main configurations, and ideally perform model selection on validation before touching the test set.","section":"§4.4 (Table 3), §4.2"}],"minor_comments":[{"comment":"There are several typos in the abstract and introduction: 'the we rethink' should be 'we rethink', 'aleviate' should be 'alleviate', and 'adn' should be 'and'.","section":"Abstract and §1"},{"comment":"The sentence 'The positional queries of can be directly obtained from the regression values at the location xi' is missing an object and should read 'The positional query of object i can be directly obtained...'.","section":"§3.1.1"},{"comment":"Equation (6) is introduced with 'and and ϕ(·, ϵw, ϵh) is a dilation function'—remove the duplicated 'and'.","section":"§3.2"},{"comment":"'multiple queries my be extracted' should be 'may be extracted'.","section":"§3.3.1"},{"comment":"The abstract states PanSR reaches '67.3 PQ' on Cityscapes, while Table 2 reports 67.2 PQ. Please reconcile these numbers.","section":"§4.3 (Table 2) and Abstract"},{"comment":"The claim that PanSR 'outperforms all state-of-the-art methods by a large margin (+3.4% PQ)' is qualified by the Cityscapes result being 'on par' with OneFormer (67.2 vs 67.2). The phrasing could more accurately distinguish the LaRS margin from the Cityscapes tie.","section":"§1"}],"recommendation":"major_revision","confidential_remarks":"The core architectural ideas are interesting and the paper is likely a good fit for this venue, but the evaluation protocol is a serious concern: all ablations and model selection on LaRS are performed on the test set, and baseline training conditions are not fully documented. If the authors can redo model selection on the validation split, report test-set results for a single frozen configuration, and provide repeated-run statistics, the paper could become a strong contribution. The Section 3.2 bounding-box constraint also deserves reframing as an inductive bias rather than an emergent learned behavior."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a well-engineered extension of Mask DINO with three sensible fixes, but don't trust the +3.4 PQ on LaRS until you see the same numbers on a held-out split. The ablations in Table 3 are all on the LaRS test set, and that's a genuine selection-bias problem, not a nitpick.\n\nWhat's actually new: The OCP module replaces top-k pixel scoring with a per-level center heatmap + NMS + box regression, which is a reasonable way to stop query selection from favoring big objects. The relaxed bbox mask constraint (Eq. 6) is a clean trick: it forces thing masks to zero outside a dilated predicted box, which both reduces merging and frees the decoder from learning global separation. Proposal-aware matching (FP removal at IoU<0.25, FN addition at IoU>0.8) is a sensible response to the query-drift issue, and the mask-conditioned queries are a cheap, sensible way to inject proposal noise during training. Each component is ablated and each shows a gain; that's real evidence the design works as intended.\n\nThe soft spots are real. First, the headline margin is the maximum over a set of design choices evaluated on the test set. LaRS has a validation split (Section 4.2), but Section 4.4 says \"We ablate the major contributions of PanSR on the LaRS test set.\" With ~4,000 images, test-set selection can easily shift PQ by a point or two, and the per-component gains (0.9, 0.3) are exactly the size of noise that can produce. The Cityscapes val result (67.2, on par with OneFormer) is encouraging and does show the architecture transfers, but it doesn't validate the magnitude of the LaRS-specific gain. Second, there are no error bars or repeated runs anywhere. Third, the bbox constraint means the reduction in instance merging for spatially separated objects is guaranteed by construction; the interesting part is that relaxed bbox still helps overall, which they show, but the rhetoric about \"mitigates instance merging\" is partly tautological. Fourth, the baseline conditions are unclear: they say they applied the same training schedule to baseline methods, but it's not stated whether Mask DINO was retrained with the same copy-paste augmentation and number of iterations, or whether the Table 1 numbers are taken from the original papers. That matters when the comparison is +3.4.\n\nThe missing CenterNet citation is minor but worth a comment; the OCP center heatmap + NMS + regression is close to keypoint detection, and the paper should acknowledge that lineage.\n\nBottom line: the paper deserves a serious referee. The architecture is coherent, the experiments are extensive, and the ideas are likely useful. But the headline number needs validation on a held-out split, error bars, released code, and a clear statement of baseline retraining. Send it to review, but with a request for those revisions rather than acceptance as-is.","headline":"Well-engineered Mask DINO extension with a real test-set bias problem: the +3.4 PQ on LaRS needs a held-out validation before it's credible.","tokens_in":12655,"tokens_out":3631,"would_cite":true,"duration_ms":30392,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A mask transformer that builds proposals from object centers claims to raise panoptic quality by 3.4 points on the maritime LaRS benchmark.","keywords":["panoptic segmentation","mask transformer","object-centric proposals","small-object detection","instance merging","maritime perception","LaRS benchmark","query drift"],"falsifier":"Run the Table 3 ablations on a fixed validation split of LaRS instead of the sequestered test set (or evaluate the released models on LaRS val); if OCP, proposal-aware matching, and dilated-box masks do not beat the pixel-level query-selection and Hungarian-matching baselines on that held-out data, the architectural explanation for the +3.4 PQ gain collapses.","tokens_in":11676,"feed_emoji":"🚢","tokens_out":9278,"duration_ms":75895,"temperature":0.7,"pith_summary":"The paper tries to establish that the main weaknesses of mask-transformer panoptic segmentation—assigning every pixel both a semantic class and an instance ID—in small-object, crowded, and scale-diverse scenes come from three design choices: proposals selected from per-pixel score maps favor large objects, one-to-one matching lets well-initialized queries drift to other objects during decoder refinement, and global mask correlation merges visually similar instances. It proposes PanSR, which replaces pixel-level query selection with an object-centric proposal module that predicts object centers, applies non-maximum suppression, and pools features from approximate object masks; it replaces pure Hungarian matching with proposal-aware matching that removes low-overlap matches and adds high-overlap alternative matches; and it forces thing-class masks to be computed within a dilated predicted bounding box. PanSR reports 57.3 PQ (panoptic quality) on the LaRS test set with a Swin-L backbone, +3.4 PQ above the previous state of the art, and a ResNet-50 version matching the Swin-L baseline, while reaching 67.2 PQ on Cityscapes val without dataset-specific tuning. The architecture's object-centric choices, not backbone size, are presented as the reason for the gain.","feed_headline":"Center-based proposals push panoptic segmentation ahead 3.4 PQ on LaRS","feed_subtitle":"PanSR finds objects by their centers, not by pixel scores, and matches masks to boxes, helping small and crowded scenes.","key_machinery":"The load-bearing machinery is the Object-Centric Proposal module (OCP). On each feature-pyramid level it predicts an object-center map, an objectness mask, and center-and-size regression maps; non-maximum suppression turns the center map into proposal locations, and each proposal's content query comes from mask-weighted pooling of features inside an approximate object mask obtained by instance voting. Around this sit three auxiliary mechanisms: proposal-aware matching, which keeps the Hungarian one-to-one matches whose box IoU with the ground truth is at least 0.25 and adds unmatched queries with IoU above 0.80 as extra supervision; object-centric mask prediction, which zeroes thing-class mask logits outside a dilated predicted box; and mask-conditioned queries sampled from ground-truth object regions during training to mimic proposal noise. The paper attributes the LaRS gains to these four pieces in its ablations.","core_discovery":"PanSR's central claim is that reworking three components around object-centric principles closes the small-object and instance-merging gaps of mask transformers. Concretely, the paper argues that thing-class proposals should be generated at object level from predicted center maps and per-level size ranges rather than from top-k pixel activations; that matching should be many-to-one and proposal-aware so redundant proposals for the same object are kept instead of pushed onto neighbors; and that thing masks should be decoded inside a dilated bounding box so the network learns local boundaries instead of global similarity. On the LaRS maritime benchmark the paper reports a +3.4 PQ improvement over Mask DINO with Swin-L, with the largest gains in thing-class recognition quality, and a detection-rate curve that is higher across all object sizes, especially the smallest. On Cityscapes the same settings reach 67.2 PQ, matching OneFormer, which the paper reads as evidence that the design generalizes beyond maritime scenes.","pith_inferences":["The center-map-plus-NMS proposal idea is not specific to panoptic segmentation; the same large-object bias in top-k selection appears in detection and counting, so an OCP-style module could be tested there.","Because the ablations are run on the sequestered LaRS test set, the individual contributions to the +3.4 PQ are not yet established on unseen data; repeating the ablation on a fixed validation split would settle that.","Bounding-box-dilated masks may hurt on highly non-convex or articulated objects, which the paper does not test; a dataset such as COCO or ADE20K would provide a harder check.","If the mask-constrained design holds up, a practical consequence is that decoder capacity is freed from global feature separation, which may improve sample efficiency on small annotation budgets."],"forward_implications":["If the reported results hold, an object-centric proposal extractor can close most of the performance gap between a ResNet-50 backbone and a Swin-L backbone on scale-diverse scenes.","Thing-class masks constrained by dilated boxes should reduce merging of visually similar instances without requiring global instance-separation features, which the qualitative comparisons support.","Proposal-aware many-to-one matching combined with test-time NMS is a workable alternative to pure one-to-one matching in mask transformers.","The same architecture and hyperparameters transfer from maritime to urban scenes, reaching 67.2 PQ on Cityscapes val with no dataset-specific tuning.","The proposed components are complementary to open-vocabulary and multi-task training, so they can be combined with those directions."],"supporting_citations":[{"why":"Mask DINO provides the transformer encoder-decoder architecture, training recipe, and the main two-stage baseline PanSR compares against.","marker":"[13]"},{"why":"LaRS supplies the maritime benchmark and sequestered test set on which the +3.4 PQ claim is measured.","marker":"[24]"},{"why":"Mask2Former establishes the mask-transformer formulation and the learnable stuff queries and Hungarian matching used in ablations.","marker":"[5]"},{"why":"DETR supplies the bipartite-matching training paradigm that proposal-aware matching refines.","marker":"[2]"},{"why":"DN-DETR provides the query-denoising approach that mask-conditioned queries are designed to improve upon.","marker":"[12]"},{"why":"The stuff/thing query split with selected thing proposals motivates PanSR's OCP design.","marker":"[21]"},{"why":"Cityscapes is the urban benchmark used to demonstrate generalization at 67.2 PQ.","marker":"[6]"}],"fun_headline_variants":["Center-based proposals lift panoptic seg 3.4 PQ on LaRS","PanSR fixes small-object misses and instance merging, +3.4 PQ","Object-centric masks beat Mask DINO by 3.4 PQ on LaRS","PanSR: object-level proposals reduce merging, find small objects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that selecting components and hyperparameters on the sequestered LaRS test set, as the ablations in Table 3 do, does not overfit that benchmark; if it does, the claimed +3.4 PQ reflects test-set tuning rather than the architecture's design.","fun_headline_variants_meta":{"raw":{"variants":["Center-based proposals lift panoptic seg 3.4 PQ on LaRS","PanSR fixes small-object misses and instance merging, +3.4 PQ","Object-centric masks beat Mask DINO by 3.4 PQ on LaRS","PanSR: object-level proposals reduce merging, find small objects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001012,"raw_usage":{"total_tokens":4271,"prompt_tokens":940,"completion_tokens":3331,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":556,"completion_tokens_details":{"reasoning_tokens":3250}},"tokens_in":556,"tokens_out":3331,"duration_ms":23098,"temperature":1.0,"reasoning_tokens":3250,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:49:16.183088+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Table 3 ablations on a fixed validation split of LaRS instead of the sequestered test set (or evaluate the released models on LaRS val); if OCP, proposal-aware matching, and dilated-box masks do not beat the pixel-level query-selection and Hungarian-matching baselines on that held-out data, the architectural explanation for the +3.4 PQ gain collapses.","supporting_citations":[{"cited_title":"Ni, and Heung-Yeung Shum","cited_arxiv_id":null,"evidence_quote":"Mask DINO provides the transformer encoder-decoder architecture, training recipe, and the main two-stage baseline PanSR compares against."},{"cited_title":"LaRS: A Diverse Panoptic Maritime Obstacle Detection Dataset and Bench- mark","cited_arxiv_id":null,"evidence_quote":"LaRS supplies the maritime benchmark and sequestered test set on which the +3.4 PQ claim is measured."},{"cited_title":"Schwing, Alexan- der Kirillov, and Rohit Girdhar","cited_arxiv_id":null,"evidence_quote":"Mask2Former establishes the mask-transformer formulation and the learnable stuff queries and Hungarian matching used in ablations."},{"cited_title":"End- to-End Object Detection with Transformers","cited_arxiv_id":null,"evidence_quote":"DETR supplies the bipartite-matching training paradigm that proposal-aware matching refines."},{"cited_title":"Ni, and Lei Zhang","cited_arxiv_id":null,"evidence_quote":"DN-DETR provides the query-denoising approach that mask-conditioned queries are designed to improve upon."},{"cited_title":"A Simple Framework for Open-V ocabulary Segmentation and Detec- tion, 2023","cited_arxiv_id":null,"evidence_quote":"The stuff/thing query split with selected thing proposals motivates PanSR's OCP design."},{"cited_title":"The Cityscapes Dataset for Semantic Urban Scene Understanding","cited_arxiv_id":null,"evidence_quote":"Cityscapes is the urban benchmark used to demonstrate generalization at 67.2 PQ."}],"review_version":1}