{"id":"50d8869d-954a-437b-87dd-6fc7df8b4589","arxiv_id":"2411.14179","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CompetitorFormer adds three competition-suppressing modules to transformer-based 3D instance segmentation, improving mAP by about +0.2 to +2.7 across four datasets and four baselines.","lead":"This paper presents three plug-and-play modules that reduce 'competition' between object queries in transformer-based 3D instance segmentation. The authors report consistent accuracy gains when adding the modules to four existing frameworks across four 3D datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significance testing or repeated-seed runs: the claim of 'consistent significant improvements' is not supported by single-run mAP differences, several of which are ≤ +0.8 mAP.","rationale":"The paper's central empirical claim is that the proposed competition-oriented modules yield consistent, significant gains across four frameworks and four datasets. The most load-bearing condition for this claim is that the reported mAP differences are real and not attributable to training noise. The paper reports only single runs, and gains range from +0.2 mAP (Mask3D on STPLS3D, Table 4b) to +3.1 mAP (SPFormer on ScanNet hidden test, Table 1); several intermediate gains are +0.3, +0.7, and +0.8 mAP. For 3D instance segmentation with transformer decoders, run-to-run variance is often comparable to these magnitudes, especially on STPLS3D and S3DIS. Without error bars, multiple seeds, or significance tests, the assertion of 'significant' improvements is not established. The reader's weakest assumption about the early mask IoU reliability (Eq. 4) is a legitimate mechanistic concern, but it is secondary: if the empirical gains were robust, the method could still work even if the early IoU signal is imperfect (e.g., by acting as a learned re-ranking). The empirical claim must be secured first. The paper does have strengths: consistent directional improvements, a systematic ablation table, and no obvious internal contradiction. Thus I recommend keeping the CONDITIONAL verdict, with the primary condition being statistical validation and code release. The proposed multi-seed test on the smallest-gain configurations would settle whether the concern lands.","tokens_in":14196,"tokens_out":10496,"duration_ms":97846,"concrete_test":"Run the baseline and the CompetitorFormer variant for the three smallest-gain configurations (Mask3D on STPLS3D, Mask3D on ScanNetv2 val, OneFormer3D on ScanNetv2 val) with at least 5 random seeds each, using identical hyperparameters and either the authors' released code or a faithful reimplementation. Report mean ± std mAP and compute a paired permutation test on per-scene mAP scores. If the 95% confidence interval for the improvement includes zero for any configuration, the 'consistent significant' claim fails; if all intervals exclude zero, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims that integrating the proposed designs 'consistently resulted in significant performance improvements' across a range of datasets. The supporting evidence is single-run mAP differences, with no error bars, no multiple seeds, and no significance tests. The smallest reported gains are C-Mask3D on STPLS3D (+0.2 mAP, +0.1 mAP50, Table 4b), C-Mask3D on ScanNetv2 val (+0.8 mAP, Table 2), C-OneFormer3D on ScanNetv2 val (+0.7 mAP, Table 2), and C-Mask3D on S3DIS (+0.3 mAP, Table 3). For transformer-based instance segmentation, run-to-run variance from random seeds, data order, and GPU nondeterminism is typically on the order of 0.5–1.5 mAP, especially for smaller or harder benchmarks. With no confidence intervals or paired tests, these gains are indistinguishable from noise. The paper also does not report per-scene variance and does not release code, so independent reproduction is impossible. The single large gain on ScanNet hidden test (+3.1 mAP, Table 1) is promising, but the central claim of consistent and significant improvements across all datasets and baselines is not established by the presented evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CompetitorFormer, a set of three plug-and-play modules (query competition layer, QCL; relative relationship encoding, RRE; and rank cross attention, RCA) designed to reduce inter-query competition in transformer-based 3D instance segmentation. The modules use per-query classification scores and mask IoUs to identify leader/laggard query pairs, reorder static embeddings, inject relative relationship biases into self-attention, and renormalize cross-attention similarities. The method is evaluated by adding the modules to SPFormer, Mask3D, MAFT, and OneFormer3D on ScanNetv2, ScanNet200, S3DIS, and STPLS3D, with ablations on ScanNetv2 val. The reported improvements are up to +3.1 mAP on the ScanNet hidden test set for C-SPFormer, but several gains are small (+0.2 to +0.8 mAP).","tokens_in":14427,"tokens_out":2139,"duration_ms":21089,"significance":"If the reported gains are reproducible, the paper makes a useful empirical contribution by showing that explicit competition-aware query interactions can improve several strong transformer-based 3D instance segmentation baselines. The plug-and-play nature of the modules, the breadth of the evaluation across four datasets and four baselines, and the per-layer analyses of matched/unmatched query scores are strengths. The paper does not claim a new state-of-the-art architecture but rather a generally applicable mechanism, and the consistent positive direction of the reported gains across most baselines suggests the idea is worth pursuing. However, the central empirical claim of 'significant performance improvements' is currently supported only by single-run mAP differences, with no seed variance, confidence intervals, or significance tests; several gains are within the typical run-to-run noise for these benchmarks.","major_comments":[{"comment":"The claim that integrating the designs 'consistently resulted in significant performance improvements' is not supported by the evidence reported. All results are single runs: no standard deviations, multiple seeds, or statistical tests are given. Several reported gains are very small, e.g., C-Mask3D on STPLS3D (+0.2 mAP, +0.1 mAP50, Table 4b), C-Mask3D on S3DIS (+0.3 mAP, Table 3), C-Mask3D on ScanNetv2 val (+0.8 mAP, Table 2), and C-OneFormer3D on ScanNetv2 val (+0.7 mAP, Table 2). For transformer-based instance segmentation, run-to-run variance on these datasets is typically on the order of 0.5-1.5 mAP, so these gains are indistinguishable from noise. The abstract and conclusion should either be reworded to 'reported improvements' without 'significant', or the authors should add error bars and a paired significance test (or at least multiple seeds) for the key comparisons. This is load-bearing because the paper's main contribution is the empirical improvement claim.","section":null},{"comment":"The notation in Eqs. (5)-(6) is not well-defined and appears to contain an error. Eq. (5) writes B^{l-1} = max_{j=0}^{n-1}(C^{l-1}_{IoU}(i,j)), but n is never defined (should presumably be N', the number of queries), and the expression as written is a scalar max rather than an index set. Eq. (6) then uses A[B] with the explanation 'arrangement of A in accordance with the index of B', which is nonstandard and ambiguous. Since the leader/laggard lists I_leader and I_laggard in Eqs. (7)-(8) are constructed from these quantities, the correctness of the QCL algorithm cannot be verified without a precise definition. Please rewrite Eqs. (5)-(6) with explicit index sets and define all variables.","section":null},{"comment":"The relative relationship encoding depends on two free hyperparameters, the quantization size v and the table length Y, but no sensitivity analysis or ablation is provided for them. The default values are not even stated in the implementation details section. Since RRE contributes a nontrivial part of the reported gain (e.g., +1.1 mAP when added alone in Table 5b), the choice of v and Y could materially affect the results, and the reader cannot judge how robust the improvement is to these settings. Please report the values used and include an ablation or sensitivity study for at least one dataset.","section":null}],"minor_comments":[{"comment":"The text says 'the sparse query distribution limit the performance of ComeptitorFormer'; 'ComeptitorFormer' is a typo for 'CompetitorFormer', and the sentence should read 'limits' for subject-verb agreement.","section":null},{"comment":"There are several spacing typos: 'V oxels are applied as superpoint on STPLS3D' should be 'Voxels are applied as superpoints on STPLS3D', and 'the query decoder' section header has an extra space.","section":null},{"comment":"In the QueryFormer row, the mAP50 column contains '57,7' with a comma instead of a decimal point; the formatting should be consistent with the rest of the table.","section":null},{"comment":"The caption says 'The color of each fraction matches the corresponding bounding box's color'; 'fraction' should likely be 'prediction' or 'box'.","section":null},{"comment":"The reference to 'SPFormer' as [24] in the related work (Jieru Mei et al., 'SPFormer: Enhancing vision transformer with superpixel representation') is confusing because the same name is used for the superpoint transformer baseline [30] in the main text; please clarify whether these are different works or use distinct names.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical plug-and-play contribution, and the direction of the reported gains is consistent, so I do not see a fundamental correctness flaw in the mechanism. However, the lack of repeated runs and significance testing is a serious gap given that several headline gains are below 1 mAP. The authors should be asked to either provide multiple-seed results and significance tests for the main tables, or substantially temper the 'significant' language in the abstract and conclusion. The paper also does not release code, which makes independent verification harder; asking for code release would be appropriate for a revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read the paper. It's a clean, well-scoped empirical study: three plug-and-play modules (QCL, RRE, RCA) that push transformer decoders for 3D instance segmentation to separate competing queries more sharply. The gains are consistent in direction across four baselines and four datasets, and the ablations support the design choices. That is worth something: the combination is genuinely new for 3D instance segmentation, even though each ingredient traces to EASE-DETR, MAFT, and QueryFormer, and the paper cites them honestly.\n\nThe main problem is the word 'significant.' The evidence is single-run mAP differences, and several of them are tiny: +0.2 mAP on STPLS3D, +0.3 on S3DIS, +0.7 and +0.8 on ScanNet val. Run-to-run variance for these models is typically on the order of 0.5–1.5 mAP, so those numbers are indistinguishable from noise. The one conspicuous exception is C-SPFormer's +3.1 mAP on the ScanNet hidden test, which is harder to dismiss, and the cumulative score distributions in Figure 3 give some mechanistic support. But the abstract's claim of 'consistent significant improvements' is not backed by the reported numbers.\n\nOther soft spots: no error bars, no seeds, no significance tests, no code release, and the RRE hyperparameters (quantization size v, table length Y) are never specified. All of these are fixable rather than fatal. The mechanism is self-referential—using the model's own classification scores and mask IoU to define competition—but that is not circular reasoning; the evaluation is on external benchmarks and the gains, where larger, are plausible.\n\nBottom line: this is a competent engineering contribution that deserves a serious referee, but the authors need to add multiple seeds, report variance or paired tests, release code, and tone down the 'significant' language. Without that, the central claim doesn't survive scrutiny. I'd take it to a reading group mainly as a case study in how to (and how not to) report empirical gains.","headline":"Consistent direction of gains, but 'significant' is not supported by single-run mAP differences; worth a referee if the authors add seeds and error bars.","tokens_in":15026,"tokens_out":2372,"would_cite":true,"duration_ms":21122,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that three plug-in modules, QCL, RRE, and RCA, reduce inter-query competition in transformer-based 3D instance segmentation and that adding them to existing baselines improves mAP on four benchmark datasets.","keywords":["3D instance segmentation","inter-query competition","query competition layer","relative relationship encoding","rank cross attention","point cloud","mask transformer","plug-and-play modules"],"falsifier":"Track, over the first training epochs of an unmodified baseline, the rank correlation between pair-wise IoU of predicted masks at decoder layer one and the event that the two queries are eventually matched to the same ground-truth instance; if the correlation is near zero early in training, the QCL and RRE signals are assigning leader/laggard labels before useful information exists and the mAP gain would need another explanation. A complementary check is to replace the mask-IoU competitor selection with random pairs while keeping the modules, which should collapse the gain if the competition signal is load-bearing.","tokens_in":14013,"feed_emoji":"🎯","tokens_out":10325,"duration_ms":82512,"temperature":0.7,"pith_summary":"Transformer-based 3D instance segmenters use a fixed number of instance queries that usually exceeds the number of objects, so several queries converge on the same instance while only one can win in the final ranking. The paper names this “inter-query competition” and argues that it slows convergence and lowers accuracy, because near-identical scores in early decoder layers stop a dominant query from emerging. To counter it, the paper proposes three plug-in components, the query competition layer, relative relationship encoding, and rank cross attention, which together mark competing pairs, label a leader and a laggard, and widen the score gap between them. Integrated into four existing transformer-based segmenters, the design is reported to improve mean average precision on ScanNetv2, ScanNet200, S3DIS, and STPLS3D, the largest gain being +3.1 mAP on the ScanNetv2 hidden test set over one baseline. If the claim holds, the practical payoff is that existing 3D instance segmentation models can be upgraded with a small, additive set of modules rather than a new architecture.","feed_headline":"Plug-in modules lift 3D instance segmentation mAP by 3.1","feed_subtitle":"Plug-in modules ease redundant queries competing for one object, raising mAP on four benchmarks.","key_machinery":"The central machinery is the per-layer competition state constructed from query pairs. It is built from two matrices: a signed leader/laggard matrix derived from the difference of each query's max classification score times predicted IoU, and an unsigned mask-IoU matrix that tells which queries are predicting the same object. QCL converts these into ordered leader and laggard lists that realign two static embeddings before fusing them with the query features; RRE quantizes the signed IoU into discrete bins and looks up a learned relation table to bias self-attention weights; RCA takes the raw dot-product similarities between queries and pooled scene features, min-max normalizes them across queries, and multiplies the similarities by this normalized rank signal before softmax. Together they widen the gap between the winning query and its rivals at each decoder step, so the winner absorbs richer features and the laggards fade.","core_discovery":"The central claim is that inter-query competition is an identifiable and correctable failure mode in transformer-based 3D instance segmentation. When multiple query tokens predict overlapping masks for the same instance, only one query is kept in the final output, yet the others still draw gradients and hold scores close to the winner in shallow decoder layers; the authors call this harmful and set out to suppress it. Their solution is to compute, before each decoder layer, a pairwise competition state from the IoU of the masks predicted by every two queries and from a leader/laggard sign based on each query's classification-confidence-times-IoU score. This state drives three modules: QCL reorders two static leader and laggard embeddings and fuses them into the queries, RRE quantizes the signed IoU into learned relative-relation biases added inside self-attention, and RCA normalizes the dot-product similarity between queries and pooling features along the query axis before the spatial softmax. The reported result is that these modules, alone or combined, shift classification scores so that matched queries score higher and unmatched queries score lower, and improve mAP across four benchmarks.","pith_inferences":["The same over-querying and bipartite-matching setup appears in 2D mask transformers, so the competition-state mechanism is likely transferable to 2D instance or panoptic segmentation; the paper only demonstrates it in 3D.","Part of the mAP gain may come from a re-ranking effect: by widening score gaps early, the model changes which query is selected at inference without necessarily improving the underlying masks; separating these effects would clarify the mechanism.","The reported gains are largest on frameworks with many queries, while Mask3D's smaller query count showed smaller gains, suggesting the benefit scales with the amount of redundant query capacity; varying query count under fixed modules would test this.","The paper's two static embeddings are randomly initialized per layer; a natural extension is to initialize them from semantic label embeddings so the leader/laggard signal incorporates class semantics from the start."],"forward_implications":["Adding the three modules to an existing transformer-based segmenter improves mAP without changing the training schedule or hyperparameters, according to the paper's experiments.","The largest reported improvement is +3.1 mAP over the SPFormer baseline on the ScanNetv2 hidden test set, with +2.7 mAP on the validation set.","Each module contributes independently: adding QCL, RRE, or RCA alone to SPFormer gives roughly +1.0 to +1.3 mAP on ScanNetv2 validation, and using all three gives +2.6 mAP.","The modules work by changing which query wins: matched-query classification scores move upward while unmatched-query scores move downward, and the IoU of the winning query with its target mask increases.","Because the modules are plug-in by design, they can be layered onto future transformer-based 3D segmenters that keep a fixed query count."],"supporting_citations":[{"why":"Supplies the SPFormer baseline, the main host for ablations and the source of the largest reported mAP gain.","marker":"[30]"},{"why":"Supplies the Mask3D baseline, integrated on ScanNetv2, S3DIS, and STPLS3D with its smaller fixed query count.","marker":"[28]"},{"why":"Supplies the MAFT baseline and the relative position encoding idea that RRE adapts into inter-query relationship encoding.","marker":"[18]"},{"why":"Supplies the OneFormer3D baseline, into which only RRE and RCA are integrated because its query count is not constant.","marker":"[16]"},{"why":"Establishes inter-query competition in 2D detection and supplies the competitive-state approach that the paper extends to 3D segmentation.","marker":"[12]"},{"why":"Defines the ScanNetv2 benchmark, where the main validation and hidden-test results are reported.","marker":"[9]"},{"why":"Defines the S3DIS benchmark used for the Area 5 evaluation.","marker":"[2]"},{"why":"Defines the STPLS3D aerial point-cloud benchmark used for outdoor evaluation.","marker":"[4]"}],"fun_headline_variants":["CompetitorFormer: settle query wars for sharper 3D instance masks","Stop query competition: plug-in modules boost 3D segmentation mAP","mAP +3.1 by resolving inter-query competition in 3D segmentation","CompetitorFormer: stop queries from stepping on each other's masks","New modules calm query rivalry to lift 3D instance segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the overlap between masks predicted by two queries in the previous decoder layer reliably indicates which queries are true competitors, and that this signal is accurate early enough in training for the leader/laggard labels to guide learning before masks stabilize.","fun_headline_variants_meta":{"raw":{"variants":["CompetitorFormer: settle query wars for sharper 3D instance masks","Stop query competition: plug-in modules boost 3D segmentation mAP","mAP +3.1 by resolving inter-query competition in 3D segmentation","CompetitorFormer: stop queries from stepping on each other's masks","New modules calm query rivalry to lift 3D instance segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001508,"raw_usage":{"total_tokens":6046,"prompt_tokens":943,"completion_tokens":5103,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":559,"completion_tokens_details":{"reasoning_tokens":5007}},"tokens_in":559,"tokens_out":5103,"duration_ms":34697,"temperature":1.0,"reasoning_tokens":5007,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:26:26.154105+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track, over the first training epochs of an unmodified baseline, the rank correlation between pair-wise IoU of predicted masks at decoder layer one and the event that the two queries are eventually matched to the same ground-truth instance; if the correlation is near zero early in training, the QCL and RRE signals are assigning leader/laggard labels before useful information exists and the mAP gain would need another explanation. A complementary check is to replace the mask-IoU competitor selection with random pairs while keeping the modules, which should collapse the gain if the competition signal is load-bearing.","supporting_citations":[{"cited_title":"Superpoint transformer for 3d scene instance segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the SPFormer baseline, the main host for ablations and the source of the largest reported mAP gain."},{"cited_title":"Mask3d: Mask trans- former for 3d semantic instance segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the Mask3D baseline, integrated on ScanNetv2, S3DIS, and STPLS3D with its smaller fixed query count."},{"cited_title":"Oneformer3d: One transformer for unified point cloud segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the OneFormer3D baseline, into which only RRE and RCA are integrated because its query count is not constant."},{"cited_title":"Ease-detr: Easing the competition among object queries","cited_arxiv_id":null,"evidence_quote":"Establishes inter-query competition in 2D detection and supplies the competitive-state approach that the paper extends to 3D segmentation."},{"cited_title":"3d seman- tic parsing of large-scale indoor spaces","cited_arxiv_id":null,"evidence_quote":"Defines the S3DIS benchmark used for the Area 5 evaluation."},{"cited_title":"Stpls3d: A large-scale synthetic and real aerial photogrammetry 3d point cloud dataset","cited_arxiv_id":null,"evidence_quote":"Defines the STPLS3D aerial point-cloud benchmark used for outdoor evaluation."}],"review_version":1}