{"id":"d858cf38-f4ad-43b4-b1ca-c23ab1e98ebc","arxiv_id":"2502.04139","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new query initialization and fusion design improves 3D instance segmentation accuracy on ScanNetV2, ScanNet200, ScanNet++, and S3DIS.","lead":"The paper introduces BFL, a transformer-based method for 3D instance segmentation that combines farthest-point sampling with learnable agents to initialize object queries, and fuses low-overlap queries across decoder layers to prevent recall decline. If the results hold, it offers a modest but consistent accuracy improvement over existing methods on four common indoor scene benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline comparisons are not post-processing-matched: Table 2's Maft baseline omits NMS while BFL uses it, and Appendix Table 14 contradicts Table 2 on Maft's test mAP (59.6 vs 57.8), shrinking the claimed gains.","rationale":"The reader's verdict of CONDITIONAL is appropriate, but the most decisive weakness is not the content-smoothness assumption in AI2M. Equations 3-4 describe a trainable inverse-distance kernel: even if object content is not globally smooth, the agent positions and content vectors can adapt during training, so the interpolation may still be useful. The directly verifiable problem is that the main comparison tables are not post-processing-matched. Section 3.5 says BFL uses NMS, and Table 7 confirms NMS contributes about 0.6 mAP. Table 2 reports Maft without NMS while Appendix Table 12 shows Maft+NMS improves by 0.6, and Appendix Table 14 appears to report Maft with NMS on the test set, giving 59.6 instead of the 57.8 shown in Table 2. This internal inconsistency is more load-bearing than the smoothness assumption because it directly affects the headline numbers, although it does not overturn the qualitative conclusion: the ablation in Table 7 shows AI2M and HQFD together improve mAP by 2.7 even without NMS. The paper also has a smaller overclaim issue: Table 2 shows Spherical Mask at 62.3 mAP on ScanNetV2 validation, above BFL's 61.7, so any SOTA statement must be restricted to transformer-based methods, which the contribution text mostly does. A conditional acceptance requiring consistent NMS reporting and released code remains the right outcome.","tokens_in":22700,"tokens_out":10225,"duration_ms":105525,"concrete_test":"Run the released Maft model on ScanNetV2 test under BFL's exact inference protocol, including the NMS post-processing described in Section 3.5, and check whether the test mAP reproduces Appendix Table 14's 59.6 rather than Table 2's 57.8. If it does, report all rows of Table 2 with NMS on or off consistently; the claimed test improvement becomes 60.6-59.6=1.0 and the validation improvement becomes 61.7-59.0=2.7, which is the number that should appear in the abstract and conclusion.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the central SOTA comparisons in Table 2 are not post-processing-matched. Section 3.5 states that BFL applies NMS at inference, and Table 7 shows NMS adds about 0.6 mAP for the full method. Yet Table 2 reports Maft at 58.4 mAP on ScanNetV2 validation and 57.8 mAP on test without disclosing whether Maft is NMS-processed, while Table 12 shows Maft+NMS reaches 59.0 on validation. More seriously, the paper's own Appendix Table 14 lists Maft test mAP as 59.6, contradicting Table 2's 57.8 for the same test set. The plausible reconciliation is that Table 14 applies NMS to Maft while Table 2 does not. If that is correct, the advertised test gain shrinks from 60.6-57.8=2.8 to 60.6-59.6=1.0, and the validation gain shrinks from 61.7-58.4=3.3 to 61.7-59.0=2.7. The module ablations in Table 7 still show a real improvement without NMS (61.1 vs 58.4), so the qualitative conclusion may survive, but the exact SOTA margin and the multi-benchmark claim rest on an inconsistent baseline. This needs to be corrected or disclosed before the headline comparisons can be taken at face value.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BFL, a transformer-based 3D instance segmentation method built on the Maft baseline. It introduces two main components: the Agent-Interpolation Initialization Module (AI2M), which combines FPS-sampled position queries with content queries interpolated from learnable agent position-content pairs (Eqs. 1-5), and the Hierarchical Query Fusion Decoder (HQFD), which carries forward low-overlap queries from earlier decoder layers (Eq. 6) to counteract the observed inter-layer recall decline. Experiments on ScanNetV2, ScanNet200, ScanNet++, and S3DIS report gains over Maft and other transformer-based methods, supported by ablations for each module, a plug-and-play generalization study, and a runtime analysis.","tokens_in":23063,"tokens_out":9224,"duration_ms":86645,"significance":"If the comparison protocol is corrected, the contribution is a solid incremental improvement. The AI2M idea is simple and well motivated, and HQFD is shown to be plug-and-play on SPFormer and Maft (Table 10) and to produce monotonic recall improvement (Table 18). The paper provides extensive ablations (Tables 5, 7-10, 18-21) and clearly describes the method. However, the headline state-of-the-art claim currently rests on an inconsistent evaluation: BFL is inferred with NMS while the main-table Maft baseline appears to be without NMS, and the paper's own appendix contradicts the test-set baseline number. These issues are fixable without changing the method, but they directly affect the advertised performance margins.","major_comments":[{"comment":"The headline comparison in Table 2 is not post-processing-matched. Section 3.5 states that BFL applies NMS at inference, and Table 7 shows that NMS adds 0.6 mAP to the full method (61.1 to 61.7). The Maft baseline in Table 2 is reported at 58.4 mAP on ScanNetV2 validation, which matches the no-NMS Maft in Table 12, while the same appendix table reports Maft+NMS at 59.0. With NMS applied to both, the validation gain would be 61.7 - 59.0 = 2.7 mAP rather than the advertised 3.3. Please re-run all baselines under the identical post-processing, or report both with-NMS and without-NMS columns in every comparison table and explicitly state which protocol each number uses. I note that the module ablation in Table 7 (row 4) still shows a 2.7 mAP gain without NMS, so the qualitative conclusion may survive, but the exact SOTA margin is currently overstated.","section":"Section 3.5, Tables 2, 7, 12"},{"comment":"The ScanNetV2 test numbers are internally inconsistent. Table 2 reports Maft at 57.8 mAP on the hidden test set, but Table 14 reports Maft at 59.6 mAP on the same test set, and Table 15 reports Maft AP@50 at 78.6 versus 77.4 in Table 2. If the appendix numbers are the NMS-applied ones, the advertised test margin shrinks from 60.6 - 57.8 = 2.8 mAP to 60.6 - 59.6 = 1.0 mAP. Please reconcile the two tables, state which evaluation protocol each number uses, and ensure the main-table baseline is the stronger of the two if NMS is part of the method's inference.","section":"Appendix Table 14 vs Table 2"},{"comment":"The multi-benchmark superiority claim rests on small margins over the Maft baseline (ScanNet++ test 22.2 vs 20.9; ScanNet200 validation 30.5 vs 29.2; S3DIS Area 5 AP@50 71.9 vs 69.1), and no error bars or repeated-seed results are reported. Given the post-processing mismatch documented above, it is currently unclear whether these differences are robust. Please report mean and standard deviation over at least three training runs for the main comparisons, or otherwise provide a stability analysis such as evaluation under different NMS thresholds or across checkpoints.","section":"Tables 3, 4, 6; Section 4.1"}],"minor_comments":[{"comment":"The reported mean distances between FPS-sampled points and instance centers are not compared with the analogous distances for learnable-based or random initialization, so the table does not by itself establish that FPS provides a stronger position-content correspondence.","section":"Section 3.3.1, Table 1"},{"comment":"The caption of Figure 5 is preceded by a stray duplicate caption line beginning with 'Figure 1: The convergence curve...'; the figure is referenced as Figure 5, so the numbering should be corrected.","section":"Section 4.3"},{"comment":"Appendix Table 17 mislabels the PointGroup row as Wu et al. (2022) and the HAIS row as Schult et al. (2022); the correct references are Jiang et al. (2020b) for PointGroup and Chen et al. (2021b) for HAIS.","section":"Appendix Table 17"},{"comment":"There are several typos: 'promose' in Section 5, 'teh' in Appendix A.3, 'moudle' in Appendix A.6, and the Maft row in Table 13 contains run-together numbers ('55.144.355.0').","section":"Throughout"},{"comment":"The roles of D1 and D2 are only partially specified in the main text; please add a one-sentence definition that D1 is the number of retained low-overlap queries per fusion step and D2 is the number of final decoder layers in which fusion is performed.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The core method is plausible and the ablations are informative, but the main comparison tables must be made internally consistent before the results can be taken at face value. The NMS mismatch and the Table 14/Table 2 contradiction are the load-bearing issues; if the authors correct these and provide repeated-run statistics, I would view the paper as publishable. The heavy self-citation to the authors' QueryFormer is noticeable but not inappropriate given the direct baseline relationship."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the core idea is good: combining FPS position sampling with learnable content agents through inverse-distance interpolation gives better initial queries than either approach alone, and the hierarchical query fusion decoder prevents the recall drop that plagues per-layer supervised transformers. The ablation evidence supports both modules, and the plug-and-play test on SPFormer and Maft is a nice touch. Second, the headline numbers are not to be trusted as reported. The main comparison table gives Maft without NMS while the proposed method uses NMS; the paper's own appendix shows Maft+NMS is 59.0 val and 59.6 test, versus 58.4 and 57.8 in the main table. That shrinks the advertised gains from 3.3/2.8 to about 2.7/1.0. The module-level ablation in Table 7, which does not use NMS for either side, still shows a real 2.7 mAP gain (61.1 vs 58.4), so the method survives, but the SOTA claim needs correcting.\n\nThe paper is clearly written and the experiments span four datasets. The AI2M design is genuinely new as a fusion of FPS and learnable queries; the smoothness assumption behind interpolation is reasonable and, more importantly, supported by the empirical gains. HQFD is also a simple and sensible fix to a known problem. The main weaknesses are the comparison inconsistency, the lack of code, and the single-run evaluation with no error bars. The appendix also has a few typos and a mislabeled table, but those are minor.\n\nThis paper is for researchers working on transformer-based 3D instance segmentation. It deserves a serious referee: the modules are promising and the experiments are broad, but the authors must fix the NMS mismatch and release code before I would trust the SOTA numbers. If the revision addresses that, it is a solid contribution.","headline":"Solid plug-and-play modules for 3D instance segmentation, but the headline SOTA margins are inflated by an NMS mismatch that the authors should fix.","tokens_in":23520,"tokens_out":3173,"would_cite":true,"duration_ms":28503,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that transformer-based 3D instance segmentation can be improved by interpolating content queries from learnable agents at farthest-point-sampled positions and by carrying low-overlap queries forward across decoder layers…","keywords":["3D instance segmentation","transformer","object queries","query initialization","farthest point sampling","learnable agents","interpolation","recall"],"falsifier":"Run the ScanNetV2 validation experiment with Eq. 3's weights replaced by uniform weights over the K nearest agents, keeping all other settings (S=400, L=400, K=3) fixed. If mAP does not drop outside the noise of Table 8 (or if the K=8 row behaves the same as K=3), then the inverse-distance interpolation is not carrying the AI2M gain. A second check is to keep HQFD but initialize all content queries from a single shared learned vector with no agent interpolation, and observe whether first-layer recall and final mAP fall back toward the FPS-based baseline.","tokens_in":22517,"feed_emoji":"🧩","tokens_out":12453,"duration_ms":105540,"temperature":0.7,"pith_summary":"Transformer-based 3D instance segmentation methods initialize a fixed set of object queries and supervise each decoder layer, but two failure modes remain: query initialization cannot keep both strong position and strong content information, and objects disappear as layers deepen because difficult instances get poorly updated. The paper proposes BFL, with an Agent-Interpolation Initialization Module that runs farthest point sampling on the scene and then interpolates a small set of learnable agent content embeddings at those sampled positions, and a Hierarchical Query Fusion Decoder that retains queries from the previous layer whose masks have low overlap with the current layer's masks. On ScanNetV2, ScanNet200, ScanNet++, and S3DIS, BFL reports higher mAP and AP than prior transformer-based methods, including 61.7 mAP on ScanNetV2 validation versus 58.4 for the Maft baseline. The paper's claim is that the query lifecycle—how queries are born and how they survive across layers—carries much of the remaining accuracy gap, and can be fixed without changing the backbone or the loss.","feed_headline":"Two query fixes lift 3D instance segmentation on four benchmarks","feed_subtitle":"Keeping low-overlap queries across decoder layers stops objects vanishing and lifts four benchmarks.","key_machinery":"The machinery is the agent-interpolation identity: for each of S farthest-point-sampled positions, find its K nearest learnable agents, weight their content vectors by inverse distance ($W_{i,j}=dis^{-1}_{i,j}/\\sum_j dis^{-1}_{i,j}$), and sum to form the content query $Q^c_i$. A scene-specific refinement rescales agent positions into the scene's bounding box, and a stop-gradient trick ($\\tilde{Q}^p = SG(Q^p - \\Phi(W,Q^p_0,idx)) + \\Phi(W,Q^p_0,idx)$) keeps agent positions trainable while leaving the actual query positions unchanged. The second mechanism is the fusion gate: between adjacent decoder layers, compute pairwise mask IoU, take each previous-layer mask's maximum IoU against all current-layer masks, keep the D1 previous-layer queries with the smallest such maxima, and concatenate them into the next layer. The first mechanism imports cross-scene content without sacrificing foreground coverage; the second prevents difficult instances from being silently dropped.","core_discovery":"The central claim is that the remaining accuracy gap in transformer-based 3D instance segmentation lies in the query lifecycle, not the backbone or the loss. The Agent-Interpolation Initialization Module treats query initialization as a balance between foreground coverage and content learning: farthest-point sampling supplies positions that cover nearly all foreground objects, while learnable agent content vectors supply dataset-level inductive bias, and inverse-distance interpolation fuses the two. The Hierarchical Query Fusion Decoder treats recall decline as a survival problem: when a query's mask after a decoder update has low maximum IoU against all masks from the previous layer, the pre-update query is copied forward and re-updated, preventing hard instances from disappearing. The paper reports that this combination makes per-layer recall improve monotonically, records the highest mAP/AP among compared methods on ScanNetV2 (61.7 validation and 60.6 test mAP), ScanNet200 (30.5), ScanNet++ (25.3 validation and 22.2 test), and S3DIS (71.9 AP@50), and shows that the fusion module also improves SPFormer and Maft when added on top.","pith_inferences":["Editorial inference: the inverse-distance weighting in Eq. 3 is an untested modeling choice; replacing it with uniform weights among the K nearest agents would show whether the smoothness prior itself, rather than having learnable content at FPS positions, drives the AI2M gain. Table 8's weak sensitivity to K hints the answer may be the latter, but the paper does not run this control.","Editorial inference: the low-overlap fusion mechanism is not inherently three-dimensional; 2D transformer-based instance segmentation also supervises every decoder layer and suffers from object disappearance, so HQFD-style retention is a transferable idea the paper does not test.","Editorial inference: the method's reliance on FPS assumes foreground objects are spread through Euclidean space; in scenes with strongly uneven instance density, such as street scenes, the fixed number of agents L may need to scale with the scene, a regime the four indoor benchmarks do not cover."],"forward_implications":["If the central claim holds, query initialization in 3D instance segmentation can be improved without redesigning the backbone, the decoder layer, or the loss function—only the way content is attached to sampled positions needs to change.","The Hierarchical Query Fusion Decoder is claimed to be plug-and-play: adding it to SPFormer and Maft raises their mAP, so other per-layer-supervised query methods may get a similar recall lift by retaining low-overlap queries.","The reported convergence curve implies that training budgets can be cut: BFL at 128 epochs beats the baseline at 512, so the initialization and fusion mechanisms shorten training time as well as improving accuracy.","The cost of the recall fix is small: query count grows by at most D1 per fused layer, runtime rises by 7.8%, and parameters grow by 0.2M relative to Maft, suggesting the gains are not bought with a larger model.","The same architecture, loss, and decoder are used on ScanNetV2, ScanNet200, ScanNet++, and S3DIS, so the reported gains extend across annotation density (198 classes in ScanNet200, sub-millimeter resolution in ScanNet++) without dataset-specific changes."],"supporting_citations":[{"why":"Supplies the Maft baseline architecture (sparse UNet, decoder layer, matching and loss) that BFL extends, and the 58.4 mAP comparison point on ScanNetV2.","marker":"Lai et al. (2023)"},{"why":"Mask3D, the FPS-based query initialization and mask-attention baseline that motivates the interpolation design and is compared at 55.2 mAP.","marker":"Schult et al. (2022)"},{"why":"SPFormer, the learnable-query baseline used for superpoint features and for the HQFD generalization test showing 59.4 mAP with HQFD.","marker":"Sun et al. (2023)"},{"why":"QueryFormer, an FPS-based transformer baseline with center-aware matching that the method follows for position-to-instance correlation.","marker":"Lu et al. (2023)"},{"why":"Supplies ScanNetV2, the primary benchmark whose validation and hidden test sets carry the headline numbers.","marker":"Dai et al. (2017)"},{"why":"Supplies ScanNet200, the 198-instance-class benchmark that tests wide-category generalization.","marker":"Rozenberszki et al. (2022)"},{"why":"Supplies ScanNet++, whose sub-millimeter dense scenes test performance under high resolution and more classes.","marker":"Yeshwanth et al. (2023)"},{"why":"Supplies S3DIS Area 5, the cross-dataset generalization benchmark.","marker":"Armeni et al. (2016)"},{"why":"Provides the Hungarian matching used in per-layer supervision, the mechanism whose instability is identified as the cause of object disappearance.","marker":"Kuhn (1955)"},{"why":"Provides NMS, the postprocessing alternative that the paper argues cannot replace the learned query-fusion mechanism for inter-layer outputs.","marker":"Neubeck & Van Gool (2006)"}],"fun_headline_variants":["Keeping hard queries alive beats SOTA on four 3D benchmarks","Agent-interpolation and fusion queries stop object vanishing","BFL: query init + fusion lift all four 3D benchmarks","Two query fixes prevent object vanish in 3D segmentation","Hierarchical fusion keeps low-overlap queries, lifts 3D seg"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that a content query formed by inverse-distance-weighted interpolation of a few learned agent vectors at farthest-point-sampled positions is genuinely more informative than a pure positional embedding—that is, that useful content varies smoothly enough in 3D within the agent set—and the reported AP gains would not materialize if that smoothness assumption failed.","fun_headline_variants_meta":{"raw":{"variants":["Keeping hard queries alive beats SOTA on four 3D benchmarks","Agent-interpolation and fusion queries stop object vanishing","BFL: query init + fusion lift all four 3D benchmarks","Two query fixes prevent object vanish in 3D segmentation","Hierarchical fusion keeps low-overlap queries, lifts 3D seg"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000783,"raw_usage":{"total_tokens":3465,"prompt_tokens":963,"completion_tokens":2502,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":2415}},"tokens_in":579,"tokens_out":2502,"duration_ms":17566,"temperature":1.0,"reasoning_tokens":2415,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T23:21:48.644244+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the ScanNetV2 validation experiment with Eq. 3's weights replaced by uniform weights over the K nearest agents, keeping all other settings (S=400, L=400, K=3) fixed. If mAP does not drop outside the noise of Table 8 (or if the K=8 row behaves the same as K=3), then the inverse-distance interpolation is not carrying the AI2M gain. A second check is to keep HQFD but initialize all content queries from a single shared learned vector with no agent interpolation, and observe whether first-layer recall and final mAP fall back toward the FPS-based baseline.","supporting_citations":[],"review_version":1}