{"id":"e99bb65a-d3c4-409d-a588-bc91ee82f1e7","arxiv_id":"2506.06412","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"NeurNCD proposes a NeRF-based framework for novel class discovery in RGB-D scenes, claiming superior mIoU on NYUv2 and Replica, though the presented implementation is internally inconsistent.","lead":"NeurNCD replaces explicit 3D segmentation maps with a neural radiance field that aggregates semantic features for novel class discovery. The paper claims state-of-the-art results on NYUv2 and Replica, but the described algorithm contains a critical flaw that undermines the method.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's feature modulation fills each segment's vector only at its own mask pixels, so Eq. 12's cosine similarity is identically zero for any two distinct segments; the Markov clustering step and the reported results cannot follow from the described method.","rationale":"The central claim requires the full pipeline to turn segment features into clusters. Algorithm 1 is the only operational specification of how segment features are formed, and Eq. 12 is the only similarity measure fed to Markov clustering. As printed, the feature vector for segment i is nonzero only on that segment's mask; masks from geometric segmentation are disjoint, so cross-segment dot products vanish identically. This is not a hyperparameter or evaluation issue; it makes the clustering input degenerate and the reported mIoU untraceable. The reader's weakest assumption is exactly this, and my independent reading agrees. I did not find an alternative assumption that is more load-bearing. Other weaknesses--under-specified entropy generation, hand-tuned MCL inflation, and the absence of a non-NeRF ablation--are real but would matter only after the pipeline can be made to run as described. There is no released code and no formal verification, so the ambiguity cannot be resolved by inspection. A concrete rerun of Algorithm 1 on two segments settles the point; if the authors' code aggregates per segment, they need to say so and reproduce the tables with that documented operation.","tokens_in":12828,"tokens_out":4235,"duration_ms":43644,"concrete_test":"Implement Algorithm 1 exactly as printed on one NYUv2 frame: take two convex segments from Eq. 10, the Embedding-NeRF output E and entropy epsilon, construct H per the pseudocode, and evaluate Eq. 12 for those two segments. The cosine similarity is 0, confirming the degenerate clustering input. Then replace the per-pixel assignment with segment-wise averaging of E and epsilon before flattening and rerun the full pipeline; if the reported NYUv2 mIoU of 51.29 changes materially or requires retuning the inflation parameter, the paper must document the corrected operation and release the code before its central claim can be assessed.","verdict_should_be":"REJECT","load_bearing_attack":"Algorithm 1 (lines 7--21) is the only specification of feature modulation, and it constructs H by copying E[mask] and epsilon[mask] into H[i] only where mask[h,w]==1. Since geometric segmentation (Eq. 10) partitions the depth image into disjoint segments, for i != j the supports of H[i] and H[j] are disjoint. The dot product in Eq. 12 therefore contains only terms in which one factor is zero, so the cosine similarity is zero for every pair of distinct segments. Feeding this matrix to Markov clustering cannot merge over-segmented pieces; it leaves each segment isolated, so the method as written cannot discover instance or class groupings. The text in Section 3.3 calls H_j^i a 'high-dimensional vector' with dimensions N x H x W x (S+1), but no per-segment aggregation (mean, max, or voting) is specified. If the implementation instead averages features within each segment, that operation is undocumented and no code is released. Either way, the central claim--that NeurNCD significantly outperforms prior methods--is not supported by the described algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes NeurNCD, a framework for novel class discovery that replaces explicit 3D segmentation maps with an implicit neural representation called Embedding-NeRF. The pipeline extracts semantic embeddings from a pretrained RGB-D segmentation network (ESANet), trains a per-scene NeRF with a photometric loss and a KL-divergence embedding loss, segments depth images into convex sub-instance segments, queries the NeRF output to obtain embeddings and entropy for each segment, concatenates these into segment-level feature vectors, and finally runs Markov clustering based on cosine similarity. The authors report state-of-the-art results on NYUv2 and Replica for both known-class segmentation and novel-class discovery, and include ablations of the major components.","tokens_in":13042,"tokens_out":5345,"duration_ms":53528,"significance":"If the method were correctly specified and reproducible, the paper would introduce a novel use of implicit neural representations for open-world semantic segmentation and novel class discovery, which is an interesting direction for embodied perception. The paper also attempts a principled KL-divergence objective for transferring semantic information into a NeRF and offers quantitative comparisons against both explicit-map baselines and supervised implicit baselines. However, the paper does not release code, and the central algorithmic step as written is degenerate: Algorithm 1 constructs segment vectors with disjoint nonzero supports, making the cosine similarities in Eq. (12) identically zero. The reported results therefore cannot be produced by the described method, and the contribution cannot be verified in its current form.","major_comments":[{"comment":"The feature modulation step as written makes the clustering input degenerate. Geometric segmentation partitions the depth image, so masks of distinct segments are disjoint; Algorithm 1 (lines 15–16) copies combined features only into the positions of each segment's own mask and leaves all other entries zero. After the reshape on line 21, for any i != j the dot product H_i · H_j in Eq. (12) contains only terms where one factor is zero, so the cosine similarity is identically zero for every pair of distinct segments. Markov clustering then receives a graph with no positive edges and cannot merge over-segmented pieces, so the reported improvement from clustering cannot be produced by the described method. If the actual implementation aggregates features per segment (e.g., by averaging), that operation is neither documented nor released as code, and the paper would need to specify and validate it before the claims can be assessed.","section":"Section 3.3, Algorithm 1 and Eq. (12)"},{"comment":"The entropy term used in feature modulation is not defined in a reproducible way. The sentence introducing Eq. (9) states that semantic embedding E_i 'obtained by fusion' is sent to 'the two upsampling modules' to obtain U_i^o, but no fusion mechanism, upsampling architecture, or input–output relation is specified anywhere in the paper. Moreover, Eq. (9) presents epsilon_i as a scalar while Algorithm 1 and Eq. (11) treat entropy as a per-pixel vector concatenated with the embedding; the discrepancy makes the exact feature vector H_j^i ambiguous.","section":"Section 3.1, Eq. (9)"},{"comment":"The evaluation protocol is transductive in a way that is not compared fairly with the baselines. The paper trains a separate Embedding-NeRF per scene on the same images that are later evaluated ('the official split of 654 images is used for testing'; 'train a separate Embedding-NeRF model for each scene'), so the model has already fitted the test frames before the segmentation metrics are computed. The comparison with [19] and [27], which are evaluated in an incremental/online setting, therefore conflates scene memorization with generalization; a held-out view or scene split is needed to support the data-efficiency claim.","section":"Section 4.2, NYUv2 evaluation"},{"comment":"The ablation table does not isolate the contribution of entropy (EP): the rows labeled with four checkmarks are ambiguous about which components are active (GS, PSSN, EP, SE, or a different combination), and the text's claim that 'incorporating only entropy features into segments yields inferior outcomes' is not backed by any row with only EP added. The reported ablation for the central components is therefore not verifiable from the table as printed.","section":"Section 4.5, Table 3"}],"minor_comments":[{"comment":"The text attributes Markov clustering to reference [37], but [37] is a survey and the deep MCL method is [39]; the exact algorithm and the hand-tuned parameter set (beyond inflation=12) should be identified for reproducibility.","section":"Section 3.4"},{"comment":"The loss weight lambda in Eq. (8) is never given; without its value and the schedule, the KL-loss contribution cannot be reproduced.","section":"Section 4.2"},{"comment":"'leverge' should be 'leverage', and the keywords list 'Neural Radiation Field' should be 'Neural Radiance Field'.","section":"Figure 2 caption and keywords"},{"comment":"The per-class numbers and mIoU totals would benefit from standard deviations over multiple runs and from a clearer statement of which classes are known versus novel; Table 2's 'Our -81.3 50.6 89.1 89.7' row also contains an unexplained dash.","section":"Tables 1 and 2"},{"comment":"Eq. (6) and the surrounding text do not define t_k, delta_k, or the coarse/fine sampling scheme; the paper should either define these or cite the original NeRF formulation with the needed notation.","section":"Section 3.1, Eq. (6)"}],"recommendation":"reject","confidential_remarks":"The reader's stress-test concern is valid: Algorithm 1 produces zero cosine similarities for distinct segments, so the central clustering step cannot operate as described. The paper is an official ICMR'24 publication, but the arXiv version contains no code, and the described algorithm is not reproducible. The topic is promising, and a corrected submission with explicit per-segment feature aggregation, a well-defined entropy module, and a proper held-out evaluation protocol could be worth considering, but the current manuscript cannot support its central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe central algorithm, as written, cannot do what the paper claims. In Algorithm 1, each segment's feature tensor is filled only at the pixels belonging to that segment's mask. Since geometric segmentation partitions the depth image into disjoint masks, any two distinct segments have disjoint supports. After flattening, the dot product in Eq. 12 gets contributions only from pixels where one of the two vectors is zero, so the cosine similarity is zero for every pair of distinct segments. The Markov clustering step receives an all-zero similarity matrix and cannot merge anything. This is not a missing detail; it is a contradiction between the described method and the reported results.\n\nThe paper deserves credit for the high-level idea. Using an implicit neural representation as a queryable, continuous replacement for explicit 3D segmentation maps in novel class discovery is a real gap in the literature, and the KL-divergence formulation of Embedding-NeRF is a sensible step in that direction. The motivation is clearly argued, and the two-dataset evaluation targets an important practical problem.\n\nThe soft spots go beyond the algorithm bug. The entropy-generation path is under-specified; the two upsampling modules materialize without definition. The ablation does not isolate the NeRF's contribution—the first row lacks all learned components, and the later rows add multiple modules at once. Clustering hyperparameters are hand-tuned on a single scene with no variance reported. There is also a mild circularity: the same pre-trained teacher features supervise the NeRF and form the final clusters, so the improvement over baselines may be mostly feature smoothing. The comparison against Semantic-NeRF is a bit apples-to-oranges because that baseline is supervised with sparse labels.\n\nIf the implementation actually averages features per segment rather than copying per-pixel values, that operation is nowhere in the text. Without code or a corrected algorithm, the numbers are unverifiable.\n\nThis paper is for researchers interested in open-world 3D perception, but only as a motivating idea, not as a reliable method. I would send it to referees rather than desk-reject, because the novel intersection deserves a careful look and the flaw is fixable in principle. But it needs major revision: a correct algorithm specification, an honest ablation, and released code.\n\nRecommendation: reject in current form; encourage resubmission after fixing the algorithm and providing code.","headline":"Algorithm 1 as written zeroes out cosine similarities between distinct segments, so the described method cannot produce the reported clustering results—though the NeRF-for-NCD idea itself merits a second look.","tokens_in":13565,"tokens_out":5789,"would_cite":false,"duration_ms":55145,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that NeurNCD, built on an Embedding-NeRF implicit scene representation, outperforms explicit-map baselines in both known-class segmentation and novel-class discovery on NYUv2 and Replica without dense labels.","keywords":["Novel class discovery","Implicit neural representation","Neural radiance fields","Semantic segmentation","Visual embedding space","Feature modulation","Markov clustering","Open-world perception"],"falsifier":"Implement Algorithm 1 and Equation (12) on two disjoint segments from any depth frame: because the masks are disjoint, the cosine similarity is zero for every pair, the Markov transition matrix is degenerate, and the reported clusters cannot emerge, so a working implementation must be doing something other than what the paper describes.","tokens_in":1606,"feed_emoji":"🔍","tokens_out":2055,"duration_ms":71995,"temperature":0.7,"pith_summary":"NeurNCD is a framework for discovering object classes that a segmentation model was never trained on, using an implicit neural representation instead of an explicit 3D segmentation map. The paper argues that explicit maps are discrete, noisy, and hole-prone, and that replacing them with an Embedding-NeRF, trained with KL divergence, produces globally consistent semantic embeddings and entropy. These features are attached to geometrically segmented regions and clustered with Markov clustering, yielding known-class segmentation and novel-class discovery without dense supervision or human interaction. The paper reports state-of-the-art results on NYUv2 and Replica, including a mean IoU of 51.29 on NYUv2 and a known-class mIoU of 81.3 on Replica with no annotations.","feed_headline":"Implicit 3D scene model finds classes it never saw","feed_subtitle":"It fuses multi-view RGB-D features into a radiance field, then clusters segments to find known and new classes without dense labels.","key_machinery":"The load-bearing component is Embedding-NeRF, an augmented neural radiance field whose multilayer perceptron maps a 3D position to color, density, and semantic embedding logits. It is trained with a KL-divergence embedding loss, which encourages the rendered embedding to match the pre-trained segmentation network's embedding whether the input is clean or noisy, plus a photometric loss; this produces globally consistent semantic embeddings and entropy across views. These features are coupled with geometric segmentation into convex sub-instance-level segments, then fused by feature modulation and grouped by Markov clustering, so that over-segmented geometric pieces of the same known or novel class are reunited.","core_discovery":"The central claim is that implicit neural representations can substitute for explicitly constructed 3D segmentation maps in open-world semantic segmentation. NeurNCD extends NeRF so that its MLP outputs radiance, volume density, and semantic embedding logits, and renders these embeddings along rays; the rendered embeddings are trained to match embeddings from a pre-trained RGB-D segmentation network via KL divergence, while a photometric loss maintains scene appearance. The resulting Embedding-NeRF produces a hole-free, low-noise semantic embedding field and an entropy field, which are queried by convex sub-instance-level segments from depth-based geometric segmentation, concatenated in a feature modulation step, and clustered with Markov clustering based on cosine similarity. The paper reports that this pipeline segments known classes more accurately than the explicit-map baseline and also discovers novel classes, outperforming state-of-the-art approaches on both NYUv2 and Replica.","pith_inferences":["The KL-divergence training recipe may transfer to other implicit scene representations, such as hash-grid or Gaussian-splatting fields; testing NeurNCD on those backbones would reveal whether the gains come from the implicit representation itself or from the specific MLP architecture.","The method inherits a reliance on convexity-based geometric segmentation, so heavily non-convex or heavily occluded objects are likely to remain fragmented; augmenting geometric segmentation with learned grouping could raise novel-class mIoU further.","The entropy field produced by Embedding-NeRF could double as an uncertainty signal for active learning, letting an embodied agent request human labels only for high-entropy regions and extend the class vocabulary incrementally.","If the feature modulation step in practice aggregates embeddings per segment rather than assigning them by mask locations, then the published Algorithm 1 and Equation (12) do not describe the implemented clustering input, and the method should be re-specified as a general per-segment feature aggregation recipe."],"forward_implications":["Open-world scene understanding can be built on a single implicit representation that simultaneously renders appearance, geometry, and semantics, rather than maintaining a separate explicit segmentation map.","The same pipeline works in closed-world settings, so a single framework could serve both known-class segmentation and incremental discovery without switching representations.","Because the method needs only a pre-trained 2D segmentation network plus posed RGB-D frames, it removes the cost of dense pixel annotation and interactive labeling for a new scene.","Training time is reported at roughly eight hours per scene on a single GPU, comparable to a supervised semantic NeRF, suggesting that the discovery capability does not add a large training overhead.","The per-class improvements on NYUv2 and the Replica comparison against sparse-label and interactive baselines indicate that implicit feature aggregation can compensate for errors in geometric over-segmentation."],"supporting_citations":[{"why":"Supplies the NeRF volume-rendering formulation and MLP scene representation that Embedding-NeRF extends.","marker":"[17]"},{"why":"Defines the explicit 3D segmentation map approach that NeurNCD replaces, and is the source of the incremental class discovery formulation and geometric segmentation concept.","marker":"[19]"},{"why":"Semantic NeRF is the supervised implicit semantic segmentation baseline that NeurNCD compares against on Replica and whose sparse-label experiments set the comparison standard.","marker":"[41]"},{"why":"Provides the KL-divergence-based open-set prototype idea used for the embedding loss that aggregates semantic features while capturing unknown classes.","marker":"[38]"},{"why":"ESANet is the pre-trained RGB-D semantic segmentation network that extracts the semantic embeddings fed into Embedding-NeRF.","marker":"[22]"},{"why":"Supplies the geometric segmentation method that divides depth images into convex sub-instance-level segments, used for querying and clustering.","marker":"[9]"},{"why":"Supplies the Markov clustering algorithm used to group sub-instance segments into known and novel semantic classes.","marker":"[37]"},{"why":"NYUv2 is the primary evaluation dataset for both known-class segmentation and novel-class discovery.","marker":"[23]"},{"why":"Replica is the reconstruction-based dataset used to evaluate novel-class discovery against sparse-label and interactive baselines.","marker":"[25]"}],"fun_headline_variants":["Implicit neural fields find unseen classes in 3D scenes","NeurNCD uses NeRF to spot novel classes without dense labels","Open-world segmentation with implicit maps uncovers new objects","Radiance fields reveal hidden classes in RGB-D data"],"cache_read_input_tokens":15744,"weakest_assumption_plain":"The entire clustering pipeline rests on the assumption that the segment feature vectors produced by Feature Modulation have meaningful pairwise cosine similarities, but as written each vector is nonzero only at its own mask pixels, so distinct segments share no nonzero entries and every pairwise cosine similarity is zero.","fun_headline_variants_meta":{"raw":{"variants":["Implicit neural fields find unseen classes in 3D scenes","NeurNCD uses NeRF to spot novel classes without dense labels","Open-world segmentation with implicit maps uncovers new objects","Radiance fields reveal hidden classes in RGB-D data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1401,"prompt_tokens":903,"completion_tokens":498,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":519,"tokens_out":498,"duration_ms":5055,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T05:58:27.130123+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Implement Algorithm 1 and Equation (12) on two disjoint segments from any depth frame: because the masks are disjoint, the cosine similarity is zero for every pair, the Markov transition matrix is degenerate, and the reported clusters cannot emerge, so a working implementation must be doing something other than what the paper describes.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the explicit 3D segmentation map approach that NeurNCD replaces, and is the source of the incremental class discovery formulation and geometric segmentation concept."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Semantic NeRF is the supervised implicit semantic segmentation baseline that NeurNCD compares against on Replica and whose sparse-label experiments set the comparison standard."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the KL-divergence-based open-set prototype idea used for the embedding loss that aggregates semantic features while capturing unknown classes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ESANet is the pre-trained RGB-D semantic segmentation network that extracts the semantic embeddings fed into Embedding-NeRF."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the geometric segmentation method that divides depth images into convex sub-instance-level segments, used for querying and clustering."}],"review_version":1}