{"id":"36bac3ad-cc4b-4714-9dd0-71b13d5b84ac","arxiv_id":"2507.14596","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A NeRF-based method jointly performs unsupervised semantic clustering and CLIP-guided relevancy to discover and segment query-relevant sub-concepts in 3D scenes, with a new Replica benchmark.","lead":"DiSCO-3D automatically discovers and segments sub-categories of objects in a 3D scene based on a user's text query, combining unsupervised clustering with open-vocabulary image-text models inside a neural radiance field. It introduces a new benchmark and beats sequential baselines, though gains over strong open-vocabulary baselines are modest.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The discovery claim depends on an unspecified Nq: Eq. (4)'s Hq requires knowing how many relevant sub-concepts to supervise, and the fixed-N=10 experiment never states how Hq is set.","rationale":"The reader's weakest assumption is the a priori specification of Nq, and my reading of Section 3.4 and Tables 2/6 confirms this is the most load-bearing point. The entire 'discovery' framing rests on the method being able to find query-relevant sub-concepts without the user providing their number or names; Eq. (4) makes Nq an explicit input through Hq, and Section 3.5 extends this requirement to per-query prototype relevance. The main fixed-N=10 experiment is presented as removing the need for GT knowledge, but the paper never specifies how Hq is computed in that setting, so the no-GT-knowledge claim is unverifiable. I do not see a deeper internal inconsistency in the loss design; the projector/prototype updates, EMA, and CLIP-guided regularization are coherent. The method also has independent support in the form of ablations showing robustness to extra prototypes (Table 2) and a clear extension to multiple feature fields, which mitigates some of the concern. However, the Nq ambiguity should be resolved before the central contribution can be fully assessed, which is consistent with the reader's CONDITIONAL verdict. My concern does not move the verdict; it strengthens the condition that the authors disclose the exact Hq construction and evaluate a non-oracle Nq setting.","tokens_in":21123,"tokens_out":7533,"duration_ms":91516,"concrete_test":"Re-run the OV-SD benchmark (Table 1) with Hq built without GT-derived Nq, e.g., fix Nq=N-Nirr=7 for every query while keeping N=10, and report PQ/mIoU/mAcc on LeRF and OpenNeRF. If the numbers drop substantially or the margin over the sequential baselines shrinks, the reported discovery performance depends on knowing Nq; the authors should also disclose the exact Hq used in the fixed-N=10 column of Table 2.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DiSCO-3D discovers query-relevant sub-concepts without the user nominating classes. The load-bearing condition is that the guidance loss Lq_irr (Eq. 4) can be built without oracle knowledge of the number of relevant sub-concepts. This condition is not secured. Hq is defined as a one-hot vector with Nq leading ones and Nirr trailing zeros, so Lq_irr requires Nq as input; Section 3.5 similarly requires the user to 'define a priori which prototypes are relevant for each query' for multiple queries. In the main benchmark (Section 4.2.1), N=10 is fixed, but the paper never states what Nq is used in Hq for each query, nor how Hq is constructed without ground-truth knowledge. The robustness ablation (Table 2) anchors N to NGT+Nadd, so it demonstrates insensitivity to excess prototypes only when NGT is known. The last column (N=10) is claimed to avoid GT knowledge, but without the Hq construction the claim is not verifiable. If Hq is set with GT sub-concept counts, the method is not discovering the number of sub-concepts; it is clustering with known k. If Hq is set to Nq=N-Nirr=7, the paper should say so and evaluate the effect of that choice across concepts with up to 9 GT sub-concepts. This is a correctness risk for the central contribution, not merely a usability caveat.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 3D Open-Vocabulary Sub-concepts Discovery (OV-SD), a segmentation task in which a NeRF-based 3D scene is decomposed into sub-concepts relevant to a user query without the user naming the classes. The proposed method, DiSCO-3D, combines a prototype-based unsupervised semantic segmentation module trained on DINO features extracted from a pretrained feature field (LeRF or OpenNeRF) with weak supervision from CLIP query relevance, via the losses Lproj, Lirr, and Lproto. The authors also introduce a Replica-based benchmark with 40 LLM-generated concepts and compare their method against sequential OVSeg-to-USS and USS-to-OVSeg baselines. They report that DiSCO-3D outperforms these baselines on PQ, mIoU, and mAcc for both LeRF and OpenNeRF, and that it also improves the edge cases of open-vocabulary segmentation and unsupervised semantic segmentation.","tokens_in":21475,"tokens_out":6987,"duration_ms":82807,"significance":"If the results hold, the paper makes a useful contribution: it defines a new task that sits between open-vocabulary and unsupervised segmentation, it adapts prototype-based clustering to neural fields, and it provides a benchmark and a consistent joint-versus-sequential comparison that controls for input features and supervision. The qualitative demonstrations across multiple feature fields and query types are informative. A key part of the claimed contribution, however, is the word 'discovery,' and that claim currently rests on an unstated choice of the number of relevant prototypes, Nq, which is load-bearing for the method's central promise.","major_comments":[{"comment":"The loss Lq_irr is defined through the one-hot vector Hq, which has Nq leading ones, so the training signal requires Nq as an input. The paper does not specify how Nq should be obtained from a user query alone, and Section 3.5 explicitly states that relevant prototypes must be defined a priori. In the main OV-SD benchmark (Section 4.2.1), the total number of prototypes is fixed to N=10, but the actual Nq used to construct Hq is never reported. If Nq comes from ground-truth sub-concept counts, the benchmark evaluates clustering with a known k and the discovery claim is weakened; if, as suggested by the supplementary hyperparameter section, Nirr=3 and hence Nq=7, that choice should be stated explicitly and its effect on queries with up to 9 ground-truth sub-concepts should be analyzed. This is a load-bearing point for the central contribution and should be resolved by either proposing an estimator for Nq or evaluating the method in a setting where Nq is not oracle-derived.","section":"Section 3.4, Eq. (4)"},{"comment":"The robustness claim in Table 2 is entangled with the same ambiguity. The ablation parameterizes N as NGT+Nadd, so it demonstrates insensitivity to excess prototypes only when NGT is known. The last column, N=10, is meant to avoid ground-truth knowledge, but without reporting how Hq is constructed the claim is not verifiable. Please report the exact values of Nq and Nirr, and describe how the first Nq prototype indices are selected, both in the ablations and in the main benchmark.","section":"Section 4.2.3, Table 2"},{"comment":"The quantitative comparison that supports the central 'joint better than sequential' claim is reported without error bars, variance, or number of runs. The absolute PQ values are small (e.g., 8.13 vs. 5.99 on LeRF), so seed-to-seed variability could affect the conclusions. The authors should report standard deviations across multiple runs, or at least justify why the margins are robust.","section":"Section 4.2.2, Table 1"},{"comment":"The K-Means baselines are missing the PCLIP columns in Table 1, making the comparison incomplete under the matching protocol that uses CLIP prototypes. The authors should either explain why these baselines cannot be evaluated under the same protocol, or provide the missing numbers.","section":"Section 4.2.1, Table 1"}],"minor_comments":[{"comment":"The statement 'no concept query exceeds 9 ground-truth sub-concepts' is indirect; please report the distribution of sub-concept counts in the benchmark so that the reader can judge whether N=10 is a safe margin.","section":"Section 4.2.1"},{"comment":"The normalization in Eq. (4) appears to use #Mq in both terms, although the second sum is over the complement of Mq. Please check whether the second denominator should be the cardinality of the complement, and clarify the notation for Mq.","section":"Equation (4)"},{"comment":"The role of the 'Relevant / Irrelevant Prototypes' block in Figure 2 is not explained until Section 3.4; adding a forward reference would improve readability.","section":"Section 3.3"},{"comment":"The row label 'Used Nadd' is defined only in the table caption; please define it explicitly in the main text before the table is referenced.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper introduces a new task and a new benchmark, which is appropriate for this venue. The main reservation is the unspecified construction of Hq and Nq, which affects the validity of the central discovery claim; I would like the authors to address this in revision. I have no concerns about citation norms or scope fit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper introduces a genuinely new task—3D open-vocabulary sub-concept discovery—and a sensible first method: run prototype-based unsupervised segmentation in a NeRF feature field while weakly steering the prototypes with CLIP relevancy. The joint optimization consistently beats the sequential baselines on their Replica-based benchmark, and the method is shown to work with two different feature fields (LeRF and OpenNeRF). The first USS for NeRF is a fair contribution, and the supplementary ablations on prototype counts and EMA updates are honest. I'd send this to a serious referee.\n\nThe soft spot is the one the stress-test flags. The central claim is that the method discovers query-relevant sub-concepts without the user nominating classes. But the loss in Eq. (4) requires Hq, a one-hot vector with Nq leading ones, and the paper never states how Nq is set in the main experiments. With N=10 fixed and three irrelevant prototypes, Nq is presumably 7, yet some concepts in the benchmark have up to 9 ground-truth sub-concepts. The robustness ablation is parameterized as N=NGT+Nadd, which sidesteps the issue: it shows insensitivity to extra prototypes only when NGT is known. The last column of Table 2 claims a fixed N=10 avoids GT knowledge, but without the Hq construction that claim is unverifiable. If Nq is chosen from GT counts, this is not discovery; it's clustering with known k.\n\nThis is fixable. The paper should either (a) present Nq as a user-provided budget and evaluate the sensitivity to it across queries, or (b) propose an estimator or a soft formulation that doesn't require a hard Nq. As written, it's a real correctness risk for the main contribution, not just a usability footnote.\n\nSecondary issues: the benchmark is new and self-created, so the absolute numbers are hard to interpret. No error bars or variance across runs, and the margins over the naive baselines are small in absolute PQ terms (8-10 vs 5-6). The \"state-of-the-art\" claims on the edge cases are overstated—on OV-Seg the gains are a few mIoU points over the feature-field backbones, and on USS the comparison to SmooSeg/GrowSP is useful but not decisive. The discussion of limitations in the supplementary is good and mitigates some of this.\n\nWho is this for? People working on neural field segmentation or open-vocabulary 3D understanding. It's a solid workshop-to-conference paper that with a clear Nq treatment and more careful claims could be a solid main-conference paper. I'd engage with it as a reviewer.\n\nRecommendation: send to peer review, but the authors need to address the Nq question head-on, add error bars, and temper the SOTA claims.","headline":"New task and a sensible joint segmentation method, but the 'discovery' claim hinges on an Nq that the paper never specifies.","tokens_in":22043,"tokens_out":2879,"would_cite":false,"duration_ms":31383,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that 3D open-vocabulary sub-concept discovery can be solved by running unsupervised prototype clustering and weak CLIP guidance jointly inside a NeRF feature field, producing segmentations that adapt to both the scene and…","keywords":["3D semantic segmentation","open-vocabulary segmentation","unsupervised semantic segmentation","neural radiance fields","feature fields","prototype clustering","sub-concept discovery","CLIP"],"falsifier":"Take a scene/query pair whose ground-truth sub-concept count is withheld, run DiSCO-3D with a deliberately wrong prototype budget (say three when the scene contains seven ground-truth sub-concepts), and measure PQ and mIoU against the benchmark; if performance collapses and the model cannot signal that the budget is wrong, then the method's discovery behavior depends on an oracle count rather than on the scene itself.","tokens_in":20929,"feed_emoji":"🎯","tokens_out":11205,"duration_ms":108925,"temperature":0.7,"pith_summary":"The paper introduces a new 3D segmentation task, Open-Vocabulary Sub-concepts Discovery (OV-SD), in which a user gives a query such as \"furniture\" but does not enumerate the classes to return; the model must decompose the scene into the query's sub-concepts that are actually present. DiSCO-3D is presented as the first method for this task, built on a pretrained NeRF feature field that provides both a spatially precise semantic feature (DINO) and a queryable open-vocabulary feature (CLIP). The core proposal is to run unsupervised prototype-based clustering and weak query guidance jointly rather than successively, so irrelevant regions are discarded while the remaining scene is split into discoverable sub-concepts. If correct, a single 3D representation serves both open-vocabulary segmentation and unsupervised segmentation, with per-query optimization of about two seconds.","feed_headline":"One NeRF model finds the sub-concepts hidden in a text query","feed_subtitle":"Joint unsupervised clustering and weak CLIP guidance make segmentation adapt to both scene and query.","key_machinery":"The central object is a set of learnable prototypes $\\{P_i\\}_{i=1}^N$ in the projected feature space, updated by an exponential moving average that weights samples by both NeRF density weights $w_k$ and prediction confidence $D_{k,i}$. Projected DINO features are assigned to prototypes through a softmax over cosine similarity with temperature $\\beta$, which decays linearly to sharpen clusters. Three losses carry the argument: $\\mathcal{L}_{\\mathrm{proj}}$ preserves DINO-space correspondence relations, $\\mathcal{L}_{\\mathrm{irr}}^q$ uses the CLIP field's query relevance mask to push query-irrelevant samples toward irrelevant prototypes while keeping relevant samples on relevant prototypes, and $\\mathcal{L}_{\\mathrm{proto}}$ pulls each projected DINO feature toward the prototype whose CLIP embedding is most similar to the sample's CLIP embedding. A parallel set of CLIP prototypes, updated with the same EMA, gives every discovered sub-concept a nameable embedding, which is what makes the unsupervised clustering and open-vocabulary guidance happen inside one optimization.","core_discovery":"The central claim is that 3D Open-Vocabulary Sub-concepts Discovery is a distinct, solvable problem and that DiSCO-3D solves it: given a pretrained feature field and one or more CLIP-encoded queries, the method jointly learns a projector over DINO features and a set of prototypes, supervised so that prototypes split into query-relevant and query-irrelevant groups. The relevant prototypes define the discovered sub-concepts, and each carries a CLIP prototype embedding that supports a posteriori naming and matching. The authors assert that this joint formulation beats the natural sequential baselines (open-vocabulary segmentation followed by unsupervised clustering, or the reverse) on a new Replica-based benchmark, and that in the edge cases of single-concept queries and no-query unsupervised segmentation, DiSCO-3D improves on or matches the best available results.","pith_inferences":["A natural next step the authors leave open is estimating the number of sub-concepts automatically; until such an estimator exists, the discovery claim depends on the user supplying a prototype budget, which the paper sets from ground-truth counts in its main ablations.","Because each sub-concept gets a nameable CLIP prototype, the method could serve as an interactive labeler: a user could query \"tools,\" inspect the discovered names, and re-query at finer granularity without retraining the feature field.","The joint-training design transfers in principle to other continuous 3D representations such as Gaussian Splatting or to 2D images, with NeRF density weights replaced by a comparable per-sample confidence measure; the paper notes the extension is theoretical.","The benchmark's groupings contain semantically overlapping classes such as armchair, sofa, and chair, so a well-calibrated prototype should reflect graded ambiguity rather than a single hard label; the paper's reported matching errors suggest this is where the method's performance ceiling lies."],"forward_implications":["Plugging DiSCO-3D into LeRF or OpenNeRF improves open-vocabulary segmentation on Replica, raising mIoU and mAcc over the raw feature fields by filling relevancy holes and reducing relevancy spilling.","In the no-query edge case, DiSCO-3D outperforms K-Means on the same feature field as well as the 2D and point-cloud unsupervised segmentation baselines SmooSeg and GrowSP.","The same one-hot formulation handles multiple simultaneous queries, including disjoint, overlapping, and nested queries, without extra constraints.","A new query only re-optimizes the projector and prototypes, converging in about two seconds on a single RTX 4090 after the feature field is pretrained.","The architecture extends to alternative segmentation fields such as a SAM-based feature field and to alternative query modalities such as visual embeddings or user clicks."],"supporting_citations":[{"why":"Supplies the pre-trained LeRF feature field, combining a multi-scale CLIP pyramid with a DINO feature field, on which DiSCO-3D's two-stream design is built.","marker":"[12]"},{"why":"Provides the DINO self-supervised features whose semantic correspondences the projector is trained to preserve.","marker":"[3]"},{"why":"Provides the CLIP text encoder used to embed user queries and compute the relevance mask that guides the irrelevant-prototype loss.","marker":"[30]"},{"why":"Introduces the prototype-based correspondence distillation idea that the unsupervised module adapts to NeRF samples.","marker":"[9]"},{"why":"Supplies the projector architecture used by DiSCO-3D and serves as a 2D unsupervised segmentation baseline.","marker":"[18]"},{"why":"Provides the OpenNeRF dense OpenSeg feature field used as a second input representation and as an open-vocabulary baseline.","marker":"[6]"},{"why":"Provides the Replica scenes whose semantic classes the paper extends into 40 sub-concept groupings for the new benchmark.","marker":"[33]"},{"why":"Serves as the point-cloud unsupervised semantic segmentation baseline in the USS edge-case comparison.","marker":"[37]"}],"fun_headline_variants":["DiSCO-3D: 3D segmentation that adapts to query and scene","Sub-concept segmentation from text in NeRF scenes","Joint clustering and CLIP guidance for 3D sub-concepts","One model: open-vocabulary 3D sub-concept discovery","NeRF segmentation that follows your text and the scene"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the user or benchmark says in advance how many relevant sub-concepts to look for; if that count is unknown or wrong, the discovery formulation has no built-in way to correct itself.","fun_headline_variants_meta":{"raw":{"variants":["DiSCO-3D: 3D segmentation that adapts to query and scene","Sub-concept segmentation from text in NeRF scenes","Joint clustering and CLIP guidance for 3D sub-concepts","One model: open-vocabulary 3D sub-concept discovery","NeRF segmentation that follows your text and the scene"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1205,"prompt_tokens":858,"completion_tokens":347,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":258}},"tokens_in":474,"tokens_out":347,"duration_ms":516668,"temperature":1.0,"reasoning_tokens":258,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:52:33.448489+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a scene/query pair whose ground-truth sub-concept count is withheld, run DiSCO-3D with a deliberately wrong prototype budget (say three when the scene contains seven ground-truth sub-concepts), and measure PQ and mIoU against the benchmark; if performance collapses and the model cannot signal that the budget is wrong, then the method's discovery behavior depends on an oracle count rather than on the scene itself.","supporting_citations":[{"cited_title":"Lerf: Language embedded radiance fields","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained LeRF feature field, combining a multi-scale CLIP pyramid with a DINO feature field, on which DiSCO-3D's two-stream design is built."},{"cited_title":"Emerg- ing properties in self-supervised vision transformers","cited_arxiv_id":null,"evidence_quote":"Provides the DINO self-supervised features whose semantic correspondences the projector is trained to preserve."},{"cited_title":"Smooseg: smoothness prior for unsupervised semantic segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the projector architecture used by DiSCO-3D and serves as a 2D unsupervised segmentation baseline."},{"cited_title":"Growsp: Unsupervised semantic segmentation of 3d point clouds","cited_arxiv_id":null,"evidence_quote":"Serves as the point-cloud unsupervised semantic segmentation baseline in the USS edge-case comparison."}],"review_version":1}