{"id":"690cc1ac-5a06-4a0d-82ec-4e21122e710e","arxiv_id":"2607.00978","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"UTTO uses uncertainty to guide test-time optimization with foundation model priors to enhance depth-only open-vocabulary 3D semantic segmentation without training, outperforming baselines on ScanNet datasets.","lead":"The paper proposes UTTO, an uncertainty-guided test-time optimization method for improving open-vocabulary 3D semantic segmentation using only depth data. This could allow privacy-preserving 3D scene understanding in indoor settings by avoiding the use of RGB images that may contain sensitive information.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Core assumption that uncertainty from depth-only inputs can be turned into reliable guidance and that foundation-model priors can regularize refinement without RGB cues remains unverified","rationale":"The reader's weakest_assumption directly identifies the same unverified mechanistic step. Because the full manuscript was unavailable to the reader, the correctness_risk remains unknown; the stress-test confirms that this assumption is the single point whose failure would invalidate the headline claim, with no other internal inconsistency visible from the abstract.","tokens_in":1707,"tokens_out":322,"duration_ms":13483,"concrete_test":"Extract the precise uncertainty formulation and the test-time loss from §3 (or equivalent); recompute the ScanNet200 mIoU after ablating the uncertainty-weighting term while keeping all other components fixed; if the gain over the non-optimized baseline disappears or reverses, the guidance mechanism is not load-bearing.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that (1) an uncertainty signal derived solely from depth geometry can correctly flag unreliable open-vocabulary predictions and (2) semantic priors from foundation models can then regularize those predictions during test-time optimization. The abstract provides no equations, no definition of the uncertainty measure (entropy, variance, or otherwise), and no description of how the regularization loss is constructed or balanced. If either step fails—e.g., if depth-only uncertainty is dominated by geometric ambiguity rather than semantic error, or if the priors introduce appearance-based hallucinations—the claimed consistent improvement on ScanNet20/40/200 would not hold.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper proposes UTTO, an uncertainty-guided test-time optimization framework for depth-only open-vocabulary 3D semantic segmentation. It converts uncertainty signals from depth geometry into guidance to identify unreliable predictions and uses semantic priors from foundation models to regularize refinement during test-time optimization, without any additional training. Experiments on ScanNet20, ScanNet40, and ScanNet200 are claimed to show consistent improvements over representative baselines under privacy-preserving conditions.","tokens_in":1835,"tokens_out":508,"duration_ms":23149,"significance":"If the central claims hold with verifiable quantitative support, the work would address an important gap in privacy-preserving 3D perception by enabling open-vocabulary segmentation from depth alone. This could have practical value for indoor scene understanding where RGB data raises privacy concerns. The use of test-time optimization with external priors is a plausible direction, but its effectiveness depends on the unverified assumptions about uncertainty guidance and regularization.","major_comments":[{"comment":"Abstract: The abstract states that UTTO consistently improves performance and outperforms baselines but provides no quantitative metrics, error bars, method details, or ablation studies, making it impossible to verify if the data supports the claim.","section":"Abstract"},{"comment":"Method (or §3): No definition is given for the uncertainty measure derived solely from depth geometry (e.g., whether it is entropy, variance, or another quantity), nor for how this signal is converted into an optimization guidance term or how the foundation-model regularization loss is constructed and balanced.","section":"Method"},{"comment":"Experiments: The central claim of consistent improvement on ScanNet20/40/200 requires evidence that depth-only uncertainty correctly flags semantic errors (rather than geometric ambiguity) and that the priors improve rather than hallucinate; without reported numbers, ablations, or failure cases, this load-bearing assumption remains untested.","section":"Experiments"}],"minor_comments":[{"comment":"Notation for the uncertainty-guided loss and optimization objective should be introduced with explicit equations early in the method section for clarity.","section":"Method"},{"comment":"The paper should include a clear statement of the privacy threat model and confirm that no RGB or appearance data is used at any stage, including during prior extraction.","section":"Introduction"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback. We address each major comment below and will revise the manuscript to enhance clarity, provide additional quantitative support, and strengthen the experimental evidence where possible.","responses":[{"response":"We agree that the abstract would benefit from quantitative support to substantiate the claims. In the revision, we will incorporate key performance metrics (e.g., mIoU improvements on the ScanNet benchmarks) along with standard deviations from repeated runs.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The abstract states that UTTO consistently improves performance and outperforms baselines but provides no quantitative metrics, error bars, method details, or ablation studies, making it impossible to verify if the data supports the claim."},{"response":"We will revise Section 3 to include an explicit definition of the uncertainty measure derived from depth geometry, along with the mathematical formulation of the guidance term and the construction/balancing of the foundation-model regularization loss, to improve reproducibility.","revision_made":"yes","referee_comment":"[Method] Method (or §3): No definition is given for the uncertainty measure derived solely from depth geometry (e.g., whether it is entropy, variance, or another quantity), nor for how this signal is converted into an optimization guidance term or how the foundation-model regularization loss is constructed and balanced."},{"response":"The manuscript already reports quantitative results and comparisons on ScanNet20/40/200 in the experiments section. To further validate the assumptions, we will add ablations and analysis in the revision showing the relationship between uncertainty signals and semantic errors, as well as discussion of cases involving potential hallucination by the priors.","revision_made":"partial","referee_comment":"[Experiments] Experiments: The central claim of consistent improvement on ScanNet20/40/200 requires evidence that depth-only uncertainty correctly flags semantic errors (rather than geometric ambiguity) and that the priors improve rather than hallucinate; without reported numbers, ablations, or failure cases, this load-bearing assumption remains untested."}],"tokens_in":1354,"tokens_out":454,"duration_ms":21771,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is that the paper introduces UTTO as an uncertainty-guided test-time optimization framework for open-vocabulary 3D semantic segmentation from depth alone. It targets privacy by skipping RGB and tries to handle the resulting high uncertainty by turning an uncertainty signal into guidance while pulling in semantic priors from foundation models for refinement during test time.\n\nThe approach is new in the sense that it applies test-time optimization specifically to the depth-only privacy-preserving case rather than the usual RGB-heavy setting. The abstract positions this as a no-training method that works on ScanNet20, ScanNet40, and ScanNet200.\n\nThe soft spot is large and central. The abstract asserts that UTTO consistently improves performance and beats baselines, yet it contains no numbers, no error bars, no ablation results, and no definition of the uncertainty measure or the regularization loss. Without those, there is no way to check whether depth-derived uncertainty actually flags semantic errors reliably or whether the priors can regularize predictions in the absence of appearance cues. The stress-test concern about those two steps is accurate on the evidence given.\n\nThis is for people working on privacy-aware 3D perception. A reader might pick up the high-level idea, but the lack of concrete evidence means it offers little usable content right now.\n\nI would not bring this to reading group. I would not cite it. It does not deserve peer review until the method and results are shown in enough detail to evaluate the claims.","headline":"The abstract presents UTTO for depth-only open-vocab 3D segmentation but supplies no metrics, equations, or method details, leaving the claimed improvements unverified.","tokens_in":2309,"tokens_out":378,"would_cite":false,"duration_ms":21179,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"An uncertainty-guided test-time optimization method improves depth-only open-vocabulary 3D semantic segmentation without additional training.","keywords":["privacy-preserving","depth-only","open-vocabulary","3D semantic segmentation","test-time optimization","uncertainty guidance","ScanNet"],"falsifier":"Applying UTTO to the ScanNet datasets and measuring no gain or a drop in segmentation accuracy relative to the unrefined depth-only baseline would show the claimed improvement does not hold.","tokens_in":2603,"feed_emoji":"🔐","tokens_out":644,"duration_ms":28755,"temperature":0.7,"pith_summary":"The paper introduces UTTO, a framework that turns uncertainty estimates into a guidance signal to spot unreliable semantic predictions from depth data alone and then refines them with semantic priors drawn from foundation models. This setup matters because it enables open-vocabulary 3D segmentation in privacy-sensitive indoor environments where RGB images cannot be captured or used. The approach requires no extra training or labeled target data and operates only at test time. Experiments across ScanNet20, ScanNet40, and ScanNet200 show consistent gains over representative baselines when only depth input is available.","feed_headline":"Test-time optimization lifts depth-only 3D segmentation","feed_subtitle":"UTTO turns uncertainty into a refinement signal and applies foundation model priors for open-vocabulary gains without training or RGB input.","key_machinery":"The UTTO framework, an uncertainty-guided test-time optimization process that flags unreliable responses via uncertainty and regularizes refinement using foundation model semantic priors.","core_discovery":"UTTO converts uncertainty into a guidance signal to identify unreliable semantic responses and uses semantic priors from foundation models to regularize their refinement, thereby improving depth-only open-vocabulary 3D semantic segmentation without additional training.","pith_inferences":["The same uncertainty-to-guidance conversion could be tested on other depth-based tasks such as 3D instance segmentation under privacy constraints.","The method hints that test-time regularization may lessen dependence on large labeled 3D datasets for new scenes.","Similar guidance signals might help in related settings where one modality is missing, such as LiDAR-only perception.","Evaluating the approach on non-indoor or outdoor depth data would check whether the foundation-model regularization generalizes beyond the reported benchmarks."],"forward_implications":["Depth-only inputs can support open-vocabulary 3D segmentation when uncertainty directs the refinement steps.","Privacy is maintained by excluding RGB data while segmentation quality still rises.","Foundation model priors can stand in for missing visual cues during test-time adaptation.","The gains appear across different label sets including 20, 40, and 200 classes on ScanNet.","No retraining or new annotations are needed to obtain the reported improvements."],"fun_headline_variants":["Uncertainty-guided test-time optimization for depth-only 3D segmentation","UTTO refines depth-only open-vocabulary segmentation using uncertainty","Foundation priors regularize uncertain depth-only 3D segmentation","Test-time optimization refines privacy-preserving depth-only segmentation"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Uncertainty estimates can be turned into a reliable signal for identifying which semantic responses need fixing and foundation model priors can correct them effectively when depth data supplies no appearance cues.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty-guided test-time optimization for depth-only 3D segmentation","UTTO refines depth-only open-vocabulary segmentation using uncertainty","Foundation priors regularize uncertain depth-only 3D segmentation","Test-time optimization refines privacy-preserving depth-only segmentation"]},"model":"grok-4.3","cost_usd":0.007491,"raw_usage":{"total_tokens":3398,"prompt_tokens":588,"num_sources_used":0,"completion_tokens":67,"cost_in_usd_ticks":74912000,"prompt_tokens_details":{"text_tokens":588,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2743,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":588,"tokens_out":67,"duration_ms":24866,"temperature":1.0,"reasoning_tokens":2743,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-02T13:36:19.365673+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Applying UTTO to the ScanNet datasets and measuring no gain or a drop in segmentation accuracy relative to the unrefined depth-only baseline would show the claimed improvement does not hold.","supporting_citations":[],"review_version":1}