{"id":"a9a62403-c7b9-4caa-9d3d-405b2a70f182","arxiv_id":"2601.13132","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"GaussExplorer (also submitted as SplatReasoner) improves embodied reasoning by using 3D Gaussian Splatting to synthesize novel, query-conditioned viewpoints that are selected by a VLM judge.","lead":"This paper integrates 3D Gaussian Splatting with vision-language models to let an agent synthesize new camera viewpoints that reveal evidence needed to answer questions about a 3D scene. The method improves embodied question answering and 3D object grounding over fixed-view memory baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 6's 'Final view' row (54.5) is indistinguishable from 3D-Mem (54.6); the reported EM-EQA gain appears only after adding a verification stage that is orthogonal to novel-view synthesis, so the central claim that novel views drive the improvement is not isolated.","rationale":"The reader's weakest_assumption was the closed-set category bottleneck, which is a real limitation but does not directly undermine the relative comparison with 3D-Mem on EM-EQA, since both methods share the same closed-set semantics. The more load-bearing concern is that the main empirical evidence for 'novel-view synthesis improves embodied reasoning' is not isolated. Table 6 shows that the full pipeline without the verification step is statistically indistinguishable from the 3D-Mem baseline, and the verification step itself is an answer-selection mechanism independent of generating novel views. The paper's own ablation tables never compare initial-view-only vs. novel-view-enabled conditions on the full benchmark while keeping all other components (prompting, verification) fixed. Therefore the central causal claim is currently supported only by a small, underpowered subset ablation and by a tiny novel-view increment on the 3D referring benchmark. This does not warrant rejection, because the framework as a whole may still be effective, but it does mean the stated contribution is not yet convincingly demonstrated. The correct verdict remains CONDITIONAL: the authors should add the missing ablation and error bars. I disagree with the reader's choice of weakest assumption because the closed-set issue is a scope limitation, not the primary threat to the central mechanism.","tokens_in":16829,"tokens_out":8462,"duration_ms":91467,"concrete_test":"On the full 1,623-question EM-EQA set, run four conditions with identical final reasoning and verification prompts: (A) initial views only, no novel views; (B) novel views only, no verification; (C) initial views + verification; (D) full pipeline (novel views + verification). If C matches D (or A+C matches the 57.8 result), then the improvement is verification-driven, not due to novel-view synthesis. Also compute variance over multiple LLM-judge runs for the 184-question subtable to assess whether the 2.3-point initial-to-final gain is significant.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the observed gains are caused by query-conditioned novel-view synthesis. The full EM-EQA evidence does not establish this. In Table 6, the full pipeline with novel views and final-view reasoning but without the verification step ('Final view') scores 54.5 LLM-Match, essentially identical to 3D-Mem's 54.6. The headline 57.8 appears only after the verification stage is added. Verification is an answer-selection/self-check step that compares initial and novel-view candidates and prefers informative answers over 'can't tell' responses; it is not specific to synthesizing new viewpoints. The paper never reports an initial-view-only condition on the full 1,623 questions with equivalent prompts and verification, so the novel-view contribution cannot be separated from the verification or prompt-design contribution. The subset ablation (Table 4, n=184) shows a 2.3-point initial-to-final gain, but with 184 questions and a stochastic LLM-Match judge this is within plausible noise (nominal SE roughly 3.7 points). For 3D referring segmentation, the novel-view increment is 0.41 mIoU on 49 curated questions. Thus, the causal role of novel-view synthesis in the headline result is not demonstrated.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper (listed as arXiv:2601.13132, though the full text is titled 'GaussExplorer') proposes a framework that uses 3D Gaussian Splatting as a semantic episodic memory for embodied question answering and 3D referring segmentation. Given a user query, the method (1) extracts evidence categories via an LLM, (2) activates and clusters semantically relevant Gaussians, (3) selects initial camera views using a visibility score, (4) generates novel candidate views through 3DGS and uses a VLM as a judge to select the most informative view, and (5) optionally verifies the final answer against the initial-view answer. The authors report 57.8 LLM-Match on EM-EQA versus 54.6 for 3D-Mem with comparable average frame count, and 12.87 3D mIoU on a newly curated 49-question 3D referring segmentation benchmark versus 10.56 for Dr. Splat.","tokens_in":17113,"tokens_out":3113,"duration_ms":38022,"significance":"If the central claim were established, the paper would make a useful contribution: it combines 3DGS with VLM-based reasoning in a way that can synthesize query-conditioned novel views and perform fine-grained 3D localization. The visibility-based initial view selection is a reasonable and partially novel idea, and the paper usefully shows that a semantic 3DGS representation can be integrated with a VLM pipeline. The new 3D referring segmentation benchmark, though small, addresses a real gap. However, the central causal claim — that novel-view synthesis is what drives the reported gains — is not currently isolated by the experiments. The largest EM-EQA improvement comes from a verification stage that is orthogonal to novel-view synthesis, and the 3D segmentation gain is 0.41 mIoU on 49 questions. The evidence is suggestive but not yet convincing. A revised version with proper ablations and statistical care could change this assessment.","major_comments":[{"comment":"The central claim that query-conditioned novel view synthesis improves embodied reasoning is not isolated. In Table 6, the full pipeline with novel views but without verification ('Final view') scores 54.5 LLM-Match, effectively identical to 3D-Mem's 54.6. The reported 57.8 gain appears only after adding a verification stage that compares initial and final-view answers and prefers informative answers over 'can't tell' responses. This verification step is not specific to novel-view synthesis. To support the abstract's claim, the paper must report an initial-view-only condition on the full 1,623 questions using the same prompts and verification, and a final-view condition without novel views, so that the contribution of synthesis is separated from the contribution of verification/prompt design.","section":"Table 6, Sec. 4.3"},{"comment":"The VLM-as-Judge/verification loop is circular in a way that biases the result. The view-selection LLM chooses the best answer from candidate views, and the verification prompt explicitly instructs the model to 'prefer the one that gives the information' and to reject answers that say 'can't tell'. This conflates 'answers with more confident/verbose content' with 'answers that are grounded in the visual evidence.' Because the same kind of model is used to select, verify, and evaluate answers, the reported gain may reflect prompt-induced answer-style preferences rather than evidence actually revealed by novel synthesis. A control with blinded view order, or with verification applied to initial views only, is needed to rule out this confound.","section":"Fig. S6, Sec. 3.3"},{"comment":"The method's query grounding is closed-set for the EM-EQA evaluation. Gaussians are activated only when their category exactly matches one of the LLM-extracted evidence categories from ScanNet200/YOLOv8. If the user query refers to an attribute, state, or object category not in the closed inventory — e.g., 'blue mug', 'dusty shelf', 'the chair with a broken leg' — no Gaussians are activated and the pipeline has no evidence to reason over. The paper does not report a sensitivity analysis or an open-vocabulary variant for EM-EQA. This is a modeling assumption that limits the generality of the claimed improvement and should be explicitly tested or acknowledged as a boundary condition.","section":"Eq. (1), Sec. 3.2"},{"comment":"The 3D referring segmentation evidence is too weak to support the grounding claim. The benchmark contains only 49 human-written questions over 24 instances in 5 scenes. The gain of the full method over Dr. Splat (question) is 12.87 vs 10.56, and the novel-view increment is only 0.41 mIoU (12.46 to 12.87). With this sample size and stochastic LLM-based components, these differences are within plausible noise; no error bars, significance tests, or per-scene breakdowns are provided. Additionally, the ReferSplat comparison uses manually generated training data (described in Sec. 4.2) because no official pipeline exists, which may disadvantage that baseline. The authors should provide variance estimates, a per-question analysis, and a fairer ReferSplat protocol before claiming superiority in 3D grounding.","section":"Sec. 4.2, Table 2, Sec. C"}],"minor_comments":[{"comment":"The arXiv metadata and abstract refer to 'SplatReasoner', while the full paper is titled 'GaussExplorer'. This identity mismatch must be fixed before publication.","section":"Title/Abstract"},{"comment":"Typos and formatting errors: 'shwon' (Sec. 4.3), 'baesd' (Fig. 6), 'Gaussain' (Suppl. A.1), 'downstars' (Fig. 8), 'dateset' (Sec. 4.2), 'LLaV A' (Sec. 4.1), and 'ReferSplat [12]' in Supp. B.2 (should be [9]).","section":"Throughout"},{"comment":"All ablations on view selection and score functions are done on only 184 questions with no error bars or significance testing. For instance, the 48.2 vs 45.4 gain in Table 3 and the 50.5 vs 48.2 gain in Table 4 are reported as deterministic numbers. The small subset and stochastic LLM-Match judge make these differences unreliable; at minimum, report standard errors or repeated evaluations.","section":"Sec. 4.3"},{"comment":"The method has several free parameters whose sensitivity is not reported: the number of evidence categories k, the similarity threshold τ in Eq. (2), the HDBSCAN parameters (mc, ms), the merge distance ε, and the number of novel-view perturbations per initial view (K = L×4). Since ε is stated to control the average number of frames, a sensitivity table is needed to show that results are not tuned to a specific operating point.","section":"Sec. 3.3, Supp. A.2"},{"comment":"The 'average frames' comparison is informative, but the footnote defines it as 'the number of initial or final views fed to VLMs.' This excludes frames used in the novel-view adjustment stage and the verification stage, so it does not reflect total VLM inference cost. Please report the actual number of VLM calls or rendering cost for a fair efficiency comparison.","section":"Table 1, Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is internally consistent and the proposed system is plausible, but the headline claim is not yet supported by the ablation design: Table 6 shows that the full gain comes from verification, not from novel-view synthesis. The 3D referring segmentation results are also fragile due to the small benchmark. I am not recommending rejection because the issues are addressable with additional experiments, but the authors should not be allowed to claim that novel-view synthesis drives the improvement until the appropriate isolated conditions are reported. Also, the title mismatch (SplatReasoner vs. GaussExplorer) suggests a lack of coordination that should be resolved editorially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper introduces a sensible way to let a VLM pick better viewpoints in a 3DGS scene, but the headline performance gain on EM-EQA is not actually attributable to novel-view synthesis. The stress-test note is right. In Table 6, the full pipeline minus the verification step ('Final view') scores 54.5 LLM-Match, statistically indistinguishable from 3D-Mem's 54.6. The 57.8 only appears after adding a verification stage that prefers 'gives information about the object' over 'cannot tell.' That is an answer-selection bias, not evidence that synthesizing new views helps.\n\nWhat is new: the combination of semantic 3DGS with a VLM-as-Judge loop for view selection. 3D-Mem uses fixed snapshots, Dr. Splat relies on static CLIP similarity, and ReferSplat does rendering-based optimization. The paper's initial-view visibility score is a reasonable engineering contribution, and the qualitative examples do show cases where a novel view reveals the object. That is worth something.\n\nWhere the soft spots are: the missing ablation is the big one. The paper never reports an initial-view-only run on the full 1,623 questions with matching prompts and verification. The 184-question subset shows a 2.3-point gain from novel views, but with no error bars and a stochastic LLM-Match judge, that is within plausible noise. The 3D referring segmentation benchmark is 49 hand-curated questions, and the novel-view increment there is 0.41 mIoU. The closed-set category activation (Eq. 1) means that if the query mentions something not in YOLOv8/ScanNet200, no Gaussians activate; that is a real generality limit and it is not discussed in the limitations section, which only mentions active navigation. Also, the paper's title and text disagree (SplatReasoner vs GaussExplorer); that should be fixed before anyone reviews it seriously.\n\nCredit: the experiments are mostly reproducible in spirit, the authors report their settings, and they do not bury Table 6. The problem is the narrative: the abstract and conclusion credit novel-view synthesis for gains that actually come from verification.\n\nBottom line: this is a good idea worth exploring, but the load-bearing claim is not demonstrated. If you want to cite it, cite it as related work on integrating VLMs with 3DGS, not as evidence that novel view synthesis improves embodied reasoning. I would send it to peer review because the idea deserves a careful look and the flaws are fixable with a proper ablation and error bars, but I'd expect heavy revision.","headline":"The paper's core selling point, that novel-view synthesis drives the EM-EQA gains, doesn't survive its own ablation: the headline number comes from a verification step that is orthogonal to view synthesis.","tokens_in":17692,"tokens_out":3789,"would_cite":false,"duration_ms":39032,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that query-conditioned novel view synthesis from a 3D Gaussian Splatting scene improves embodied reasoning and 3D grounding over fixed-view memory and language-embedded 3DGS baselines.","keywords":["embodied question answering","3D Gaussian Splatting","novel view synthesis","vision-language models","3D referring segmentation","view selection","episodic memory","spatial reasoning"],"falsifier":"Run the pipeline on a set of questions whose key objects are deliberately absent from the closed-set category list (for example 'rug' when the detector has no such class); if activated Gaussians drop to zero and the model cannot answer while a fixed-view VLM baseline can, the claimed query-conditioning advantage fails for out-of-vocabulary queries.","tokens_in":16682,"feed_emoji":"🧠","tokens_out":12147,"duration_ms":107516,"temperature":0.7,"pith_summary":"The paper sets out to show that a Vision-Language Model (VLM) answers spatial and compositional questions about an indoor scene better when it is shown novel camera views synthesized from a 3D Gaussian Splatting reconstruction, rather than only the originally recorded frames. The proposed pipeline finds which 3D Gaussians match the question, groups them into object instances, selects the recorded view where each instance is most visible, and then renders perturbed versions of that view, letting a VLM judge which one best supports an answer. On an episodic-memory question answering benchmark this raises the semantic match score from 54.6 for the best fixed-view memory baseline to 57.8 at the same average number of input frames, and on a new 3D referring segmentation benchmark it raises 3D mIoU from 10.56 to 12.87. This matters because it offers a concrete way to give VLMs spatial memory without requiring the agent to physically move.","feed_headline":"Rendering new 3D views lifts embodied question answering","feed_subtitle":"Synthesizing views a VLM deems informative beats fixed-view memory, lifting the benchmark score from 54.6 to 57.8.","key_machinery":"Semantic 3D Gaussian Splatting with a visibility-based view scoring function and a VLM-as-Judge novel-view adjustment loop. The visibility score renders an activation map from the target Gaussian cluster and checks, at each pixel, whether the highest-weight Gaussian belongs to the cluster, giving an occlusion-robust view selection criterion. The VLM-as-Judge stage renders four perturbed views per initial view, obtains a per-view VLM answer, uses an LLM to pick the best answer/view, and then verifies against the initial view to decide whether the novel view is truly needed.","core_discovery":"The central claim is that query-conditioned novel view synthesis is an effective way to extract visual evidence for VLM reasoning in 3D scenes. Starting from a semantic 3D Gaussian scene, the method activates Gaussians whose categories match evidence categories extracted from the question, clusters them into instances, and selects the best initial view per instance via a visibility score based on rendering weights. It then synthesizes candidate novel views by shifting or zooming the camera, and a VLM-as-Judge selects the viewpoint whose rendered answer is most informative; a final verification step compares the refined answer against the initial one and chooses the better. The selected viewp","pith_inferences":["The reported gain may partly reflect VLMs' preference for centered or zoomed-in content rather than true 3D understanding; a control that decouples viewpoint informativeness from VLM composition bias would clarify the mechanism.","Because evidence extraction is category-based, the method likely handles compositional queries only within a closed vocabulary; extending it to open-vocabulary evidence retrieval (e.g., via the open-set embeddings the pipeline already uses for grounding) is the natural next step.","The VLM-as-Judge score could be reused as a reward for active next-best-view planning, turning this static-memory system into a stepping stone for embodied navigation."],"forward_implications":["Embodied agents can answer 'where is X' or 'what is the color of the X in the kitchen' by actively rendering better views instead of being locked to whatever the camera happened to record.","The VLM-as-Judge selection provides a general mechanism for picking the most informative viewpoint for any view-dependent question, not just the specific benchmarks tested here.","The same pipeline yields 3D referring segmentation: once the final answer view is chosen, the activated Gaussians inside its frustum localize the referred instance, so question answering and grounding share one stage.","The gain is not bought with more frames: the approach uses about the same average number of input images as the fixed-view memory baseline (2.6 vs 2.7)."],"fun_headline_variants":["Novel 3D view synthesis boosts robot question answering","SplatReasoner: Add views to VLMs, gain 3D grounding","Query-driven new views beat fixed memory in embodied AI","Score jumps to 57.8 with synthesized viewpoints","3D Gaussian views sharpen VLM reasoning and grounding"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The pipeline only activates Gaussians whose semantic category appears in the pre-defined object category list, so any query whose key object or attribute is absent from that list activates no evidence and the reasoning chain has nothing to look at.","fun_headline_variants_meta":{"raw":{"variants":["Novel 3D view synthesis boosts robot question answering","SplatReasoner: Add views to VLMs, gain 3D grounding","Query-driven new views beat fixed memory in embodied AI","Score jumps to 57.8 with synthesized viewpoints","3D Gaussian views sharpen VLM reasoning and grounding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000224,"raw_usage":{"total_tokens":1265,"prompt_tokens":678,"completion_tokens":587,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":422,"completion_tokens_details":{"reasoning_tokens":504}},"tokens_in":422,"tokens_out":587,"duration_ms":6141,"temperature":1.0,"reasoning_tokens":504,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T09:36:23.450896+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the pipeline on a set of questions whose key objects are deliberately absent from the closed-set category list (for example 'rug' when the detector has no such class); if activated Gaussians drop to zero and the model cannot answer while a fixed-view VLM baseline can, the claimed query-conditioning advantage fails for out-of-vocabulary queries.","supporting_citations":[],"review_version":1}