{"id":"8fcbae8d-26bb-42fd-9322-da4678191afa","arxiv_id":"2607.06620","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Adaptive manifold keyframe sampling plus an instruction-pose-aware geometry MoE raises sparse-RGB 3D spatial reasoning to 63.5 average on VSI-Bench, beating strong baselines by 7.8 points.","lead":"SpaR3D-MoE lets multimodal language models do 3D spatial reasoning from sparse RGB video by picking topology-preserving keyframes and routing features through specialized geometry experts. It posts large gains on navigation-style and relative-direction tasks without needing depth sensors or point clouds.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"SOTA gains rest on VGGT geometry/pose quality that the paper's own noise ablation already shows is fragile for the claimed tasks.","rationale":"The reader's weakest_assumption correctly isolates the load-bearing dependency: ASMS and the pose-aware experts are only as good as the VGGT signals they consume. The paper's own Tab. 7 already quantifies sensitivity, and the largest claimed gains sit exactly on the tasks most exposed to that sensitivity. No internal contradiction is present—the engineering is coherent and the ablations are informative—but the numerical SOTA claim cannot be cleanly attributed to the proposed modules until the geometry/pose quality is controlled. That keeps the verdict at CONDITIONAL rather than ACCEPT, matching the reader's assessment. No stronger objection (e.g., data leakage or broken math) is visible in the manuscript.","tokens_in":22453,"tokens_out":591,"duration_ms":6448,"concrete_test":"Re-evaluate the full VSI-Bench suite (especially Route Plan and Relative Direction) under three controlled geometry conditions with the same 32-frame budget: (1) frozen VGGT as published, (2) VGGT features/poses replaced by ScanNet/ARKitScenes ground-truth poses + depth-derived geometry, and (3) the published σ²=0.1 noise. If the relative gains over Qwen3VL-8B fall below ~15% under (2) or collapse under (3), the SOTA claim is not robust to the upstream estimator the method depends on.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (63.5 avg on VSI-Bench with 32 sparse frames, +7.8 over Qwen3VL-8B, +35.4% Route Plan / +51.4% Relative Direction) requires that VGGT-predicted geometry features and 6-DoF poses are accurate enough to (i) define the composite distance D(i,j) and quality-gated FPS of ASMS (Eqs. 2–4) and (ii) condition the IPAR and experts E2/E3. The supplementary noise-injection ablation (σ²=0.1) already drops the average by 0.9–1.7 points, with pose noise alone costing 1.7. Because Route Plan and Relative Direction are precisely the tasks that rely most on pose-conditioned alignment and topological connectivity, any realistic degradation of VGGT (domain shift, motion blur, textureless walls) would shrink or erase the headline relative gains. The paper freezes VGGT and never reports oracle-pose or GT-pose upper bounds, so the contribution of ASMS+HGI-MoE versus the quality of the frozen geometry estimator remains entangled.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"SpaR3D-MoE equips an MLLM (Qwen3VL-8B) with geometry-aware spatial reasoning from sparse RGB video by combining two modules: Adaptive Spatiotemporal Manifold Sampling (ASMS), which builds a composite distance graph from VGGT poses/geometry and quality-gated FPS to select informative keyframes, and a Heterogeneous Geometry-Inductive Mixture-of-Experts (HGI-MoE) with an instruction-pose-aware router that dispatches tokens to four architecturally distinct experts (additive fusion, cross-attention, pose-conditioned HyperNet adapter, gravity-aligned structural probes). The paper reports SOTA on VSI-Bench (63.5 avg with 32 frames, +7.8 over Qwen3VL-8B; large relative gains on Route Plan and Relative Direction), competitive or SOTA results on ScanQA and SQA3D among video-input models, and ablations on expert masking, routing inputs, sampling strategy, and (in the supplement) component cross-ablation and geometry/pose noise.","tokens_in":22836,"tokens_out":1469,"duration_ms":26867,"significance":"If the reported gains hold under fair controls, the work is a solid systems contribution to RGB-only 3D spatial MLLMs: it replaces topology-agnostic sampling and monolithic fusion with adaptive keyframe selection and task/pose-conditioned sparse expert fusion, and it is among the first to bring heterogeneous MoE fusion into this setting. Strengths include standardized evaluation (lmms-eval, greedy decoding), multi-benchmark coverage, expert-masking results that align with the intended roles of E0–E3 (Table 4), consistent ASMS gains over uniform sampling at 8/16/32 frames (Table 5), and a supplementary 2×2 cross-ablation showing complementary benefits of ASMS and HGI-MoE. The large margin over other geometry-aware RGB baselines (e.g., Spatial-MLLM, VG LLM) suggests the architectural choices matter beyond merely adding a frozen geometry encoder.","major_comments":[{"comment":"§3.2 Eqs. (2)–(4) and §3.3 (E2/E3, IPAR): ASMS distance/quality scoring and the pose-conditioned experts/router all depend on frozen VGGT geometry and 6-DoF poses. Route Plan and Relative Direction—the headline relative gains—are exactly the tasks most sensitive to pose alignment (Table 4: masking E2 drops Rel. Dir. by 9.7 and Route Plan by 5.2). The supplementary noise injection (σ²=0.1) already costs 0.9–1.7 avg points, with pose noise alone −1.7. The manuscript never reports an oracle/GT-pose or alternative-geometry upper/lower bound, so the contribution of ASMS+HGI-MoE remains partially entangled with VGGT quality. Please add (i) a main-text discussion of this dependence with the noise results, and (ii) at least one control that better isolates architecture from the geometry estimator (e.g., shared VGGT features with a monolithic baseline already partially does this via Spatial-MLLM/","section":null},{"comment":"§1.1 Implementation / training data: training uses a 288K subset of VSI-590K plus VICA, ScanQA, and SQA3D train sets, while primary SOTA claims are on VSI-Bench. The paper should explicitly state that VSI-Bench evaluation videos/QA pairs are held out from this subset (no scene or question leakage) and, if possible, report a brief ablation trained without VSI-family data or with a fixed public split, so the +7.8 over Qwen3VL-8B cannot be attributed to in-family supervised exposure rather than ASMS/HGI-MoE.","section":null},{"comment":"§3.3 and abstract: the experts are described as exhibiting “emergent specialization,” but E0–E3 are architecturally heterogeneous by design (additive residual, cross-attention, HyperNet pose adapter, gravity probes). What is learned is the router’s assignment, not the expert operators themselves. Please rephrase to “designed specialization with learned routing” (or provide routing-statistics / expert-usage analyses that demonstrate specialization beyond the hard-wired operators), so the MoE claim is not overstated.","section":null}],"minor_comments":[{"comment":"Tables 1–5 report point estimates only; a short note on run-to-run variance (or at least seed sensitivity for the MoE router) would strengthen confidence in the 7.8-point margin.","section":null},{"comment":"§3.2: free parameters γ, ω, λ, τ, Kv are set without full sensitivity analysis in the main text (only ω is ablated in the supplement). A compact sensitivity table or justification for defaults would help reproducibility.","section":null},{"comment":"Fig. 1 and Fig. 3: expert numbering in the overview (Expert1–4 vs E0–E3) is slightly inconsistent with the text; align labels.","section":null},{"comment":"Abstract/intro claim “first work that introduces MoE into 3D spatial reasoning” should be softened to “to our knowledge” with a brief check against concurrent multimodal MoE scene-understanding work, or left as is with that qualifier already present—ensure consistency across abstract and §1.","section":null},{"comment":"Supplementary efficiency table: peak memory 27.7 GB vs 17.9 GB for Qwen3VL-8B is a real deployment cost; a one-sentence main-text mention of memory/latency trade-offs would set expectations.","section":null},{"comment":"Minor prose issues: occasional missing spaces after commas in compound adjectives (e.g., “diversespatial,” “spatiotemporalconnectivity” in the abstract/intro) and inconsistent hyphenation of “state-of-the-art.”","section":null}],"recommendation":"minor_revision","confidential_remarks":"The skeptic’s VGGT-fragility concern is real but overstated relative to the manuscript: Spatial-MLLM and VG LLM already use similar frozen geometry, and SpaR3D-MoE still leads them by a large margin; noise drops leave the method well above prior SOTA. I would not reject on that basis. The VSI-590K training overlap is the more important disclosure item for the editor—if leakage exists, the VSI-Bench headline would need re-evaluation. Overall fit for a solid CV systems venue is good after the requested clarifications."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The headline numbers are real and useful: 63.5 average on VSI-Bench with only 32 non-uniform RGB frames, +7.8 over Qwen3VL-8B, with the biggest lifts on Route Plan and Relative Direction. That is the result that matters for embodied work that wants to avoid depth sensors and full reconstruction.\n\nWhat is actually new is the pairing of two clean engineering pieces. ASMS builds a spatiotemporal graph from VGGT translations, geometric cosine, and time, then does quality-gated FPS with a motion-aware gate; it consistently beats uniform sampling at 8/16/32 frames and keeps topology that uniform sampling breaks. HGI-MoE is the first geometry-inductive MoE in this setting: an instruction-pose router feeding four heterogeneous experts (simple residual, cross-attention, pose HyperNet adapter, gravity-aligned probes). Expert-masking ablations line up with the claimed specializations, especially E2 for pose-sensitive tasks. Cross-ablation of sampling vs fusion shows they are complementary. Training protocol, lmms-eval, and greedy decoding are transparent; ScanQA and SQA3D also look competitive among video-input models.\n\nThe soft spot the stress-test flags is real but proportionate. Everything rides on frozen VGGT geometry and poses for both sampling distances and routing/experts E2/E3. Their own σ²=0.1 noise ablation costs 0.9–1.7 points, pose noise the larger hit. They never give an oracle/GT-pose upper bound, so the contribution of ASMS+HGI-MoE versus the quality of the geometry estimator stays partly entangled. That is a standard systems limitation, not a circularity or internal contradiction; the method still works under the noise they inject, and the gains remain large. Other ordinary gaps: free parameters (γ, ω, λ, τ, Top-K, λ_moe), no released code, no error bars, and the usual mixed training data that includes VSI-family material. None of these erase the experimental pattern.\n\nThis is for people building RGB-only 3D-aware MLLMs or sparse-view embodied agents. The math is straightforward, the ablations are honest, and the citation pattern is normal. I would send it to peer review; a referee can push on the VGGT dependence and ask for code/oracle bounds, but the core result is already strong enough to deserve that time. Worth reading and, for anyone in the subfield, worth citing.","headline":"Solid systems paper: topology-aware sparse sampling + geometry-specialized MoE delivers real SOTA gains on VSI-Bench/ScanQA/SQA3D under a strict sparse-RGB protocol; the VGGT dependence is a real but ordinary soft spot, not a collapse of the claim.","tokens_in":23453,"tokens_out":632,"would_cite":true,"duration_ms":7412,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"SpaR3D-MoE gives multimodal language models 3D spatial intelligence from only sparse RGB frames, reaching 63.5 average on VSI-Bench by adaptive keyframe sampling and specialized expert fusion.","keywords":["3D spatial reasoning","mixture-of-experts","multimodal large language models","sparse RGB views","spatiotemporal sampling","geometry-aware fusion","embodied AI"],"falsifier":"On the same VSI-Bench videos, replace the adaptive sampler with uniform sampling of the identical frame count and replace the MoE with a single monolithic fusion layer; if the 7.8-point average gain and the large Route-Plan and Relative-Direction gains disappear, the claim that ASMS plus HGI-MoE are responsible is falsified.","tokens_in":23335,"feed_emoji":"🧭","tokens_out":1034,"duration_ms":21861,"temperature":0.7,"pith_summary":"Multimodal language models handle 2D images and video well but still fail at real 3D tasks such as estimating distances, directions, and routes. Prior fixes either demand expensive depth or point-cloud data, or they sample RGB frames by crude heuristics and fuse visual and geometric features in one rigid layer, breaking scene connectivity and mixing signals that different tasks need. SpaR3D-MoE claims that two adaptive pieces close the gap from ordinary RGB alone: a manifold sampler that picks sparse keyframes while keeping the scene’s topological links, and a mixture-of-experts router guided by the question and camera pose that sends tokens to specialized fusion experts instead of a single shallow mixer. On the main spatial benchmark the method scores 63.5—7.8 points above the strongest open baseline—with especially large gains on route planning and relative direction, and it remains competitive on ScanQA and SQA3D. A reader cares because the approach removes the need for 3D sensors while still delivering measurable physical grounding, a practical step toward agents that understand space from everyday video.","feed_headline":"Sparse RGB alone lifts 3D spatial reasoning to 63.5 on VSI-Bench","feed_subtitle":"Adaptive keyframe graphs plus specialized experts close the 2D-to-3D gap without depth sensors","key_machinery":"Adaptive Spatiotemporal Manifold Sampling (ASMS)—quality-gated farthest-point sampling on a pose-and-geometry distance graph—together with the Heterogeneous Geometry-Inductive Mixture-of-Experts (HGI-MoE) driven by an Instruction-Pose Aware Router that dispatches tokens to four specialized experts (simple additive fusion, geometric-semantic cross-attention, pose-conditioned dynamic adapter, and gravity-aligned structural probes).","core_discovery":"An end-to-end multimodal language model can acquire physically grounded spatial intelligence from sparse RGB views alone by first constructing a geometry-aware spatiotemporal graph that adaptively extracts informative keyframes while preserving manifold connectivity, then routing multimodal tokens through an instruction-pose-aware heterogeneous Mixture-of-Experts whose specialized fusion experts resolve the cross-modal contention of monolithic fusion, producing state-of-the-art averages of 63.5 on VSI-Bench (with large relative gains on Route Plan and Relative Direction) and strong results on ScanQA and SQA3D.","pith_inferences":["The same sampling-plus-specialized-experts pattern could be extended to online video streams for continuous embodied agents rather than offline clips.","Because scores drop under injected pose and geometry noise, any stronger foundation geometry model would lift performance without retraining the MoE.","Failure cases on repetitive instances imply that geometric self-verification or reinforcement feedback may be needed before reliable instance-level metric reasoning.","Emergent expert specialization offers a reusable template for other multimodal contention problems where one fusion layer is insufficient."],"forward_implications":["As few as 16–32 non-uniform RGB frames can outperform dense-video baselines and some explicit 3D-input models on navigation and metric spatial tasks.","Topology-preserving keyframe selection alone improves Route Plan by roughly 10 percent over uniform sampling.","Task- and pose-conditioned expert routing removes modality contention that a single fusion layer cannot resolve.","MLLMs can be given physically grounded spatial intelligence without depth sensors or reconstructed meshes.","The same sparse-view pipeline generalizes across general spatial reasoning, scene QA, and situated reasoning benchmarks."],"fun_headline_variants":["Sparse RGB plus adaptive MoE hits 63.5 on VSI-Bench","Geometry-aware keyframe graphs lift spatial scores 7.8 points","SpaR3D-MoE routes sparse views to 63.5 VSI-Bench SOTA","Instruction-pose experts close 2D-3D gap from RGB alone","Manifold sampling and MoE deliver 35%+ Route Plan gains"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"The method assumes that camera poses and implicit 3D geometry features predicted from RGB by a frozen visual-geometry model are accurate enough both to choose the keyframes and to steer the expert router and spatial experts.","fun_headline_variants_meta":{"raw":{"variants":["Sparse RGB plus adaptive MoE hits 63.5 on VSI-Bench","Geometry-aware keyframe graphs lift spatial scores 7.8 points","SpaR3D-MoE routes sparse views to 63.5 VSI-Bench SOTA","Instruction-pose experts close 2D-3D gap from RGB alone","Manifold sampling and MoE deliver 35%+ Route Plan gains"]},"model":"grok-4.5","effort":"low","cost_usd":0.00407,"raw_usage":{"total_tokens":1303,"prompt_tokens":889,"num_sources_used":0,"completion_tokens":108,"cost_in_usd_ticks":40700000,"prompt_tokens_details":{"text_tokens":889,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":306,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":889,"tokens_out":108,"duration_ms":4060,"temperature":1.0,"reasoning_tokens":306,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T01:28:24.565398+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the same VSI-Bench videos, replace the adaptive sampler with uniform sampling of the identical frame count and replace the MoE with a single monolithic fusion layer; if the 7.8-point average gain and the large Route-Plan and Relative-Direction gains disappear, the claim that ASMS plus HGI-MoE are responsible is falsified.","supporting_citations":[],"review_version":1}