{"id":"0f19513c-2202-4625-a2f6-153f487d9a3f","arxiv_id":"2608.12683","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"FUSE, an entropy-gated planner that combines amortized viewpoint prediction with explicit semantic-geometric exploration, achieves the best non-oracle active functional grounding results on a new Habitat benchmark while cutting compute by 1.33x versus full explicit exploration.","lead":"This paper introduces active functional affordance grounding, a task in which a robot must explore a scene to find an object that satisfies a functional request such as 'something to pour milk in'. The proposed FUSE system alternates between fast learned viewpoint selection and explicit semantic-geometric search, reaching 72% success on a new Habitat benchmark while using about 25% less compute than full explicit search.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The amortized planner is trained on ground-truth target-mask IoU and target visibility/distance terms, so the claimed non-oracle advantage may reflect optimization of the SAM3 success metric rather than unbiased evidence acquisition.","rationale":"The strongest claim depends on FUSE being a non-oracle method. The most load-bearing assumption is not merely that SAM3 confidence measures functional grounding, but that the amortized planner's decisions are learned without the evaluation target. The appendix's training objective includes Δq_t (SAM3 IoU with ground-truth target mask), Δv_t (target visibility), and Δg_t (inverse target distance), which are oracle measurements. Because the planner is invoked for 66.59% of decisions, the adaptive system's headline success may be driven by a policy fitted to the success criterion rather than by the proposed semantic-geometric evidence acquisition. The paper's paired statistics are careful, and the explicit exploration baseline provides useful evidence for the task's viability; the runtime reduction is also plausible. However, the central non-oracle performance claim needs to be re-established with a cleanly trained planner. The reader's CONDITIONAL verdict is appropriate; my concern selects a specific missing condition that should be checked before accepting the claim.","tokens_in":19813,"tokens_out":8246,"duration_ms":91510,"concrete_test":"Retrain the amortized planner from scratch using the same trajectories, architecture, hyperparameters, and entropy threshold, but with the oracle-dependent terms removed: y_t = w_sem Δs_t + w_unc Δu_t (i.e., drop Δq_t, Δv_t, and Δg_t). Evaluate this FUSE variant on the same 100 episodes and compare Success and IoU with Explicit Exploration and with the published FUSE. If the success gap over Explicit Exploration (+2pp) or the improvement over the planner-only variant disappears or reverses, the non-oracle grounding claim is not established. If the gap persists, the privileged training terms are not the source of the reported advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FUSE achieves the best non-oracle grounding performance requires that the learned amortized planner select viewpoints without privileged information about the target. The appendix's 'Planner Training Targets' violates this requirement. The training target is y_t = w_sem Δs_t + w_unc Δu_t + w_iou Δq_t + w_vis Δv_t + w_geo Δg_t, where Δq_t is explicitly defined as the change in 'IoU between the highest-confidence SAM3 region and the ground-truth target mask'; Δv_t is target visibility normalized by the scene-specific maximum target pixel count; and Δg_t is normalized inverse camera-to-target distance. All three terms require knowing the target object's mask and position during training. FUSE uses this planner for 66.59% of online decisions. Success and IoU at evaluation are computed from SAM3 masks against the same ground-truth target masks, so the policy is trained on a reward that is essentially the evaluation metric. At test time no oracle is fed to the planner, but the policy has been optimized with privileged target information. Consequently, the +6pp improvement over Amortized Planning (p=0.109) and the assertion of the highest observed non-oracle performance may reflect supervised fitting to the metric rather than an emergent benefit of semantic-geometric evidence acquisition. This is a validity concern, not an internal inconsistency; explicit Exploration, which is hand-crafted, provides a fairer lower-bound comparison, but the learned branch's contribution is unsupported until the training target is re-examined.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Active Functional Affordance Grounding, a sequential perception task in which an agent must move a camera to localize an object satisfying a functional query under partial observability. The proposed FUSE framework combines an explicit exploration policy that scores candidate viewpoints from SAM3 semantic evidence and SSNR-based geometric uncertainty (Eqs. 4-5) with a learned amortized planner, switching between the two on the basis of action entropy. The authors contribute a Habitat-based benchmark with 100 fixed episodes, evaluate against passive, random, VLM-driven, explicit, and amortized baselines, and report that FUSE reaches 72% success and 70.91% average IoU (Table 1), a 1.33x end-to-end speedup over Explicit Exploration (Table 5), and robustness across seven affordance knowledge sources (Table 2). The appendix includes paired McNemar tests, bootstrap intervals, ablations of the semantic and geometric components, and detailed runtime profiling.","tokens_in":20198,"tokens_out":10966,"duration_ms":110431,"significance":"The task definition and benchmark are a useful addition to embodied perception: they make explicit the distinction between grounding from a fixed view and actively acquiring functional evidence. The explicit semantic-geometric explorer is a reasonable heuristic, and the entropy-gated integration of amortized and explicit planning is architecturally sensible. The paper is also careful in several ways: all methods are evaluated on the same 100 episode definitions; paired exact McNemar tests and bootstrap confidence intervals are reported; the failure of the success advantage over Explicit Exploration to reach significance is admitted in the appendix; and the cross-knowledge-source comparison is a good stress test. If the central claim of non-oracle performance survives the concerns below, the paper would make a moderate but solid contribution. The main unresolved issues are the privileged training signal used for the amortized planner and the use of the same SAM3 model for both viewpoint selection and grounding evaluation; both are fixable with additional experiments, and neither invalidates the benchmark or the explicit-exploration results.","major_comments":[{"comment":"The learned planner is not trained without oracle information. In the appendix, the utility target is y_t = w_sem Δs_t + w_unc Δu_t + w_iou Δq_t + w_vis Δv_t + w_geo Δg_t, where Δq_t is the change in IoU between the highest-confidence SAM3 region and the ground-truth target mask, Δv_t is target visibility normalized by the scene-specific maximum target pixel count, and Δg_t is normalized inverse camera-to-target distance. These are privileged quantities that a non-oracle agent should not have. Since FUSE routes 66.59% of online decisions through the amortized branch (Table 6), and since Success and IoU in 'Metric Computation' are computed by comparing SAM3 masks with the same ground-truth masks, the planner is effectively trained on the evaluation metric. The claim of 'highest observed non-oracle grounding performance' (Abstract; Table 1) is therefore not established: the improvement over Amortized Planning (+6pp, p=0.109) and part of FUSE's gain over Explicit Exploration may reflect supervised fitting to ground-truth target masks rather than an emergent benefit of semantic-geometric evidence acquisition. I ask the authors to retrain the planner using only observable targets, for example changes in SAM3 score and geometric uncertainty without any ground-truth mask, visibility, or distance terms, and to re-report the FUSE comparisons, or to clearly label the current variant as an oracle-assisted bound.","section":"Appendix 'Planner Training Targets'; 'Amortized Evidence Planning' and Table 6"},{"comment":"SAM3 is used both as the semantic evidence source that drives viewpoint selection and as the model whose masks define Success and Average IoU. In Eq. (4), the acquisition map is built from SAM3 confidence and geometric uncertainty; in 'Metric Computation', success is the fraction of episodes in which the highest-scoring SAM3 mask at the returned viewpoint has IoU above 0.5 with a valid ground-truth target. The evaluation therefore rewards finding viewpoints that make SAM3 confident and well-aligned, but it does not independently validate that the grounded object is functionally suitable. All compared methods use SAM3, so the relative comparisons are not unfair, but the absolute claim about reliable functional grounding is weakened. I recommend a control experiment with an independent semantic segmenter at evaluation time, a downstream functional interaction check, or at least an analysis of how the FUSE-vs-baseline differences change when masks for the success metric come from a different model.","section":"Eq. (4) and 'Metric Computation'"},{"comment":"The headline '72% success, highest observed non-oracle grounding performance' is a point estimate on a single 100-episode benchmark. The paper's own paired statistics show that FUSE versus Explicit Exploration is not significant (exact McNemar p=0.688, CI [-3,7]pp) and FUSE versus Amortized Planning is not significant at the conventional level (p=0.109, CI [0,12]pp, including zero). The appendix states that 'the current evaluation does not establish a statistically distinguishable success advantage over fully explicit reasoning,' yet the abstract and Table 1 present the point estimate without this qualification. Given the small benchmark and the risk of metric overfitting discussed above, the main claim should be rephrased as comparable grounding performance with a 1.33x computation reduction; additional episodes or a Bayesian analysis would be needed before claiming the highest non-oracle success rate.","section":"Table 7 and Abstract"}],"minor_comments":[{"comment":"The form of the evidence map in Eq. (4) does not visibly match the appendix's E_t(u) = w_s S_t(u) + w_g U^geo_t(u) with w_s = w_g = 1 and U^geo scaled by alpha = 0.8; the main text should define the relationship between the gradient-ratio term and the SSNR-based U^geo, and state where alpha enters, to avoid a seeming inconsistency.","section":"Eq. (4) and Appendix 'Semantic and Geometric Evidence'"},{"comment":"The entropy threshold eta is selected from {0.6, 0.7, 0.8, 0.9} using only five validation episodes, and no sensitivity analysis of eta is reported. Because eta controls the amortized/explicit branch rate and the computational savings, a small sweep of eta on the development set would make the design choice more robust.","section":"Appendix 'Confidence Gate'"},{"comment":"The amortized planner is trained for a fixed 20 epochs with no checkpoint selection, as the validation loss is monitored but not used for checkpoint selection. Given that this planner handles roughly two-thirds of online decisions, reporting training and validation loss curves or using early stopping would increase confidence that the learned policy has not merely memorized the 416 training trajectories.","section":"Appendix 'Planner Training Targets'"}],"recommendation":"major_revision","confidential_remarks":"The main validity risk is the oracle-trained amortized planner. If the authors can retrain an oracle-free planner and show that FUSE still matches Explicit Exploration while saving computation, the paper could be acceptable after major revision. The SAM3 dual-use issue further suggests that an independent evaluation model is needed. I would not reject outright because the benchmark and explicit planning variant are useful and the issues are fixable with additional experiments. Please weigh the possibility that the +6pp over Amortized Planning is an artifact of training on target-mask IoU, visibility, and distance; the 100-episode benchmark is too small to resolve that question by itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe new thing here is the task: active functional affordance grounding — deciding where to look when a functional query rather than a category defines the target. That's a real gap, and the benchmark (100 episodes, Habitat, six held-out configs) is a reasonable first cut. FUSE's confidence gate between an amortized planner and explicit 3DGS-based exploration is also a sensible efficiency idea, and the paper is unusually transparent with paired statistics and bootstrap CIs. It openly tells you that FUSE does not significantly beat Explicit Exploration on success (p=0.688), and that the IoU gain over Amortized is suggestive rather than conclusive. Credit where due: they don't oversell what the numbers show in the main comparisons.\n\nBut the stress-test note is right, and it matters. The amortized planner is trained on a target that includes Δq_t, the change in IoU between the highest-confidence SAM3 region and the ground-truth target mask, plus normalized target visibility and inverse camera-to-target distance. That is privileged information. The planner is used for 66.6% of online decisions. So FUSE is not a purely non-oracle system: its learned branch has been fitted to the evaluation metric. The paper calls it non-oracle because the oracle is not fed at test time, but the policy has seen the target's mask and position during training. That undermines the headline claim, and the +6pp over Amortized Planning (p=0.109) is what you'd expect from fitting the success metric. The problem is not internal inconsistency — the appendix is honest about the training targets — but the main text's claim is not supported.\n\nThere are also smaller issues: SAM3 supplies both the semantic evidence for action selection and the masks scored for Success/IoU, so there's a partial feedback loop; the evaluation is entirely simulated; and no code or data is released despite the promise. The speedup (1.33x vs Explicit Exploration) is real but modest, and the wall-clock advantage comes mostly from skipping 3DGS refinements — that part holds up.\n\nWho gets value: embodied-perception researchers who want a task formulation and a benchmark for functional grounding under partial observability. The FUSE architecture itself needs another pass — retrain the amortized planner without ground-truth mask/position targets, or present it as an oracle-informed upper bound, and rerun the comparisons.\n\nRecommendation: send to peer review, but with major-revision expectations. The task and benchmark are worth refereeing; the non-oracle claim is not ready as written.","headline":"The task and benchmark are worth a look, but the 'non-oracle' claim is undercut by the amortized planner's privileged training targets.","tokens_in":20683,"tokens_out":2997,"would_cite":false,"duration_ms":30429,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FUSE introduces active functional affordance grounding: an entropy-gated planner selects viewpoints from semantic-geometric uncertainty, reaching 72% grounding success at a 1.33x computation saving.","keywords":["active functional affordance grounding","functional uncertainty","viewpoint selection","semantic-geometric evidence","amortized planning","explicit exploration","3D Gaussian scene representation","partial observability"],"falsifier":"Run the benchmark again but judge success by whether the agent can physically perform the queried function with the grounded object (pour, cut, scoop, sip) at the returned viewpoint, instead of by mask IoU; if high-confidence, high-IoU viewpoints do not reliably yield objects that pass the interaction test, the evidence signal is not tracking functional suitability.","tokens_in":19629,"feed_emoji":"🤖","tokens_out":12245,"duration_ms":113690,"temperature":0.7,"pith_summary":"This paper defines a new embodied perception task: given a functional request such as \"something to pour milk in,\" an agent must move through a partially observed scene until it can identify and spatially locate an object that satisfies the request without being told the object's category in advance. The authors propose FUSE, a framework that decides where to look next by maintaining a spatial map of unresolved semantic and geometric evidence, and that alternates between a fast learned planner and a slower explicit exploration routine depending on how confident the learned planner's action choice is. On a simulated tabletop benchmark of 100 episodes, FUSE succeeds in 72% of episodes with 70.91% mean mask overlap (IoU), the best non-oracle grounding performance reported in the paper, while cutting end-to-end computation by 1.33 times relative to running explicit exploration at every step. The result matters because agents that act on function rather than category need a perception loop that actively resolves ambiguity instead of committing from a single fixed view.","feed_headline":"Robots that choose their view ground functional queries at 72%","feed_subtitle":"A learned-plus-explicit view planner beats fixed-view grounding and runs 1.33x faster than full search.","key_machinery":"The load-bearing object is the functional uncertainty field, a viewpoint-conditioned spatial map of where the evidence needed to ground the query's candidate objects is still unresolved. It is the sum of a semantic component, built from per-pixel segmentation confidence for each query-induced candidate label, and a geometric component, defined as the inverse of a structural signal-to-noise ratio between the current observation and the render of an incrementally refined Gaussian 3D scene model. The field becomes an evidence-acquisition map that scores candidate camera actions by the evidence expected in the left, right, upper, or lower image region those actions would reveal. FUSE's adaptive rule is an entropy gate: when the softmax distribution over the amortized planner's predicted action values is below a threshold, the agent follows the fast prediction; otherwise it refines the 3D model and invokes explicit exploration. A grounding-based stopping rule, combining segmentation confidence with depth proximity to the target, ends exploration after K consecutive non-improving views.","core_discovery":"FUSE's central claim is that task-directed sensing should be driven by a fused functional-uncertainty map instead of generic scene coverage, fixed viewpoints, or a known target category. The map decomposes unresolved evidence into a semantic part, where segmentation confidence for the query's candidate object labels is weak in the current view, and a geometric part, where the observed image disagrees with a render from an incrementally refined Gaussian 3D scene model. Explicit planning scores each candidate camera action by the average acquisition value in the image region that action would expose; a learned amortized planner imitates these scores in a single forward pass; and normalized action entropy decides which of the two to trust. On the introduced benchmark, FUSE reaches 72% grounding success and 70.91% mean IoU, above static grounding (42%), random active exploration (56%), a label-provided VLM active baseline (65%), and either planning mode alone (66% amortized, 70% explicit), while using 1.33 times less computation than full explicit exploration.","pith_inferences":["If FUSE is transferred to a physical robot, the natural next test is to replace mask overlap with physical interaction success; an agent that actually pours, cuts, or scoops with the grounded object would tell whether the semantic-geometric confidence signal tracks function or merely visibility.","The benchmark's discrete elliptical camera loop and three pitch levels are restrictive; allowing continuous or six-degree-of-freedom viewpoint control would increase the number of candidate actions and likely raise the value of the amortized planner as a pre-filter.","The same confidence-gated fast/deliberative architecture could apply to other active perception problems, such as task-oriented grasping or object search, where a cheap policy should handle routine sensing and expensive model updates should be reserved for ambiguous states.","One should not read the 72% figure as an upper bound: the gap to the oracle-label variant (77%) suggests that improved functional hypothesis generation, not only sensing, is a separate route to higher grounding success."],"forward_implications":["Functional grounding no longer needs a predefined target class: the query generates candidate labels that serve as hypotheses, and exploration resolves which hypothesis is grounded.","Fixed or predetermined viewpoints are not enough under occlusion, since static and single-canonical-view grounding reach only 42% and 32% success on the benchmark.","The entropy gate provides most of explicit search's reliability at lower cost, because FUSE routes about two-thirds of decisions through the amortized branch and still matches or exceeds explicit planning's accuracy.","Because gains persist across several affordance knowledge sources, active evidence acquisition and upstream functional-hypothesis generation act as complementary stages rather than interchangeable alternatives."],"supporting_citations":[{"why":"Supplies the pixel-level semantic evidence and per-mask confidence used for viewpoint selection, candidate scoring, and the success/IoU metrics.","marker":"Carion et al. 2025"},{"why":"Default knowledge source that turns a functional query into candidate object hypotheses in the main experiments.","marker":"Chen et al. 2025b"},{"why":"Provides the incrementally refined Gaussian scene representation whose renders define geometric uncertainty.","marker":"Kerbl et al. 2023"},{"why":"Initializes the scene model and registers each newly acquired observation into the shared coordinate frame.","marker":"Schonberger and Frahm 2016"},{"why":"Supports the benchmark's scenes, sensors, and discrete camera action space.","marker":"Savva et al. 2019; Szot et al. 2021; Puig et al. 2023"},{"why":"Symbolic affordance grounding source used in the knowledge-source robustness comparison.","marker":"Chen, Lin, and Aakur 2025"},{"why":"Serves as an alternative knowledge source and as the model behind the VLM-driven active baseline.","marker":"Comanici et al. 2025"}],"fun_headline_variants":["FUSE: Uncertainty-driven view planning grounds affordances at 72%","Active affordance grounding picks views, hits 72% success","FUSE learns where to look for functional objects, 1.33x faster","Semantic-geometric evidence acquisition beats fixed views, 72%","Uncertainty-guided exploration grounds functional queries at 72%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the segmentation model's mask confidence and the overlap of its mask with a ground-truth object mask actually measure whether the found object satisfies the requested function.","fun_headline_variants_meta":{"raw":{"variants":["FUSE: Uncertainty-driven view planning grounds affordances at 72%","Active affordance grounding picks views, hits 72% success","FUSE learns where to look for functional objects, 1.33x faster","Semantic-geometric evidence acquisition beats fixed views, 72%","Uncertainty-guided exploration grounds functional queries at 72%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1468,"prompt_tokens":910,"completion_tokens":558,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":465}},"tokens_in":526,"tokens_out":558,"duration_ms":5731,"temperature":1.0,"reasoning_tokens":465,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:39:23.888880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the benchmark again but judge success by whether the agent can physically perform the queried function with the grounded object (pour, cut, scoop, sip) at the returned viewpoint, instead of by mask IoU; if high-confidence, high-IoU viewpoints do not reliably yield objects that pass the interaction test, the evidence signal is not tracking functional suitability.","supporting_citations":[{"cited_title":"Aakur , booktitle=","cited_arxiv_id":null,"evidence_quote":"Symbolic affordance grounding source used in the knowledge-source robustness comparison."}],"review_version":1}