{"id":"feab1b9f-3e05-49b7-9c8f-3e6493b64256","arxiv_id":"2609.04193","paper_version":1,"verdict":"ACCEPT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"GIFT guides intermediate visual features to preserve geometry, affordance, and goal structure via auxiliary training losses, improving zero-shot robustness and precision across VLA and WAM policies by 5-13 points on challenging benchmarks.","lead":"This paper introduces GIFT, a framework that trains intermediate visual features in robot policies to encode geometry, affordance, and goal information through auxiliary losses during training. A generalist reader should care because GIFT consistently improves robustness and precision across three different policy architectures (VLA, direct-action WAM, inverse-dynamics WAM) on simulation and real robots, without needing extra predictions at test time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek/deepseek-v4-flash","headline":"Geometry alignment may be redundant; missing baseline comparison for depth-decoding diagnostic weakens claim of complementary geometric knowledge.","rationale":"The reader identified geometry guidance redundancy as weakest assumption, and I agree that the missing baseline depth-decoding diagnostic is the most load-bearing concern. The paper's claim that VGGT alignment provides complementary geometric knowledge is not decisively validated: the only diagnostic (Fig. 4) shows GIFT features decode depth, but no comparable measurement from raw student features exists. Without that control, the gains from geometry guidance could be partly regularization. However, the overall body of evidence—consistent improvements across three policy families, ablations isolating each signal, real-world validation—still supports the core contribution. The verdict should remain UNCHANGED because the concern is non-fatal but worth flagging for future work or a small revision.","tokens_in":27549,"tokens_out":3329,"duration_ms":31377,"concrete_test":"Run the depth-decoding diagnostic (frozen VGGT depth head with student features substituted) on baseline Fast-WAM-IDM and StarVLA-OFT features (without GIFT) and compare depth reconstruction error (e.g., RMSE or abs relative) against the GIFT variants. If baseline depth error is within 10% of GIFT's, geometry guidance is largely redundant, weakening the claim that geometric knowledge transfer drives the improvements.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper claims geometry alignment with frozen VGGT transfers control-relevant geometric knowledge not already present in student features. The only evidence is a depth-decoding proxy (Fig. 4) applied to GIFT-trained features. This diagnostic is not run on baseline student features (without GIFT). If baseline already decodes depth comparably well, geometry guidance might be redundant and its gains could stem from generic regularization rather than complementary geometric structure. The ablations (Table 4) show geometry alone improves camera robustness, but without a control for the extra supervisory signal itself, the specific benefit of VGGT alignment over any auxiliary prediction task is unquantified. This is load-bearing because the central principle—that structuring features with geometry, affordance, and goal is uniquely effective—requires demonstrating that each signal provides complementary information beyond what native objectives already encode.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek/deepseek-v4-flash","summary":"The paper proposes GIFT (Guided Intermediate Feature Training), a framework that augments intermediate visual features of robot policies with three auxiliary supervision signals: geometry alignment (via cosine similarity and log-scale regression to a frozen VGGT teacher), object-centric affordance prediction, and goal-region reconstruction. These losses are applied to the intermediate tokens of the policy encoder during training only; the default no-injection design does not route auxiliary predictions into the action head. GIFT is instantiated on three policy families—a semantics-centered VLA (StarVLA-OFT variant), a direct-action World-Action Model (Fast-WAM), and an inverse-dynamics WAM (Fast-WAM-IDM)—without changing their action formulations. Extensive simulations on LIBERO, LIBERO-Plus, and RoboCasa, as well as real-robot experiments on single-arm and dual-arm platforms, show consistent improvements over matched baselines, with especially large gains under distribution shifts, articulation tasks, and high-precision manipulation. Ablations confirm the complementarity of the three signals and show that the no-injection variant outperforms the injection variant.","tokens_in":27842,"tokens_out":3225,"duration_ms":31012,"significance":"If the claims hold, GIFT provides a reusable principle for bridging the action-sufficiency gap across diverse policy architectures. The key strengths are: (1) the same three objectives improve three fundamentally different action-generation mechanisms, suggesting that structuring intermediate features is architecture-agnostic; (2) the no-injection design cleanly separates representation shaping from auxiliary conditioning; (3) the real-robot results with limited demonstrations (100 per task) show practical relevance for precision-sensitive tasks. The paper is well-executed methodologically, with thorough ablation studies, attention visualizations, and failure analysis. The gains—often 5–13 points in challenging zero-shot benchmarks—are practically meaningful.","major_comments":[{"comment":"The paper claims that geometry alignment transfers control-relevant geometric knowledge not already present in the student's native features. The only diagnostic evidence is the depth-decoding proxy in Figure 4, which shows that GIFT-trained features, when substituted into a frozen VGGT depth head, produce plausible depth maps. However, this diagnostic is not applied to baseline student features (without GIFT). If the baseline already decodes depth comparably well, the geometric guidance may be redundant and its benefits could stem from generic regularization rather than complementary geometric structure. Since the claim of complementary geometry is load-bearing for the motivation of the geometry guidance, the authors should either provide the same depth-decoding comparison on baseline features or add an ablation that replaces VGGT alignment with an alternative auxiliary objective (e.g., random feature matching, a simple depth regression) to isolate the specific benefit of the geometric teacher. Without this, the paper's attribution of gains to complementary geometric knowledge is not fully supported.","section":"§3.2.1, Fig. 4"},{"comment":"The affordance supervision requires ground-truth object poses and end-effector poses in the object frame. In simulation, these are obtained from privileged information. In the real-world, they are obtained via a human-in-the-loop annotation pipeline (Grounding DINO + SAM2 + Orient Anything V2). The paper should discuss the potential failure modes of this pipeline more explicitly than the single sentence in Section 5.1.3, especially how annotation errors (e.g., tracking drift after SAM2 propagation) might affect affordance supervision quality and whether the method is robust to such noise. A brief quantitative analysis (e.g., percentage of corrected masks in the validation set) would strengthen the practical claim.","section":"§3.2.2, Eq. (9)"}],"minor_comments":[{"comment":"The definition of ρ_{t,i} = log(1 + ||g_{t,i}||_2) and the reconstruction in Eq. (8) are clear; however, the scale weight λ_scale = 0.05 is significantly smaller than λ_ang = 0.2. Provide a brief justification for this ratio or a sensitivity study in the supplement.","section":"§3.2.1, Eq. (4)"},{"comment":"The paper states that the VGGT teacher is frozen and used only during training. This is stated but could be emphasized earlier in the method section to avoid confusion with the ablation variant that uses auxiliary features as inputs.","section":"§4.2, implementation details"},{"comment":"The paper uses video inpainting to remove the demonstrator's green gloves from half of the real-world data. How does this mixture affect the distribution of visual features? Is there any risk that the inpainting artifacts become a shortcut? A brief discussion or a small ablation comparing training on purely gloved vs. mixed data would be helpful.","section":"§5.1.2, data collection"},{"comment":"In the failure case examples (Fig. 7), the affordance predictions use colors to distinguish anchor and other objects, but the caption says 'Colors in the affordance predictions distinguish anchor and other objects.' The figure itself does not include a legend; consider adding color labels directly on the figure or in the caption.","section":"Fig. 7 and caption"},{"comment":"The denoising step sensitivity shows that 2 steps suffice, with very little variation up to 15 steps. This is a strong result; consider noting that the performance is robust to step count, which reduces inference cost.","section":"§4.4, Table 6"},{"comment":"The paper uses the term 'no-injection' and 'w/o inject' interchangeably; pick one for consistency.","section":"General"}],"recommendation":"minor_revision"},"author_rebuttal":null,"desk_editor":{"model":"deepseek/deepseek-v4-flash","letter":"Bottom line: this is a solid, unusually well-controlled empirical paper, and the GIFT recipe is worth taking seriously. The reader's accept verdict is reasonable; I would send it to review rather than desk-reject, but I would ask for one missing control in revision.\n\nThe genuinely new thing is the architecture-flexible framing. Geometry distillation, affordance prediction, and goal segmentation have each appeared before, but GIFT packages them as a single training-time principle and tests it across three policy families while keeping each model's action formulation intact. That cross-formulation evidence is the paper's real contribution, and it holds up. The controlled comparisons are impressive: matched backbones, a same-head no-guidance control for the VLA, injection versus no-injection variants, per-signal ablations, full LIBERO-Plus perturbation sweeps, RoboCasa, and real-robot tasks with progressive perturbations. The failure analysis is honest and a plus. The citation pattern is fine, including the authors' own prior PokeVLA and concurrent affordance work.\n\nThe soft spots are proportionate. The stress-test concern about the geometry diagnostic is correct. The depth-decoding proxy in Figure 4 is only run on GIFT-trained features; there is no baseline-student-features decoding, and no control auxiliary task. So the claim that VGGT alignment transfers complementary geometric knowledge not already present in the student is under-supported. Table 4 shows that the geometry loss helps, but not that VGGT alignment specifically is better than, say, depth supervision or a generic auxiliary prediction head. This is a moderate gap in the central principle, not a fatal one. The gains could partly come from extra regularization, but the consistency across three architectures and the factor-specific perturbation patterns make pure regularization an unlikely full explanation.\n\nA second, more practical limitation is annotation cost. Affordance targets need privileged poses and instruction-relevant object roles; real-world labels require Grounding DINO, SAM2, human verification, mask propagation, and pose estimation. The paper reports all this, but it means the recipe is not cheap to apply to new settings. No code or data release is mentioned, which matters for a method whose value is mainly reproducibility of the training recipe.\n\nWho is this for: robot learning practitioners training VLA or WAM policies who want a robustness boost without inference-time overhead, and researchers working on intermediate representations. The paper deserves a serious referee. My recommendation is to accept after revision, with the depth-decoding baseline and a generic-supervision control added.","headline":"A well-controlled empirical paper that makes a genuine cross-architecture claim: three auxiliary training-time losses on intermediate features improve zero-shot robustness across a VLA and two WAM formulations, and it deserves serious peer review with one missing control added.","tokens_in":28580,"tokens_out":2374,"duration_ms":24450},"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek/deepseek-v4-flash","headline":"A framework called GIFT bridges the action-sufficiency gap by shaping intermediate visual features with geometry, affordance, and goal objectives, achieving consistent gains in zero-shot robot manipulation.","keywords":["Vision-Language-Action Models","World-Action Models","Intermediate Feature Training","Geometry Alignment","Affordance Prediction","Goal-Region Reconstruction","Zero-Shot Transfer","Robotic Manipulation"],"falsifier":"Train a GIFT variant on LIBERO where the geometry alignment objective uses a randomly initialized, frozen teacher that has never been trained on geometric tasks. If this variant still yields similar gains (i.e., the benefit comes from regularization rather than teacher knowledge), then the geometry guidance is not leveraging geometric structure. Conversely, if the random-teacher variant performs worse than the VGGT variant, the specific geometric content of VGGT is load-bearing.","tokens_in":29141,"feed_emoji":"🤖","tokens_out":4434,"duration_ms":42360,"temperature":0.7,"pith_summary":"The paper argues that vision-language and world-model policies learn visual features that are rich but not optimally useful for control, a mismatch they call the action-sufficiency gap. GIFT closes this gap by attaching three training-time objectives to intermediate visual tokens: geometry alignment with a frozen geometric foundation model, affordance prediction of object-centric interaction configurations, and goal-region reconstruction of task-relevant areas. These objectives are applied without modifying the action generation interface and without feeding auxiliary predictions to the action head in the default design. The same three objectives are instantiated in a Vision-Language-Action policy, a direct-action World-Action Model, and an inverse-dynamics World-Action Model, each preserving its native action formulation. Across LIBERO-Plus, RoboCasa, and real-robot experiments, GIFT improves zero-shot robustness by 5 to 13 percentage points over matched baselines, with especially large gains on articulated-object and high-precision tasks.","feed_headline":"Geometry, affordance, goal: structured supervision boosts robot policies by 5–13 points","feed_subtitle":"The GIFT framework applies the same three objectives to three different policy families without changing action generation, improving…","key_machinery":"Three auxiliary objectives applied to intermediate visual tokens: (1) Geometry alignment distills the direction and magnitude of features from a frozen VGGT geometric foundation model into shallow student tokens using cosine direction and log-scale regression losses. (2) Affordance prediction uses learned queries to regress 20-dimensional targets encoding entity roles (anchor object, end effector, other objects), anchor-relative poses, end-effector-relative poses, and gripper closure states. (3) Goal prediction uses attended visual features to decode a binary mask of instruction-relevant image regions. All objectives backpropagate into shared intermediate representations but are excluded from the action head in the default no-injection design, leaving the action-generation mechanism unchanged.","core_discovery":"GIFT demonstrates that training intermediate visual features to preserve three complementary types of control-relevant structure—geometry for motion feasibility, affordance for instruction-relevant interaction configurations, and goals for task-conditioned spatial localization—consistently improves robot policy performance under zero-shot distribution shifts, without requiring auxiliary predictions as action inputs. The principle is architecture-flexible: it transfers across a semantics-centered VLA, a direct-action diffusion WAM, and a future-conditioned inverse-dynamics WAM, each gaining 4.6–12.6 percentage points over its baseline on the LIBERO-Plus benchmark. Ablations confirm that all three objectives contribute, and that the no-injection variant (where auxiliary predictions are not routed to the action head) outperforms the variant that feeds them in, establishing representation shaping itself as the mechanism.","pith_inferences":["The success of no-injection over injection challenges the common practice of feeding auxiliary predictions into the action head and suggests that intermediate features may already encode the necessary structure once properly shaped; injecting them may add noise or cause the head to overfit to training-time prediction patterns.","The three guidance signals correspond to three distinct failure modes under distribution shifts: geometry helps camera and viewpoint changes, affordance helps robot initialization shifts, and goal helps background and sensor noise. This decomposition suggests a diagnostic tool for identifying which structure a policy lacks.","Because GIFT uses a frozen VGGT teacher for geometry, but the teacher's features are not independently validated to be complementary to the student's, a natural extension is to ablate the teacher choice (e.g., swap VGGT for a depth-estimation model or DINOv2) to test whether the benefit is specific to VGGT or generalizes to any geometric representation.","The principle could be extended to other control-relevant structures not covered here, such as contact force, object dynamics, or temporal consistency, potentially further narrowing the action-sufficiency gap."],"forward_implications":["GIFT can be added as a plug-in module to most existing VLA and WAM policies, requiring only small decoder heads and frozen teacher features during training, with no inference overhead.","The no-injection design implies that shaping intermediate features alone is sufficient; injecting auxiliary predictions into the action head is not only unnecessary but can slightly hurt performance.","The consistent improvement across three different action formulations suggests that the principle of structured intermediate supervision generalizes beyond the specific networks tested.","On articulated-object tasks and high-precision real-robot tasks, the gains are larger than on simple pick-and-place, indicating that geometry and affordance guidance are especially valuable when contact dynamics and spatial accuracy matter.","The geometry teacher can be discarded at inference, meaning the policy retains its original input modality while benefiting from geometric knowledge absorbed during training."],"supporting_citations":[{"why":"Provides the frozen VGGT geometric foundation model whose patch features serve as training targets for geometry alignment.","marker":"[47]"},{"why":"Defines the Fast-WAM baseline (direct-action and inverse-dynamics variants) that GIFT extends and compares against.","marker":"[5]"},{"why":"Defines the StarVLA-OFT baseline that GIFT-VLA builds on and outperforms.","marker":"[49]"},{"why":"Provides the LIBERO-Plus benchmark with seven controlled distribution shifts used for zero-shot robustness evaluation.","marker":"[16]"},{"why":"Inspires the object-centric affordance representation with anchor-frame and end-effector poses used in the affordance prediction target.","marker":"[48]"},{"why":"Provides the RoboCasa benchmark with humanoid tasks, including articulated-object manipulation, used for additional evaluation.","marker":"[17]"},{"why":"Represents a prior approach that aligns visual features with a geometric teacher, providing context for GIFT's geometry guidance.","marker":"[12]"}],"fun_headline_variants":["Geometry, affordance, goals: three signals boost robot policies 5–13 points","GIFT trains intermediate features to preserve structure for zero-shot gains","Without changing action generation, GIFT lifts VLA, WAM, and WAM-IDM 4.6–12.6 points","Representation shaping alone beats feeding auxiliary predictions into robot policies","Three control-relevant structures lift robot policy under distribution shift"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The assumption that the frozen VGGT teacher's geometric features complement the student's existing visual features, rather than being redundant or delivering no additional control-relevant information; if the teacher features are already well-encoded by the student's native objectives, geometry alignment would add little benefit.","fun_headline_variants_meta":{"raw":{"variants":["Geometry, affordance, goals: three signals boost robot policies 5–13 points","GIFT trains intermediate features to preserve structure for zero-shot gains","Without changing action generation, GIFT lifts VLA, WAM, and WAM-IDM 4.6–12.6 points","Representation shaping alone beats feeding auxiliary predictions into robot policies","Three control-relevant structures lift robot policy under distribution shift"]},"model":"deepseek/deepseek-v4-flash","effort":"low","cost_usd":0.000717,"raw_usage":{"cost":0.000716548,"is_byok":false,"cost_details":{"upstream_inference_cost":0.000716548,"upstream_inference_prompt_cost":0.000122668,"upstream_inference_completions_cost":0.00059388},"total_tokens":3202,"prompt_tokens":1081,"completion_tokens":2121,"prompt_tokens_details":{"audio_tokens":0,"video_tokens":0,"cached_tokens":256,"cache_write_tokens":0},"completion_tokens_details":{"audio_tokens":0,"image_tokens":0,"reasoning_tokens":2015}},"tokens_in":1081,"tokens_out":2121,"cost_source":"openrouter_usage","duration_ms":19149,"temperature":1.0,"reasoning_tokens":2015,"transport_provider":"openrouter","cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-09-08T19:08:59.396706+00:00","model_set":{"reader":"deepseek/deepseek-v4-flash"},"pipeline_version":"pith-pipeline@v1.4.1-alltime-deepseek-or-medium","input_kind":null,"input_chars":null,"input_total_chars":null,"passes_missing":null,"published_standing":null,"published_confidence":null,"objection_audit":null,"run_record":null,"basis":{"kind":"not_recorded","source":null,"chars_given":null,"chars_total":null},"receipt":null,"falsifier":"Train a GIFT variant on LIBERO where the geometry alignment objective uses a randomly initialized, frozen teacher that has never been trained on geometric tasks. If this variant still yields similar gains (i.e., the benefit comes from regularization rather than teacher knowledge), then the geometry guidance is not leveraging geometric structure. Conversely, if the random-teacher variant performs worse than the VGGT variant, the specific geometric content of VGGT is load-bearing.","supporting_citations":[{"cited_title":"Vggt: Visual geometry grounded transformer,","cited_arxiv_id":null,"evidence_quote":"Provides the frozen VGGT geometric foundation model whose patch features serve as training targets for geometry alignment."},{"cited_title":"Fast-WAM: Do World Action Models Need Test-time Future Imagination?","cited_arxiv_id":"2603.16666","evidence_quote":"Defines the Fast-WAM baseline (direct-action and inverse-dynamics variants) that GIFT extends and compares against."},{"cited_title":"StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing","cited_arxiv_id":"2604.05014","evidence_quote":"Defines the StarVLA-OFT baseline that GIFT-VLA builds on and outperforms."},{"cited_title":"LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models","cited_arxiv_id":"2510.13626","evidence_quote":"Provides the LIBERO-Plus benchmark with seven controlled distribution shifts used for zero-shot robustness evaluation."},{"cited_title":"HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos","cited_arxiv_id":"2605.24934","evidence_quote":"Inspires the object-centric affordance representation with anchor-frame and end-effector poses used in the affordance prediction target."},{"cited_title":"Robocasa: Large-scale simulation of everyday tasks for generalist robots,","cited_arxiv_id":null,"evidence_quote":"Provides the RoboCasa benchmark with humanoid tasks, including articulated-object manipulation, used for additional evaluation."},{"cited_title":"Spatial forcing: Implicit spatial representation alignment for vision- language-action model.arXiv preprint arXiv:2510.12276, 2025","cited_arxiv_id":null,"evidence_quote":"Represents a prior approach that aligns visual features with a geometric teacher, providing context for GIFT's geometry guidance."}],"lane_accuracy":{"status":"not yet measured","lane":{"model":"deepseek/deepseek-v4-flash","input_kind":null},"url":"https://pith.science/accuracy"},"review_version":1}