{"id":"43cc7e0d-d9fa-4dbd-909c-38f2d3422e90","arxiv_id":"2505.24103","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":4,"one_line_summary":"A pseudo-supervised pipeline with an affordance-to-part mapping, label refinement, cross-view alignment, and a reasoning module achieves state-of-the-art weakly supervised affordance grounding on AGD20K.","lead":"This paper trains an affordance grounding model using pseudo labels generated by off-the-shelf part segmentation, guided by a hand-defined mapping from actions to object parts. It reports large gains over prior weakly supervised methods on AGD20K and demonstrates a robot grasping application.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pseudo-label quality is the load-bearing condition: the paper's own Table 12 shows the choice of part segmenter swings seen KLD from 0.890 to 0.976, so part of the reported breakthrough is inherited from an external foundation model; the claim is plausible but should be conditional.","rationale":"The reader identified the same load-bearing assumption: the correctness and completeness of the manual mapping P plus the reliability of VLPart+SAM. This is confirmed by the paper's own admission of systematic VLPart failures and by Table 12, where swapping the external segmenter shifts seen KLD from 0.890 to 0.976 or 0.863. The proposed modules (refinement, alignment, reasoning) do produce consistent improvements over the pseudo-label baseline, so the paper is not invalidated. However, the central 'breakthrough' claim is substantially attribution-dependent: part of the gain comes from an external foundation model that improves independently of the authors' contribution. This does not warrant rejection, because the comparison against prior CAM-based methods is fair (all use the same weakly supervised inputs), the code is released, and the method is reproducible in structure. But it does warrant a conditional verdict requiring reproducibility details: full mapping release, error bars, sensitivity of thresholds, and a clarified Eq. (7). The typo in Eq. (7) is minor but should be corrected because the reasoning loss is a claimed contribution. Overall, the central claim is plausible and mostly well-supported, but its strength is overstated given the external-model dependence.","tokens_in":26846,"tokens_out":1681,"duration_ms":22635,"concrete_test":"Re-run the full pipeline exactly as specified, but replace the pseudo-label generator with PartGLEE+SAM (or a third independent part segmenter) while keeping all proposed modules and hyper-parameters fixed. If the resulting seen-split KLD remains below 0.9 and unseen KLD below 1.15, the central claim is robust to the specific pseudo-label generator; if performance degrades to the VLPart+FastSAM level, the headline gains are largely inherited from the external segmenter. Also report standard deviations across the five seeds and re-derive Eq. (7) to confirm which text feature (object name or part name) is actually used in the object-classification loss.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim--that pseudo-supervised training yields a breakthrough over CAM-based WSAG--depends critically on the quality of pseudo labels produced by the manually authored (object, affordance) to part-name mapping P and the VLPart+SAM pipeline. The paper explicitly concedes systematic failures (e.g., VLPart struggles on tennis racket handles, Appendix B.1), and Table 12 shows large performance swings when the label generator changes: seen KLD is 0.976 with VLPart+FastSAM, 0.890 with VLPart+SAM, and 0.863 with PartGLEE+SAM, with corresponding unseen KLD 1.219/1.153/1.084. This confirms that a substantial share of the headline improvement is inherited from the off-the-shelf part segmenter, not from the proposed learning modules. That is a legitimate engineering contribution, but it weakens the novelty framing and makes the claim contingent on an external model the authors do not control. Additionally, the label-adaptation heuristics (box threshold 0.5, mask-inversion rule, SAM region selection threshold max(0.1, 0.9*max-ratio)) are not sensitivity-analyzed, and no error bars are reported despite averaging over five seeds. A further internal inconsistency: Eq. (7) defines the object loss as 1 - Cos-Sim(f_pred_obj, Enc_T(p)), where p is the part name, which contradicts the text 'predict the object class' and the stated balance between object and part learning; this appears to be a typo but it obscures which supervision the reasoning module actually receives.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses weakly supervised affordance grounding (WSAG) on AGD20K. It replaces CAM-based training with supervised training on pseudo labels: for each (object, affordance) pair, a manually authored mapping P selects a part name, an off-the-shelf VLPart detector plus SAM produces a part mask, and the mask is used as a pseudo-heatmap. Three extensions are added to the baseline: a label-refinement stage based on occlusion cues and SAM post-processing, a masked exocentric feature alignment loss, and a lightweight noun/part reasoning module with a stitching augmentation. The full model reports large improvements over prior CAM-based methods on both seen and unseen splits (e.g., seen KLD 0.890 vs 1.176 for WSMA; unseen KLD 1.153 vs 1.335), with ablations, a hard-split evaluation, a study of alternative label generators, and a real-robot grasping deployment.","tokens_in":27150,"tokens_out":8532,"duration_ms":90601,"significance":"If the results are reproducible, the paper makes a strong case for pseudo-supervised training as an alternative to CAM-based WSAG and for using part-level semantics as a bridge between objects and actions. The empirical work is unusually complete: ablations for every module, an additional hard split, variation of the visual encoder and the pseudo-label generator, and a real-robot deployment. The authors are also candid about label-quality failures. The main caveat is that the headline gains are not solely due to the proposed learning modules: Table 12 shows that swapping VLPart for PartGLEE changes seen KLD from 0.890 to 0.863 and unseen KLD from 1.153 to 1.084, while the weaker VLPart+FastSAM still outperforms all CAM baselines. The contribution is therefore best understood as a pipeline whose performance scales with an external, rapidly evolving label generator; this is valuable engineering but weakens the \"breakthrough\" framing.","major_comments":[{"comment":"The reasoning loss as written appears to have its two cosine-similarity terms swapped: f_pred-part is compared with Enc_T(o) and f_pred-obj with Enc_T(p), while the text states that the first MLP predicts the object class and the second predicts the target part. Because this objective defines the reasoning module that drives the unseen-split gains in Tables 2 and 10, please correct the equation (or, if the equation is intentional, explain why part prediction is matched to the object embedding and vice versa). Also update the cross-reference in Appendix A.1, which cites Eq. (6) for this loss.","section":"§3.6, Eq. (7)"},{"comment":"All reported numbers are means over five seeds, but no standard deviations or confidence intervals are given. Without variance estimates, the reader cannot judge whether the margins between Ours-full and Ours-baseline, or among label-generator choices in Table 12, are statistically meaningful. Please report standard deviations for at least the main tables.","section":"§4.2 / Tables 1, 2, 7, 8–12"},{"comment":"The pseudo-labeling pipeline is the load-bearing component of the method, yet the complete affordance-to-part mapping P is not released (Appendix B contains only five examples in Table 5), and none of the heuristic thresholds (box confidence 0.5, the mask-inversion perimeter rule, the SAM region-selection threshold max(0.1, 0.9·max-ratio), N_exo_pool=10, λ1, λ2) is sensitivity-analyzed. Please make the full mapping available with the code and add a sensitivity study for the most consequential thresholds, or explicitly argue that the reported results are insensitive to them.","section":"§3.3 / Appendix B"}],"minor_comments":[{"comment":"The appendix cites Eq. (6) for f_pred-part; the reasoning loss is defined in Eq. (7).","section":"Appendix A.1"},{"comment":"The caption uses 'VLpart' while the text uses 'VLPart'; please standardize the spelling.","section":"Figure 2 caption"},{"comment":"The text says '8×8 gird points'; this should be 'grid points'.","section":"Appendix D.2"},{"comment":"The phrase 'fully supervised training process' is imprecise because the supervision comes from pseudo labels; 'pseudo-supervised' would be more accurate.","section":"§3.3"},{"comment":"ManipVQA's NSS of 1.735 with KLD of 12.67 is surprising; please explain how such a high NSS coexists with a KLD an order of magnitude larger than other methods.","section":"Table 7"},{"comment":"The robot results are based on five attempts per object; please state explicitly that these are illustrative and not statistically powered.","section":"Table 13"}],"recommendation":"major_revision","confidential_remarks":"The paper is already published at ICLR 2025, so the journal version should be expected to resolve the reproducibility issues identified here: the Eq. (7) inconsistency, missing standard deviations, and the unreleased full P mapping. If the authors address these, the paper would be a solid contribution; without them, the 'breakthrough' claim remains hard to verify independently."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper does something genuinely new for weakly supervised affordance grounding: instead of CAMs, it trains on pseudo labels produced by an off-the-shelf part segmenter (VLPart+SAM) guided by a hand-written (object, affordance)-to-part mapping. That shift alone buys a large chunk of the reported improvement, and the three add-ons (label refinement, object-masked cross-view alignment, part reasoning) each contribute something in the ablations. On AGD20K the full model beats the best CAM-based method by a wide margin on KLD/SIM/NSS on both splits, and the hard-split results strengthen the case. The paper is also honest about its weak spots: it shows failure cases, admits VLPart struggles on some categories, and includes the Table 12 sensitivity analysis across label generators.\n\nThe main caveat is exactly that Table 12. Seen KLD goes from 0.890 with VLPart+SAM to 0.976 with VLPart+FastSAM, and to 0.863 with PartGLEE+SAM. So a substantial part of the \"breakthrough\" is inherited from the external foundation model, not from the proposed learning pipeline. That is still a legitimate engineering contribution—and it is the right direction for the field—but the novelty framing should be scaled back. The paper's own framing as a \"breakthrough improvement\" is a bit strong given that the zero-shot pseudo-label generator is doing a lot of the work.\n\nOther soft spots are smaller but real. No standard deviations are reported despite averaging over five seeds. The full mapping P is only partially released, which matters for reproducibility since the whole pipeline depends on it. The SAM post-processing thresholds (0.5, max(0.1, 0.9*max-ratio), etc.) are heuristic and not sensitivity-analyzed. And Eq. (7) has an apparent typo: the object loss uses Enc_T(p), the part name, while the text says the object class. That should be fixed but does not undermine the method.\n\nWho is this for? Researchers working on affordance grounding, weakly supervised dense prediction, or using foundation models as label generators. They will get a clear, well-tested recipe and a good baseline for future work. I would cite it. The paper deserves a serious referee: the central claim is plausible and mostly supported, and the reproducibility issues are addressable in revision. Recommendation: send to peer review, and ask for error bars, the full mapping, and a corrected Eq. (7) before acceptance.","headline":"A solid pseudo-supervised WSAG pipeline with large, reproducible-looking gains; the main caveat is that part of the win is inherited from the off-the-shelf segmenter, not the proposed modules.","tokens_in":27700,"tokens_out":1242,"would_cite":true,"duration_ms":15195,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that weakly supervised affordance grounding can be turned into pseudo-supervised segmentation by mapping every (object, affordance) pair to a part name that off-the-shelf part segmenters can localize, and that this recipe…","keywords":["affordance grounding","weakly supervised learning","pseudo labels","part-level semantics","foundation models","vision-language models","AGD20K","heatmap prediction"],"falsifier":"Run the same training recipe with pseudo labels replaced by whole-object masks (or by randomly permuted part labels): if the seen-split KLD stays near 0.890 instead of degrading toward the CAM baselines' ~1.18, the claimed gains are not actually coming from the part-level priors. A second check: apply the mapping P(o,a) to a category the authors cite as hard, e.g. 'hold' on a tennis racket, and verify whether the refinement stage systematically repairs the missing-handle labels; if the repaired labels are still wrong on a large share of images, the generalization claims for the reasoning module lose their foundation.","tokens_in":26590,"feed_emoji":"🎯","tokens_out":8523,"duration_ms":75563,"temperature":0.7,"pith_summary":"The paper claims that weakly supervised affordance grounding—training a model to mark where an action like ``hold'' or ``open'' happens on an object, using only image-level labels plus interaction images—should stop relying on class activation maps and instead be treated as a supervised segmentation problem with pseudo labels. It builds those labels by translating each (object, affordance) pair into a concrete part name (handle, cap, grip, ...), having an off-the-shelf part detector plus segmenter localize that part, and training a CLIP-based encoder–decoder to reproduce the resulting heatmaps. On the AGD20K benchmark, the pseudo-supervised baseline already cuts the KLD error well below the best CAM method, and the full model—with label refinement, cross-view feature alignment, and a light reasoning module—drops the seen-split KLD from 1.176 to 0.890 and the unseen-split KLD from 1.335 to 1.153. A robot-grasping deployment suggests the learned affordance maps transfer to real scenes and novel objects, lifting grasp success from 50% to 77.5%. If this holds, the field's bottleneck shifts from CAM quality to the quality and coverage of the (object, affordance) to part mapping.","feed_headline":"Part-name pseudo labels beat CAM for affordance grounding","feed_subtitle":"Object+action pairs become part names, so off-the-shelf models write the labels, and AGD20K scores jump.","key_machinery":"The load-bearing mechanism is the (object, affordance) to part-name mapping P(o,a), a hand-authored table of about 100 entries that converts verb-like affordances into noun-like part queries such as ``handle of the knife'' or ``cap of the bottle''. Off-the-shelf VLPart detects that part's box and SAM segments it into a mask, which becomes the pseudo-label heatmap; three enhancement modules—a pretraining stage that refines labels via occlusion-aware similarity to interaction images, masked-pooled cross-view alignment using object masks, and an MLP reasoning module that predicts object and part features (plus a 2x2 stitching augmentation)—carry the rest of the training. The decoder is a SAM-style mask decoder driven by a CLIP text feature of the affordance query, keeping the architecture plain and putting the weight of the method on the supervision itself.","core_discovery":"The paper's central discovery is that action-related affordance grounding can be driven by noun-level semantic priors: for a given object o and affordance a, the affordance region is usually one named part of the object, so a mapping P(o, a) = p turns the verb-like affordance into a part query that open-vocabulary part segmentation models can execute. The authors show that training a heatmap-predicting network with Kullback–Leibler supervision against these VLPart+SAM pseudo labels—without any pixel-level human annotation—already outperforms all previous CAM-based weakly supervised methods on the seen split of AGD20K, and that their full model improves every metric on both splits. They further claim each enhancement contributes: the refinement stage exploits the fact that a human hand occludes the affordance region in interaction images to repair bad pseudo labels; masked pooling inside VLPart/SAM object masks aligns egocentric and exocentric features without background or human noise; and a small noun/part reasoning module with a stitching augmentation lets the model generalize to unseen object categories and even novel affordance words. Taken together, the paper positions affordance learning as a beneficiary of foundation-model part segmentation rather than a task that needs bespoke action supervision.","pith_inferences":["A consequence the paper leaves implicit is that the pipeline's ceiling is set by the coverage of P(o,a): any pair whose part name is ambiguous, missing, or not segmentable becomes a noisy or empty label, so scaling to new objects means either standardizing the mapping table or letting an LLM generate and validate part queries.","Because the method is agnostic to the specific part detector and segmenter, gains in generic part segmentation should transfer almost mechanically to affordance grounding; a testable prediction is that the next generation of open-vocabulary part models will push AGD20K numbers further without any architectural change to the grounding model.","The same object-to-part translation could be applied to other verb-driven dense prediction tasks, such as 'wearable regions' on clothing or 'graspable handles' on tools in robot manipulation benchmarks, reusing the exact pseudo-label-and-align recipe.","The empirical requirement that the stitching augmentation be needed suggests the model would otherwise shortcut the text query by memorizing object-affordance co-occurrence; one testable extension is to measure how much the KL gap between conditioning on the correct versus an incorrect affordance query widens after removing the stitching."],"forward_implications":["A purely weakly supervised pipeline trained on part-prior pseudo labels can beat CAM-based WSAG by a wide margin on both seen and unseen object splits of AGD20K (KLD 0.890 vs. 1.176 seen; 1.153 vs. 1.335 unseen).","The model accepts free-form text affordance queries at inference, so the same weights can respond to words never seen during training (e.g., 'grasp' with similar heatmaps to 'hold').","Better off-the-shelf part segmenters translate directly into better affordance grounding: swapping VLPart for PartGLEE while keeping everything else fixed sets new state-of-the-art numbers on both splits.","Affordance heatmaps are usable as a prior for grasp planning: combining the heatmap with a 6-DoF grasp generator raises real-world grasp success on 8 objects from 50.0% to 77.5%, including objects and an affordance word absent from training.","The method's three modules each pull in the intended direction in the ablations: refinement helps the seen split, cross-view alignment helps both, and the reasoning module plus stitching augmentation matter most for the unseen split."],"supporting_citations":[{"why":"Defines the WSAG task and contributes the AGD20K dataset that every experiment and comparison in the paper uses.","marker":"Luo et al. (2022)"},{"why":"VLPart, the open-vocabulary part detector that converts part-name queries into bounding boxes for pseudo-label generation.","marker":"Sun et al. (2023b)"},{"why":"SAM, the segmenter that turns VLPart boxes into pixel masks and supplies the Auto Mask Generator used in label post-processing.","marker":"Kirillov et al. (2023)"},{"why":"CLIP, whose visual and text encoders provide the backbone and the affordance/part features used throughout training.","marker":"Radford et al. (2021)"},{"why":"LOCATE, the strongest localized-alignment baseline whose masked-pooling idea the paper extends by using VLPart/SAM object masks.","marker":"Li et al. (2023)"},{"why":"WSMA, the previous state of the art whose CAM-based numbers the paper's baseline and full model are measured against on both splits.","marker":"Xu et al. (2024)"},{"why":"WorldAfford, a CAM-based foundation-model-augmented baseline compared in Table 1.","marker":"Chen et al. (2024a)"}],"fun_headline_variants":["Pseudo labels from part names beat CAM in affordance grounding","Turn actions into part names: weak supervision wins","Noun priors replace pixel labels for affordance grounding","From object parts to actions: no hand-labeled pixels needed","Use part segmentation to teach affordance grounding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the hand-written table that maps each (object, affordance) pair to a part name being right and complete, and on the off-the-shelf detector and segmenter actually finding that part; the paper itself reports that the handle of a tennis racket is often missed, and the full table is not released.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo labels from part names beat CAM in affordance grounding","Turn actions into part names: weak supervision wins","Noun priors replace pixel labels for affordance grounding","From object parts to actions: no hand-labeled pixels needed","Use part segmentation to teach affordance grounding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000551,"raw_usage":{"total_tokens":2647,"prompt_tokens":984,"completion_tokens":1663,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":1587}},"tokens_in":600,"tokens_out":1663,"duration_ms":10344,"temperature":1.0,"reasoning_tokens":1587,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:34:51.562764+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training recipe with pseudo labels replaced by whole-object masks (or by randomly permuted part labels): if the seen-split KLD stays near 0.890 instead of degrading toward the CAM baselines' ~1.18, the claimed gains are not actually coming from the part-level priors. A second check: apply the mapping P(o,a) to a category the authors cite as hard, e.g. 'hold' on a tennis racket, and verify whether the refinement stage systematically repairs the missing-handle labels; if the repaired labels are still wrong on a large share of images, the generalization claims for the reasoning module lose their foundation.","supporting_citations":[{"cited_title":"Learning affordance grounding from exocentric images","cited_arxiv_id":null,"evidence_quote":"Defines the WSAG task and contributes the AGD20K dataset that every experiment and comparison in the paper uses."}],"review_version":1}