{"id":"fc1ef6b6-f1e6-4122-b53e-4fc87243c118","arxiv_id":"2509.03893","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A weakly-supervised pipeline that distills VLM functional part knowledge and multi-view spatial structure into a model for dense cross-category functional correspondence, outperforming baselines on new synthetic and real benchmarks.","lead":"This paper trains a model to find matching functional parts across pictures of objects that do the same job, such as the spout of a kettle and the neck of a bottle. It does so with almost no human labels, using a vision-language model to roughly mark parts and a self-supervised model to refine dense matches, and it shows the result works on synthetic and real photos.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential train/eval asset overlap may inflate synthetic benchmark; paper does not state that the 950 synthetic evaluation pairs are disjoint from the 8,285 curated assets (80% used for training).","rationale":"The reader's weakest assumption was ambiguity in the ground-truth functional alignment, which the paper acknowledges in Section 11 and partially mitigates by manual filtering. I agree that ambiguity is a valid concern and reinforces the conditional verdict. However, I find a more concrete and potentially more damaging issue: the paper never states that the synthetic evaluation assets are disjoint from the training assets. The wording in Section 3.2 ('selected for quality and diversity' with reference to Section 4.1) and the explicit '80% of which are used for training' without a described evaluation split leave this ambiguous. If overlap exists, the synthetic evaluation does not measure cross-object generalization, undermining the headline numbers. The real benchmark is independent and shows a similar margin, so the core method may still be sound, but the synthetic claim as stated would be unsupported. The concrete test (checking asset overlaps and re-evaluating on disjoint assets) would settle this. I do not think this changes the reader's CONDITIONAL verdict; rather, it adds a specific condition that the authors must verify to make the claim credible. I also credit the paper for multiple ablations, scaling experiments, and the independent real benchmark, which provide partial support even if the synthetic set is flawed.","tokens_in":29643,"tokens_out":9666,"duration_ms":96014,"concrete_test":"Obtain the asset identifiers for the 950 synthetic evaluation pairs and the 8,285 curated assets with the 80% training split. Compute the intersection. If any evaluation asset appears in the training set, re-run all models using only pairs whose assets are entirely in the held-out 20% and recompute the Table 1 synthetic metrics. If the AP@10p margin drops substantially or vanishes, the synthetic claim is unsupported and the paper must be revised to use a disjoint split.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest quantitative claim is the synthetic benchmark margin (AP@10p 0.330 vs 0.188 for CogVLM+DINO). Section 4.1 defines a curated Objaverse set of 8,285 assets, '80% of which are used for training.' Section 3.2 describes the synthetic evaluation as '950 pairs of assets from Objaverse ... selected for quality and diversity' and refers to Section 4.1 for asset selection, but never states that these evaluation assets are held out from the training split. If the 950 pairs are sampled from the same 8,285 assets without excluding the training 80%, the model may have been trained on multi-view renders of the exact same 3D meshes used for evaluation. The spatial contrastive loss (Eq. 2) is applied across views of the same asset, so the model could memorize object-specific geometry, inflating both label transfer and discovery metrics on the synthetic set. The real HANDAL benchmark is disjoint and partially mitigates, but the claim explicitly includes synthetic numbers. This is a concrete, checkable methodological gap, not a critique of the ambiguity issue alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces the task of dense functional correspondence: establishing dense pixel-level matches between images of objects from different categories whose parts support the same function (e.g., kettle spout to bottle neck). The authors propose a weakly-supervised training pipeline that (i) uses GPT-4 to build a function/object taxonomy, (ii) pseudo-labels functional parts on multi-view renders of Objaverse assets with CogVLM bounding boxes aggregated on 3D point clouds, and (iii) trains a function-conditioned MLP on top of frozen DINOv2/CLIP features with two contrastive losses—a functional part loss and a multi-view spatial loss—plus an optional mask loss. For evaluation, they define ground truth by aligning 3D object pairs in functionally equivalent poses and derive dense 2D correspondences; they curate 950 synthetic Objaverse pairs and 190 real HANDAL pairs. Experiments compare against DINOv2, Stable Diffusion, and VLM-grounded baselines (CogVLM+DINO, ManipVQA+DINO), reporting consistent improvements on label transfer and correspondence discovery metrics.","tokens_in":29971,"tokens_out":2276,"duration_ms":25037,"significance":"If the results hold, the paper makes a useful contribution: it gives a clean, actionable definition of dense functional correspondence, a scalable pseudo-labeling recipe that combines VLM semantic grounding with self-supervised spatial features, and two new benchmarks. The method is clearly specified, the grounding of pseudo-labels in an external VLM avoids the most obvious circularity in training, and the ablations (functional-only, spatial-only, mask loss, backbone scaling, loss-weight sensitivity) are extensive. The main uncertainty is not the method itself but the validity of the synthetic benchmark and the benchmark curation process, which the authors partly acknowledge. The real HANDAL evaluation provides some external anchor, but the headline synthetic numbers are the strongest quantitative claim.","major_comments":[{"comment":"The synthetic evaluation set may overlap with the training set, and this is load-bearing for the main claim. Section 4.1 states that 8,285 curated assets are used, with 80% for training, but Section 3.2 says the 950 evaluation pairs are 'assets from Objaverse selected for quality and diversity' and refers to Section 4.1 for selection without stating that the evaluation assets are held out from the training split. Since the spatial contrastive loss (Eq. 2) is applied across rendered views of the same training assets, the model could memorize object-specific geometry, inflating both label-transfer and discovery metrics on the synthetic set. Please state explicitly whether the 950 pairs are drawn exclusively from the held-out 20% (or otherwise exclude all training assets), and if not, report the evaluation on a disjoint held-out split. This is a concrete, checkable methodological requiremen","section":"§3.2 and §4.1"},{"comment":"Hyperparameters λspatial and λmask appear to be selected on the same benchmark used for final reporting. Table 4 sweeps λspatial ∈ {1,5,10} and λmask ∈ {1,5,10} and reports test metrics on the synthetic and real evaluation sets, with λspatial=10 chosen as 'best result.' No validation split is described. If these choices were made by looking at the evaluation set, the reported numbers are optimistic. Please describe the model-selection protocol (e.g., a separate validation subset, or a fixed choice made before evaluation).","section":"§9.3, Table 4"},{"comment":"The ground-truth annotation procedure involves manual disambiguation and filtering. The appendix says ambiguity is resolved 'based on the objects' orientation' and that annotations are 'manually filter[ed] ... to disambiguate and ensure high quality,' and the Limitations section concedes that radial symmetry and multiple functional regions make the task ambiguous. This is reasonable for a first benchmark, but the evaluation is partly defined by the annotators' judgment. Please report inter-annotator agreement or at least quantify how many pairs were filtered and whether the reported metrics are stable under alternative disambiguation choices. Without this, it is hard to know how much of the measured performance reflects the model versus the annotation protocol.","section":"§8.1 and §11"}],"minor_comments":[{"comment":"The row labels 'DINO' and 'DINOv2' are used inconsistently; the text cites DINOv2 but the table says DINO. Please unify.","section":"Table 1"},{"comment":"The definition of PCK@k is missing an indicator function: it should read 1[||p2^{j(i)} - p2^i|| < k], not the expression as written.","section":"§5.1"},{"comment":"The negative set for the spatial contrastive loss is described only as 'any other pixel on I'. Please clarify whether negatives are sampled uniformly, how many are used, and whether hard-negative mining is applied; this affects reproducibility.","section":"Eq. (2)"},{"comment":"The ablation labeled 'Ours (functional only)' underperforms DINOv2 on several metrics. The text says the full model 'consistently outperforms,' but the functional-only variant does not; please qualify the claim to refer to the full model.","section":"§4.2 and Table 1"},{"comment":"ManipVQA is described as a 7B model, while CogVLM is described as 17B. It would be useful to note the parameter-count gap when interpreting the comparison, since the baselines have very different capacities.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The central method and task formulation are sound and interesting, but the benchmark integrity question in §3.2/§4.1 is exactly the kind of issue that can undermine an otherwise solid paper. I would encourage the editor to require the authors to release or clearly describe the train/eval split of the Objaverse assets and to document the model-selection protocol. The real HANDAL results are reassuring, but the synthetic numbers are the headline claim. If the overlap is confirmed, the paper would still be publishable with corrected numbers, but the margins would likely shrink."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely useful: it defines dense functional correspondence as a task, gives a 3D-alignment-based procedure for dense 2D ground truth, and builds both synthetic and real benchmarks. The method is also sensible — distill VLM pseudo-labels into a function-conditioned feature extractor, add multi-view spatial contrastive learning to avoid mode collapse, and optionally predict a functional part mask. The ablations support the story: functional-only collapses, spatial-only lacks semantics, together they beat DINOv2, Stable Diffusion, and CogVLM-grounded baselines on both benchmarks. The within/across-category breakdown is a nice touch and shows the method's edge is largest where the task is hardest.\n\nThe biggest problem is one the authors do not flag: I cannot find any statement that the 950 synthetic evaluation asset pairs are disjoint from the 8,285 curated assets used for training. Section 4.1 says 80% of the curated set is used for training; Section 3.2 says evaluation pairs come from Objaverse 'selected for quality and diversity' and defers to Section 4.1, but never says those assets are held out. If the evaluation pairs are drawn from the same pool without exclusion, the spatial contrastive loss — which pulls together pixels from different views of the same asset — could memorize object-specific geometry and inflate every synthetic number, including the headline AP@10p gain (0.330 vs 0.188). The real HANDAL benchmark is disjoint and shows the same ranking, so the central claim probably survives, but the synthetic margin should be treated as unverified until the authors confirm the split. This is a concrete, checkable issue, not an ambiguity nitpick.\n\nOther soft spots are minor by comparison: no error bars, hyperparameters chosen via sensitivity analysis without a validation split, and the ground-truth ambiguity is real but acknowledged and partially handled by manual filtering. The benchmark curation and the 3D aggregation of noisy VLM boxes are contributions worth crediting, and the writing is refreshingly direct about limitations.\n\nWorth a serious referee. If I were reviewing, I would ask for the train/eval overlap check, error bars, and code or data release. For a reading group, the task definition and benchmark design are good discussion material; I would cite this as a new benchmark and baseline.","headline":"Defines a new task with useful benchmarks and a solid weakly-supervised method, but the synthetic evaluation may have a train/eval asset overlap that needs checking.","tokens_in":30384,"tokens_out":2123,"would_cite":true,"duration_ms":22160,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dense functional correspondence across object categories can be learned with weak supervision.","keywords":["dense correspondence","functional correspondence","weakly supervised learning","contrastive learning","vision-language pseudo-labels","cross-category matching","3D object alignment","affordance grounding"],"falsifier":"Run an inter-annotator study in which several annotators independently align each object pair; if their dense correspondences disagree on a substantial fraction of pairs, then the ground-truth benchmark and reported gains would be tied to one particular alignment convention.","tokens_in":29615,"feed_emoji":"🧩","tokens_out":5255,"duration_ms":46674,"temperature":0.7,"pith_summary":"This paper tries to establish that dense pixel-level correspondence between objects of different categories—e.g., matching a kettle's spout to a bottle's neck for pouring—can be defined and learned without manual dense labeling. The authors derive the task from 3D alignment of functionally equivalent parts, then train a model by combining pseudo-labeled functional parts from a vision-language model with multi-view contrastive learning that preserves spatial structure. They build synthetic and real benchmarks for evaluation and report that the trained model outperforms off-the-shelf self-supervised features and vision-language grounding baselines on both. If correct, the result matters for applications like transferring a demonstrated manipulation from one object to another.","feed_headline":"Weak supervision maps functional parts across object categories","feed_subtitle":"A distilled vision-language model plus contrastive learning beats off-the-shelf features on synthetic and real tests.","key_machinery":"The load-bearing mechanism is a function-conditioned dense feature extractor: frozen self-supervised image features are combined at each pixel with a text embedding of the function, then passed through a small MLP. Two contrastive objectives shape the feature space. The functional-part contrastive loss distills semantic knowledge from the vision-language pseudo-labels by pulling pixels on matching functional parts together and pushing other pixels apart. The spatial multi-view contrastive loss prevents mode collapse by requiring a pixel to match its projection in a second view of the same object and to differ from all other pixels, preserving the internal geometry of a part such as the tip v","core_discovery":"The central claim is that object function—the effect an object can cause on another object or substance—provides a reliable guide for dense correspondence across categories, because parts that carry out the same function tend to be similar in shape and appearance even when whole objects are visually different. The paper formalizes dense functional correspondence as a pixel-level mapping induced by aligning the functional parts of two 3D objects in space, and proposes a weakly supervised training scheme: use an off-the-shelf grounded vision-language model to propose functional-part bounding boxes on multi-view renders, aggregate those proposals into 3D pseudo-labels, then train a function-con","pith_inferences":["A natural extension would be to condition on finer descriptions than a single function, such as 'pour-with while keeping the container upright,' to resolve the ambiguities the paper acknowledges.","The same vision-language pseudo-label plus 3D aggregation recipe could be applied to functions that are not object-part based, such as those involving object-object or object-substance interactions, as long as prompts can be written.","The learned feature space could be used as a reward or cost signal for motion planning, not just for matching, because it encodes functionally equivalent alignment.","The benchmark construction implies that inter-annotator agreement on functional alignment should be measured; if agreement is low, a probabilistic correspondence model would be needed."],"forward_implications":["A model can transfer a functional-part label from one object category to another without ever seeing human-annotated correspondence pairs.","Because training uses rendered synthetic images, the resulting representation generalizes to real photographs of unseen objects.","The pseudo-labeling pipeline converts noisy vision-language bounding-box proposals into spatially consistent pixel-level part masks by aggregating over many views in 3D.","Correspondence discovery—finding which pixels correspond without being given either part mask—becomes feasible, a capability needed for aligning a robot's actions to a new object.","The method's performance improves with larger backbones and higher spatial resolution, suggesting headroom from scaling."],"supporting_citations":[{"why":"Provides the large 3D object asset collection from which rendered training images and synthetic evaluation pairs are drawn.","marker":"[7]"},{"why":"Supplies frozen self-supervised image features that form the base of the function-conditioned extractor.","marker":"[56]"},{"why":"Generates the functional-part bounding-box pseudo-labels that are aggregated into training masks.","marker":"[82]"},{"why":"Provides text embeddings used to condition the pixel features on the function name.","marker":"[63]"},{"why":"Defines the earlier sparse functional-correspondence problem and supplies part of the function taxonomy.","marker":"[38]"},{"why":"Supplies real-world object scans and images used to build the real evaluation benchmark.","marker":"[20]"},{"why":"Motivates dense visual descriptor learning from multi-view correspondence, the basis of the spatial contrastive loss.","marker":"[17]"},{"why":"Provides captions that let the pipeline match noisy 3D assets to object categories for training.","marker":"[47]"}],"fun_headline_variants":["Function-aware weak supervision densifies cross-category matches","Pseudo-labeled functional parts train dense correspondence without full supervision","Vision-language priors distill into dense functional matching","Learning pixel-level function matching from sparse labels","Weakly supervised model maps object functions across categories"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The whole evaluation rests on the assumption that one human 3D alignment of functional parts, such as a kettle spout against a bottle neck, gives the unambiguous ground truth for dense pixel correspondence; the paper notes this can be ambiguous for multi-functional parts or radially symmetric objects.","fun_headline_variants_meta":{"raw":{"variants":["Function-aware weak supervision densifies cross-category matches","Pseudo-labeled functional parts train dense correspondence without full supervision","Vision-language priors distill into dense functional matching","Learning pixel-level function matching from sparse labels","Weakly supervised model maps object functions across categories"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1196,"prompt_tokens":678,"completion_tokens":518,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":422,"completion_tokens_details":{"reasoning_tokens":445}},"tokens_in":422,"tokens_out":518,"duration_ms":5726,"temperature":1.0,"reasoning_tokens":445,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:33:28.039664+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an inter-annotator study in which several annotators independently align each object pair; if their dense correspondences disagree on a substantial fraction of pairs, then the ground-truth benchmark and reported gains would be tied to one particular alignment convention.","supporting_citations":[{"cited_title":"The functional correspondence problem","cited_arxiv_id":null,"evidence_quote":"Defines the earlier sparse functional-correspondence problem and supplies part of the function taxonomy."},{"cited_title":"Scalable 3d captioning with pretrained models","cited_arxiv_id":null,"evidence_quote":"Provides captions that let the pipeline match noisy 3D assets to object categories for training."}],"review_version":1}