{"id":"08f57d64-0e17-4150-8b83-0f8bc93b88dc","arxiv_id":"2603.04419","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Seven different agent personas made a vision-language model describe the same COCO image with under 10% lexical overlap, which the paper interprets as >90% context-dependent affordance computation.","lead":"The paper measures how much a vision-language model's descriptions of the same photo change when it is asked to act as a chef, security guard, child, or other persona, reporting under 10% lexical overlap between personas. The authors read this as evidence that affordances are computed context-dependently and suggest robotics build task-specific scene representations at query time rather than static world models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 90% claim conflates prompt wording with agent context: absent paraphrase/same-prompt controls, the observed Jaccard drop may be instruction-following rather than affordance drift.","rationale":"Read in good faith: the paper’s goal is to quantify context-dependent affordance computation in VLMs. The strongest claim is the 90% lexical drift. For that claim to hold, the experimental conditions must isolate agent context. They do not: the primes confound persona with prompt wording, output format, and requested object count. The authors explicitly defer the decisive controls to future work (§4.4, §6.6), so the paper itself flags the missing evidence. This is not an internal inconsistency—the paper is candid about its limitations—but it is a correctness risk because the headline magnitude is exactly what the missing controls would test. The LLaVA replication and stochastic baseline show the phenomenon is reproducible and not pure sampling noise, but they share the same confound; the Tucker decomposition is stable, but it decomposes the same confounded outputs. I therefore agree with the reader’s weakest-assumption identification: the reported 90% figure could collapse to instruction-following surface variation. Since the reader already rejected, my read does not move the verdict; the rejection stands.","tokens_in":20336,"tokens_out":3669,"duration_ms":34931,"concrete_test":"Run a paraphrase-control on the same 479 images: for each prime Pi, construct 3 paraphrases that preserve the persona/goal but change wording and, crucially, match output format (same object count, same JSON keys). For example, P1 variants: “You are a professional chef...” vs “Imagine you are an expert cook preparing a meal...”; P5 variants all ask for the single most urgent survival tool. Compute mean Jaccard within each persona’s paraphrase pairs. If within-persona Jaccard ≈ cross-persona Jaccard (~0.095), the reported drift is driven by prompt lexicons/format rather than agent context; if within-persona Jaccard is high (>0.7), the context interpretation survives. Also report the same comparison restricted to the 'name' field to remove reasoning-text wording.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central inference—from mean Jaccard J̄=0.095 to “>90% of lexical scene description is context-dependent”—requires that the seven primes vary only in agent context. They do not. Appendix A.1 (Table 12) shows the primes differ in surface task demands: P0 requests “3 most prominent objects, their geometric properties, and standard functions”; P1–P4 request “3 critical/interesting objects” for a specific role; P5 drops the count and asks “What do you see first and how do you use it?”; P6 asks what “catches your eye.” Because word-level Jaccard (Eq. 8) is computed over token sets of concatenated object name + affordance + reasoning, any change in requested object category, output length, or instruction vocabulary directly lowers J between conditions. P5’s single-object output alone caps Jaccard vs. three-object outputs near 1/3. The paper itself lists “same-prompt, different-image” and “prompt paraphrase sensitivity” as remaining controls (§4.4, §6.6)—i.e., the confound is acknowledged and untested. 1−J is also not a variance decomposition, so even with clean prompts the 90% figure is an overlap statistic, not a causal share. The qualitative finding that VLMs follow persona instructions is plausible and consistent with cited robotics work, but the headline magnitude is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript characterizes context-dependent affordance computation in vision-language models (VLMs). Using 479 COCO images and seven persona prompts (neutral, chef, security, child, wheelchair-user, urgent, leisure), it asks Qwen3-VL-30B-A3B to output JSON affordance descriptions and computes pairwise Jaccard similarity across primes. The mean word-level Jaccard of 0.095 is interpreted as '>90% of lexical scene description is context-dependent'; sentence-level cosine similarity of 0.415 is interpreted as 58.5% context-dependence. A LLaVA-1.5-13B replication, stochastic sampling controls, and Tucker decomposition of sentence embeddings are reported, and the paper proposes a 'semantic-first' processing architecture and Just-In-Time Ontology for robotics. The paper explicitly limits its claims to behavioral output and lists several untested controls.","tokens_in":20644,"tokens_out":9123,"duration_ms":74680,"significance":"The qualitative phenomenon—that a VLM's reported objects and affordances shift when the user prompt changes—is plausible and consistent with prior work on instruction-following and task-conditioned affordance ranking. The paper has notable strengths: public code/data, a large pairwise sample, cross-model replication, stochastic baselines, bootstrap stability analysis, and unusually explicit statements about what is not claimed. If the 90% magnitude were valid, it would be an important quantitative anchor for debates about static versus dynamic scene representations in robotics and cognitive AI. However, the central magnitude is not currently established: the primes differ in surface task demands, the key same-prompt/paraphrase controls are acknowledged as future work, and 1−Jaccard is not a variance-explained quantity. The headline contribution therefore requires substantial additional support.","major_comments":[{"comment":"The headline inference from J̄=0.095 to '>90% of functional scene ontology is context-dependent' requires that the seven primes isolate agent context. They do not. P0 asks for the 3 most prominent objects, geometric properties, and standard functions; P1–P4 ask for 3 role-specific critical/interesting objects; P5 asks for a single survival tool; P6 asks what 'catches your eye.' Word-level Jaccard (Eq. 8) is computed over token sets of concatenated name+affordance+reasoning, so changes in requested object count, output length, and instruction vocabulary directly deflate J; P5's single-object output alone caps overlap with three-object outputs near 1/3. The paper itself lists same-prompt/different-image and prompt-paraphrase sensitivity as 'Remaining Controls for Future Work' (§4.4, §6.6), i.e., the control that would separate agent context from prompt wording is absent. As it stands, the","section":"§4.4, §6.6, Appendix A.1 (Table 12)"},{"comment":"The paper treats 1−J and 1−cosine as 'proportion of variance explained' and as 'Ctx-Dep.' percentages. Jaccard is a set-overlap coefficient; its complement is not a variance decomposition and cannot be interpreted as 'context explains approximately 90% of the variance.' The same holds for cosine similarity: 0.415 similarity does not imply 58.5% of variance is context-dependent. The only variance-explained quantity in the paper is the Tucker decomposition's 46.6%, and that is tensor-approximation variance, not a decomposition of the Jaccard contrast. This is a category error in the central statistic.","section":"§5.6, Table 10"},{"comment":"Table 7 explicitly defines 'Var. %' as each factor's share of captured variance (summing to 100% of explained variance). Yet §5.4 says Dim1 'accounts for only 0.9% of variance,' and §6.4 cites 'Tucker Dim1's low variance contribution (0.9%)' to argue geometry-invariant features are a small residual. Because only 46.6% of total tensor variance is explained, Dim1's 0.9% is a share of that captured variance, not of total variance. The architectural inference drawn from this number is therefore not supported by the reported decomposition.","section":"§5.4, §6.4, Table 7"}],"minor_comments":[{"comment":"The abstract reports LLaVA mean J=0.160, whereas Table 3 reports 0.1807; the abstract reports 2,384 stochastic inference runs, while §5.7 describes 7,000 runs. Please harmonize these numbers.","section":"Abstract vs. Table 3 / §5.7"},{"comment":"§6.6 refers to 'Lemmatized Jaccard' as an alternative metric, but §5.8/Table 10 implements stopword-filtered Jaccard, not lemmatization; Section 4.3.4 explicitly states that no lemmatization/stemming is used.","section":"§6.6 vs. §5.8"},{"comment":"The Visual Genome human baseline is not collected under goal priming, so it cannot validate context-dependent extraction; the comparison is suggestive only and should be labeled as such.","section":"§5.3"},{"comment":"The null threshold J<0.5 is arbitrary; since the observed J=0.095 is within 2–9× of the paper's own random-baseline estimate (J≈0.01–0.05, §4.4), a more informative benchmark would be random-pair or same-prompt overlap.","section":"§4.4, Table 2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is transparent and reproducible, but the central quantitative claim is confounded by prompt-wording variation and by the misinterpretation of 1−J as variance explained. A major revision requiring the missing same-prompt and paraphrase controls is appropriate; if those controls cannot be supplied, the paper should be rejected. The theoretical framework is largely analogical, and the machine-learning contribution would be modest without the 90% magnitude claim, but the qualitative finding and the explicit scope limitations give the authors a reasonable path to a defensible paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The 90% headline doesn't survive contact with the actual prompts. The Jaccard drop is at least partly instruction-following, not measured agent-context drift. That said, this is not a careless paper: it is unusually honest about its limitations, it ships code and data, and the qualitative phenomenon—VLMs produce different affordance lists when you change the task instruction—is real and consistent with the robotics literature it cites.\n\nWhat is actually new: the attempt to quantify context-dependence at scale, the cross-model replication with LLaVA, the Tucker decomposition with bootstrap stability (Culinary and Access factors are interpretable), and the stochastic control idea (within-prime vs cross-prime variance). The paper explicitly disclaims processing order, lists the missing controls in Sections 4.4 and 6.6, and acknowledges that the cosine and Jaccard numbers are metric-dependent. That is good scientific practice, and it earns the paper a real read.\n\nThe soft spots are load-bearing for the central number. 1 minus Jaccard is a pairwise overlap statistic, not a variance decomposition; the claim in Section 5.6 that context \"explains approximately 90% of the variance\" is unsupported. More seriously, the seven primes differ in surface task demands (Appendix A.1): P0 asks for geometric properties, P1–P4 ask for role-specific items with a count, P5 drops the count entirely and asks \"what do you see first,\" P6 asks what catches your eye. Word-level Jaccard over token sets therefore drops for reasons that have nothing to do with persona—P5 alone caps Jaccard with three-object outputs near 1/3. The controls needed to separate persona from prompt wording (same-prompt/different-image, prompt paraphrase sensitivity) are explicitly deferred. The run-count inconsistency (7,000 in Section 5.7 vs 2,384 in the abstract and Section 6.6) is a concrete red flag. The semantic cosine number (58.5%) is less sensitive to exact lexical surface, but it still inherits the task-demand confound.\n\nIf the claim were rewritten as \"affordance lists vary with task instructions,\" that would be a solid, useful empirical statement, already partly established by RoboPoint, VoxPoser, and the TAR/CGR work. The Tucker factors are a nice exploratory addition but not a load-bearing contribution.\n\nThis paper is for researchers studying VLM behavior and robotic affordance grounding. I would not cite the 90% figure without heavy caveats, but I would send this to a serious referee: the topic is important, the confound is fixable with paraphrase and same-prompt controls, and the author's transparency makes a good revision possible.","headline":"The 90% context-dependence headline is confounded by prompt surface differences and should not be cited as-is, but the paper is transparent, ships code/data, and has a real qualitative finding; it deserves a serious referee.","tokens_in":21131,"tokens_out":2809,"would_cite":false,"duration_ms":26794,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Vision-language models describe the same image with less than ten percent lexical overlap when the observer's goal changes, and the paper argues this context-dependence reflects genuine computation rather than sampling noise.","keywords":["vision-language models","affordances","context-dependent processing","scene understanding","functional semantics","Jaccard similarity","Tucker decomposition","just-in-time ontology"],"falsifier":"A decisive check is the same-prompt/different-persona control: keep the wording of the prime fixed and vary only the named persona, or vary the wording while keeping the persona fixed. If pairwise Jaccard similarity follows the wording rather than the persona, the 90 percent drift is prompt sensitivity, not agent context; if it follows the persona, the paper's interpretation survives.","tokens_in":20166,"feed_emoji":"🤖","tokens_out":10157,"duration_ms":85398,"temperature":0.7,"pith_summary":"The paper tries to establish that vision-language models do not produce a stable, context-free description of a visual scene: the same image receives a different functional reading depending on who the model is told is looking. Affordances — the action possibilities a scene offers an agent — are computed in a strongly context-dependent way: across 479 images and seven agent personas, word-level Jaccard similarity (a standard overlap measure between token sets) averaged 0.095 between contexts, meaning more than 90 percent of the lexical scene description shifts with context; sentence-level cosine similarity was 0.415, so about 58.5 percent of semantic content shifts as well. The drift survives replication on a second model and survives stochastic sampling controls, and tensor decomposition reveals stable latent factors such as a culinary-oriented dimension and an access dimension. The authors present this as behavioral evidence for context-dependent affordance computation and draw a design consequence for robotics: query-dependent, just-in-time ontologies rather than static world models, while explicitly declining to claim any processing order.","feed_headline":"Same image, seven goals, nine-tenths of the description changes","feed_subtitle":"Vision-language models reweight what a scene affords depending on who is looking; the lexical overlap between contexts is under 10 percent.","key_machinery":"The carrying mechanism is a controlled comparison: the same image is prompted with seven agentic persona primes; outputs are parsed into object/affordance lists; and pairwise Jaccard similarity over word tokens and object names measures how much of the functional scene ontology changes with context. Stochastic baselines across four temperatures and five seeds separate genuine context effects from sampling noise, and Tucker decomposition — a three-mode factor analysis separating image, context, and embedding dimensions — reveals the latent structure of the shift. The Jaccard coefficient is the load-bearing statistic: the claim that more than 90 percent of lexical scene description is context-","core_discovery":"The central discovery is 'affordance drift': the set of objects and uses a vision-language model reports for one image changes massively when the prompt changes the agent's identity or goal, from neutral observer to chef, security professional, child, wheelchair user, urgent survivor, or leisured explorer. The mean word-level Jaccard similarity between context conditions is 0.095 (95% CI [0.093, 0.096], p<0.0001), object-level 0.119, and a replication on a second, architecturally distinct model gives 0.181. Because sentence-embedding cosine similarity is much higher (0.415), the authors conclude that surface vocabulary shifts more than underlying meaning, leaving a semantic invariant core. T","pith_inferences":["Extension: the paper's own list of remaining controls — same prompt with different personas, and prompt paraphrase — is the decisive test; if those controls show low drift, the 90% figure is surface prompt sensitivity, not agentic context.","Extension: the roughly 40% semantic overlap that survives context shifts suggests a hybrid architecture: a small stable geometric core plus a large dynamic task projection, rather than the pure just-in-time alternative the paper sketches.","Extension: the access axis separating child from wheelchair-user contexts offers a concrete testbed for the paper's untested action-distance prediction: compare spatial reasoning or navigation judgments under the two personas and see whether behavior tracks Euclidean or action-based distance."],"forward_implications":["If the drift is real, vision pipelines that first build a context-free geometric representation are spending most of their computation on the roughly 10 percent of functional signal that is invariant across tasks.","The lexical-versus-semantic gap (90.5% vs 58.5% context-dependent) implies that scene meaning retains a substantial invariant core even as vocabulary changes, so context re-weights salience rather than creating or destroying affordances.","The stable orthogonal factors (a culinary manifold and an access axis) mean context-dependent affordance structure is organized around functional categories, not diffuse variation.","The effect replicates across two architecturally distinct vision-language models, suggesting it is a general property of models trained on naturalistic image-text data.","For embodied AI, the result motivates just-in-time ontology: construct task-specific functional projections at query time rather than maintaining a single static world model, though the paper labels this a suggested direction rather than a proven design."],"fun_headline_variants":["What a scene affords shifts 90% with who's looking","Affordance drift: 90% of VLM scene descriptions change with context","VLM scene descriptions share only 9.5% overlap across goals","Meaning stays 60% stable while vocabulary shifts 90%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the seven differently worded prompts really create seven different agent contexts; because the paper has not yet run the same-prompt/different-persona and paraphrase controls it lists as future work, the 90 percent drift could in principle be instruction-following surface variation.","fun_headline_variants_meta":{"raw":{"variants":["What a scene affords shifts 90% with who's looking","Affordance drift: 90% of VLM scene descriptions change with context","VLM scene descriptions share only 9.5% overlap across goals","Meaning stays 60% stable while vocabulary shifts 90%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001064,"raw_usage":{"total_tokens":4376,"prompt_tokens":902,"completion_tokens":3474,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":646,"completion_tokens_details":{"reasoning_tokens":3409}},"tokens_in":646,"tokens_out":3474,"duration_ms":21830,"temperature":1.0,"reasoning_tokens":3409,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T23:28:39.526615+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check is the same-prompt/different-persona control: keep the wording of the prime fixed and vary only the named persona, or vary the wording while keeping the persona fixed. If pairwise Jaccard similarity follows the wording rather than the persona, the 90 percent drift is prompt sensitivity, not agent context; if it follows the persona, the paper's interpretation survives.","supporting_citations":[],"review_version":1}