{"id":"f101a6d6-8304-4368-b5ba-540790bfb0af","arxiv_id":"2608.08663","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A listener-side symbolic layer using three pact-state sets and a web-image retrieval pipeline grounds referring expressions at 67.8% top-5 accuracy after leakage filtering, below human top-1.","lead":"This paper builds a machine listener that tracks what a speaker and listener have agreed to call each object, using three explicit sets of pact states updated by formal rules. It evaluates the system on a large corpus of human tangram-description games and reports accuracy that is above chance but below human performance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.4's update rules, read literally, erase prior committed pacts on each new commitment; the symbolic pact layer cannot accumulate state, and no multi-turn experiment tests it.","rationale":"The paper is unusually candid: Sections 5.2, 6.1, and 6.3 explicitly disclaim interactivity, recovery, and full entrainment, and Section 4.3 quantifies leakage. In good faith, the contribution is a component, not a complete entrainment system. However, the abstract and Section 1 still make the stronger claim that the framework 'recovers the structure of lexical entrainment turn by turn.' That claim requires the formal update layer to work as specified and to be exercised. The Section 3.4 equations, on the paper's own definition of Γ as a set of pacts, use ∩ where ∪ is needed; the worked example's 'Γ gains' language confirms the intended semantics but contradicts the equations. This is an internal inconsistency in the core contribution, not a disagreement with the field. Additionally, the evaluation never runs the bookkeeping across turns: all reported numbers are single-utterance top-k, so the entrainment-structure claim is unsupported by direct evidence. The retrieval-leakage concern identified by the reader is real and is handled transparently, but the more fundamental problem is that the symbolic layer—the claimed novel component—is neither correctly specified nor empirically exercised. These issues are addressable (fix the update operations; add a multi-turn evaluation), so the conditional verdict remains appropriate, but the conditions should be stated explicitly.","tokens_in":14191,"tokens_out":10347,"duration_ms":108733,"concrete_test":"Trace Eqs. (1)–(3) on a two-turn toy: turn 1 commits (r1←oA), turn 2 commits (r2←oB). If Γ after turn 2 is {(r2←oB)} rather than {(r1←oA),(r2←oB)}, the rules as written cannot maintain pact state; the fix is to use ∪ for new commitments and ∩ only for narrowing the hypothesis context.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that the Γ/Ξ/Ω layer 'recovers the structure of lexical entrainment turn by turn' requires that the update rules accumulate commitments across turns and that the system is actually evaluated over sequences of turns. Neither holds. Section 2.4 defines Γ as the set of pacts the MCP has committed to, yet Eq. (1) updates it as Γ ← Γ ∩ □(rφ ← o_i). On the set-of-pacts reading, intersecting with the singleton {□(rφ ← o_i)} deletes every prior pact; accumulation requires ∪, not ∩. The same issue affects Ω in Eq. (3) and Ξ in the pre-commit rule. Section 3.5's worked example uses union language ('Γ gains □(rφ3 ← o_A)'), directly contradicting Eq. (1). Independently, the experiments (Tables 1–3) report only single-utterance top-k scores; no experiment runs the multi-turn bookkeeping, so the 'turn by turn' recovery claim is asserted rather than demonstrated. The paper's own Sections 5.2 and 6.3 concede both the absence of interactivity and the lack of any recovery mechanism, but the abstract and Section 1 still claim the framework recovers entrainment structure. If the equations are read as context-of-worlds updates, then Γ is not a set of pacts and the 'inspectable pact state' contribution is misdescribed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a symbolic listener-side framework for grounding referring expressions in the Stanford Repeated Reference Game corpus. A dynamic-semantics layer maintains three sets of pact bindings (Γ, Ξ, Ω) updated by context-change rules, and a perceptual layer converts director utterances into web-image queries, aligns retrieved images to tangram candidates via SIFT homographies, and scores them with UQI. The authors report top-5 single-utterance accuracy of 83.56% (unfiltered) and 67.8% under a conservative leakage filter, with top-1 accuracy of 41.66% and 29.2%, respectively, against a human top-1 baseline of roughly 77–80%. They also present ablations of the retrieval pipeline and an explicit internal-validity discussion of retrieval leakage, and they position the contribution as an inspectable, auditable pact-state bookkeeping layer that could be integrated into interactive emergent-communication or RSA-style systems.","tokens_in":14502,"tokens_out":5678,"duration_ms":61826,"significance":"If the claims were fully supported, the paper would supply a useful, transparent component for a problem that recent LVLM studies identify as a gap: stable, inspectable pact-state tracking. The manuscript is unusually honest: it reports the previously missing ablation tables, includes a conservative leakage-filtered analysis, explicitly quantifies the retrieval-leakage threat, and releases code. The main contribution is the symbolic Γ/Ξ/Ω layer together with a reproducible perceptual-alignment pipeline. However, the significance as stated is limited by two gaps: the formal update rules as written do not obviously implement accumulation of pacts, and the empirical evaluation is single-utterance only, so the central 'turn by turn recovery' claim is not demonstrated. With appropriate revisions to the formalism, the experiment, and the claims, the component could be a valuable contribution to multimodal grounding and emergent-communication research.","major_comments":[{"comment":"Under the stated interpretation of Γ as 'the set of pacts the MCP has committed to,' the update Γ ← Γ ∩ □(rφ ← o_i) is internally inconsistent with accumulation: intersecting a set of pacts with a singleton deletes every prior pact. The same issue affects Ω in Eq. (3), and the Ξ update in Eq. (2) similarly removes rather than accumulates. Section 3.5's worked example uses union-like language ('Γ gains □(rφ3 ← o_A)'), and Section 2.7 says the layer 'accumulates hypothesized pacts over turns.' If the ∩ notation is intended as a context-of-worlds update, then Γ/Ξ/Ω are not sets of pacts and the paper's 'inspectable pact state' description is misleading. This is the central formal contribution, and the inconsistency must be resolved before the symbolic layer can be assessed.","section":"Section 3.4, Eqs. (1)–(3); Section 3.5; Section 2.7"},{"comment":"The abstract and introduction claim that the framework 'recovers the structure of lexical entrainment turn by turn,' but the experiments in Tables 1–3 and Section 4 evaluate only single director utterances; no multi-turn sequence is passed through the Γ/Ξ/Ω bookkeeping. Sections 5.2 and 6.3 explicitly concede that the system is not interactive and that no recovery mechanism exists. The 'turn by turn' claim is therefore asserted rather than demonstrated. Either remove it from the abstract and contribution list, or add an actual multi-turn evaluation on the repeated-reference corpus that tracks commitment, refinement, and pruning across consecutive utterances.","section":"Abstract and Section 1 vs. Sections 4, 5.2, 6.3"},{"comment":"All accuracy numbers are point estimates from a single retrieval pass, and the hyperparameters (k, the UQI threshold ε, the augmentation configuration, the similarity metric, and the 0.85 conservative-filter threshold) are selected by performance on the same evaluation corpus. The paper acknowledges variance but still reports 83.56% and 67.8% as headline figures. Without bootstrap confidence intervals, multiple independent retrieval runs, or a held-out hyperparameter split, the empirical claims are not quantitatively robust; the conservative condition, in particular, is defined by a threshold that is itself tuned on the data being scored.","section":"Sections 4.2, 4.3, and 6.4"},{"comment":"The comparison of MCP top-5 accuracy (83.56%) with the human top-1 baseline (77–80%) is an apples-to-oranges comparison: human top-k scores for k > 1 are not available, and top-1 is a stricter success criterion than top-5. The statement in Section 5.1 that the unfiltered top-5 'matches or slightly exceeds the human top-1 baseline' should be either removed or accompanied by the explicit caveat that the two metrics are not directly comparable, and the abstract should not juxtapose the 83.56% top-5 figure with the human top-1 figure without that caveat.","section":"Table 3 and Section 5.1"}],"minor_comments":[{"comment":"The caption contains a typo: 'T able 1' should read 'Table 1.'","section":"Table 1 caption"},{"comment":"The sentence 'The common ground is the set of possible worlds consistent with Γ∩Ξ∩Ω' is unclear because Γ, Ξ, and Ω are defined as sets of pacts, not sets of possible worlds; the formal type of each set and the meaning of '∩' between them should be clarified.","section":"Section 2.4"},{"comment":"The 16% and 32% gaps used as proxies for utterances that cannot be grounded are acknowledged as proxies, but the text should state explicitly that these are estimates derived from the accuracy gaps, not from a direct annotation of ungroundable utterances.","section":"Section 5.3"},{"comment":"The citation 'Anonymous: Lvlms and humans ground differently in referential communication' should be replaced with the full author list and venue information if available, so that readers can locate the work.","section":"Reference [15]"},{"comment":"The note 'Chance on 12 tangrams is 8.33%' applies to top-1 accuracy only; for top-k accuracy with k>1, chance-level performance is k/12 and should be reported separately to help readers interpret the top-3 and top-5 numbers.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript contains repeated references to 'earlier drafts' and to anonymous reviewer probes; this is unusual in a submission but reads as an attempt at transparency rather than as a substantive problem. The main issue is formal: the update rules in Section 3.4 appear inconsistent with the accumulation language used elsewhere, and the 'turn by turn' recovery claim is not tested. I see no evidence of misbehavior; the self-critical limitations section is a strength. I would encourage the authors to fix the formalism and either provide a multi-turn experiment or substantially soften the entrainment-recovery claim before resubmission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things worth knowing about arXiv:2608.08663. It is a genuinely honest paper — it quantifies its own leakage, prints ablations for every design choice, and explicitly retracts earlier overclaims. It is also, read closely, formally inconsistent at its core: the update rules erase the very state they are supposed to accumulate, and no experiment tests the multi-turn behavior that the abstract claims.\n\nWhat is new: the idea of a listener-side symbolic bookkeeping layer (Γ/Ξ/Ω) that externalizes pact state in an auditable way, sitting on top of a classical perceptual pipeline (Bing retrieval, SIFT homography, UQI). Single-utterance grounding results are modest but the evaluation is unusually transparent — top-5 is 83.56% unfiltered, 67.8% under a conservative leakage filter, versus human top-1 of 77–80%. The ablations in Tables 1–2 are a genuine strength.\n\nThe soft spot is in Section 3.4. Γ is defined as the set of committed pacts, then updated via Γ ← Γ ∩ □(rφ←o_i). Intersection with a singleton deletes any prior commitments; accumulation requires ∪. The same applies to Ω and Ξ. The worked example says 'Γ gains □(rφ3←o_A)', which is union language — a direct contradiction. If you reinterpret Γ as a set of possible worlds, the intersection standard makes sense, but then Γ is no longer an inspectable set of pacts, which is the paper's stated contribution. Either way, the formal account does not hold up.\n\nOn top of that, all experiments are single-utterance top-k. No multi-turn run exercises the bookkeeping, so the abstract's 'recovers the structure of lexical entrainment turn by turn' is unsupported. The Limitations sections concede the lack of interactivity and recovery, which makes the abstract overreach.\n\nThe empirical numbers are point estimates with hyperparameters tuned on the evaluation corpus and no error bars; the conservative filter is a reasonable sensitivity analysis, not a rigorous held-out design. The paper acknowledges most of this.\n\nWho this is for: people working on common-ground tracking and human-AI reference games, and anyone who wants a good example of honest evaluation practice. It deserves a serious referee because the motivating gap is real and the author's candor is valuable, but it needs major revision: fix or rewrite the update rules, either provide multi-turn evidence or soften the claim, and report variance. My recommendation: engage with it, but do not take the formal layer at face value.","headline":"A well-meant, transparent component paper whose central formal claim is undercut by an internal set-operation bug and a missing multi-turn experiment.","tokens_in":15014,"tokens_out":6935,"would_cite":false,"duration_ms":68778,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that explicit, inspectable pact-state bookkeeping — three sets of referent-object bindings updated by dynamic-semantics rules — is the missing component for machine grounding, and shows that one utterance puts the right…","keywords":["lexical entrainment","conceptual pacts","dynamic semantics","referring expressions","multimodal grounding","visual grounding","tangram reference game","Universal Quality Index"],"falsifier":"Re-run the pipeline with a retrieval corpus built to exclude all images resembling the target tangrams; if top-5 accuracy falls to the 8.33% chance level under that exclusion, the reported grounding signal is an artifact of retrieval memorization rather than cross-modal grounding.","tokens_in":13981,"feed_emoji":"🧩","tokens_out":7406,"duration_ms":69774,"temperature":0.7,"pith_summary":"The paper argues that the missing ingredient for machines that must interpret human referring expressions is not better perception but explicit bookkeeping of what the listener and speaker have agreed to call each object. It proposes a listener-side layer that maintains three sets of pact bindings — committed ($\\Gamma$), under negotiation ($\\Xi$), and ruled out ($\\Omega$) — and updates them with a dynamic-semantics context-change rule after every utterance. Stacked on a perceptual channel that aligns web-retrieved images to abstract silhouettes, the framework places the correct target in its top-5 hypothesis set 83.56% of the time from a single utterance, or 67.8% after removing retrievals that could be memorized. The defensible claim is deliberately narrow: a transparent, auditable symbolic component for pact tracking, not a full interactive entrainment system.","feed_headline":"Pact-state layer grounds tangram references at 83.56 percent top-5","feed_subtitle":"Three transparent sets track agreed names turn by turn, beating chance while staying below human top-1.","key_machinery":"The load-bearing object is the triple $\\Gamma, \\Xi, \\Omega$ together with the update rule $C[\\varphi] = C \\cap \\Diamond B$, where $B$ is the set of candidate referent-object bindings surviving perceptual alignment. $\\Diamond$ (might) adds candidates to $\\Xi$, $\\square$ (must) commits a unique binding to $\\Gamma$, and $\\square\\neg$ moves rejected bindings into $\\Omega$; the rule makes the common ground inspectable as three explicit sets rather than an opaque embedding. The perceptual layer supplies $B$ by converting each utterance into a search query, retrieving images, aligning them to each tangram with a scale-invariant feature transform homography, and thresholding the Universal Quality Index comparison, with rotation and grayscale-inversion augmentation handling mirror and polarity variation.","core_discovery":"The central discovery is that lexical entrainment's pact state can be externalized into three explicit sets — $\\Gamma$ for finalized commitments, $\\Xi$ for hypotheses still under negotiation, and $\\Omega$ for rejected bindings — with a context-change-potential update rule that narrows $\\Xi$ by intersection across turns and promotes a binding to $\\Gamma$ once a single hypothesis survives. On top of this symbolic layer, a perceptual pipeline estimates the hypothesis set for each utterance by retrieving crowd-sourced images, aligning them to each candidate with a scale-invariant feature transform, and scoring alignment quality with the Universal Quality Index. The paper shows this combination recovers the structure of lexical entrainment turn by turn and grounds a single director utterance with 83.56% top-5 accuracy, dropping to 67.8% under a conservative leakage filter designed to strip out retrieved copies of the target itself.","pith_inferences":["A reader could expect the $\\Gamma/\\Xi/\\Omega$ layer to transfer directly to other referent types — actions, plans, abstract concepts — because the update semantics is agnostic about what the similarity function measures; the bottleneck is the perceptual channel, not the bookkeeping.","Because the commit rule is deterministic and monotone, the framework would fail to recover from an early false commitment; adding probabilistic commit thresholds is a natural extension that the paper itself flags.","The conservative filter is a proxy for leakage; a stronger test would compare against a learned perceptual similarity baseline and use a retrieval corpus constructed to exclude the target stimulus class entirely, which the paper identifies as the most informative next experiment."],"forward_implications":["If the framework is right, vision-language agents can carry an inspectable, auditable record of what has been agreed with a partner, directly addressing the absence of stable pact state documented in current neural systems.","The 83.56% top-5 result, and the 67.8% figure after conservative filtering, imply that a genuine cross-modal grounding signal survives even when obvious memorization routes are removed; both are well above the 8.33% chance level.","The ablation results isolate the contribution of each component, so future systems know that query preprocessing, SIFT alignment, UQI scoring, and rotation/inversion augmentation each earn their place.","The result is explicitly a component, not a closed system: a full entrainment model would need a generation-side partner model and interactive repair, which this paper does not attempt."],"supporting_citations":[{"why":"Supplies the repeated-reference-game corpus of over 15,000 director-matcher utterances used for evaluation and the human error-rate baselines.","marker":"[5]"},{"why":"Provides independent round-one human accuracy on the same corpus and documents that agent-agent dyads fail to form conventions, motivating the pact-state layer.","marker":"[9]"},{"why":"Supplies the dynamic-semantics notation and context-change-potential formalism on which the update rules are based.","marker":"[10]"},{"why":"Documents a metric suite showing that leading vision-language models fail to build common ground, a gap the paper's symbolic layer addresses.","marker":"[14]"},{"why":"Documents systematic failures of large vision-language models to shorten references, reuse expressions, and maintain pact state across turns.","marker":"[15]"},{"why":"Defines the Universal Quality Index that serves as the perceptual similarity measure between retrieved images and tangram silhouettes.","marker":"[28]"},{"why":"Describes scale-invariant feature matching used for homography alignment of retrieved images to tangram targets.","marker":"[27]"},{"why":"Provides the image-retrieval tool that yields the external visual prior for each query.","marker":"[12]"},{"why":"Frames the contribution as a listener-side bookkeeping component within the broader emergent-communication modeling tradition.","marker":"[7]"}],"fun_headline_variants":["Three-set pact state yields 83.56% top-5 on tangram reference game","Externalized pact state: transparent, auditable, 83.56% top-5","SIFT-grounding plus three pact sets: 83.56% top-5","Pact-state leakage quantified; top-5 hits 83.56%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the web images retrieved for each phrase, after preprocessing, fairly sample what a human matcher would visualize, so that alignment between those images and a tangram indicates real grounding rather than retrieval of the corpus itself.","fun_headline_variants_meta":{"raw":{"variants":["Three-set pact state yields 83.56% top-5 on tangram reference game","Externalized pact state: transparent, auditable, 83.56% top-5","SIFT-grounding plus three pact sets: 83.56% top-5","Pact-state leakage quantified; top-5 hits 83.56%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000514,"raw_usage":{"total_tokens":2556,"prompt_tokens":1065,"completion_tokens":1491,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":681,"completion_tokens_details":{"reasoning_tokens":1400}},"tokens_in":681,"tokens_out":1491,"duration_ms":14174,"temperature":1.0,"reasoning_tokens":1400,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:28:26.787680+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the pipeline with a retrieval corpus built to exclude all images resembling the target tangrams; if top-5 accuracy falls to the 8.33% chance level under that exclusion, the reported grounding signal is an artifact of retrieval memorization rather than cross-modal grounding.","supporting_citations":[{"cited_title":"In: Proceedings of the 39th Conference on Neural Information Processing Systems (NeurIPS) Workshop: Dynamics of Behavior and Mind (2025).https://openreview.net/forum?id=0xWIIC0Ibh","cited_arxiv_id":null,"evidence_quote":"Provides independent round-one human accuracy on the same corpus and documents that agent-agent dyads fail to form conventions, motivating the pact-state layer."},{"cited_title":"Mind128(511), 795–835 (2019) https://doi.org/10.1093/mind/fzy076","cited_arxiv_id":null,"evidence_quote":"Supplies the dynamic-semantics notation and context-change-potential formalism on which the update rules are based."},{"cited_title":"Measuring How (Not Just Whether) VLMs Build Common Ground","cited_arxiv_id":"2509.03805","evidence_quote":"Documents a metric suite showing that leading vision-language models fail to build common ground, a gap the paper's symbolic layer addresses."},{"cited_title":"IEEE Signal Processing Letters9(3), 81–84 (2002) https://doi.org/10.1109/97.995823","cited_arxiv_id":null,"evidence_quote":"Defines the Universal Quality Index that serves as the perceptual similarity measure between retrieved images and tangram silhouettes."},{"cited_title":"pypi (Feb 10, 2022)","cited_arxiv_id":null,"evidence_quote":"Provides the image-retrieval tool that yields the external visual prior for each query."},{"cited_title":"Trends in Cognitive Sciences7(7), 308–312 (2003)","cited_arxiv_id":null,"evidence_quote":"Frames the contribution as a listener-side bookkeeping component within the broader emergent-communication modeling tradition."}],"review_version":1}