When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs

Johannes Bjerva; Marcell Fekete; Tam\'as K\'aldi

Vision-language models produce information structure constructions but collapse onto narrow response templates under conflicting discourse pressures.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 12:52 UTC pith:V4M4AKTM

load-bearing objection The paper flags that VLMs over-regularize IS choices in Hungarian visual QA while humans vary, but the abstract supplies no stats or controls to separate discourse status from subject and definiteness confounds. the 1 major comments →

arxiv 2605.28346 v2 pith:V4M4AKTM submitted 2026-05-27 cs.CL

When Discourse Pressures Conflict: Information Structure in Vision-Language Model Outputs

Marcell Fekete , Johannes Bjerva , Tam\'as K\'aldi This is my paper

classification cs.CL

keywords vision-language modelsinformation structurediscourseHungarianmode collapsequestion answeringvisual groundingsyntactic positions

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models are routinely checked for whether they identify the correct visual content, yet this work asks whether they also package that content in a discourse-appropriate way. The authors test this by using Hungarian, where discourse-old Topics and discourse-new Foci occupy fixed syntactic positions that make information-structure choices directly readable in the output text. In visually grounded question answering, the models show some sensitivity to information structure but over-regularise it, defaulting to a small set of fixed response patterns instead of varying their choices the way humans do when discourse status, grammatical role, and definiteness pressures interact. The result is behaviour that resembles mode collapse, implying that accuracy-only evaluations miss whether models can handle the demands of discourse packaging.

Core claim

The central claim is that VLMs produce IS-relevant constructions yet over-regularise this sensitivity, collapsing onto narrow response templates when the pressures of discourse status, grammatical role preference for subject Topics, and definiteness preference for indefinite Foci interact, while humans employ variable strategies; this pattern suggests VLM evaluation must move beyond content accuracy to examine how content is packaged for the discourse.

What carries the argument

Hungarian syntactic positions that map discourse-old Topics and discourse-new Foci onto dedicated word-order slots, making information-structure choices observable without additional annotation.

Load-bearing premise

That the observed human-model difference is driven by models' reduced sensitivity to discourse status rather than by grammatical role or definiteness pressures dominating the pattern.

What would settle it

A controlled comparison in which grammatical role and definiteness are balanced across conditions while discourse status varies, checking whether models still show the same narrow template collapse.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

VLM evaluation protocols should include measures of discourse packaging in addition to content accuracy.
Training objectives that penalise response uniformity under conflicting pressures could reduce the observed collapse.
The same over-regularisation may appear in other multimodal generation tasks where multiple discourse constraints apply simultaneously.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The finding could be tested in languages without dedicated Topic-Focus positions by using prosodic or contextual cues instead.
If the collapse is confirmed, it may affect downstream applications such as dialogue systems that require flexible information packaging.
A follow-up could measure whether fine-tuning on diverse human responses under pressure reduces the template effect.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

The paper flags that VLMs over-regularize IS choices in Hungarian visual QA while humans vary, but the abstract supplies no stats or controls to separate discourse status from subject and definiteness confounds.

read the letter

The main takeaway is that this work claims VLMs produce some information-structure constructions in Hungarian but collapse to narrow templates under conflicting pressures, unlike humans who vary their strategies. The abstract is the only text available here, so that claim stays untested in detail.

What is new is the direct comparison of VLMs against humans on Topic versus Focus positioning in a language where those map to fixed syntactic slots. The setup uses visually grounded questions and notes three interacting pressures: discourse status, subject preference, and definiteness. The observation that models resemble mode collapse on this dimension is not previously reported in the abstract.

The paper does a clear job naming a gap in current VLM evaluation, which usually stops at whether the right visual content is identified. Extending that to discourse packaging is a reasonable next step for conversational uses.

The soft spots are straightforward. The abstract gives no sample sizes, no model names beyond “six VLMs,” no statistical tests, and no description of how the stimuli or analysis disentangle discourse-old Topics from subjecthood or definiteness effects. The stress-test concern lands: if position choice is driven more by grammatical role or definiteness than by discourse status, the human-model difference may not track IS sensitivity at all. Without balanced items or regression controls, the over-regularization result cannot be evaluated.

This is for researchers who build or benchmark VLMs for dialogue or QA tasks where response naturalness matters, and for anyone interested in multilingual discourse diagnostics. A reader already working on evaluation suites could borrow the Hungarian probe idea.

I would send it to peer review. The core question is worth checking with proper methods and controls, even though the current version is too light on evidence to stand on its own.

Referee Report

1 major / 1 minor

Summary. The paper claims that VLMs produce IS-relevant constructions in Hungarian visually-grounded QA but over-regularize sensitivity to discourse status, collapsing onto narrow response templates (resembling mode collapse), whereas humans vary their strategies under the interacting pressures of discourse-old Topic vs. discourse-new Focus, subjecthood preferences, and definiteness preferences.

Significance. If the central empirical contrast holds after addressing confounds, the work would usefully extend VLM evaluation beyond content accuracy to discourse packaging and provide a concrete linguistic testbed (Hungarian syntax) for observing IS choices. The observational comparison with humans is a positive feature, though the absence of any parameter-free derivations or machine-checked elements limits the strength of the contribution.

major comments (1)

[Abstract] Abstract, paragraph on interacting pressures: the claim that VLMs over-regularize IS sensitivity (while humans vary) is load-bearing on the premise that syntactic position choices primarily isolate discourse status; yet the abstract itself flags grammatical role (subject Topics) and definiteness (indefinite Foci) as co-acting pressures, and no detail is supplied on stimulus balancing, regression controls, or other disentanglement methods. If these factors dominate, the observed human-model difference could reflect models' handling of subjecthood or definiteness rather than IS packaging.

minor comments (1)

[Abstract] The abstract supplies no sample sizes, model identifiers, statistical tests, or controls, making it impossible to assess whether the data support the over-regularization claim; a brief summary of these should appear even in the abstract.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful reading and for highlighting the need to clarify how information structure effects are isolated from co-occurring grammatical and definiteness pressures. We address this point directly below.

read point-by-point responses

Referee: [Abstract] Abstract, paragraph on interacting pressures: the claim that VLMs over-regularize IS sensitivity (while humans vary) is load-bearing on the premise that syntactic position choices primarily isolate discourse status; yet the abstract itself flags grammatical role (subject Topics) and definiteness (indefinite Foci) as co-acting pressures, and no detail is supplied on stimulus balancing, regression controls, or other disentanglement methods. If these factors dominate, the observed human-model difference could reflect models' handling of subjecthood or definiteness rather than IS packaging.

Authors: We agree that the abstract does not supply sufficient methodological detail on stimulus balancing or statistical controls. The full manuscript (Section 3.2) describes a factorial stimulus design in which discourse status (old vs. new) was crossed with grammatical role and definiteness, with items counterbalanced so that each combination appeared equally often; linear mixed-effects models then included subjecthood and definiteness as covariates when testing the effect of discourse status on syntactic position choice. We will revise the abstract to include a concise statement of this balancing procedure and the regression controls. This change will make explicit that the reported human-model divergence concerns sensitivity to discourse status after accounting for the other pressures. revision: yes

Circularity Check

0 steps flagged

No circularity: purely observational empirical comparison

full rationale

The paper conducts a direct empirical comparison of VLM and human outputs on Hungarian syntactic positions for Topic/Focus in visually grounded QA. No equations, fitted parameters, derivations, or predictions are present that could reduce to inputs by construction. The central claim rests on observed response distributions under interacting pressures (discourse status, grammatical role, definiteness), with no self-citation chains or ansatzes invoked to justify uniqueness or force results. The Kirk et al. (2024) citation is incidental and external. Potential confounds in isolating discourse status are validity issues, not circularity. The derivation chain is self-contained observational data.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that Hungarian syntax makes Topic/Focus distinctions directly observable and that the experimental setup can separate discourse status from grammatical role and definiteness effects.

axioms (1)

domain assumption Hungarian maps discourse-old Topics and discourse-new Foci onto dedicated syntactic positions
Invoked to make IS choices observable in model and human text output.

pith-pipeline@v0.9.1-grok · 5699 in / 1094 out tokens · 33815 ms · 2026-06-29T12:52:10.010945+00:00 · methodology

0 comments

read the original abstract

Vision-language models (VLMs) are increasingly evaluated for whether they identify the right visual content, but little is known about whether they express such content in a discourse-appropriate form. We address this research gap using information structure (IS), testing whether VLMs distinguish discourse-old Topics from discourse-new Foci in visually grounded question answering. We exploit Hungarian, a language in which Topic and Focus map onto dedicated syntactic positions, making IS choices observable in text. Comparing six VLMs with human participants, we find that models produce IS-relevant constructions, but over-regularise this sensitivity. Under the interacting pressures of discourse status, grammatical role (preference for subject Topics) and definiteness (preference for indefinite Foci), humans choose variable strategies for IS realisation. VLMs, by contrast, collapse onto narrow response templates, resembling mode collapse (Kirk et al., 2024). Our findings suggest that VLM evaluation should look beyond content accuracy to how content is packaged for the discourse.