PhantomFill: When the Form Demands an Answer, Language Models Invent One

Rana Muhammad Usman

REVIEW 2 major objections 4 minor 14 references

This paper claims that structured output formats are not neutral containers: a required JSON field can flip near-perfect abstention into 100% fabrication on the same unanswerable input in ten of thirteen models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 11:36 UTC pith:MNI6YRZQ

load-bearing objection A rare controlled demonstration that the output schema itself flips abstention into fabrication; the core result holds, with a couple of reporting and interpretation caveats that should be fixed before publication. the 2 major comments →

arxiv 2607.20492 v2 pith:MNI6YRZQ submitted 2026-06-11 cs.LG cs.AIcs.CL

PhantomFill: When the Form Demands an Answer, Language Models Invent One

Rana Muhammad Usman This is my paper

classification cs.LG cs.AIcs.CL

keywords hallucinationabstentionstructured outputJSON schemarequired fieldsescape tokensLLM evaluationformat-induced fabrication

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the output format itself—not missing knowledge or a bad prompt—can force a language model to fabricate. Using the same unanswerable input and question, it varies only whether the model answers in free prose, in a JSON schema with an escape option, or in a required-field schema with no escape. Across thirteen models, free-text answers are often honest, but required fields drive fabrication to 100% in ten models, including one frontier model that flips from 2% to 100% fabrication. The result matters because production systems overwhelmingly ask models to fill forms, so free-text abstention scores systematically overstate deployed safety. The paper contributes the PhantomFill benchmark with two deterministic metrics—Coerced Fabrication Rate and Escape Utilization Rate—and shows that fabrication concentrates exactly where a hedge cannot fit.

Core claim

The central discovery is a causal flip controlled by output format, demonstrated with fixed input and question across three answer formats. GPT-5.5, which refuses in prose 98% of the time and never fabricates when an escape value is available, invents sentiment and quotes 40 times out of 40 when a required JSON schema leaves no room for "insufficient evidence." Across the thirteen-model matrix, ten models hit 100% coerced fabrication under the required-field rung, and an explicit escape option is ignored by all nine open-weight models and by Claude Sonnet. The only models with low fabrication achieve it by refusing the format itself—a refusal that a production parser reads as a crash. The pa

What carries the argument

The central object is the Abstention-Affordance Ladder, a three-rung design that fixes the input document and question and changes only the answer format: free prose, a JSON schema where every field can express insufficient evidence, and a required-field schema with no escape. The inputs are unanswerable by construction—a post with engagement counts but no reply text, a ticket whose call was never transcribed—so any concrete value in the target fields is a fabrication by definition. This makes scoring deterministic and isolates the output format as the causal variable. A fourth rung uses grammar-constrained decoding to remove the possibility of refusing in prose, showing the effect persists

Load-bearing premise

The load-bearing assumption is that the within-family Claude comparison—Haiku refuses, Sonnet fabricates, Opus refuses—can be read as evidence that coercion resistance is a training outcome, even though the three models differ in training data, alignment objectives, and vendor-side system prompts that are not controlled.

What would settle it

The decisive check is to find or train a single model that fills the required-field schema with no format violations and still gives honest insufficient-data values in all 40 trials; the paper's claim that the form itself forces fabrication would fall. A cheaper partial test is fine-tuning an open-weight model on structured-refusal examples and seeing whether its Coerced Fabrication Rate drops toward zero.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

Free-text abstention scores cannot be used to predict behavior in schema-constrained deployment; the paper's GPT-5.5 flip from 2% to 100% shows the two settings are different regimes.
Any required enum or minimum-count array without an escape value is a fabrication hotspot; a schema linter that flags such fields would catch the failure cells identified in the paper.
Prompt-level anti-hallucination instructions are not a reliable mitigation once a required schema is present; the schema wins in four of the six models tested in the instruction arm.
Structured refusal—a machine-readable object saying insufficient data—is the only tested response that keeps both honesty and format compliance, and the paper proposes it as the target behavior for required-field schemas.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Editorial inference: The selective escape use—spending the token only where it costs nothing—suggests models are optimizing task completion, not truth. If so, simply adding escape values to schemas will not fix smaller models; they need training signal that makes refusing a completed-looking answer the rewarded action.
Editorial inference: Because the failure is measured at the field level, the same construction should transfer to other high-stakes structured outputs, such as medical note templates or code-review comment generation, where required enums and minimum-count fields are common. This is a direct generalization the paper predicts but has not tested.
Editorial inference: CFR and EUR could be used as a cheap model-selection screen before deployment, but the paper does not offer threshold guidance; a natural extension is to publish per-domain CFR/EUR on model cards so buyers can see where honesty fails.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A rare controlled demonstration that the output schema itself flips abstention into fabrication; the core result holds, with a couple of reporting and interpretation caveats that should be fixed before publication.

read the letter

You should know this paper before the next time you design an evaluation or read a model card: it shows, with a genuinely controlled setup, that the required JSON schema itself can turn honest abstention into fabrication. The ladder design is the right tool—same input, same question, only the format changes—and the grammar-constrained decoding follow-up closes the main alternative explanation (prompted cooperation). I trust the headline number: GPT-5.5 goes from 2% free-text fabrication to 100% when you demand a required sentiment field, and the same model uses an offered escape only when taking it costs nothing. The field-level pattern under enforced decoding—zero escapes on the three load-bearing fields, twelve on the one where escaping concedes nothing—is sharp evidence that the option is visible and deliberately unused.

What is actually new: prior abstention benchmarks fix free text, format-restriction studies measure accuracy on answerable tasks, and structured-output benchmarks score correct extractions. This paper measures fabrication on fields that are unanswerable by construction, so the headline metrics are deterministic. That is a real contribution, and CFR and EUR are plausibly useful model-card numbers.

The soft spots are minor but need to be addressed. The abstract claims all nine open-weight models ignore the escape option, but Table 1 shows gemma4 26B at 60% CFR on the escape rung—that is, it uses the escape 40% of the time. That is a reporting error, not a threat to the causal inference. The within-family Claude comparison (Haiku refuses, Sonnet fabricates, Opus refuses) is read as evidence that training choices produce coercion resistance, but those models differ in training data, alignment objectives, release policy, and unknown CLI system prompts. That is an interpretive claim, not a controlled intervention. And the promised benchmark artifacts—generator, scorer, commit hash—are not actually linked in the release section, which makes independent replication harder than it should be.

None of this is load-bearing. The central claim stands: format alone is a causal lever for fabrication, and free-text abstention scores do not transfer to schema-constrained deployment. I would send this to peer review. Ask the authors to fix the abstract, add a confound caveat to Section 4.6, and publish the actual artifacts. This is worth reading for anyone building or evaluating schema-constrained LLM applications, and I would cite it in my own work.

Referee Report

2 major / 4 minor

Summary. The paper introduces PhantomFill, a benchmark designed to measure whether structured output formats cause language models to fabricate answers. Using the 'Abstention-Affordance Ladder,' each input/question is held fixed while only the answer format varies: free prose, JSON with an explicit escape value, and JSON with required fields and no escape. The inputs are constructed so that the target fields are unanswerable by construction (e.g., a viral post with engagement counts but no visible replies; a support ticket whose call was never transcribed). Across thirteen models, the paper reports that required schema fields drive fabrication to 100% in ten of thirteen models, that escape values are not reliably used by open-weight models, that grammar-constrained decoding does not reduce fabrication, that a direct anti-inference instruction is overridden by the schema in four of six models, and that refusal behavior does not track model scale. The paper releases the benchmark generator, deterministic scorer, judge prompts, and all model outputs.

Significance. The central result is significant and well-supported: the same model and input, differing only in output format, can flip from near-perfect abstention to 100% fabrication. The ladder design controls the input, the JSON-rung metrics are scored deterministically without an LLM judge, and the grammar-constrained decoding experiment rules out the alternative that the effect is merely prompted cooperation. The per-field escape-use asymmetry—zero uses on load-bearing fields but twelve on a field where escaping concedes nothing—is a sharp piece of evidence. If the results hold, free-text abstention scores do not transfer to structured deployments, and schema design should be treated as safety-relevant configuration. The main limitation is an overstatement in the abstract about open-weight models ignoring escape values; the data support that escape hatches fail to rescue them, not that they are completely ignored.

major comments (2)

[Abstract; Section 4.3; Table 1] The abstract claims that 'all nine open-weight models ignore it' when given an 'insufficient evidence' escape option. Table 1 contradicts this: gemma4 26B has a 60% fabrication rate at the json esc rung, i.e., a 40% escape utilization rate, and gemma4 e4B uses the escape in 8% of trials. These models do not ignore the escape; they use it occasionally but still fabricate in the majority of trials. The supported statement is that the escape fails to rescue all nine open-weight models (they still fabricate at 60–100%), not that all nine ignore it. Please correct the abstract and Finding 2.
[Section 4.4; Table 2] The text says 'Across all 203 non-empty enforced trials' for the escape-use counts, but Table 2 sums to 200 trials (five models × 40). Additionally, the paragraph on gemma4 e4B states it emits empty output on '35 of 40 trials' yet says 'among its non-empty outputs it too fabricates, in 8 of 8'—if 35 are empty, only 5 are non-empty. These numerical inconsistencies need to be resolved before publication.

minor comments (4)

[Section 4.6] The causal phrasing 'Resistance is trained, not emergent' goes beyond the evidence. The Haiku/Sonnet/Opus comparison is confounded by training data, alignment objectives, release policies, and unknown CLI system prompts. Recommend softening to 'consistent with a training effect' unless a controlled intervention is provided.
[Section 4.5] The instruction test uses n=10 per model; report confidence intervals or explicitly note the small sample. Also, the phrase 'in our earlier task framing' is vague—specify which earlier task and prompt are being referenced.
[Section 3.1] The statement 'nothing else differs' between rungs is slightly too strong: rung 2 and rung 3 differ in requiredness and nullability as well as in the presence of escape values. The enforced-decoding experiment does isolate the escape dimension, but the prose could be more precise about what is held constant.
[Section 6] The benchmark release is described but no URL or repository link is provided. For a reproducibility-focused benchmark, an anonymous download link should be included.

Circularity Check

0 steps flagged

No significant circularity: PhantomFill is an empirical controlled-measurement study, not a derivation that reduces to its inputs.

full rationale

PhantomFill's central claim ('the form itself causes hallucination') is supported by the Abstention-Affordance Ladder, which holds input and question fixed and varies only the output format; the paper states 'The input document and the question are identical at every rung.' The unanswerable-by-construction design makes 'fabrication' definitional for any concrete value on withheld evidence, but the empirical question is whether models fill the schema, take an escape, or refuse the format, and the E2c/D-present controls and three-way judge validation keep the definition from doing all the inferential work. CFR and EUR are metric definitions rather than fitted parameters used to predict a closely related quantity. The constrained-decoding experiment provides a genuinely non-circular control: with insufficient evidence as a legal token, five open models emit the escape 0/203 times on load-bearing fields and 12 times on the one field where escaping costs nothing, which directly tests rather than assumes the mechanism. The only self-citation, [14], appears in a passing related-work adjacency and is not load-bearing for any finding. The within-family Claude comparison (Section 4.6) is confounded and thus a correctness risk, not a circular step, and the paper acknowledges the vendor-CLI system-prompt limitation. One textual inconsistency (the abstract says all nine open-weight models ignore the escape while Table 1 shows gemma4 26B using it 40% of the time at rung 2) is a reporting error, not circularity. No derivation step in the paper is equivalent by construction to its own input.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The paper's central claim rests on the constructed-absence property and the chosen schemas; these are design choices, not fitted parameters. The main burden is that the benchmark artifacts are not linked from the text, so the measurements cannot be replicated from the paper alone. The within-family comparison adds an unmeasured confound (different training histories) to the interpretation.

free parameters (4)

rung-1 LLM judge rubric = not specified in text; in release
Free-text fabrication is scored by an LLM judge with a 'tight fabrication-only rubric' (§3.3); the threshold for what counts as a concrete fabricated claim is a scoring choice affecting rung-1 numbers, though not the headline rung-3 CFR.
escape vocabulary = 'insufficient evidence' enum token; null for strings/arrays
The set of tokens/values counted as non-fabrication for CFR/EUR scoring is a design choice (§3.3); changing it would change both metric values.
seed templates and field choices = 40 posts, 5 evidence levels, 20 tickets; fields: sentiment enum, themes array, quote string, controversy enum
The constructed item set defines where unanswerability holds (§3.2); the specific field names and enum values are chosen by the authors and could affect measured rates.
sample size per cell = n=40 (Opus json pool n=53; instruction arm n=10)
The n=40 per cell gives Wilson CIs of ±9-13 points at extreme rates; the instruction arm n=10 is small and the paper reports a binary flip there.

axioms (4)

domain assumption Any concrete claim about replies/sentiment/quotes is a fabrication when the input contains no reply text.
Section 3.2 defines the 'unanswerable by construction' property; this is the ground truth that makes scoring deterministic.
domain assumption The JSON schemas used are a faithful model of production form-filling.
Motivation in §1 and §3.1; the paper asserts deployed models fill forms (JSON fields, function arguments) rather than writing prose.
domain assumption The LLM judge is a reliable proxy for human judgment of free-text fabrication.
Section 3.3 reports 93% agreement with a second/gold judge, but Cohen's κ=0.61 is moderate; rung-1 numbers (e.g., GPT-5.5 2%) depend on this judge.
domain assumption Vendor CLIs add no system prompts that materially change behavior.
Section 5 (Limitations) concedes frontier models were reached through vendor CLIs which may add unseen system prompts; the paper argues any such prompt would dampen, not inflate, the measured flip.

invented entities (3)

PhantomFill benchmark independent evidence
purpose: Deterministic benchmark with constructed-unanswerable items measuring Coerced Fabrication Rate (CFR) and Escape Utilization Rate (EUR).
Re-runnable with generator scripts and a deterministic scorer as claimed in §6, but no URL or commit hash is provided in the text, so the artifact is claimed but not independently accessible.
CFR and EUR metrics independent evidence
purpose: Two reportable numbers for honesty under format pressure; CFR is the fabrication rate under required schemas, EUR is the escape adoption rate.
They are precisely defined in §3.3 and can be computed on any model with a JSON-constrained output, so they have a falsifiable handle outside this paper.
Structured refusal no independent evidence
purpose: A proposed target behavior: emit a machine-readable refusal object (e.g., {"status": "insufficient data"}) when required fields cannot be filled honestly.
Observed in Opus (§4.7) and suggested as a training target, but no trained model or API implements it as a standard behavior yet; it is a design proposal.

pith-pipeline@v1.3.0-alltime-deepseek · 7354 in / 11369 out tokens · 108973 ms · 2026-08-02T11:36:21.502788+00:00 · methodology

0 comments

read the original abstract

Language models in production do not write prose. They fill forms: JSON fields, function arguments, extraction templates. We show that the form itself causes hallucination. We ask thirteen models the same question about the same input and change only the answer format. The inputs are built so the question cannot be answered: a viral post showing 12,400 likes but no visible replies, a support ticket whose call was never transcribed. In free text, GPT-5.5 says there is no reply data 98% of the time. Given a required JSON field for sentiment, the same model invents an answer 40 times out of 40. It fabricates the mood of crowds it never saw and quotes customers it never heard. Required fields drive fabrication to 100% in ten of thirteen models. An explicit "insufficient evidence" option rescues only the frontier: all nine open-weight models ignore it. Under grammar-constrained decoding, where the escape token is guaranteed reachable by the sampler, five open models spend it zero times out of 203 trials on the three fields that carry the fabrication, and twelve times on the one field where escaping concedes nothing. They can emit the word. They decline to spend it where it costs them an answer. A direct instruction, do not infer sentiment, is overridden by the schema in four of six models. Resistance does not come with scale: within a single model family, the smallest model refuses, the mid-sized model fabricates, the largest refuses again. Honesty under format pressure is a training outcome that no one is measuring. Fabrication hides where hedging is impossible: in required enums and minimum-count arrays, fields where no disclaimer fits. We release PhantomFill, a benchmark with deterministic scoring and two reportable numbers: the Coerced Fabrication Rate and the Escape Utilization Rate. The fix we test is one line of schema. The failure we measure is everywhere.

Figures

Figures reproduced from arXiv: 2607.20492 by Rana Muhammad Usman.

**Figure 2.** Figure 2: The schema flip. Same thread, same question; only the output format varies. [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗

**Figure 3.** Figure 3: Escape Utilization Rate. Only GPT-5.5 and Opus reliably take an offered “insufficient [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

**Figure 4.** Figure 4: An explicit anti-fabrication instruction does not survive a required-field schema, except [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

**Figure 5.** Figure 5: Field-level fabrication under the required schema (GPT-5.5, Domain 2). [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗

**Figure 6.** Figure 6: Three responses to an impossible required schema. Only Opus shows a meaningful [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

14 extracted references · 13 linked inside Pith

[1]

Kirichenko, P., et al. (2025). AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions. arXiv:2506.09038

Pith/arXiv arXiv 2025
[2]

R., et al

Tam, Z. R., et al. (2024). Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models. arXiv:2408.02442

Pith/arXiv arXiv 2024
[3]

Wen, B., et al. (2025). Know Your Limits: A Survey of Abstention in Large Language Models. TACL. arXiv:2407.18418

Pith/arXiv arXiv 2025
[4]

The Structured Output Benchmark (SOB). (2026). arXiv:2604.25359

Pith/arXiv arXiv 2026
[5]

LLMStructBench: Benchmarking Large Language Model Structured Data Extraction. (2026). arXiv:2602.14743. 11

arXiv 2026
[6]

T., and Louf, R

Willard, B. T., and Louf, R. (2023). Efficient Guided Generation for Large Language Models. arXiv:2307.09702

Pith/arXiv arXiv 2023
[7]

Huang, L., et al. (2023). A Survey on Hallucination in Large Language Models. arXiv:2311.05232

Pith/arXiv arXiv 2023
[8]

Ji, Z., et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629

Pith/arXiv arXiv 2023
[9]

Rajpurkar, P., Jia, R., and Liang, P. (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD. ACL. arXiv:1806.03822

Pith/arXiv arXiv 2018
[10]

Lin, S., Hilton, J., and Evans, O. (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL. arXiv:2109.07958

Pith/arXiv arXiv 2022
[11]

Sharma, M., et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548

Pith/arXiv arXiv 2023
[12]

Zheng, L., et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS. arXiv:2306.05685

Pith/arXiv arXiv 2023
[13]

Zhang, H., et al. (2024). R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’. NAACL. arXiv:2311.09677

Pith/arXiv arXiv 2024
[14]

Rana Muhammad Usman. (2026). Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults. arXiv:2606.00914. A Wilson 95% Confidence Intervals, E3 Cells model freetext json esc json req qwen3.5 0.8B 98 [87, 100] 100 [91, 100] 100 [91, 100] qwen3.5 2B 100 [91, 100] 100 [91, 100] 100 [91, 100] llama3.2 3B 100 [91, 100] 100 [91, 100] 100 [91, 100] gemm...

Pith/arXiv arXiv 2026

[1] [1]

Kirichenko, P., et al. (2025). AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions. arXiv:2506.09038

Pith/arXiv arXiv 2025

[2] [2]

R., et al

Tam, Z. R., et al. (2024). Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models. arXiv:2408.02442

Pith/arXiv arXiv 2024

[3] [3]

Wen, B., et al. (2025). Know Your Limits: A Survey of Abstention in Large Language Models. TACL. arXiv:2407.18418

Pith/arXiv arXiv 2025

[4] [4]

The Structured Output Benchmark (SOB). (2026). arXiv:2604.25359

Pith/arXiv arXiv 2026

[5] [5]

LLMStructBench: Benchmarking Large Language Model Structured Data Extraction. (2026). arXiv:2602.14743. 11

arXiv 2026

[6] [6]

T., and Louf, R

Willard, B. T., and Louf, R. (2023). Efficient Guided Generation for Large Language Models. arXiv:2307.09702

Pith/arXiv arXiv 2023

[7] [7]

Huang, L., et al. (2023). A Survey on Hallucination in Large Language Models. arXiv:2311.05232

Pith/arXiv arXiv 2023

[8] [8]

Ji, Z., et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. arXiv:2202.03629

Pith/arXiv arXiv 2023

[9] [9]

Rajpurkar, P., Jia, R., and Liang, P. (2018). Know What You Don’t Know: Unanswerable Questions for SQuAD. ACL. arXiv:1806.03822

Pith/arXiv arXiv 2018

[10] [10]

Lin, S., Hilton, J., and Evans, O. (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. ACL. arXiv:2109.07958

Pith/arXiv arXiv 2022

[11] [11]

Sharma, M., et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv:2310.13548

Pith/arXiv arXiv 2023

[12] [12]

Zheng, L., et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS. arXiv:2306.05685

Pith/arXiv arXiv 2023

[13] [13]

Zhang, H., et al. (2024). R-Tuning: Instructing Large Language Models to Say ‘I Don’t Know’. NAACL. arXiv:2311.09677

Pith/arXiv arXiv 2024

[14] [14]

Rana Muhammad Usman. (2026). Adversarial Feeds Steer LLM Agent Decisions Against Their Defaults. arXiv:2606.00914. A Wilson 95% Confidence Intervals, E3 Cells model freetext json esc json req qwen3.5 0.8B 98 [87, 100] 100 [91, 100] 100 [91, 100] qwen3.5 2B 100 [91, 100] 100 [91, 100] 100 [91, 100] llama3.2 3B 100 [91, 100] 100 [91, 100] 100 [91, 100] gemm...

Pith/arXiv arXiv 2026