{"paper":{"title":"Mind the Gap: How Elicitation Protocols Shape the Stated-Revealed Preference Gap in Language Models","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Elicitation protocols determine the correlation between stated and revealed preferences in language models.","cross_cats":["cs.ET"],"primary_cat":"cs.AI","authors_text":"Ihor Kendiukhov, Lydia Nottingham, Pranav Mahajan, Syed Hussain","submitted_at":"2026-01-29T16:51:43Z","abstract_excerpt":"Recent work identifies a stated-revealed (SvR) preference gap in language models (LMs): a mismatch between the values models endorse and the choices they make in context. Existing evaluations rely heavily on binary forced-choice prompting, which entangles genuine preferences with artifacts of the elicitation protocol. We systematically study how elicitation protocols affect SvR correlation across 24 LMs. Allowing neutrality and abstention during stated preference elicitation allows us to exclude weak signals, substantially improving Spearman's rank correlation ($\\rho$) between volunteered stat"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"SvR correlation is highly protocol-dependent and that preference elicitation requires methods that account for indeterminate preferences.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the changes in Spearman's rho are driven by genuine preference indeterminacy rather than by specific properties of the AIRiskDilemmas dataset or the 24 models tested.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Allowing neutrality in stated preference questions improves correlation with revealed choices across 24 LMs, but allowing abstention in revealed preferences drives correlation to near zero or negative due to high neutrality rates.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Elicitation protocols determine the correlation between stated and revealed preferences in language models.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"1c1cfb6e778a88a8bbc393e2a52947e3186d2e2d8e6b8fc71afc49fbec280119"},"source":{"id":"2601.21975","kind":"arxiv","version":2},"verdict":{"id":"642510bd-c011-43ac-a563-812134b8ccbc","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-16T09:41:41.953760Z","strongest_claim":"SvR correlation is highly protocol-dependent and that preference elicitation requires methods that account for indeterminate preferences.","one_line_summary":"Allowing neutrality in stated preference questions improves correlation with revealed choices across 24 LMs, but allowing abstention in revealed preferences drives correlation to near zero or negative due to high neutrality rates.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the changes in Spearman's rho are driven by genuine preference indeterminacy rather than by specific properties of the AIRiskDilemmas dataset or the 24 models tested.","pith_extraction_headline":"Elicitation protocols determine the correlation between stated and revealed preferences in language models."},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}