Pith. sign in

REVIEW 3 major objections 5 minor 45 references

It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM agreement with false claims depends on how the claim is phrased, not just what it says.

desk verdict A genuinely useful controlled benchmark for how linguistic framing changes LLM context-following, with one real construct-validity caveat that should be fixable in revision. read the letter →

arxiv 2607.18232 v2 pith:BGHT4J25 submitted 2026-07-20 cs.CL

Kevin Du , Clara Kümpel , Michelle Wastl , Alex Warstadt This is my paper
classification cs.CL
keywords expressionsofbeliefknowledgeconflictscontext-followingrateinstructiontuningmodelscalelinguistictypologyLLMrobustnesspromptengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the linguistic surface form of a user's belief—whether it is phrased as a command, a question, a counterfactual, or a casual aside—systematically changes whether a large language model abandons its prior knowledge and follows the context. To test this, the authors build a typology of expressions of belief across four dimensions and generate roughly 66,000 controlled belief–query pairs, then measure a context-following rate across 16 models spanning three families, sizes, and training stages. They find that instruction tuning and larger scale both reduce context-following, and that certain phrasings—imperatives, child-directed tone, formal tone, appeals to authority, suppositions, and emotional appeals—are statistically significantly more persuasive, while counterfactuals, belief reports, weak epistemic stance, and interrogatives are significantly less. The work matters because it shows that prompt engineering and model robustness evaluations cannot treat 'context' as a single undifferentiated input: the same proposition lands differently depending on its packaging.

What carries the argument

The central machinery is the EoB typology—four linguistically motivated dimensions (form, evidentiality, epistemic stance, tone) covering 19 fine-grained types—combined with a parameterized template system that holds propositional content constant while varying surface form. Each test item is a semantic tuple (subject, relation, false object, true object, plus optional extra material for specific types), from which ten templates per EoB type generate expressions like 'Remember that London is the capital of France' versus 'My brother believes the capital of France is London.' The outcome measure is the context-following rate (CFR), defined as P(answer follows context | answer follows context

What would settle it

Re-run the benchmark with templates stripped of the extra semantic content (appositives without additional predicates, authority/belief frames with neutral placeholders, conditionals with tautological antecedents) and check whether the significant CFR rankings persist; if they vanish or reverse, the typology-level claim is falsified. A second check: a human-subject rating study of the same EoB pairs should show the same ordering if the effect is about linguistic form rather than artifacts of template wording.

Watch

Extended reading notes

Core claim

The paper's central claim is that expressions of belief (EoBs) are a real, measurable axis of variation in how LLMs resolve knowledge conflicts. Using EoBench, it shows that the same false proposition, phrased differently, shifts the context-following rate reliably across 16 models, three families, and size/instruction axes. Two global trends emerge: instruction-tuned models and larger models each follow context less than their counterparts (one exception: Qwen3-8B instruct is more context-following than base). Type-level results rank child-directed, imperative, formal, authority, supposition, and emotional-appeal EoBs as significantly more persuasive than random, and counterfactual, belief-

Load-bearing premise

The load-bearing premise is that the template-based generation isolates linguistic form while holding propositional content constant; several EoB types inject extra semantic material (extra information, authority or belief sources, antecedent conditions) that could, rather than the intended dimension, drive the observed CFR differences.

Editorial extensions

If this is right

  • If phrasing systematically shifts CFR, then misinformation-resistance tests that use a single canonical phrasing will misestimate LLM susceptibility; benchmarks should sample across EoB types.
  • Instruction tuning is a partial but uneven safeguard: it lowers overall context-following, yet certain phrasings (child-directed, imperative, authority) remain effective on instruct models.
  • Scaling is not a uniform cure: larger models follow context less on average, but the spread across dimensions widens with size, so some phrasings still win on big models.
  • For chat assistants and retrieval-augmented systems, the same corrective fact, phrased differently by different users, will be accepted at different rates—so user-facing robustness may require normalizing or hedging phrasing.
  • The significant rankings give a concrete, testable shortlist of high- and low-persuasion phrasings that future work can use as a control variable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A cleaner test of the typology's construct validity would strip the extra semantic content that some templates inject (appositive extra information, authority/belief sources, conditional antecedents) and check whether the CFR rankings survive; the paper's own limitation section flags that templates mix dimensions in places.
  • The scale effect may be partly a memory effect: larger models have stronger parametric priors, so lower CFR could reflect better fact storage rather than better epistemic judgment; separating these would require testing with novel, unmemorized facts.
  • If these EoB effects hold under adversarial use, then phrasing a false premise as a child-directed command or an appeal to authority could become a practical jailbreak lever, and the typology offers a map for probing that.
  • The paper's data could be used to build a small 'persuasion profile' per model family, predicting CFR from EoB type alone; that would be a direct, testable extension of the reported rankings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces EoBench, a template-generated set of 65,778 expressions of belief (EoBs) that pair incorrect assertions with PopQA-derived facts across 19 EoB types in four dimensions (form, evidentiality, epistemic stance, tone). It evaluates 18 Gemma3/Llama3/Qwen3 models (1B–30B, base and instruct), filters to 16 with sufficient prior knowledge, and measures context-following rate (CFR) via two yes/no questions per item. Main findings: child-directed, imperative, formal, authority, supposition, and emotional-appeal EoBs rank as significantly more context-following than random; counterfactual, belief-report, weak, and interrogative EoBs rank significantly lower; instruction tuning and model size broadly reduce CFR; Llama3 is relatively less context-following.

Significance. The benchmark is a useful new resource: it is public, spans 19 linguistically motivated EoBs, covers three model families and base/instruct checkpoints, and applies per-model prior-knowledge filtering with bidirectional yes/no scoring. If the confound issue is addressed, the ranking findings would be of practical value for prompt engineering and for understanding how training choices affect context sensitivity. The paper does not claim to have fitted any free parameters; the results are empirical measurements, which is a strength.

major comments (3)
  1. [§4.2, Listings 1–2] the templates do not hold propositional content constant. Not-at-issue appends extra_information; authority/belief-report/hearsay insert sources; counterfactual/material-conditional insert antecedents. These are extra propositions/credibility signals, not merely form. The ranking results for authority, belief reports, counterfactual (all central to Fig. 1) may therefore be driven by content confounds rather than by the intended linguistic dimension. An explicit-baseline matched-control ablation (adding the same extra material as a separate sentence, or varying source credibility) is needed before claiming the benchmark isolates the effect of linguistic variation.
  2. [§6.1] the permutation test is underspecified. 'Mean ranking' is not defined (per-model ranking of the 19 EoBs? item-level? how ties are handled?), the shuffling procedure and test statistic are not given, and no p-values/effect sizes are reported. Because the significance claims are the paper's headline, the test must be described completely (including which subset of models survives filtering) and the code for exactly this analysis referenced.
  3. [§6.2–6.3 and §7] the categorical statements 'instruction-tuning and increasing model size, both reduce context-following' overstate the results. Fig. 3 shows Qwen3-8B-Instruct has CFR 18.1% higher than its base, and Fig. 4 shows Qwen3 Instruct is flat across scale; no significance test or confidence interval is reported for these aggregate comparisons. Please either qualify the claims (as 'tend to' in the abstract) or add statistical support.
minor comments (5)
  1. [Abstract] '17 fine-grained types' vs. 19 in Section 3/Figure 2; align the numbers.
  2. [§5] 'We ask in both directions' is ambiguous — specify whether each yes/no pair is administered in both orders and how conflicting responses are resolved.
  3. [Figure 1] the stars for FDR significance levels have no legend; report p-values or q-values in text/appendix.
  4. [Listing 3] counterfactual_condition 'Berlin were the capital of Germany' looks like the true conditional antecedent, not the counterfactual used in §3.1 ('Berlin weren't ...'); check consistency with the templates.
  5. [Appendix B] the filtering criterion 'answers <10% of facts correctly' excludes Gemma3-1B and Qwen3-1.7B-Base, but the paper later says '18 models' in the abstract and '16 total models'; clarify that the benchmark covers 18 but analysis 16.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central EoB rankings and scale/tuning trends are empirical measurements, not consequences of the framework's definitions or self-citations.

full rationale

The paper's core claims are empirical: it constructs EoBench from templates, measures context-following rates (CFR) on 16 LLMs, and reports statistical patterns across EoB types, model scale, and instruction tuning. No parameter is fitted to the outcome and then renamed as a prediction; no EoB type is defined in terms of the CFR it produces; and no uniqueness theorem or ansatz is imported from prior self-cited work to force the conclusions. The self-citations (e.g., Du et al. 2024, Nguyen et al. 2025, Minder et al. 2025) supply background framing and prior knowledge-conflict methodology, but the typology, templates, and reported trends are new contingent measurements that would stand even if those citations were removed. The Limitations section explicitly notes that the template-based generations are synthetic and may not capture natural language variability, and the reviewer-level construct-validity concern about extra semantic content in some templates (appositives, authority sources, conditional antecedents) is a threat to whether the experiment isolates linguistic form—not a circularity, because the empirical CFR values are not entailed by the template definitions. Therefore no circular step meets the evidentiary standard required here.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No fitted numerical parameters; the central claims rest on measurement and representativeness assumptions about template-generated EoBs and yes/no probes. The main unvalidated premise is that the generation scheme isolates linguistic framing from extra semantic content.

assumptions (4)
  • domain assumption PopQA ground-truth facts represent the model's prior knowledge, and asserting a different object within the same relation reliably contradicts that knowledge.
    Section 4.3: triples with multiple objects are filtered and a conflicting object is sampled within the same relation. If some 'conflicting' objects are plausibly true or ambiguous, CFR labels are noisy.
  • domain assumption Answering yes/no to two direct questions after an EoB measures whether the model follows the EoB; the CFR denominator of C+M correctly handles 'other' responses.
    Section 5, Evaluation Metrics. Binary yes/no mapping could miss mixed or evasive responses; asking both directions mitigates token bias but does not fully characterize model belief.
  • domain assumption The 19 EoB types are representative instances of their linguistic categories, and the 10 templates per type are interchangeable.
    Section 3 and Section 4.2: types are hand-selected representative instances; template-level variation is treated as noise rather than a factor.
  • domain assumption Holding propositional content constant is achieved by the template system despite some types embedding extra semantic material.
    Section 4.2, Listings 1-2: not-at-issue appositives add extra_information, evidentiality adds authority/belief sources, and conditionals add antecedent conditions; this extra content may affect CFR independently of the intended linguistic dimension.

how reviews work

0 comments
Cite this review

Pith. "Pith review of It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief." pith.science (2026). https://pith.science/paper/BGHT4J25

@misc{pith2026260718232,
  author       = {Pith},
  title        = {Pith review of: It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BGHT4J25}},
  note         = {Machine review of arXiv:2607.18232}
}
read the original abstract

Users frequently express their beliefs to large language models (LLMs). In some situations, the LLM should accept these contextual beliefs as true. In others, they should stick to their prior knowledge. Notably, users' expressions of belief (EoBs) can take linguistically diverse forms - using presuppositions, evidential and certainty markers, or varied tones - each of which may have a different persuasiveness over the LLMs. We introduce a typology to systematically evaluate how different EoBs affect whether models follow context versus prior knowledge. The typology is grounded in four linguistically motivated dimensions: form, evidentiality, epistemic stance, and tone, spanning 17 fine-grained types. By pairing these EoBs with world knowledge facts, we generate controlled EoB-query pairs that isolate the effect of linguistic variation. Using this benchmark, we evaluate 16 LLMs that differ in architecture (Llama3, Qwen3, Gemma3), scale (1B-30B parameters), and training stages (base vs instruct). We identify meaningful variations in response behavior across these axes, e.g., that bigger models and instruction models tend to be less context-following than smaller models and base models. We further identify specific EoBs that statistically significantly persuade LMs more consistently than others. Our work reveals systematic patterns in how linguistic framing affects LLM context integration, with implications for prompt engineering and model robustness.

Figures

Figures reproduced from arXiv: 2607.18232 by the authors.

Figure 1
Figure 1. Certain expressions of belief (EoBs) are sta [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Our EoB typology spanning four linguistic dimensions (19 total types). Each dimension varies in how [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. CFR across dimensions for base vs. instruct [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Llama3 and Gemma3 decrease in CFR as model size increases. Further, we tease out in these plots that, [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: CFR for all models (aggregated over all EoBs). [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Prior prompting results for all 18 models grouped by the responses. We filter the results for ’always correct’ to guarantee correct model behavior. Dimension Description Form How the assertion is syntactically structured and presented. Explicit Direct statement asserti…
Figure 7
Figure 7. Figure 7: The CFR for all 19 EoBs, for all evaluated models. The dashed black line/black bar represents a baseline [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: We provide a second visualization for the CFR for all 19 EoBs, for all evaluated models. Here, we can [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 9 canonical work pages

  1. [1]

    Aikhenvald

    Alexandra Y. Aikhenvald. 2004. https://global.oup.com/academic/product/evidentiality-9780199204335 Evidentiality . Oxford University Press, Oxford

    2004

  2. [2]

    Yoav Benjamini and Yosef Hochberg. 1995. http://www.jstor.org/stable/2346101 Controlling the false discovery rate: A practical and powerful approach to multiple testing . Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289--300

    arXiv 1995

  3. [3]

    Douglas Biber and Susan Conrad. 2019. https://www.cambridge.org/core/books/register-genre-and-style/55217DB70BD87C93FA2BBA70DB4C2575 Register, genre, and style . Cambridge University Press

    2019

  4. [4]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. https://proceedings.neurips.cc/paper/202...

    2020

  5. [5]

    Marta Carretero. 2002. https://revistas.ucm.es/index.php/EIUC/article/download/EIUC0202110011A/8052 The influence of genre and register on epistemic modality in spoken english: A preliminary study . Estudios Ingleses de la Universidad Complutense, 2002

    2002

  6. [6]

    Vanessa Cheung, Maximilian Maier, and Falk Lieder. 2025. https://doi.org/10.1073/pnas.2412015122 Large language models show amplified cognitive biases in moral decision-making . Proceedings of the National Academy of Sciences, 122(25):e2412015122

    doi:10.1073/pnas.2412015122 2025

  7. [7]

    Filippo Domaneschi, Paolo Canal, Viviana Masia, Edoardo Lombardi Vallauri , and Valentina Bambini. 2018. https://doi.org/10.1016/j.jneuroling.2017.08.002 N400 and P 600 modulation in presupposition accommodation: The effect of different trigger types . Journal of Neurolinguistics, 45:13--35

    work page doi:10.1016/j.jneuroling.2017.08.002 2018

  8. [8]

    Kevin Du, V \'e steinn Sn bjarnarson, Niklas Stoehr, Jennifer White, Aaron Schein, and Ryan Cotterell. 2024. https://doi.org/10.18653/v1/2024.acl-long.714 Context versus prior knowledge in language models . In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13211--13235, Bangkok, Thail...

    doi:10.18653/v1/2024.acl-long.714 2024

Show all 45 references
  1. [9]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha...

    2024 arXiv

  2. [10]

    Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. https://aclanthology.org/2021.emnlp-main.446 Transformer feed-forward layers are key-value memories . In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5484--5495

    2021

  3. [11]

    H. P. Grice. 1975. https://doi.org/10.1163/9789004368811_003 Logic and Conversation . Speech Acts, pages 41--58

    1975 doi

  4. [12]

    John Heritage. 2012. https://doi.org/10.1002/9781118325001.ch18 Epistemics in Conversation , chapter 18. John Wiley & Sons, Ltd

    2012 doi

  5. [13]

    Paloma Jeretic, Alex Warstadt, Suvrat Bhooshan, and Adina Williams. 2020. https://doi.org/10.18653/v1/2020.acl-main.768 Are natural language inference models IMPPRESsive ? L earning IMPlicature and PRESupposition . In Proceedings of the 58th Annual Meeting of the Association f...

    2020 doi

  6. [14]

    Mehran Kazemi, Quan Yuan, Deepti Bhatia, Najoung Kim, Xin Xu, Vaiva Imbrasaite, and Deepak Ramachandran. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/7adce80e86aa841490e6307109094de5-Paper-Datasets_and_Benchmarks.pdf BoardgameQA : A dataset for natural lang...

    2023

  7. [15]

    Andrew K Lampinen, Ishita Dasgupta, Stephanie C Y Chan, Hannah R Sheahan, Antonia Creswell, Dharshan Kumaran, James L McClelland, and Felix Hill. 2024. https://doi.org/10.1093/pnasnexus/pgae233 Language models, like humans, show content effects on reasoning tasks . PNAS Nexus,...

    2024 doi

  8. [16]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. 2020. https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f78...

    2020

  9. [17]

    Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.565 Entity-based knowledge conflicts in question answering . In Proceedings of the 2021 Conference on Empirical Methods in Natural L...

    2021 doi

  10. [18]

    John Lyons. 1977. https://www.cambridge.org/9780521291651 Semantics . Cambridge University Press

    1977

  11. [19]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/v1/2023.acl-long.546 When not to trust language models: Investigating effectiveness of parametric and non-parametric memories . In Proceedings of the 6...

    2023 doi

  12. [20]

    Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. https://proceedings.neurips.cc/paper_files/paper/2022/file/6f1d43d5a82a37e89b0665b33bf3a182-Paper-Conference.pdf Locating and editing factual associations in GPT . 35:17359--17372

    2022

  13. [21]

    Julian Minder, Kevin Du, Niklas Stoehr, Giovanni Monea, Chris Wendler, Robert West, and Ryan Cotterell. 2025. https://openreview.net/forum?id=Igm9bbkzHC Controllable context sensitivity and the knob behind it . In The Thirteenth International Conference on Learning Representations

    2025

  14. [22]

    Tu Nguyen, Kevin Du, Alexander Miserlis Hoyle, and Ryan Cotterell. 2025. https://doi.org/10.18653/v1/2025.emnlp-main.1633 How persuasive is your context? In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32097--32123, Suzhou, Chin...

    2025 doi

  15. [23]

    Jan Nuyts and Johan van der Auwera. 2016. https://doi.org/10.1093/oxfordhb/9780199591435.001.0001 The Oxford Handbook of Modality and Mood . Oxford University Press

    2016

  16. [24]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 GPT-4 technical report . arXiv

    2023 arXiv

  17. [25]

    Peter Pagin and Neri Marsili. 2021 . https://plato.stanford.edu/archives/win2021/entries/assertion/ Assertion . In Edward N. Zalta , editor, The Stanford Encyclopedia of Philosophy , W inter 2021 edition. Metaphysics Research Lab, Stanford University

    2021

  18. [26]

    Frank R. Palmer. 2001. https://www.cambridge.org/core/books/mood-and-modality/70959BB4A71A4C305A0A0695634A91EC Mood and Modality , 2nd edition. Cambridge Textbooks in Linguistics. Cambridge University Press, Cambridge

    2001

  19. [27]

    Bowman, and Tal Linzen

    Alicia Parrish, Sebastian Schuster, Alex Warstadt, Omar Agha, Soo-Hwan Lee, Zhuoye Zhao, Samuel R. Bowman, and Tal Linzen. 2021. https://doi.org/10.18653/v1/2021.conll-1.28 NOPE : A corpus of naturally-occurring presuppositions in E nglish . In Proceedings of the 25th Conferen...

    2021 doi

  20. [28]

    Fabio Petroni, Tim Rockt \"a schel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. https://doi.org/10.18653/v1/D19-1250 Language models as knowledge bases? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language P...

    2019 doi

  21. [29]

    Limited access’ as a “fishing

    Anita Pomerantz. 1980. https://doi.org/10.1111/j.1475-682X.1980.tb00020.x Telling my side: “Limited access’ as a “fishing” device . Sociological Inquiry, 50(3-4):186--198

    1980 doi

  22. [30]

    Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.437 How much knowledge can you pack into the parameters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages...

    2020 doi

  23. [31]

    Robert Stalnaker. 1978. https://archive.nytud.hu/program/szuperkurzus2006/Stalnaker_78.pdf Assertion . In Peter Cole, editor, Syntax and Semantics, volume 9, pages 315--332. Academic Press

    1978

  24. [32]

    Zhaochen Su, Jun Zhang, Xiaoye Qu, Tong Zhu, Yanshu Li, Jiashuo Sun, Juntao Li, Min Zhang, and Yu Cheng. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/baf4b960d118f838ad0b2c08247a9ebe-Paper-Datasets_and_Benchmarks_Track.pdf Conflictbank: A benchmark for eval...

    2024

  25. [33]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...

    2025 arXiv

  26. [34]

    Michael Tomasello. 2008. https://mitpress.mit.edu/9780262515207/origins-of-human-communication/ Origins of Human Communication . MIT press

    2008

  27. [35]

    Nicole Van Hoeck, Ning Ma, Lisa Ampe, Kris Baetens, Marie Vandekerckhove, and Frank Van Overwalle. 2012. https://doi.org/10.1093/scan/nss031 Counterfactual thinking: An f MRI study on changing the past for a better future . Social Cognitive and Affective Neuroscience, 8(5):556--564

    2012 doi

  28. [36]

    Oriol Vinyals and Quoc Le. 2015. https://arxiv.org/abs/1506.05869 A neural conversational model . arXiv

    2015 arXiv

  29. [37]

    Albert Webson, Alyssa Loo, Qinan Yu, and Ellie Pavlick. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.514 Are language models worse than humans at following prompts? it`s complicated . In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 7662...

    2023 doi

  30. [38]

    Ludwig Wittgenstein. 1953. https://static1.squarespace.com/static/54889e73e4b0a2c1f9891289/t/564b61a4e4b04eca59c4d232/1447780772744/Ludwig.Wittgenstein.-.Philosophical.Investigations.pdf Philosophical Investigations . Wiley-Blackwell, New York, NY, USA

    1953

  31. [39]

    Jian Xie, Kai Zhang, Jiangjie Chen, Renze Lou, and Yu Su. 2024. https://openreview.net/forum?id=auKAUJZMO6 Adaptive chameleon or stubborn sloth: Revealing the behavior of large language models in knowledge conflicts . In The Twelfth International Conference on Learning Representations

    2024

  32. [40]

    Rongwu Xu, Brian Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu. 2024. https://doi.org/10.18653/v1/2024.acl-long.858 The earth is flat because...: Investigating LLM s' belief towards misinformation via persuasive conversation . In...

    2024 doi

  33. [41]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. https://arxiv.org/abs/2505.09388 Qw...

    2025 arXiv

  34. [42]

    Wenhao Yu, Meng Jiang, Peter Clark, and Ashish Sabharwal. 2023 a . https://doi.org/10.18653/v1/2023.emnlp-main.515 I f QA : A dataset for open-domain question answering under counterfactual presuppositions . In Proceedings of the 2023 Conference on Empirical Methods in Natural...

    2023 doi

  35. [43]

    Xinyan Yu, Sewon Min, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.583 CREPE : Open-Domain question answering with false presuppositions . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistic...

    2023 doi

  36. [44]

    Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. 2024. https://arxiv.org/abs/2403.17336 Don't listen to me: Understanding and exploring jailbreak prompts of large language models . arXiv

    2024 arXiv

  37. [45]

    Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, and Baobao Chang. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.296 Can we edit factual knowledge by in-context learning? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Pro...

    2023 doi

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.