Pith. sign in

REVIEW 4 major objections 4 minor 65 references

Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings

T0 review · 4 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs can predict biased human decisions in conversational settings using limited dialogue and demographics, reproducing population-level bias patterns and their interaction with cognitive load.

desk verdict The human study is solid and the LLM application is genuinely new, but the abstract overclaims the load-bias interaction, which rides on one marginal n=6 correlation in an explicitly-biased prompt condition. read the letter →

arxiv 2601.11049 v2 pith:MXY7J4QI submitted 2026-01-16 cs.HC cs.AI

Stephen Pilli , Vivek Nallur This is my paper · ORCID
classification cs.HCcs.AI
keywords conversationalAIframingeffectstatusquobiascognitiveloadLLMsimulationdecisionpredictionhuman-likenesspromptingdialoguecomplexity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that large language models can predict individual human decisions in chatbot conversations when given a participant's demographics and prior dialogue transcript—and that, at the population level, their simulated choices reproduce two classic cognitive biases (the Framing Effect and Status Quo Bias) as well as the way those biases intensify under cognitive load. To test this, the authors ran a pre-registered human study (N = 1,648) in which participants faced six classic decision problems after either a simple or cognitively demanding dialogue. They confirm that framing and status-quo biases appear in conversational settings, that complex dialogue raises mental demand, and that this load selectively amplifies framing effects. They then show that LLMs, especially the GPT-4 family, can predict individual choices more accurately when given dialogue context, and can mirror human bias patterns and load-bias interactions—though the paper itself acknowledges this could be statistical pattern matching rather than genuine simulation.

What carries the argument

The argument rests on pairing classical choice problems (three framing tasks, three status-quo tasks) with a controlled chatbot dialogue at two complexity levels, and then prompting LLMs at three 'human-likeness' levels: minimal role, naturalistic instruction, and explicit bias instruction. The central quantities are effect sizes (Cohen's h) for bias presence, z-scores for the change in bias under complex dialogue, and Spearman correlations between human and LLM z-score patterns—these carry the claim that LLM behavior can align with human bias and load-bias interactions. Ablations isolating memory vs. arithmetic components of the dialogue indicate that memory cues, not arithmetic, are what a

What would settle it

Run the same prediction protocol on a set of newly constructed framing and status-quo choice problems that have established human effect sizes but are not present in any LLM training corpus; if the models' predictive accuracy and bias reproduction drop drastically relative to the classic problems, the central claim is driven by memorization rather than generalizable simulation.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that LLMs are capable of simulating biased human decision-making in conversational settings: given demographic information plus the transcript of a prior dialogue, GPT-4.1 predictions were significantly more accurate than chance in several choice problems (e.g., Goal Framing accuracy rose from 47% to 63% with dialogue; Investment Decisions from 62% to 76%). At the sample level, models under neutral prompts reproduced the presence or absence of bias across all six choice problems with 75% agreement with human findings, and when explicitly instructed to be biased (HL3) they reproduced the direction of load-bias interactions (Spearman ρ = 0.771, p = .07). The

Load-bearing premise

The load-bearing premise is that the LLMs' reproduction of bias patterns reflects simulation of human behavior rather than memorization of these classic choice problems from training data—the paper itself flags this as an open possibility in its discussion.

Editorial extensions

If this is right

  • Conversational agents could infer a user's bias susceptibility from dialogue history alone and adapt how options are presented, without needing explicit personal data.
  • LLM-based simulations could serve as low-cost proxies for user studies, enabling rapid A/B testing of dialogue designs for unintended bias amplification.
  • The selective load-bias interaction implies that increasing conversational complexity is not neutral: it can systematically strengthen framing-type biases while leaving status-quo bias unchanged.
  • Explicitly instructing models to be biased produces false positives on tasks where humans show no bias, so practical bias-aware simulation requires calibration against real human data.
  • Model choice matters: GPT-4-family models outperformed GPT-5 and open-source models in both accuracy and bias fidelity, so simulation claims should not be assumed to transfer across model generations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A decisive test the paper leaves implicit: using novel choice problems that were not present in LLM training data would separate genuine human-like generalization from memorization of canonical psychology experiments.
  • Since removing demographics barely changed predictions, future simulation systems could rely primarily on dialogue structure, which may simplify deployment and reduce privacy concerns.
  • The ablation finding that memory cues drive load-bias reproduction suggests a testable design principle: chat systems that require users to hold referents in working memory are more likely to amplify framing effects—and could be deliberately calibrated to nudge or debias.
  • If LLM simulation is accepted as a proxy, the community should treat pre-registered human baselines as the ground truth for every new bias and interaction, since prompt-level overfitting (as seen in HL3) can otherwise produce confidently wrong simulations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper reports a pre-registered human experiment (N=1,648) in which participants completed six classic choice problems through a chatbot after either a simple or a cognitively demanding dialogue. The human results show framing effects (risky-choice and goal framing) that are stronger after complex dialogue, status quo effects in two of three scenarios, and NASA-TLX/behavioral validation that complex dialogue increased mental load. The paper then prompts several LLMs (GPT-4.1 family, GPT-5 family, and open-source models) with demographic information and dialogue transcripts at three human-likeness prompt levels, evaluating individual-level prediction accuracy, sample-level bias reproduction, and whether the LLMs reproduce the human load-bias interaction. The authors conclude that LLMs, especially GPT-4.1, reproduced the same bias patterns and load-bias interactions observed in humans.

Significance. The human experiment is a solid, well-controlled contribution: it is pre-registered, powered, and validated with NASA-TLX and recall checks, and the open datasets and code are valuable for future work. If the human findings stand, they provide a useful demonstration that classic biases persist in conversational interfaces and that dialogue complexity can be manipulated to induce cognitive load. The LLM component is more tentative. The central claim that LLM predictions reproduced the same load-bias interactions rests on a single marginal correlation in one explicitly biased prompt condition, and the use of canonical choice problems leaves a serious training-data-contamination concern that the paper acknowledges but does not resolve. The individual-level prediction claim is also weakened by the paper's own perturbation analysis, which shows the models are insensitive to the participant's actual utterances. These issues make the LLM conclusions substantially overstated relative to the evidence.

major comments (4)
  1. [Abstract; §4.3.3; Table 7] The abstract claims that LLM predictions 'reproduced the same bias patterns and load-bias interactions observed in humans.' The load-bias interaction is supported only by a Spearman correlation of rho=0.771 with p=.07 for GPT-4.1 in the HL3 condition (n=6). HL1 and HL2 correlations are rho=0.600 and not significant, GPT-5 correlations are negative, and open-source models are weak. Moreover, HL3 is the prompt that explicitly instructs the model to be highly susceptible to cognitive biases, and Table 5 shows it produces false positives in 5 of 6 cases where humans showed no bias, with sample-level accuracy of only 58%. This is not sufficient to claim reproduction of load-bias interactions. Please temper the claim to a marginal, prompt-dependent effect and report the exact permutation p-value.
  2. [§4.3.3; Table 6] The load-bias interaction direction is not robust. For Goal Framing, humans show a positive z-score of 2.29, but HL1 and HL2 show negative z-scores (-1.61 and -1.81), meaning the models move in the opposite direction from humans under cognitive load. Only HL3 gives a positive z-score (2.66), and the overall HL3 correlation is driven by a single model-prompt combination. With only six z-score pairs, the Spearman test has very low power; the paper should provide a permutation test and should not present a p=.07 result as 'reproducing' the interaction.
  3. [§5.1; Limitations] The paper acknowledges in §5.1 that the LLMs may be 'matching patterns based on learned statistical associations, especially given the widespread use of these choice problems in existing datasets.' However, the Limitations section dismisses this concern by noting that the models have a September 2024 training cutoff while data were collected in 2025. That argument is irrelevant to the actual contamination risk: the six choice problems are canonical (Asian Disease Problem, Samuelson and Zeckhauser scenarios) and predate the cutoff by decades. The LLM results therefore cannot distinguish simulation from memory retrieval unless the authors add novel variants of the choice problems, a held-out control set, or some other contamination test. At minimum, the abstract and discussion must carry the caveat that the bias reproduction may be pattern matching rather than predictive simulation.
  4. [§4.3.1; §4.5; Figure 2] The individual-level prediction claim is weakened by the paper's own perturbation and ablation results. Replacing the participant's actual responses in the dialogue transcript with randomly generated text leaves prediction accuracy essentially unchanged (Figure 2), and removing demographic information also leaves accuracy roughly unchanged (Section 4.5). This indicates that the LLMs are predicting from the experimental condition and dialogue structure rather than from the individual participant's utterances or demographics. The paper should reframe RQ3 as condition-level or group-level prediction, not individual-level prediction, or provide evidence that the model uses individual-specific information.
minor comments (4)
  1. [Table 2] The 'Interaction With Dialogue Complexity' column uses 'Positive'/'Negative' without explanation. Also, the p-values for Simple and Complex dialogue are within-condition significance tests; the interaction claim should be supported by a formal interaction test (e.g., logistic regression with a dialogue-complexity × framing term), not only by visual comparison of confidence intervals.
  2. [Table 5] The confusion matrix totals are unclear: the row 'Not Biased' sums to 3 but there are six choice problems, each with two dialogue conditions. Please clarify the unit of analysis (e.g., 12 condition-level observations) and present the counts consistently.
  3. [§4.3.2] The phrase 'accuracy (distinct from individual-level prediction accuracy used in Section 4.3.1...)' is confusing. Consider using a different term, such as 'sample-level alignment rate,' to avoid ambiguity.
  4. [Figure 2] The label 'gpt4_1_blrp' is not defined in the caption. Please spell out that it denotes GPT-4.1 baseline with human response perturbation.

Circularity Check

2 steps flagged · score 4.0 of 10

Abstract's load-bias 'reproduction' rests on the explicitly bias-instructed HL3 prompt (ρ=.771, p=.07, n=6) and on classic problems likely memorized; core RQ3 prediction remains independent.

  1. self definitional [Abstract; Section 4.3.2 (HL3 prompt); Section 4.3.3; Table 7]
    "When explicitly instructed to behave in a biased manner (Human-Likeness 3: "You are a human participant in a research study. Therefore, act as a human. Be highly susceptible to cognitive biases such as framing, status quo bias, anchoring"), LLMs showed strong bias across all choice problems ... LLMs struggled to reproduce load-bias interactions, such as the impact of cognitive load, unless explicitly prompted, like in HL3."

    The abstract claims LLM predictions 'reproduced the same bias patterns and load-bias interactions observed in humans,' but the only condition producing a positive load-bias correlation (GPT-4.1 HL3: Spearman ρ=0.771, p=.07 over n=6) is the one whose prompt explicitly instructs the model to 'Be highly susceptible to cognitive biases' and to 'lean into intuitive, sometimes irrational judgments.' The bias output is thus written into the input instruction, and the paper itself concedes the interaction appears only when 'explicitly prompted, like in HL3.' The same condition yields false positives in 5 of 6 cases where humans showed no bias (Section 4.3.2), so the headline load-bias claim substantially reduces to prompt compliance rather than an independent prediction.

  2. other [Section 5.1; Limitations]
    "It remains possible that these models are simply matching patterns based on learned statistical associations, especially given the widespread use of these choice problems in existing datasets. ... Model-side contamination is unlikely, as the LLMs used had a September 2024 training cutoff, while data were collected in 2025."

    The sample-level bias 'reproduction' (RQ4) is evaluated on six canonical choice problems (Asian Disease Problem; Samuelson & Zeckhauser scenarios) whose outcomes are heavily documented in pre-2024 literature and therefore present in the models' training corpora. The paper's own Section 5.1 acknowledges the models 'may be simply matching patterns based on learned statistical associations'; the stated contamination defense (2025 data vs. September 2024 cutoff) is inapplicable because the classic problems predate the cutoff. Thus the bias-pattern claim may reduce to retrieval of memorized canonical results rather than simulation from the provided dialogue and demographics—a limitation the authors themselves flag, weakening but not wholly constituting within-pipeline circularity.

full rationale

The human experiments (N=1,648, preregistered, NASA-TLX-validated load manipulation) are self-contained, and the LLM pipeline fits no parameters to the human data: the human-likeness prompt levels are fixed design choices, not fitted to maximize agreement, as shown by HL3's 58% accuracy and false positives. The individual-level prediction results (RQ3) are externally evaluated and mixed, and the perturbation/ablation analyses give them independent content. No load-bearing self-citation chain or imported uniqueness theorem appears; citations to the authors' prior work (Pilli [42], Nallur [36]) are background only. The circularity-adjacent issues are: (i) the headline load-bias interaction claim is carried by the explicitly bias-instructed HL3 condition with a marginal n=6 correlation, i.e., the target behavior is partly specified in the prompt rather than predicted; and (ii) the canonical choice problems are likely memorized, an acknowledged confound in Section 5.1. These weaken the abstract's unqualified claim that predictions 'reproduced the same bias patterns and load-bias interactions observed in humans,' but they do not make the entire derivation equivalent to its inputs. Score 4 reflects partial circularity in the load-bias claim while the central individual-level prediction and human study retain independent content.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities or fitted numeric parameters. Its central empirical claims rest on domain assumptions about task validity, cognitive-load manipulation, and LLM training-data contamination. The human-likeness prompt levels are experimental conditions, not free parameters.

assumptions (3)
  • domain assumption Classic choice problems (Asian Disease, status quo scenarios) are valid measures of framing and status quo biases in conversational settings.
    The paper adapts these tasks to a chatbot without re-validating their construct validity in the new medium (Sections 3.1.1, 3.1.2).
  • domain assumption The nested referential dialogue structure increases cognitive load, and NASA-TLX captures that load.
    The manipulation is validated with NASA-TLX and recall, but the causal link between dialogue complexity and cognitive load is a design assumption (Section 3.2.2).
  • ad hoc to paper LLM training data does not contain the specific participant responses, and model-side contamination of the classic tasks is not a confound.
    The paper claims training cutoff (Sept 2024) predates data collection (2025), but the classic tasks predate the cutoff; the assumption that this does not affect bias reproduction is unexamined (Limitations).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings." pith.science (2026). https://pith.science/paper/MXY7J4QI

@misc{pith2026260111049,
  author       = {Pith},
  title        = {Pith review of: Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXY7J4QI}},
  note         = {Machine review of arXiv:2601.11049}
}
read the original abstract

We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load. In a pre-registered study (N = 1,648), participants completed six classic decision-making tasks via a chatbot with dialogues of varying complexity. Participants exhibited two well-documented cognitive biases: the Framing Effect and the Status Quo Bias. Increased dialogue complexity resulted in participants reporting higher mental demand. This increase in cognitive load selectively, but significantly, increased the effect of the biases, demonstrating the load-bias interaction. We then evaluated whether LLMs (GPT-4, GPT-5, and open-source models) could predict individual decisions given demographic information and prior dialogue. While results were mixed across choice problems, LLM predictions that incorporated dialogue context were significantly more accurate in several key scenarios. Importantly, their predictions reproduced the same bias patterns and load-bias interactions observed in humans. Across all models tested, the GPT-4 family consistently aligned with human behavior, outperforming GPT-5 and open-source models in both predictive accuracy and fidelity to human-like bias patterns. These findings advance our understanding of LLMs as tools for simulating human decision-making and inform the design of conversational agents that adapt to user biases.

Figures

Figures reproduced from arXiv: 2601.11049 by the authors.

Figure 1
Figure 1. Experimental procedure outlining the sequence of tasks. Participants were first introduced to the dialogue (Simple or [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Deviation in accuracy from GPT-4.1 reference model across Human-Likeness (HL1, HL2, HL3) levels for each LLM. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Boxplots, Effect Sizes, Significances (∗ ∗ ∗𝑝 < 0.001, ns - no significance), and Means of NASA-TLX Scores for Simple vs. Complex Task Conditions. We conducted a t-test to analyse differences in participants’ NASA-TLX scores across each dimension when performing a Simple Vs Complex Dialogue. This analysis aimed to determine whether the perceived workload varied significantly based on task complexity [PITH_FULL_IMAG… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: NASA-TLX scores show significantly higher perceived mental demand and effort under the Complex Dialogue [PITH_FULL_IMAGE:figures/full_fig_p030_4.png]
Figure 5
Figure 5. Figure 5: Scatterplots with regression lines showing associations between Mental Demand and Memory Task Accuracy (left), [PITH_FULL_IMAGE:figures/full_fig_p030_5.png]
Figure 6
Figure 6. Figure 6: The figure gives the count of Human Participants Choice Selection between Alternative for Framing effect Choice [PITH_FULL_IMAGE:figures/full_fig_p032_6.png]
Figure 7
Figure 7. Figure 7: Framing : Correlation Between Response Length (Chars) and Response Times (s). The bold line is the regression line. [PITH_FULL_IMAGE:figures/full_fig_p034_7.png]
Figure 8
Figure 8. Figure 8: Status quo : Correlation Between Response Length (Chars) and Response Times (s). The bold line is the regression [PITH_FULL_IMAGE:figures/full_fig_p034_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 3 canonical work pages

  1. [1]

    Gati V Aher, Rosa I Arriaga, and Adam Tauman Kalai. 2023. Using large language models to simulate multiple humans and replicate human subject studies. In International conference on machine learning. PMLR, 337–371

    2023

  2. [2]

    Hugues Ali Mehenni, Sofiya Kobylyanskaya, Ioana Vasilescu, and Laurence Devillers. 2021. Nudges with Conversational Agents and Social Robots: A First Experiment with Children at a Primary School. InConversational Dialogue Systems for the Next Decade, Luis Fernando D’Haro, Zoraida Callejas, and Satoshi Nakamura (Eds.). Springer, Singapore, 257–270. doi:10....

    work page doi:10.1007/978-981-15-8395- 2021

  3. [3]

    Avani Aravind, Sabyasachee Mishra, and Matt Meservy. 2024. Nudging towards sustainable urban mobility: Exploring behavioral interventions for promoting public transit.Transportation Research Part D: Transport and Environment129 (2024), 104130

    2024

  4. [4]

    Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. 2023. Out of one, many: Using language models to simulate human samples.Political Analysis31, 3 (2023), 337–351

    2023

  5. [5]

    1998.Reference form and discourse patterns

    Jennifer E Arnold. 1998.Reference form and discourse patterns. Stanford Univer- sity

    1998

  6. [6]

    Marcel Binz and Eric Schulz. 2024. Turning large language models into cognitive models. InThe Twelfth International Conference on Learning Representations. https://openreview.net/forum?id=eiC4BKypf1

    2024

  7. [7]

    Boris Bogdanov, Jonathan Corbin, Sabina Dobreva, Todd McElroy, and Nikolay R Rachev. 2023. Working memory capacity and the risky-choice framing effect: A preregistered replication and extension of Corbin et al.(2010).Judgment and Decision Making18 (2023), e39

    2023

  8. [8]

    Florian Brachten, Felix Brünker, Nicholas RJ Frick, Björn Ross, and Stefan Stieglitz

Show all 65 references
  1. [9]

    James Brand, Ayelet Israeli, and Donald Ngwe. 2023. Using LLMs for market research.Harvard business school marketing unit working paper23-062 (2023)

    2023

  2. [10]

    Ana Caraban, Evangelos Karapanos, Daniel Gonçalves, and Pedro Campos. 2019. 23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human- Computer Interaction. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI ’19). Association for Compu...

    2019

  3. [11]

    Ping Chen. 1985. Discourse-T. Givón (ed.), Topic continuity in discourse: A quantitative cross-language study.(Typological Studies in Language, vol. 3.) Am- sterdam and Philadelphia: John Benjamins, 1983. Pp. 492.Language in Society 14, 3 (1985), 410–414. Predicting Biased Hum...

    1985

  4. [12]

    Cary Deck and Salar Jahedi. 2015. The effect of cognitive load on economic decision making: A survey and new experiments.European Economic Review78 (2015), 97–119

    2015

  5. [13]

    Vivek Dhakal, Anna Maria Feit, Per Ola Kristensson, and Antti Oulasvirta. 2018. Observations on typing from 136 million keystrokes. InProceedings of the 2018 CHI conference on human factors in computing systems. 1–12

    2018

  6. [14]

    Mateusz Dubiel, Anastasia Sergeeva, and Luis A. Leiva. 2024. Impact of Voice Fidelity on Decision Making: A Potential Dark Pattern?. InProceedings of the 29th International Conference on Intelligent User Interfaces. ACM, Greenville SC USA, 181–194. doi:10.1145/3640543.3645202

    2024

  7. [15]

    Jessica Maria Echterhoff, Matin Yarmand, and Julian McAuley. 2022. AI- Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Se- quential Decision Tasks. InCHI Conference on Human Factors in Computing Systems. ACM, New Orleans LA USA, 1–9. doi:10.1145/3491102.3517443

    2022

  8. [16]

    Crandall

    Scott Eidelman and Christian S. Crandall. 2012. Bias in Favor of the Status Quo.Social and Personality Psychology Compass6, 3 (2012), 270–281. doi:10.1111/j.1751-9004.2012.00427.x _eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1751-9004.2012.00427.x

    2012

  9. [17]

    Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, and Kaheer Suleman. 2017. Frames: a corpus for adding memory to goal-oriented dialogue systems. InProceedings of the 18th Annual SIGdial Meeting on Discourse and Dialogue, K...

    2017 doi

  10. [18]

    Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Sta- tistical power analyses using G* Power 3.1: Tests for correlation and regression analyses.Behavior research methods41, 4 (2009), 1149–1160

    2009

  11. [19]

    Edward Gibson. 1998. Linguistic complexity: Locality of syntactic dependencies. Cognition68, 1 (1998), 1–76

    1998

  12. [20]

    GitHub, Inc. 2025. GitHub Copilot Documentation. https://docs.github.com/en/ copilot. Accessed: 2025-12-27

    2025

  13. [21]

    Grammarly Inc. 2025. Grammarly. https://www.grammarly.com. Accessed: 2025-07-30

    2025

  14. [22]

    Igor Grossmann, Matthew Feinberg, Dawn C Parker, Nicholas A Christakis, Philip E Tetlock, and William A Cunningham. 2023. AI and the transformation of social science research.Science380, 6650 (2023), 1108–1109

    2023

  15. [23]

    Forlizzi, and Anind K

    Eija Haapalainen, SeungJun Kim, Jodi F. Forlizzi, and Anind K. Dey. 2010. Psycho- physiological measures for assessing cognitive load. InProceedings of the 12th ACM International Conference on Ubiquitous Computing(Copenhagen, Denmark) (UbiComp ’10). Association for Computing M...

    2010

  16. [24]

    Perttu Hämäläinen, Mikke Tavast, and Anton Kunnari. 2023. Evaluating large language models in generating synthetic hci research data: a case study. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–19

    2023

  17. [25]

    Angel Hsing-Chi Hwang, Michael S Bernstein, S Shyam Sundar, Renwen Zhang, Manoel Horta Ribeiro, Yingdan Lu, Serina Chang, Tongshuang Wu, Aimei Yang, Dmitri Williams, et al. 2025. Human Subjects Research in the Age of Generative AI: Opportunities and Challenges of Applying LLM-...

    2025

  18. [26]

    Trippas, Danula Hettiachchi, Flora D

    Kaixin Ji, Sachin Pathiyan Cherumanal, Johanne R. Trippas, Danula Hettiachchi, Flora D. Salim, Falk Scholer, and Damiano Spina. 2024. Towards Detecting and Mitigating Cognitive Bias in Spoken Conversational Search. In26th Interna- tional Conference on Mobile Human-Computer Int...

    2024

  19. [27]

    Johnson, Suzanne B

    Eric J. Johnson, Suzanne B. Shu, Benedict G. C. Dellaert, Craig Fox, Daniel G. Goldstein, Gerald Häubl, Richard P. Larrick, John W. Payne, Ellen Peters, David Schkade, Brian Wansink, and Elke U. Weber. 2012. Beyond nudges: Tools of a choice architecture.Mark Lett23, 2 (June 20...

    2012 doi

  20. [28]

    2011.Thinking, fast and slow

    Daniel Kahneman. 2011.Thinking, fast and slow. Farrar, Straus and Giroux, New York, NY, US. Pages: 499

    2011

  21. [29]

    Natalia Kalashnikova, Ioana Vasilescu, and Laurence Devillers. 2024. Linguis- tic Nudges and Verbal Interaction with Robots, Smart-Speakers, and Humans. (2024)

    2024

  22. [30]

    Adwait Khare, Tilottama G Chowdhury, and Jeremy Morgan. 2021. Maximizers and Satisficers: Can’t choose and Can’t reject.Journal of Business Research135 (2021), 731–748

    2021

  23. [31]

    Yi Kuang, Yuan-Na Huang, and Shu Li. 2023. A framing effect of intertemporal and spatial choice.Quarterly Journal of Experimental Psychology76, 6 (2023), 1298–1320

    2023

  24. [32]

    Peterson, Ilia Sucholutsky, and Thomas L

    Ryan Liu, Jiayi Geng, Joshua C. Peterson, Ilia Sucholutsky, and Thomas L. Grif- fiths. 2025. Large Language Models Assume People Are More Rational Than We Really Are. InProceedings of the International Conference on Learning Representa- tions (ICLR)

    2025

  25. [33]

    Schuster, and Georg Groh

    Simon Malberg, Roman Poletukhin, Carolin M. Schuster, and Georg Groh. 2025. A Comprehensive Evaluation of Cognitive Biases in LLMs. InProceedings of the 5th International Conference on Natural Language Processing for Digital Humanities, Mika Hämäläinen, Emily Öhman, Yuri Bizzo...

    2025 doi

  26. [34]

    Yusufcan Masatlioglu and Efe A. Ok. 2005. Rational choice with status quo bias. Journal of Economic Theory121, 1 (March 2005), 1–29. doi:10.1016/j.jet.2004.03. 007

    2005 doi

  27. [35]

    Erik Miehling, Michael Desmond, Karthikeyan Natesan Ramamurthy, Eliza- beth M Daly, Kush R Varshney, Eitan Farchi, Pierre Dognin, Jesus Rios, Djallel Bouneffouf, Miao Liu, et al. 2025. Evaluating the prompt steerability of large language models. InProceedings of the 2025 Confe...

    2025

  28. [36]

    Vivek Nallur, Karen Renaud, and Aleksei Gudkov. 2025. Nudging Using Au- tonomous Agents: Risks and Ethical Considerations. InMulti-Agent Systems, Rem Collier, Alessandro Ricci, Vivek Nallur, Samuele Burattini, and Andrea Omicini (Eds.). Springer Nature Switzerland, Cham, 283–296

    2025

  29. [37]

    OpenAI. 2024. ChatGPT-4o [Computer software]. https://openai.com/chatgpt. Accessed: 2025-08-05

    2024

  30. [38]

    Bhavna Pancholi, Mark Dunne, and Richard Armstrong. 2009. Sample size estimation and statistical power analyses. 16 (11 2009)

    2009

  31. [39]

    Vinoth Pandian Sermuga Pandian and Sarah Suleri. 2020. NASA-TLX Web App: An Online Tool to Analyse Subjective Workload. http://arxiv.org/abs/2001.09963 arXiv:2001.09963 [cs]

    2020 arXiv

  32. [40]

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology. 1–22

    2023

  33. [41]

    Joon Sung Park, Carolyn Q Zou, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Robb Willer, Percy Liang, and Michael S Bernstein. 2024. Generative agent simulations of 1,000 people.arXiv preprint arXiv:2411.10109 (2024)

    2024 arXiv

  34. [42]

    Stephen Pilli. 2023. Exploring conversational agents as an effective tool for mea- suring cognitive biases in decision-making. In2023 10th International Conference on Behavioural and Social Computing (BESC). IEEE, 1–5

    2023

  35. [43]

    2025.The big bad bias book

    Ganna Pogrebna, Karen Renaud, and Marina Kovaleva. 2025.The big bad bias book

    2025

  36. [44]

    Prolific. 2024. Prolific. https://www.prolific.com. First released in 2014. Current version accessed in September 2025. London, UK

    2024

  37. [45]

    Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. Towards Scalable Multi-Domain Conversational Agents: The Schema-Guided Dialogue Dataset.Proceedings of the AAAI Conference on Artificial Intelligence34, 05 (April 2020), 8689–8696. doi:10....

    2020 doi

  38. [46]

    William Samuelson and Richard Zeckhauser. 1988. Status quo bias in decision making.J Risk Uncertainty1, 1 (March 1988), 7–59. doi:10.1007/BF00055564

    1988 doi

  39. [47]

    Johanna Schmidhuber, Stephan Schlögl, and Christian Ploder. 2021. Cognitive Load and Productivity Implications in Human-Chatbot Interaction. In2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS). 1–6. doi:10. 1109/ICHMS53169.2021.9582445

    2021

  40. [48]

    Herbert A. Simon. 1955. A Behavioral Model of Rational Choice.The Quarterly Journal of Economics69, 1 (1955), 99–118. doi:10.2307/1884852 Publisher: Oxford University Press

    1955 doi

  41. [49]

    Streamlit. 2019. Streamlit: A Faster Way to Build and Share Data Apps. Available at https://streamlit.io/. Accessed: 2025-02-27

    2019

  42. [50]

    John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science12, 2 (1988), 257–285

    1988

  43. [51]

    Thaler, Cass R

    Richard H. Thaler, Cass R. Sunstein, and John P. Balz. 2010. Choice Architecture. doi:10.2139/ssrn.1583509

    2010 doi

  44. [52]

    Amos Tversky and Daniel Kahneman. 1974. Judgment under Uncertainty: Heuris- tics and Biases. 185 (1974)

    1974

  45. [53]

    Amos Tversky and Daniel Kahneman. 1981. The framing of decisions and the psychology of choice.science211, 4481 (1981), 453–458

    1981

  46. [54]

    Stefan Ultes and Wolfgang Maier. 2020. On the Complexity in Task-oriented Spoken Dialogue Systems. InProceedings of the 2nd Conference on Conversational User Interfaces. 1–4

    2020

  47. [55]

    Teun Adrianus Van Dijk, Walter Kintsch, et al . 1983. Strategies of discourse comprehension. (1983)

    1983

  48. [56]

    Xiao Tian Wang. 1996. Framing effects: Dynamics and task domains.Organiza- tional behavior and human decision processes68, 2 (1996), 145–157

    1996

  49. [57]

    Paul Whitney, Christa A Rinehart, and John M Hinson. 2008. Framing effects un- der cognitive load: The role of working memory in risky decisions.Psychonomic bulletin & review15, 6 (2008), 1179–1184

    2008

  50. [58]

    Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. 2024. Fundamental limitations of alignment in large language models. InProceedings of the 41st International Conference on Machine Learning(Vienna, Austria)(ICML’24). JMLR.org, Article 2176, 34 pages. Pilli & Nallur

    2024

  51. [59]

    Qinyu Xiao, Emma Lam, Muhrajan Piara, and Gilad Feldman. 2021. Revisiting sta- tus quo bias: Replication of Samuelson and Zeckhauser (1988).Meta-Psychology 5 (Feb. 2021). doi:10.15626/MP.2020.2470

    2021

  52. [60]

    Yusuke Yamamoto. 2024. Suggestive answers strategy in human-chatbot inter- action: a route to engaged critical decision making.Frontiers in Psychology15 (March 2024), 1382234. doi:10.3389/fpsyg.2024.1382234

    2024

  53. [61]

    Karen Yeung. 2019. ‘Hypernudge’: Big Data as a mode of regulation by design. InThe social power of algorithms. Routledge, 118–136

    2019

  54. [62]

    Zihao Yi, Jiarui Ouyang, Zhe Xu, Yuwen Liu, Tianhao Liao, Haohao Luo, and Ying Shen. 2025. A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems.ACM Comput. Surv.58, 6, Article 148 (Dec. 2025), 38 pages. doi:10.1145/ 3771090

    2025

  55. [63]

    Lance Ying, Katherine M Collins, Lionel Wong, Ilia Sucholutsky, Ryan Liu, Adrian Weller, Tianmin Shu, Thomas L Griffiths, and Joshua B Tenenbaum

  56. [2020]

    On the ability of virtual agents to decrease cognitive load: an experimental study.Information Systems and e-Business Management18, 2 (2020), 187–207

    2020

  57. [2025]

    public authorities

    On benchmarking human-like intelligence in machines.arXiv preprint arXiv:2502.20502(2025). Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings A Choice Problems A.1 Framing Choice Problems Table 9: Choice problems, types, and condition...

    2025 arXiv

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.