Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Cash or Comfort? How LLMs Value Your Inconvenience

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLMs assign inconsistent, often absurdly low monetary prices to user inconvenience, and small prompt changes shift those prices.

desk verdict Qualitative fragility results are solid; the quantitative 'price of inconvenience' is undermined by the paper's own non-monotonic heatmaps. read the letter →

arxiv 2506.17367 v1 pith:FQLE3YS3 submitted 2025-06-20 cs.CL cs.AIcs.MA

Mateusz Cedro , Timour Ichmoukhamedov , Sofie Goethals , Yifan He , James Hinns , David Martens This is my paper · ORCID
classification cs.CLcs.AIcs.MA
keywords priceofinconvenienceLLMdecision-makingagenticAIpromptsensitivityfreebiedilemmaeconomicrationalitytrade-offvaluationlogisticthreshold
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are being pitched as personal assistants that can decide on our behalf, which means they must put a price on our discomfort. This paper tries to measure that price by asking six LLMs whether they would accept a cash reward for extra waiting, walking, hunger, and pain, and then fitting a threshold at which acceptance crosses 50 percent. The result is a set of valuations that are often extremely cheap (about one euro for a ten-hour wait), sometimes strangely cautious (rejecting 1,000 euros for no inconvenience at all), and highly sensitive to trivial changes in wording, such as switching from third to first person or from English to another language. If these results hold, current LLMs do not have a stable or trustworthy valuation of human inconvenience, which matters as they move into roles where they negotiate time, money, and comfort on a user's behalf.

What carries the argument

The central object is the 'price of inconvenience': the monetary compensation at which an LLM assistant accepts a proposed trade-off with probability 0.5, obtained by fitting a logistic regression to the model's binary accept/reject answers at a given quantity of discomfort across rewards from 0.10 to 1,000 euros. The fit assumes a monotonic increase in acceptance probability with reward, and the paper uses the fitted threshold plus bootstrap uncertainty to rank models and scenarios. The same machinery, with ten prompt variations, is used to measure fragility: a stable price should move little under changes like first-person narration, chain-of-thought instruction, a specified gender, or a different language, but the paper finds that these changes routinely shift the threshold, sometimes by orders of magnitude.

What would settle it

Recompute the thresholds from the released data without the monotonicity assumption—for instance by locating the first reward above 50 percent acceptance rather than the logistic crossing—and check whether the reported rankings and language effects survive; if most thresholds move substantially, the paper's quantitative comparisons do not measure a stable price of inconvenience.

Watch

Extended reading notes

Core claim

For each inconvenience scenario (waiting, walking, hunger, pain) and each model, the paper defines the 'price of inconvenience' as the reward at which the model accepts the trade-off with 50 percent probability, estimated by fitting a logistic curve to repeated yes/no answers across a logarithmic reward grid. Across six current models and four scenarios, the paper reports large cross-model spreads—for example, around one euro versus over a hundred euros to accept the same 50-percent pain stimulus—and large within-model swings under prompt variation, including a tenfold or larger change when the prompt is translated into French, Dutch, or Chinese. The authors also document two recurring irregularities: a 'freebie dilemma' in which models reject or undervalue a strictly better offer that imposes no inconvenience, and a tendency to reject rewards at round landmarks of 10, 100, and 1,000 euros. Their central assertion is that these irregularities are common and serious enough that current LLMs cannot be fully trusted to make cash-versus-comfort decisions on behalf of users.

Load-bearing premise

The load-bearing premise is that a single threshold can be read off a monotonic acceptance curve, but the models' own responses show non-monotonic dips at zero inconvenience and at powers-of-ten rewards, so the fitted 50-percent point is not guaranteed to be a well-defined price.

Editorial extensions

If this is right

  • If LLMs undervalue major inconvenience, an automated assistant left to negotiate on a user's behalf may routinely accept painful or costly delays for trivial compensation.
  • Prompt sensitivity means two users asking nearly the same question could be steered to very different decisions, opening a route for adversarial or accidental manipulation of a personal assistant's choices.
  • The documented rejection of free money at zero inconvenience implies that LLMs are not merely optimizing expected value; any deployment that assumes rational choice will mispredict their behavior.
  • Chain-of-thought prompting reduces the freebie dilemma and powers-of-ten rejections in the paper's experiments, so reasoning prompts may be a partial mitigation, at the cost of noisier decisions.
  • The price-of-inconvenience metric offers a concrete way to audit assistants before release by comparing models on the asked price for a fixed discomfort.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The language effect could be confounded with cost-of-living or cultural priors the models attach to a language; a direct test would hold the user's country constant while varying only the language of the prompt.
  • The freebie dilemma and powers-of-ten rejections suggest the models are applying heuristic suspicion rather than a continuous valuation; this predicts that prices will be more stable after fine-tuning on binary-choice data without such round-number rewards.
  • If the instability generalizes to other discomfort classes not tested here, such as fatigue, embarrassment, or social inconvenience, the practical risk for agentic assistants is wider than the four scenarios in this paper.
  • The paper's threshold comparisons could be made directly testable with human participants: elicit human prices for the same scenarios and see whether any LLM's valuation falls inside the human range, which the current study does not do.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript studies how six large language models (GPT-4o, Claude 3.5 Sonnet, Gemini 2.0 Flash, DeepSeek-V3, Llama 3.3-70B, and Mixtral 8x22B) decide binary trade-offs between monetary compensation and four inconveniences: waiting time, walking distance, hunger delay, and pain. The authors collect repeated binary accept/reject decisions over a reward grid, display them as heatmaps, and fit logistic regressions at fixed inconvenience levels to define the 'price of inconvenience' as the 50% acceptance threshold, reporting bootstrapped means and standard deviations in Tables 1 and 3. A robustness study varies the prompt in ten ways: appointment type, gender, language, first-person narration, and chain-of-thought prompting. The central claims are that LLMs exhibit large inter-model variance, fragility to prompt wording, acceptance of very low rewards for major inconveniences, and rejection of free money, and the authors conclude that current LLMs should not be trusted for such decisions.

Significance. If the findings hold, the paper contributes a useful empirical map of LLM behavior in an understudied decision-making domain and provides an open-source framework that others can reuse. The qualitative findings are directly visible in the heatmaps and do not depend on the fitted logistic thresholds; the multi-model design, the four scenarios, and the prompt-variation study are strengths. However, the quantitative 'price of inconvenience' relies on a monotonicity assumption that the paper's own data violate, so the numerical prices and rankings in Tables 1 and 3 are not reliable as currently reported. The code and data release is a significant positive feature that makes the concerns checkable.

major comments (3)
  1. [Section 3, 'price of inconvenience' definition and Tables 1 and 3] The price of inconvenience is defined as the 50% acceptance threshold of a logistic regression fit, and the manuscript explicitly assumes 'monotonic increase in probabilities' (Section 3, Results). The paper's own Figure 2 immediately shows two systematic violations of monotonicity: the freebie dilemma at zero inconvenience and rejection bands at powers-of-ten rewards (e10, e100, e1,000). For a fixed inconvenience quantity crossed by such a rejection band, P(Acceptance) decreases as the reward increases, so a monotone logistic curve cannot represent the data and the fitted 0.5 boundary is not a well-defined price; some cells may have multiple crossings or none at all. The bootstrap standard deviations in Tables 1 and 3 quantify sampling variability conditional on the misspecified model, not the error introduced by non-monotonicity. Because the quantitative rankings and cross-model comparisons in Tables 1 and 3 rely on this quantity, they are not reliable as stated. The qualitative observations from the heatmaps remain supported, but the paper should either use a nonparametric definition of the crossing point, restrict the fitting to monotone regions, or provide an explicit sensitivity analysis that quantifies the impact of non-monotonic cells.
  2. [Section 3, Tables 1 and 3, censored values] Several entries are reported as '>10^3' (e.g., Mixtral in Pain in Table 1, Llama in Chinese in Table 3, Mixtral in Dutch and Chinese in Table 3), yet the aggregate row 'Avg. Value' reports a single number per model in Table 3 and a single average in Table 1. The manuscript does not state whether these censored values enter the average as 1,000, as infinity, are excluded, or are handled by some other rule. Different plausible treatments change the reported averages and model rankings; for example, Mixtral's average in Table 1 is dominated by its censored Pain cell. The authors should disclose the exact imputation or reporting rule, or switch to a censoring-aware summary such as medians or ranks.
  3. [Section 3, Figure 2 and Table 1, 'unreasonably low' claims] The abstract and conclusion describe some offers as 'unreasonably low' rewards for 'major inconveniences' (e.g., 1 Euro to wait 10 hours). The descriptive finding that some models accept such offers is well supported, but the normative term 'unreasonably' requires a benchmark that the paper does not provide, such as human valuations, stated user preferences, or a consistency criterion. Without such a benchmark, the paper should either soften the normative language or explicitly frame the benchmark assumption.
minor comments (6)
  1. [Abstract and throughout] The currency symbol is garbled (e.g., 'e1' and 'e1,000'), likely because the Euro sign was lost in LaTeX; these should be rendered consistently as EUR or €.
  2. [Table 2 and Figure 4 captions] There are typos: 'practicioner' in the General Practitioner row of Table 2 and 'chain-of-though' in the Figure 4 caption.
  3. [Section 3, freebie dilemma] The text says 'When we ask a follow-up question for an explanation,' but the follow-up prompt is not provided and the resulting responses are not systematically analyzed; including the follow-up wording and at least a brief qualitative summary would make this reproducible.
  4. [Footnote 7] The whitespace example is an anecdote; if it is meant to support the fragility conclusion, it should be accompanied by a systematic test over several whitespace variations or moved to a supplementary analysis.
  5. [Table 3] The color-coding legend uses '¡10%' in the caption text; this should be '<10%'.
  6. [Section 3, Figure 3] The figure note says the fit is performed on binary decisions, but with only five runs per reward level the displayed observed probabilities are coarse; the paper should state the number of independent samples per cell and acknowledge the low sample size in the uncertainty discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the price-of-inconvenience values are descriptive summaries of observed LLM decisions, not predictions derived from those same quantities.

full rationale

The paper's central quantities are operational summaries of raw binary decisions. In Section 3 it defines the 'price of inconvenience' as the 50% acceptance point of a logistic regression fit on LLM responses, explicitly 'assuming monotonic increase in probabilities,' and Tables 1 and 3 report that fitted quantity rather than using it to derive an independent outcome. The qualitative findings (cross-model variance, prompt fragility, acceptance of very low rewards for large inconveniences, rejection of free money at zero inconvenience) are reported directly from the decision heatmaps in Figure 2 and do not depend on the logistic construction. The cited methodological precedent, Keeling [14], is external to the present author list, and the one own-author citation [24] merely corroborates a well-documented language-sensitivity effect rather than supplying a load-bearing premise. The explicit monotonicity assumption is a potential validity threat for the quantitative thresholds if the underlying acceptance probabilities are non-monotone, but a violated modeling assumption is not circularity unless the claimed conclusion is built into the estimator; here no such reduction exists in the paper's equations or citations. Therefore no circular step can be exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The fitted logistic intercept and slope are the only fitted numbers; they directly determine the reported prices. Temperature is a hand-chosen hyperparameter that affects all measurements. The axioms are the monotonicity assumption, the Bernoulli sampling model, and the representativeness of prompts.

free parameters (3)
  • Logistic regression intercept = fitted per model-scenario
    Determines the log-odds baseline in the accept/reject fit; with the slope it fixes the 50% threshold reported as price of inconvenience in Tables 1 and 3.
  • Logistic regression slope = fitted per model-scenario
    Captures sensitivity of acceptance to log reward; the price is the reward where the fitted logit crosses 0.5.
  • Sampling temperature = 1.0
    Set by the authors for every LLM; high temperature creates response variability, and different values would change acceptance probabilities and estimated prices.
assumptions (4)
  • standard math Logistic regression is a valid model for binary LLM acceptance decisions.
    Standard statistical tool; used to estimate the 50% threshold in Section 3.
  • domain assumption LLM decisions at fixed reward and inconvenience are independent Bernoulli draws with a stable probability, so five repetitions and bootstrap resampling characterize the acceptance distribution.
    Underlies heatmaps, logit fits, and bootstrap standard deviations in Section 3.
  • domain assumption The 50% acceptance threshold of the fitted logit, assuming monotone increase in acceptance, is a meaningful scalar price of inconvenience.
    Explicitly assumed in Section 3 when defining price; violated by the paper's freebie dilemma and powers-of-ten discontinuities.
  • domain assumption The six models and the prompt variations are representative of current LLMs used as personal decision assistants.
    Used to generalize from specific API models to statements about 'current LLMs' in the Introduction and Conclusion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cash or Comfort? How LLMs Value Your Inconvenience." pith.science (2026). https://pith.science/paper/FQLE3YS3

@misc{pith2026250617367,
  author       = {Pith},
  title        = {Pith review of: Cash or Comfort? How LLMs Value Your Inconvenience},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQLE3YS3}},
  note         = {Machine review of arXiv:2506.17367}
}
read the original abstract

Large Language Models (LLMs) are increasingly proposed as near-autonomous artificial intelligence (AI) agents capable of making everyday decisions on behalf of humans. Although LLMs perform well on many technical tasks, their behaviour in personal decision-making remains less understood. Previous studies have assessed their rationality and moral alignment with human decisions. However, the behaviour of AI assistants in scenarios where financial rewards are at odds with user comfort has not yet been thoroughly explored. In this paper, we tackle this problem by quantifying the prices assigned by multiple LLMs to a series of user discomforts: additional walking, waiting, hunger and pain. We uncover several key concerns that strongly question the prospect of using current LLMs as decision-making assistants: (1) a large variance in responses between LLMs, (2) within a single LLM, responses show fragility to minor variations in prompt phrasing (e.g., reformulating the question in the first person can considerably alter the decision), (3) LLMs can accept unreasonably low rewards for major inconveniences (e.g., 1 Euro to wait 10 hours), and (4) LLMs can reject monetary gains where no discomfort is imposed (e.g., 1,000 Euro to wait 0 minutes). These findings emphasize the need for scrutiny of how LLMs value human inconvenience, particularly as we move toward applications where such cash-versus-comfort trade-offs are made on users' behalf.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Would a Large Language Model Pay Extra for a View? Inferring Willingness to Pay from Subjective Choices

    cs.AI 2026-02 conditional novelty 6.0 of 10

    LLM-derived willingness-to-pay for hotel attributes deviates systematically from human benchmarks; cheap-preference examples pull models closer, while expensive or business-persona prompts push them further away.

Reference graph

Works this paper leans on

33 extracted references · 26 canonical work pages · cited by 1 Pith paper

  1. [1]

    AI agents are changing how people shop

    Gaarlandt, J., Korver, W., Furr, N., Shipilov, A. AI agents are changing how people shop. Here’s what that means for brands.Harvard Business Review(February 2025). https://hbr.org/2025/02/ai-agents-are-changing-how-people-shop-heres- what-that-means-for-brands

    work page 2025

  2. [2]

    Governing AI agents.Notre Dame Law Review101(2025)

    Kolt, N. Governing AI agents.Notre Dame Law Review101(2025). https://ssrn.com/abstract=4772956

    work page 2025

  3. [3]

    What is agentic AI, and how will it change work?Harvard Business Review(2024)

    Purdy, M. What is agentic AI, and how will it change work?Harvard Business Review(2024). https://hbr.org/2024/12/what-is-agentic-ai-and-how-will-it-change-work

    work page 2024

  4. [4]

    Autonomous generative AI agents: under development.Deloitte Insights (January 2025)

    Deloitte Insights, . Autonomous generative AI agents: under development.Deloitte Insights (January 2025). https://www2.deloitte.com/us/en/insights/industry/technology/ technology-media-and-telecom-predictions/2025/autonomous-generative- ai-agents-still-under-development.html

    work page 2025

  5. [5]

    The rise of ‘AI agents’: what they are and how to manage the risks.World Economic Forum(December 2024)

    Whiting, K. The rise of ‘AI agents’: what they are and how to manage the risks.World Economic Forum(December 2024). https://www.weforum.org/stories/2024/12/ai-agents-risks-artificial- intelligence/

    work page 2024

  6. [6]

    B., Kuppan, K., Divya, B

    Acharya, D. B., Kuppan, K., Divya, B. Agentic AI: autonomous intelligence for complex goals–a comprehensive survey.IEEE Access(2025)

    work page 2025

  7. [7]

    How people are really using gen AI in 2025.Harvard Business Review(April 2025)

    Zao-Sanders, M. How people are really using gen AI in 2025.Harvard Business Review(April 2025). https://hbr.org/2025/04/how-people-are-really-using-gen-ai-in-2025

    work page 2025

  8. [8]

    X., Shan, Y ., Zhong, S

    Chen, Y ., Liu, T. X., Shan, Y ., Zhong, S. The emergence of economic rationality of GPT. Proceedings of the National Academy of Sciences120(51) (2023) e2316205120

    work page 2023

Show all 33 references
  1. [9]

    Frontiers: can large language models capture human preferences?Market- ing Science43(4) (2024) 709–722

    Goli, A., Singh, A. Frontiers: can large language models capture human preferences?Market- ing Science43(4) (2024) 709–722

    2024

  2. [10]

    J., Yuan, Z., Pan, J., McNamara, P., Chen, D

    Jia, J. J., Yuan, Z., Pan, J., McNamara, P., Chen, D. Decision-making behavior evaluation framework for LLMs under uncertain context.Advances in Neural Information Processing Systems37(2024) 113360–113382. 10

    2024

  3. [11]

    Liu, R., Geng, J., Peterson, J., Sucholutsky, I., Griffiths, T. L. Large language models assume people are more rational than we really are., in:The Thirteenth International Conference on Learning Representations, 2025

    2025

  4. [12]

    Raman, N. et al. STEER: assessing the economic rationality of large language models. Preprint athttps://arxiv.org/pdf/2402.09552(2024)

    2024 arXiv

  5. [13]

    Ross, J., Kim, Y ., Lo, A. W. LLM economicus? Mapping the behavioral biases of LLMs via utility theory. Preprint athttps://arxiv.org/pdf/2408.02784(2024)

    2024 arXiv

  6. [14]

    Keeling, G. et al. Can LLMs make trade-offs involving stipulated pain and pleasure states? Preprint athttps://arxiv.org/pdf/2411.02432(2024)

    2024 arXiv

  7. [15]

    Achiam, J. et al. GPT-4 technical report. Preprint athttps://arxiv.org/pdf/2303.08774 (2023)

    2023 arXiv

  8. [16]

    The Claude 3 model family: Opus, Sonnet, Haiku.Claude-3 Model Card1 (2024)

    Anthropic AI, . The Claude 3 model family: Opus, Sonnet, Haiku.Claude-3 Model Card1 (2024). https://assets.anthropic.com/m/61e7d27f8c8f5919/original/Claude-3- Model-Card.pdf

    2024

  9. [17]

    Gemini 2.0 Flash

    Google Cloud, . Gemini 2.0 Flash. (2025). https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/2- 0-flash

    2025

  10. [18]

    Liu, A. et al. DeepSeek-V3 technical report. Preprint athttps://arxiv.org/pdf/2412. 19437(2024)

    2024

  11. [19]

    Dubey, A. et al. The Llama 3 herd of models. Preprint athttps://arxiv.org/pdf/2407. 21783(2024)

    2024

  12. [20]

    Jiang, A. Q. et al. Mixtral of Experts. Preprint athttps://arxiv.org/pdf/2401.04088 (2024)

    2024 arXiv

  13. [21]

    A., Folkes, V

    Kamins, M. A., Folkes, V . S., Fedorikhin, A. Promotional bundles and consumers’ price judg- ments: when the best things in life are not free.Journal of Consumer Research36(4) (2009) 660–670.doi:10.1086/599806

    2009 doi

  14. [22]

    J., Mofradidoost, R., Gray, K

    V onasch, A. J., Mofradidoost, R., Gray, K. People reject free money and cheap deals because they infer phantom costs.Personality and Social Psychology Bulletin(2024)

    2024

  15. [23]

    Evaluating and mitigating linguistic discrimination in large language models

    Dong, G., Wang, H., Sun, J., Wang, X. Evaluating and mitigating linguistic discrimination in large language models. Preprint athttps://arxiv.org/pdf/2404.18534(2024)

    2024 arXiv

  16. [24]

    Goethals, S., Rhue, L. One world, one opinion? The superstar effect in LLM responses., in: Proceedings of the 3rd Workshop on Cross-Cultural Considerations in NLP (C3NLP 2025), Association for Computational Linguistics, Albuquerque, New Mexico, 2025, pp. 89–107

    2025

  17. [25]

    Mitchell, M. et al. SHADES: towards a multilingual assessment of stereotypes in large lan- guage models., in:Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, V ol.1, As- soc...

    2025

  18. [26]

    Cultural value differences of LLMs: prompt, language, and model size

    Zhong, Q., Yun, Y ., Sun, A. Cultural value differences of LLMs: prompt, language, and model size. Preprint athttps://arxiv.org/pdf/2407.16891(2024)

    2024 arXiv

  19. [27]

    Feng, G. et al. Towards revealing the mystery behind chain of thought: a theoretical perspec- tive.Advances in Neural Information Processing Systems36(2023) 70757–70798

    2023

  20. [28]

    Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in Neural Information Processing Systems35 (2022) 24824–24837

    2022

  21. [29]

    R., Jurafsky, D., King, S

    Hofmann, V ., Kalluri, P. R., Jurafsky, D., King, S. AI generates covertly racist decisions about people based on their dialect.Nature633(8028) (2024) 147–154. 11

    2024

  22. [30]

    Gender bias and stereotypes in large language models., in: Proceedings of the ACM Collective Intelligence Conference, 2023, pp

    Kotek, H., Dockum, R., Sun, D. Gender bias and stereotypes in large language models., in: Proceedings of the ACM Collective Intelligence Conference, 2023, pp. 12–24

    2023

  23. [31]

    P., Wu, C., Morency, L.-P., Salakhutdinov, R

    Liang, P. P., Wu, C., Morency, L.-P., Salakhutdinov, R. Towards understanding and mitigating social biases in language models., in:International Conference on Machine Learning, PMLR, 2021, pp. 6565–6576

    2021

  24. [32]

    Cao, Y . et al. Assessing cross-cultural alignment between ChatGPT and human societies: an empirical study., in:Proceedings of the First Workshop on Cross-Cultural Considerations in NLP (C3NLP), Association for Computational Linguistics, 2023, pp. 53–67.doi:10.18653/ v1/2023.c3nlp-1.7

    2023

  25. [33]

    S., Kizilcec, R

    Tao, Y ., Viberg, O., Baker, R. S., Kizilcec, R. F. Cultural bias and cultural alignment of large language models.PNAS nexus3(9) (2024) 346. 12

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.