Greedy or not, here I come: Language production under vocabulary constraints in humans and resource-rational models
Pith reviewed 2026-05-19 15:29 UTC · model grok-4.3
pith:CIUNC33H Add to your LaTeX paper
What is a Pith Number?\usepackage{pith}
\pithnumber{CIUNC33H}
Prints a linked pith:CIUNC33H badge after your title and writes the identifier into PDF metadata. Compiles on arXiv with no extra files. Learn more
The pith
Humans produce language more like greedy step-by-step choices than global planning when vocabulary is restricted.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
When vocabulary is constrained to the most frequent 250 words, human responses to questions align more closely with greedy sampling than with globally optimal sampling, as approximated through Sequential Monte Carlo inference on large language models, except that more skilled participants exhibit greater backtracking and revision.
What carries the argument
Sequential Monte Carlo inference on large language models used to approximate both greedy word-by-word selection and global search for the best message under vocabulary limits.
If this is right
- Resource-rational models can explain everyday language production when speakers face lexicon limits.
- Greater skill correlates with more backtracking, suggesting expertise enables non-greedy planning.
- Reliance on light words emerges automatically from either sampling strategy.
- The same modeling approach may apply to communication under other constraints such as time pressure or aphasia.
Where Pith is reading between the lines
- Default reliance on local decisions may be a general feature of how minds handle complex sequential tasks.
- Training or practice could shift behavior toward more revision even for less skilled speakers.
- The findings point toward testable predictions for second-language learners facing similar vocabulary bottlenecks.
Load-bearing premise
Sequential Monte Carlo sampling from language models gives a faithful stand-in for what truly greedy or truly optimal human production would look like under the same word limits.
What would settle it
A direct comparison showing that human word sequences under the 250-word limit match the globally optimal model outputs more closely than the greedy ones, or that speaker skill does not predict rates of revision.
Figures
read the original abstract
Communicating using only a limited vocabulary is a common but challenging cognitive phenomenon, requiring an ideal communicator to plan carefully to optimize for intelligibility while circumventing a constrained lexicon. In this work, we investigate how humans respond to a broad array of questions under variable vocabulary limitations, consisting of only 250 highly frequent words at the most restrictive. We provide theoretically motivated comparisons to greedy and globally optimal sampling algorithms using Sequential Monte Carlo inference with large language models. Humans generally resemble greedy sampling more than globally optimal sampling, though more skilled humans are more likely to backtrack and revise -- a non-greedy behavior. An observed human pattern of leaning on semantically light words in high-constraint settings falls out of both greedy and globally optimal sampling. We discuss the results and their broader implications for resource-rational cognition, psycholinguistics, L2 communication, and language impairments.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates human language production when restricted to limited vocabularies (most restrictively, the 250 most frequent words). It compares human responses to questions under these constraints against greedy sampling and globally optimal sampling, the latter approximated via Sequential Monte Carlo inference on large language models. The central claim is that humans generally align more closely with greedy sampling, although more skilled humans exhibit non-greedy behaviors such as backtracking and revision; the increased reliance on semantically light words under high constraint emerges from both sampling regimes.
Significance. If the SMC approximations are shown to be faithful, the work would strengthen resource-rational accounts of language production by demonstrating how humans trade off planning depth against computational cost under lexical constraints, with direct implications for psycholinguistics, L2 communication, and models of language impairment. The parameter-free character of the model predictions and the use of SMC to operationalize global optimality constitute methodological strengths that could support falsifiable predictions.
major comments (2)
- [§4 (SMC implementation and global-optimality approximation)] §4 (SMC implementation and global-optimality approximation): no diagnostic is reported that compares log-probability, intelligibility, or downstream task performance of the SMC-derived 'globally optimal' trajectories against the greedy samples under the 250-word mask. Without evidence that the particle filter recovers meaningfully higher-utility sequences, the claimed distinction between the two model classes is not established and the human-greedy alignment result becomes difficult to interpret.
- [Results section (human-model alignment)] Results section (human-model alignment): the directional resemblance between human data and greedy sampling is asserted without reported sample sizes for participants, statistical tests, alignment metrics (e.g., correlation or divergence measures), or details on how model-generated sequences were matched to human responses, preventing quantitative evaluation of the central claim.
minor comments (2)
- [Abstract and §2] Abstract and §2: the exact vocabulary sizes and constraint levels used in each condition should be stated explicitly rather than summarized as 'variable vocabulary limitations.'
- [Discussion] Discussion: the implications for L2 communication and language impairments are noted but would benefit from concrete links to one or two key prior studies in those literatures.
Simulated Author's Rebuttal
We thank the referee for their constructive comments, which identify key areas where additional evidence and quantitative detail would strengthen the manuscript. We address each major comment below and outline planned revisions.
read point-by-point responses
-
Referee: [§4 (SMC implementation and global-optimality approximation)] §4 (SMC implementation and global-optimality approximation): no diagnostic is reported that compares log-probability, intelligibility, or downstream task performance of the SMC-derived 'globally optimal' trajectories against the greedy samples under the 250-word mask. Without evidence that the particle filter recovers meaningfully higher-utility sequences, the claimed distinction between the two model classes is not established and the human-greedy alignment result becomes difficult to interpret.
Authors: We agree that direct diagnostics would better substantiate the SMC approximation to global optimality. The manuscript motivates SMC as a standard method for approximating exhaustive search over sequences under the vocabulary mask and reports the specific particle count and resampling procedure used, but it does not include explicit utility comparisons. In the revision we will add to §4 a comparison of mean log-probabilities (under the base LLM) for SMC trajectories versus greedy samples in the 250-word condition, along with a brief note on sequence intelligibility via human ratings of a small subset. This will provide concrete evidence that the particle filter recovers higher-utility sequences. revision: yes
-
Referee: [Results section (human-model alignment)] Results section (human-model alignment): the directional resemblance between human data and greedy sampling is asserted without reported sample sizes for participants, statistical tests, alignment metrics (e.g., correlation or divergence measures), or details on how model-generated sequences were matched to human responses, preventing quantitative evaluation of the central claim.
Authors: We accept that the Results section would be improved by explicit quantitative reporting. The current text describes qualitative alignment in word-choice patterns and revision rates but omits sample sizes, formal tests, and matching procedures. We will revise the Results section to report the total number of participants and trials, describe the sequence-matching protocol (pairing by question and constraint level), and include alignment metrics such as type-overlap proportions and a distributional divergence measure between human and model outputs. Where sample sizes permit, we will also add simple statistical comparisons of key behavioral measures. revision: yes
Circularity Check
Independent LLM-based SMC simulations; no reduction of predictions to human-data fits
full rationale
The paper compares human responses under vocabulary constraints to greedy and globally optimal sampling implemented via Sequential Monte Carlo on large language models. These model procedures operate on pretrained LLMs with a fixed 250-word mask and do not fit parameters or define optimality from the human behavioral data itself. No self-definitional loops, fitted-input predictions, or load-bearing self-citations appear in the derivation chain. The observed human resemblance to greedy sampling is therefore evaluated against an external computational benchmark rather than being forced by construction from the same data.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Humans approximate resource-rational agents when producing language under constraints
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We provide theoretically motivated comparisons to greedy and globally optimal sampling algorithms using Sequential Monte Carlo inference with large language models.
-
IndisputableMonolith/Foundation/BranchSelection.leanbranch_selection unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Humans generally resemble greedy sampling more than globally optimal sampling, though more skilled humans are more likely to backtrack and revise
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Lombrozo, Tania and Wilkenfeld, Daniel A. , editor =. Mechanistic versus Functional Understanding , booktitle =
-
[2]
Adolphs, Svenja and Schmitt, Norbert , year = 2003, journal =. Lexical
work page 2003
-
[3]
Arnon, Inbal and Clark, Eve V. , year = 2011, journal =. Why
work page 2011
-
[4]
Beeke, Suzanne , year = 2012, pages =. 13. 13
work page 2012
- [5]
-
[6]
Handbook of Psycholinguistics , author =
Language Production:. Handbook of Psycholinguistics , author =
-
[7]
Perceptual Adaptation to Non-Native Speech , author =. Cognition , volume =. doi:10.1016/j.cognition.2007.04.005 , pmcid =
-
[8]
Brysbaert, Marc and Stevens, Micha. How. Frontiers in Psychology , volume =. doi:10.3389/fpsyg.2016.01116 , pmcid =
-
[9]
Calderon, Nitay and Reichart, Roi and Dror, Rotem , year = 2025, number =. The. doi:10.48550/arXiv.2501.10970 , archiveprefix =. 2501.10970 , primaryclass =
-
[10]
Carnie, Andrew , year = 2013, publisher =. Syntax:
work page 2013
-
[11]
Cooperative Explanation as Rational Communication , booktitle =
Chandra, Kartik and Chen, Tony and Li, Tzu-Mao and. Cooperative Explanation as Rational Communication , booktitle =
-
[12]
Journal of Language Evolution , volume =
Discrete and Systematic Communication in a Continuous Signal-Meaning Space , author =. Journal of Language Evolution , volume =
- [13]
-
[14]
Chomsky, Noam , year = 2014, publisher =. The
work page 2014
-
[15]
Behavioral and Brain Sciences , volume =
Rules and Representations , author =. Behavioral and Brain Sciences , volume =
-
[16]
IRE Transactions on Information Theory , volume =
Three Models for the Description of Language , author =. IRE Transactions on Information Theory , volume =
-
[17]
Clark, Thomas Hikaru and Vigly, Jacob Hoover and Gibson, Edward and Levy, Roger , year = 2025, journal =. A
work page 2025
-
[18]
Clark, Thomas Hikaru and Vigly, Jacob Hoover and Gibson, Edward and Levy, Roger P. , editor =. Resource-. Proceedings of the 2025
work page 2025
-
[19]
Nature Human Behaviour , volume =
How Laypeople Evaluate Scientific Explanations Containing Jargon , author =. Nature Human Behaviour , volume =. doi:10.1038/s41562-025-02227-0 , copyright =
-
[20]
and Graf, Caroline and Kreiss, Elisa and Goodman, Noah D
Degen, Judith and Hawkins, Robert D. and Graf, Caroline and Kreiss, Elisa and Goodman, Noah D. , year = 2020, journal =. When Redundancy Is Useful:. doi:10.1037/rev0000186 , pmid =
-
[21]
D. Communication. Language Learning , volume =. doi:10.1111/0023-8333.51997005 , copyright =
-
[22]
Agrammatic Output in Non-Fluent, Including
Fedorenko, Evelina and Ryskin, Rachel and Gibson, Edward , year = 2022, journal =. Agrammatic Output in Non-Fluent, Including
work page 2022
-
[23]
Language Is Primarily a Tool for Communication Rather than Thought , author =. Nature , volume =. doi:10.1038/s41586-024-07522-w , copyright =
-
[24]
Good-Enough Representations in Language Comprehension , author =
-
[25]
Florian Jaeger, T. , year = 2010, journal =. Redundancy and Reduction:. doi:10.1016/j.cogpsych.2010.02.002 , pmcid =
-
[26]
Proceedings of the National Academy of Sciences , volume =
Information-Theoretic Principles in Incremental Language Production , author =. Proceedings of the National Academy of Sciences , volume =
- [27]
-
[28]
and Dautriche, Isabelle and Mahowald, Kyle and Bergen, Leon and Levy, Roger , year = 2019, journal =
Gibson, Edward and Futrell, Richard and Piantadosi, Steven P. and Dautriche, Isabelle and Mahowald, Kyle and Bergen, Leon and Levy, Roger , year = 2019, journal =. How. doi:10.1016/j.tics.2019.02.003 , pmid =
-
[29]
Goulden, Robin and Nation, Paul and Read, John , year = 1990, journal =. How
work page 1990
-
[30]
and Lieder, Falk and Goodman, Noah D
Griffiths, Thomas L. and Lieder, Falk and Goodman, Noah D. , year = 2015, journal =. Rational
work page 2015
-
[31]
Grogger, Jeffrey and Steinmayr, Andreas and Winter, Joachim , year = 2020, series =. The. doi:10.3386/w26719 , archiveprefix =. 26719 , publisher =
-
[32]
Gu, Jiawei and Jiang, Xuhui and Shi, Zhichao and Tan, Hexiang and Zhai, Xuehao and Xu, Chengjin and Li, Wei and Shen, Yinghan and Ma, Shengjie and Liu, Honghao and Wang, Saizhuo and Zhang, Kun and Wang, Yuanzhuo and Gao, Wen and Ni, Lionel and Guo, Jian , year = 2025, number =. A. doi:10.48550/arXiv.2411.15594 , archiveprefix =. 2411.15594 , primaryclass =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2411.15594 2025
-
[33]
Lexical Teddy Bears and Advanced Learners: A Study into the Ways
Hasselgren, Angela , year = 1994, journal =. Lexical Teddy Bears and Advanced Learners: A Study into the Ways
work page 1994
-
[34]
Charles , year = 2010, journal =
Healey, E. Charles , year = 2010, journal =. What the. doi:10.1055/s-0030-1265756 , copyright =
-
[35]
Heim, Irene and Kratzer, Angelika , year = 1998, publisher =. Semantics in
work page 1998
-
[36]
Cognitive Psychology , volume =
Sources of. Cognitive Psychology , volume =. doi:10.1006/cogp.1998.0713 , copyright =
-
[37]
Honnibal, Matthew and Montani, Ines and Van Landeghem, Sofie and Boyd, Adriane , year = 2020, doi =
work page 2020
-
[38]
Journal of speech, language, and hearing research : JSLHR , volume =
The Relationship between Listener Comprehension and Intelligibility Scores for Speakers with Dysarthria , author =. Journal of speech, language, and hearing research : JSLHR , volume =. doi:10.1044/1092-4388(2008/040) , pmcid =
-
[39]
Kagan, A. and Black, S. E. and Duchan, F. J. and. Training Volunteers as Conversation Partners Using ". Journal of speech, language, and hearing research: JSLHR , volume =. doi:10.1044/1092-4388(2001/051) , pmid =
-
[40]
Kahneman, Daniel , year = 2003, journal =. Maps of
work page 2003
- [41]
-
[42]
Keuleers, Emmanuel and Stevens, Micha. Word Knowledge in the Crowd:. Quarterly Journal of Experimental Psychology , volume =. doi:10.1080/17470218.2015.1022560 , pmid =
-
[43]
Klein, Wolfgang and Perdue, Clive , year = 1997, journal =. The
work page 1997
-
[44]
Kleinschmidt, Dave F. and Jaeger, T. Florian , year = 2015, journal =. Robust Speech Perception: Recognize the Familiar, Generalize to the Similar, and Adapt to the Novel , shorttitle =. doi:10.1037/a0038695 , pmcid =
-
[45]
Kong, Anthony Pak-Hin and Wong, Cherie Wan-Yin , year = 2018, journal =. An. doi:10.1044/2018_AJSLP-18-0015 , pmcid =
-
[46]
and Weinheimer, Sidney , year = 1964, journal =
Krauss, Robert M. and Weinheimer, Sidney , year = 1964, journal =. Changes in Reference Phrases as a Function of Frequency of Usage in Social Interaction: A Preliminary Study , shorttitle =
work page 1964
-
[47]
Florian , year = 2015, journal =
Kurumada, Chigusa and Jaeger, T. Florian , year = 2015, journal =. Communicative Efficiency in Language Production:
work page 2015
-
[48]
Behavioral and Brain Sciences , volume =
Building Machines That Learn and Think like People , author =. Behavioral and Brain Sciences , volume =
- [49]
-
[50]
Is Bilingualism Associated with Enhanced Executive Functioning in Adults?
Lehtonen, Minna and Soveri, Anna and Laine, Aini and J. Is Bilingualism Associated with Enhanced Executive Functioning in Adults?. Psychological Bulletin , volume =. doi:10.1037/bul0000142 , pmid =
-
[51]
A Functionalist Typology of Redundancy , author =. Revista da ABRALIN , pages =. doi:10.25189/rabralin.v19i3.1722 , copyright =
-
[52]
Lew, Alexander K. and. Sequential. doi:10.48550/arXiv.2306.03081 , archiveprefix =. 2306.03081 , primaryclass =
-
[53]
Lieder, Falk and Griffiths, Thomas L. , year = 2020, journal =. Resource-Rational Analysis:
work page 2020
-
[54]
Lipkin, Benjamin and LeBrun, Benjamin and Vigly, Jacob Hoover and Loula, Jo. Fast. doi:10.48550/arXiv.2504.05410 , archiveprefix =. 2504.05410 , primaryclass =
-
[55]
Functional Explanation and the Function of Explanation , author =. Cognition , volume =
-
[56]
Inferring Design: Evidence of a Preference for Teleological Explanations in Patients with
Lombrozo, Tania and Kelemen, Deborah and Zaitchik, Deborah , year = 2007, journal =. Inferring Design: Evidence of a Preference for Teleological Explanations in Patients with. doi:10.1111/j.1467-9280.2007.02015.x , pmid =
-
[57]
Syntactic and semantic control of large language models via sequential
Loula, Jo. Syntactic and. doi:10.48550/arXiv.2504.13139 , archiveprefix =. 2504.13139 , primaryclass =
-
[58]
Proceedings of Meetings on Acoustics , volume =
Human Language Diversity and the Acoustic Adaptation Hypthesis , author =. Proceedings of Meetings on Acoustics , volume =
-
[59]
Grammatical Cues to Subjecthood Are Redundant in a Majority of Simple Clauses across Languages , author =. Cognition , volume =
-
[60]
Marr, David , year = 1982, publisher =. Vision:
work page 1982
-
[61]
McCarthy, Amanda M. and Keil, Frank C. , year = 2023, journal =. A Right Way to Explain?
work page 2023
-
[62]
Morley, Brendan Arkell , year = 2022, journal =. 27184953 , eprinttype =
work page 2022
-
[63]
Na, Liu and Nation, I.S.P. , year = 1985, journal =. Factors
work page 1985
-
[64]
The Behavioral and Brain Sciences , volume =
An Integrated Theory of Language Production and Comprehension , author =. The Behavioral and Brain Sciences , volume =. doi:10.1017/S0140525X12001495 , pmid =
-
[65]
Behavioral and Brain Sciences , volume =
Natural Language and Natural Selection , author =. Behavioral and Brain Sciences , volume =
-
[66]
Poulisse, Nanda , year = 2011, pages =. A. The
work page 2011
-
[67]
Proceedings of the National Academy of Sciences of the United States of America , volume =
A Syntax--Lexicon Trade-off in Language Production , author =. Proceedings of the National Academy of Sciences of the United States of America , volume =. doi:10.1073/pnas.2120203119 , pmcid =
-
[68]
Rissman, Lilia and Liu, Qiawen and Lupyan, Gary , year = 2023, journal =. Gaps in the
work page 2023
-
[69]
A Mathematical Theory of Communication , author =
-
[70]
Rspeer/Wordfreq: V3.0 , shorttitle =
Speer, Robyn , year = 2022, doi =. Rspeer/Wordfreq: V3.0 , shorttitle =
work page 2022
- [71]
-
[72]
and Delogu, Francesca and Crocker, Matthew W
Tourtouri, Elli N. and Delogu, Francesca and Crocker, Matthew W. , year = 2021, journal =. Rational. doi:10.1111/cogs.13071 , copyright =
-
[73]
Current biology: CB , volume =
Evidence for Cognition without Grammar from Causal Reasoning and 'theory of Mind' in an Agrammatic Aphasic Patient , author =. Current biology: CB , volume =. doi:10.1016/s0960-9822(00)00538-8 , pmid =
-
[74]
Wang, Hao and Shimizu, Hirofumi and Kawahara, Daisuke , year = 2024, number =. Kanbun-. doi:10.48550/arXiv.2305.12759 , archiveprefix =. 2305.12759 , primaryclass =
-
[75]
and Kirkovski, Melissa and Lum, Jarrad A
Ware, Anna T. and Kirkovski, Melissa and Lum, Jarrad A. G. , year = 2020, journal =. Meta-. doi:10.3389/fpsyg.2020.01458 , pmcid =
-
[76]
Wicha, Nicole Y. Y. and Moreno, Eva M. and Kutas, Marta , year = 2004, journal =. Anticipating. doi:10.1162/0898929041920487 , pmcid =
-
[77]
Wilcox, Ethan Gotlieb and Hu, Michael Y. and Mueller, Aaron and Warstadt, Alex and Choshen, Leshem and Zhuang, Chengxu and Williams, Adina and Cotterell, Ryan and Linzen, Tal , year = 2025, journal =. Bigger Is Not Always Better:
work page 2025
-
[78]
Yurtsever, Asl. Bilingual Children Outperform Monolingual Children on Executive Function Tasks Far More Often than Chance:. Developmental Review , volume =
-
[79]
Zechmeister, Eugene B. and Chronis, Andrea M. and Cull, William L. and D'Anna, Catherine A. and Healy, Noreen A. , year = 1995, journal =. Growth of a
work page 1995
-
[80]
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , year = 2023, number =. Judging. doi:10.48550/arXiv.2306.05685 , archiveprefix =. 2306.05685 , primaryclass =
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2306.05685 2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.