Evolutionary ecology of words

Reiji Suzuki; Takaya Arita

REVIEW 3 major objections 6 minor 15 references

Evolutionary ecology of words

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An LLM can act as both mutation source and fitness judge, making word evolution open-ended.

desk verdict A promising but under-evidenced demonstration of LLM-driven word evolution; the lack of order-randomized controls makes the selection signal partly suspect, but the model is worth a serious round of revision. read the letter →

arxiv 2505.05863 v1 pith:E5FIDF2C submitted 2025-05-09 q-bio.PE cs.AIcs.CL

Reiji Suzuki , Takaya Arita This is my paper · ORCID

classification q-bio.PEcs.AIcs.CL MSC 91A22

keywords largelanguagemodelsagent-basedevolutionarygametheoryartificiallifeopen-endedevolutionwordecologyspatialnoveltygeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a large language model can replace both the mutation operator and the fitness function in an evolutionary system, making the space of possible strategies effectively open-ended. The proposed model places agents on a two-dimensional grid, gives each agent a short word as its genome, and resolves encounters by asking the LLM which word is stronger; the loser's word is overwritten by the winner's, and occasional mutations are generated by prompting the LLM for similar-but-different words. In experiments where 'strong animal species' are supposed to survive, populations that start from familiar animals diverge into terrestrial, marine, and extinct megafauna, with over a hundred species appearing per trial and dominance shifting in bursts. A longer, larger run produces coexistence of poison frogs, fungal pathogens, and malaria parasites, which the authors present as evidence that LLM-based novelty plus LLM-based selection supports open-ended evolution.

What carries the argument

The load-bearing mechanism is the LLM-as-oracle: a cached prompt call, 'Which one is stronger ...?', supplies the selection pressure for every pair of words, and a second prompt call supplies mutations by listing similar-but-different words. Because these calls draw on the LLM's open vocabulary, the set of possible strategies is effectively unbounded and the payoff structure is never fixed in advance, which is what the authors claim distinguishes this model from standard matrix-based evolutionary game theory. The spatial grid and the rule that occupied cells block movement add population viscosity, letting new words spread locally before they take over globally.

What would settle it

Re-run the full simulation twice with the same initial population and parameters, once using the cached LLM judgments and once using a fixed random tournament matrix over the same word pairs; if the random-tournament version still produces hundreds of species and punctuated equilibria, the LLM's semantic judgments are not the cause of the claimed evolution. A cheaper check is to query the same pair ('whale' versus 'eagle') many times with nonzero temperature and count how often the winner flips.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an LLM can serve simultaneously as the source of new variants and as the arbiter of which variants spread, and that this minimal setup is enough to reproduce qualitative signatures of biological evolution: gradual drift interrupted by takeover events, ecological specialization, and the appearance of organism types that were never in the initial pool. The payoff matrix is never written down; it exists implicitly in the LLM's pairwise comparisons, so the strategy space is not fixed in advance. Under the prompt 'strong animal species survive,' each trial converges on a different large, specialized winner, and the long experiment shows that species substantially outside the initial category—toxin-producing amphibians, fungi, a malaria parasite—can appear and persist.

Load-bearing premise

The whole process depends on the LLM answering 'Which one is stronger?' consistently enough to act as a fitness function; if those pairwise judgments are noisy, context-dependent, or unstable across re-queries, the observed evolutionary trajectories are partly artifacts of the oracle rather than results of the model's ecological rules.

Editorial extensions

If this is right

If the LLM's pairwise judgments are stable enough, evolutionary game theory can move beyond finite, pre-specified strategy sets: new actions can arise mid-run and the payoff matrix can be implicit rather than explicit.
The same experimental apparatus can be pointed at a different selection criterion simply by changing the prompt (for instance 'cute' or 'witty'), so the model offers a cheap way to build artificial-life worlds with arbitrary fitness semantics.
The LLM's latent knowledge—about size, toxicity, habitat, extinction—becomes an implicit fitness landscape, meaning observed 'ecological' outcomes reflect the model's world model rather than an external biological truth.
The long experiment's coexistence of diverse species suggests that LLM-generated novelty plus spatial structure can sustain diversity, a combination the authors propose as a step toward open-ended artificial evolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper leaves untested whether the dynamics rely on the LLM's semantic content or simply on having a mostly consistent, non-transitive tournament. An editorial conjecture is that cyclic dominance (A beats B, B beats C, C beats A) in the cached judgments could itself sustain diversity, so the 'ecology' may be a projection of the oracle's opinion graph.
A natural extension would be to replace the LLM judge with a randomly generated pairwise matrix and see whether similar punctuated equilibria appear; that would separate the contribution of the model's rules from the contribution of its world knowledge.
The same mutation-plus-judgment loop could be applied to non-biological word spaces, such as evolving product names or slogans against an LLM-elicited preference, since the mechanism never depends on animal-specific semantics.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a plausible minimal model for using an LLM as both mutation source and fitness function in a spatial evolutionary game. The authors set up words as organisms on a grid, compete them pairwise via Gemma-2, and let winners overwrite losers. The novel bit is the combination — LLM-generated mutations plus LLM-judged competition inside an explicit spatial ecology. That combination isn't in the cited literature, and the paper shows it can produce rich, qualitative dynamics: 10 trials each yielding over a hundred unique species, and a long run with 3,704 distinct names, including parasites and fungi. The code and data are on figshare, which is real evidence and helps.

What the paper does well: it's clearly written, the model is minimal and reproducible, and the authors are upfront that these are preliminary experiments. They also identify plausible next steps (draws, crossovers, environmental descriptions).

The soft spots are substantial, though. The central claims — punctuated equilibrium, convergence to large/extinct species, open-endedness — rest entirely on visual inspection of UMAP plots. There are no repeated runs with identical initial conditions, no measures of diversity or takeover times, and no baseline comparisons (e.g., random judgment, fixed payoff matrices, or word-order controls). The stress-test note is right: in the competition prompt, the acting agent's word is always placed first. Since agent order is random, the first encounter of any word pair caches a result that depends on who happened to act first that step. If Gemma-2 has any positional preference, the cached payoff matrix is partially arbitrary, and the "selection" driving the dynamics could reflect stochastic turn order more than word semantics. The same issue applies to any LLM pairwise comparison, and the authors don't report any control for it. This is a gap in the evidence, not an accusation.

There's also an inherent circularity: the LLM supplies the mutations and the fitness signal, so the dynamics are largely a projection of its priors. That's not fatal — the authors explicitly frame the goal as "virtual" evolution using LLM judgments — but it does mean the "prediction" that large/extinct species dominate is essentially a restatement of what Gemma-2 already believes about strength. The paper would be much stronger with a sensitivity analysis on the LLM prompt, temperature, and order.

Who should read it: anyone working on LLM-based artificial life or open-ended evolution. It's a good starting point for discussion, but it should not be taken as a validated result. I'd send it to peer review, but with a clear expectation of major revision: add controls, repeated runs, and quantitative metrics.

Verdict: worth reading, worth citing as an early example, but only if the revision addresses the positional-bias control and baselines.

Referee Report

3 major / 6 minor

Summary. The paper proposes an agent-based model in which each agent carries a single word (e.g., an animal species name) as its genome, moves on a two-dimensional toroidal grid, and interacts with neighboring agents. The outcome of each pairwise interaction is determined by an LLM (Gemma-2) answering the prompt 'Which one is stronger "#word1#" or "#word2#"?' with the winner's word replacing the loser's; mutations are also generated by the LLM when asked to list similar but different species. Two sets of experiments are reported: ten 300-step trials with 80 agents, and one 2000-step run with 200 agents. The authors report qualitative observations of word replacement, punctuated equilibrium, and convergence to large, extinct, or poisonous species, and interpret this as evidence that LLMs can serve as both the source of novelty and the arbiter of selection in an evolutionary ecology model.

Significance. If the central claim is supported, the paper makes a novel contribution by using an LLM simultaneously as a mutation generator and a fitness oracle in an evolutionary game-theoretic setting, potentially expanding the scope of open-ended evolution research. The model is simple, clearly described, and the accompanying code and data release (Figshare) is a reproducible and transparent asset. However, the evidence is largely qualitative, the long-term result rests on a single run, and a likely positional bias in the competition prompt directly threatens the interpretation of the LLM as a semantic fitness function. With appropriate controls and quantitative analyses, the model could become a useful platform for studying LLM-driven evolutionary dynamics; as it stands, the paper is better viewed as a preliminary report requiring substantial additional evidence.

major comments (3)

[Section 2, Figure 2c] The competition prompt always places the initiating agent's word first: "Which one is stronger \"#word1#\" or \"#word2#\"?" and the outcome for a word pair is cached after the first encounter. If Gemma-2 exhibits any positional bias in pairwise comparisons, the cached payoff matrix will depend on the stochastic agent-update order at the first meeting rather than on word semantics, and the claim that the LLM acts as an ecological selector would be compromised. The paper provides no order-randomized control or test for positional bias; I request an experiment with randomized word order and a report of the direction and magnitude of any bias.
[Section 3, Figures 3-5] The central claims of 'punctuated equilibrium' and 'emergence and coexistence of diverse species' are supported only by visual inspection of UMAP projections and frequency plots. No quantitative measures of diversity, turnover, or punctuation are provided, and the long-term experiment (Figure 5) consists of a single run. To justify the general conclusions, the authors should supply repeated long-term runs, diversity indices, and statistical tests, or explicitly reframe the results as qualitative observations from a preliminary model.
[Section 3 (entire experimental section)] No control conditions are reported. Without a baseline using a random payoff matrix or a fixed non-LLM ranking, it is not possible to distinguish the effect of the LLM's semantic knowledge from the dynamics of the spatial replacement process itself (e.g., the tendency for local populations to become homogeneous or for transient species to be replaced). Adding such controls is essential for the claim that the observed dominance of large or extreme species arises from the LLM's evaluation of word meanings.

minor comments (6)

[Abstract and Introduction] The phrase 'infinite options' overstates the model's capability; since the word space is generated by a finite LLM vocabulary and list sizes, 'unbounded' or 'open-ended' would be more accurate.
[Figure 3] The figure is dense and difficult to read at the available resolution; the traces of average word vectors are hard to distinguish from the background gray words. Consider separate panels per trial or clearer trajectory markers.
[Figure 4 and accompanying text] The figure caption refers to 'word frequency in the final step' while the text refers to 'top 10 frequent species in the final step'; please align the terminology and state clearly what the frequency plot shows.
[Section 3, paragraph after Figure 4] The sentence 'As for the top 10 species, With the exception of the Siberian tiger...' has a grammatical error (capitalization after a comma) and becomes a run-on; it should be split into cleaner sentences.
[Section 2, Model] The paper does not specify the LLM's decoding parameters (e.g., temperature, top-p, random seed), even though these affect reproducibility of the LLM-generated judgments and mutations. Please add these details or point to them in the code repository.
[Section 2, Figure 1d] The text states 'whale obtained X and eagle obtained Y, and whale won the game because X > Y', but X and Y are not defined; the reader is left to guess that they represent payoffs. Please clarify or omit the X/Y notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in this LLM-oracle demonstration model.

full rationale

The paper's derivation chain is self-contained as a model demonstration. It explicitly defines the scenario as 'strong animal species' survive and uses the LLM's answer to 'Which one is stronger #word1# or #word2#?' as the competition outcome and the LLM's similarity lists as the mutation source. The observed dominance of large, dangerous, or poisonous species is therefore the intended consequence of the chosen fitness criterion, not a hidden prediction smuggled in after the fact. No parameter is fitted to a subset of data and then renamed as a prediction; the LLM is a pretrained oracle whose judgments are the model's fitness function. The paper is also transparent that it 'aim[s] to utilize the creativity of LLMs and their grasp of complex contextual word relationships to generate virtual evolutionary and ecological processes rather than simulating actual biological evolution.' The only self-citation is to the authors' prior related work in the introduction, which is not load-bearing for the present claims. Potential LLM positional bias in the competition prompt is an experimental-validity concern, not a circularity in the derivation. The claimed diversity and punctuated-equilibrium-like dynamics are empirical outputs of the stochastic spatial process and are not equivalent by construction to any single input, so no circular step is exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central result depends on hand-chosen simulation parameters and on the LLM's behavior as a black-box oracle. No new physical or mathematical entities are introduced. The main untested assumption is the reliability of the LLM as a stable fitness function.

free parameters (6)

population size N = 80 (short trials), 200 (long trial)
Hand-chosen; not swept or justified.
grid size W = 16 (short), 90 (long)
Hand-chosen; affects spatial viscosity and encounter rates.
mutation probability pm = 0.05
Hand-chosen; no sensitivity analysis.
initial word list size A = 35
Hand-chosen; initial species pool.
mutation list size B = 10
Hand-chosen; mutation diversity.
number of steps = 300 (short), 2000 (long)
Hand-chosen; no convergence criterion.

assumptions (4)

domain assumption LLM pairwise strength judgments are transitive and stable enough to define a fitness landscape
The model uses a single query per word pair, cached in a dictionary, assuming deterministic outcomes; Section 2 'Competition'.
domain assumption Word mutation via LLM similarity lists explores a relevant neighborhood of the current word
Mutation prompt asks for similar or significantly different species, Section 2 'Mutation'.
domain assumption Spatial movement creates viscosity and clustering as intended
Movement rule: individuals do not move onto occupied cells; Section 2 'Movement'.
domain assumption The base prompt gives the LLM a stable role across all queries
Figure 2a, base prompt prepended to all prompts.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Evolutionary ecology of words." pith.science (2026). https://pith.science/paper/E5FIDF2C

@misc{pith2026250505863,
  author       = {Pith},
  title        = {Pith review of: Evolutionary ecology of words},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E5FIDF2C}},
  note         = {Machine review of arXiv:2505.05863}
}

read the original abstract

We propose a model for the evolutionary ecology of words as one attempt to extend evolutionary game theory and agent-based models by utilizing the rich linguistic expressions of Large Language Models (LLMs). Our model enables the emergence and evolution of diverse and infinite options for interactions among agents. Within the population, each agent possesses a short word (or phrase) generated by an LLM and moves within a spatial environment. When agents become adjacent, the outcome of their interaction is determined by the LLM based on the relationship between their words, with the loser's word being replaced by the winner's. Word mutations, also based on LLM outputs, may occur. We conducted preliminary experiments assuming that ``strong animal species" would survive. The results showed that from an initial population consisting of well-known species, many species emerged both gradually and in a punctuated equilibrium manner. Each trial demonstrated the unique evolution of diverse populations, with one type of large species becoming dominant, such as terrestrial animals, marine life, or extinct species, which were ecologically specialized and adapted ones across diverse extreme habitats. We also conducted a long-term experiment with a large population, demonstrating the emergence and coexistence of diverse species.

Figures

Figures reproduced from arXiv: 2505.05863 by the authors.

**Figure 2.** Prompts for “strong animal species” survives. The base prompt (a), defines the basic behavior of the LLM, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Transition of populations in the semantic space of words over 10 trials. Each dot represents the average word [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Spatial distribution of individuals in the final step of Trial 2 (top left), word frequency in the final step (top [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Transition of populations in the semantic space of words in a long-term trial. Each dot represents the [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

[1]

Generative Agent-Based Models for Complex Systems Research: a review

Y . Lu, A. Aleta, C. Du, L. Shi, and Y . Moreno, “Generative agent-based models for complex systems research: a review,” arXiv e-prints, arXiv:2408.09175, 2024. 7 A PREPRINT - MAY 12, 2025

work page Pith review arXiv 2024
[2]

From text to life: On the reciprocal relationship between artificial life and large language models,

E. Nisioti, C. Glanois, E. Najarro, A. Dai, E. Meyerson, J. W. Pedersen, L. Teodorescu, C. F. Hayes, S. Sud- hakaran, and S. Risi, “From text to life: On the reciprocal relationship between artificial life and large language models,” Proc. of the 2024 Conference on Artificial Life, isal 2024, 39, 2024

work page 2024
[3]

Machine psychology,

T. Hagendorff, I. Dasgupta, M. Binz, S. C.Y . Chan, A. Lampinen, J. X. Wang, Z. Akata, and E. Schulz, “Machine psychology,” arXiv e-prints, arXiv:2303.13988, 2023

arXiv 2023
[4]

Large language model based multi-agents: A survey,

T. Guo, X. Chen, Y . Wang, R. Chang, S. Pei, N. V . Chawla, O. Wiest, and X. Zhang, “Large language model based multi-agents: A survey,” arXiv e-prints, arXiv:2402.01680, 2024

arXiv 2024
[5]

Large language models empowered agent- based modeling and simulation: a survey and perspectives,

C. Gao, X. Lan, N. Li, Y . Yuan, J. Ding, Z. Zhou, F. Xu, and Y . Li , “Large language models empowered agent- based modeling and simulation: a survey and perspectives,” Humanities and Social Sciences Communications, 11, 1259 (2024)

work page 2024
[6]

Evolutionary computation in the era of large language model: Survey and roadmap,

X. Wu, S. Wu, J. Wu, L. Feng, and K. C. Tan, “Evolutionary computation in the era of large language model: Survey and roadmap,” arXiv e-prints, arXiv:2401.10034, 2024

arXiv 2024
[7]

Language model crossover: Variation through few-shot prompting,

E. Meyerson, M. J. Nelson, H. Bradley, A. Gaier, A. Moradi, A. K. Hoover, and J. Lehman, “Language model crossover: Variation through few-shot prompting,” arXiv e-prints, arXiv:2302.12170, 2023

arXiv 2023
[8]

Promptbreeder: Self-referential self-improvement via prompt evolution,

C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rockt ¨aschel, “Promptbreeder: Self-referential self-improvement via prompt evolution,” arXiv e-prints, arXiv:2309.16797, 2023

arXiv 2023

Show all 15 references

[9]

Large language models as in-context AI generators for quality-diversity,

B. Lim, M. Flageat, and A. Cully, “Large language models as in-context AI generators for quality-diversity,” Proc. of the 2024 Conference on Artificial Life, isal 2024, 48, 2024

2024
[10]

Evolutionary latent space search for driving human portrait gener- ation,

B. Mach ´ın, S. Nesmachnow, and J. Toutouh, “Evolutionary latent space search for driving human portrait gener- ation,” arXiv e-prints, arXiv:2204.11887, 2022

2022 arXiv
[11]

A critical examination of large language model capabilities in iteratively refining differential evolution algorithm,

M. Pluhacek, J. Kovac, P. Janku, T. Kadavy, R. Senkerik, and A. Viktorin, “A critical examination of large language model capabilities in iteratively refining differential evolution algorithm,” Proc. of the Genetic and Evolutionary Computation Conference Companion, pp. 1855-1862, 2024

2024
[12]

An evolutionary model of personality traits related to cooperative behavior using a large language model,

R. Suzuki, and T. Arita, “An evolutionary model of personality traits related to cooperative behavior using a large language model,” Scientific Reports, 14, 5989, 2024

2024
[13]

Collective innovation in groups of large language models,

E. Nisioti, S. Risi, I. Momennejad, P.-Y . Oudeyer, and C. Moulin-Frier, “Collective innovation in groups of large language models,” Proc. of the 2024 Conference on Artificial Life, isal2024, 16, 2024

2024
[14]

Gemma 2: Improving open language models at a practical size,

M. Riviere, et al. (Gemma team), “Gemma 2: Improving open language models at a practical size,” arXiv e- prints, arXiv:2408.00118, 2024

2024 arXiv
[15]

UMAP: Uniform manifold approximation and projection

L. McInnes, J. Healy, N. Saul, and L. Großberger, “UMAP: Uniform manifold approximation and projection.” Journal of Open Source Software, 3, 861, 2018. 8

2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Generative Agent-Based Models for Complex Systems Research: a review

Y . Lu, A. Aleta, C. Du, L. Shi, and Y . Moreno, “Generative agent-based models for complex systems research: a review,” arXiv e-prints, arXiv:2408.09175, 2024. 7 A PREPRINT - MAY 12, 2025

work page Pith review arXiv 2024

[2] [2]

From text to life: On the reciprocal relationship between artificial life and large language models,

E. Nisioti, C. Glanois, E. Najarro, A. Dai, E. Meyerson, J. W. Pedersen, L. Teodorescu, C. F. Hayes, S. Sud- hakaran, and S. Risi, “From text to life: On the reciprocal relationship between artificial life and large language models,” Proc. of the 2024 Conference on Artificial Life, isal 2024, 39, 2024

work page 2024

[3] [3]

Machine psychology,

T. Hagendorff, I. Dasgupta, M. Binz, S. C.Y . Chan, A. Lampinen, J. X. Wang, Z. Akata, and E. Schulz, “Machine psychology,” arXiv e-prints, arXiv:2303.13988, 2023

arXiv 2023

[4] [4]

Large language model based multi-agents: A survey,

T. Guo, X. Chen, Y . Wang, R. Chang, S. Pei, N. V . Chawla, O. Wiest, and X. Zhang, “Large language model based multi-agents: A survey,” arXiv e-prints, arXiv:2402.01680, 2024

arXiv 2024

[5] [5]

Large language models empowered agent- based modeling and simulation: a survey and perspectives,

C. Gao, X. Lan, N. Li, Y . Yuan, J. Ding, Z. Zhou, F. Xu, and Y . Li , “Large language models empowered agent- based modeling and simulation: a survey and perspectives,” Humanities and Social Sciences Communications, 11, 1259 (2024)

work page 2024

[6] [6]

Evolutionary computation in the era of large language model: Survey and roadmap,

X. Wu, S. Wu, J. Wu, L. Feng, and K. C. Tan, “Evolutionary computation in the era of large language model: Survey and roadmap,” arXiv e-prints, arXiv:2401.10034, 2024

arXiv 2024

[7] [7]

Language model crossover: Variation through few-shot prompting,

E. Meyerson, M. J. Nelson, H. Bradley, A. Gaier, A. Moradi, A. K. Hoover, and J. Lehman, “Language model crossover: Variation through few-shot prompting,” arXiv e-prints, arXiv:2302.12170, 2023

arXiv 2023

[8] [8]

Promptbreeder: Self-referential self-improvement via prompt evolution,

C. Fernando, D. Banarse, H. Michalewski, S. Osindero, and T. Rockt ¨aschel, “Promptbreeder: Self-referential self-improvement via prompt evolution,” arXiv e-prints, arXiv:2309.16797, 2023

arXiv 2023