REVIEW 4 major objections 4 minor 30 references
The paper establishes that two conditional acceptance rates, π and φ, compress the stochasticity of LLM listeners in a naming game, and that the sign of 3π−2φ−1 predicts whether a population orders — making decoding temperature a tunable co
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 12:49 UTC pith:XS2FJ4OY
load-bearing objection First systematic temperature scan of LLM Naming Games with a genuinely useful (π, φ) decomposition and a legitimate mean-field critical line, undercut by an internal consensus contradiction and an inventory-size caveat that limits its predictive reach in the conservative regime. the 4 major comments →
Microscopic dynamics of consensus formation in multi-agent LLM Naming Games
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that the LLM Naming Game admits a two-rate microscopic description: at fixed temperature, the listener's decision is governed by π(T)≡P(YES|w∈P_j) and φ(T)≡P(YES|w∉P_j). The ordering–disordering balance of the population is set by the drift ∆=mπ−(1−m)φ, and a complete-graph mean-field analysis of the two-word sector yields the critical line π_c(φ)=(1+2φ)/3, i.e. R=3π−2φ−1>0, which generalizes the known consensus threshold of the stochastic Naming Game. Across three architectures and temperatures T∈[0.05,2.0], consensus is always reached, but through three distinct listener regimes with different scaling exponents and temperature sensitivities: permissive (repaint-noise d
What carries the argument
The central object is the (π,φ) pair of conditional acceptance rates, defined by a single-token YES/NO LLM call at temperature T. π is the probability of a correct collapse trigger when the proposed word is already in the listener's inventory; φ is the probability of an erroneous 'repaint' when the word is new. Their balance defines the drift proxy ∆=mπ−λ(1−m)φ, and the mean-field two-word sector yields the ordering condition R=3π−2φ−1>0, where R plays the role of effective distance from the transition. The mechanism: legitimate collapses (rate mπ) build order, repaints (rate (1−m)φ) destroy it, and the critical line quantifies how much repaint noise raises the threshold for consensus.
Load-bearing premise
The effective two-rate rule treats π and φ as constants depending only on inventory membership, not on inventory size, word identity, or time; the paper's own conservative-regime data show that inventory size co-determines macroscopic convergence, so if these rates vary with inventory size, the critical line and regime classification lose predictive force for those models.
What would settle it
Simulate a population of several hundred agents with a model whose measured plateau rates satisfy R=3π−2φ−1<0 at some temperature; if strict consensus is still reached with high probability across many seeds, the mean-field critical line fails as a finite-size predictor. Conversely, for a model with R>0, if consensus time does not follow the predicted exponential temperature dependence t_c∼e^{αT} with α tied to the rate trajectories, the two-rate parametrization is not sufficient.
If this is right
- Decoding temperature becomes a quantitative control parameter for decentralized LLM populations; its effect on consensus time is exponential with architecture-dependent rate α, ranging from about 0.67 to about 0 across the three models tested.
- A model's regime can be predicted from offline measurements of π and φ alone: permissive (large φ), near-deterministic (π≈1, φ≈0), and conservative (low π, φ≈0).
- The critical condition R=3π−2φ−1>0 generalizes the stochastic Naming Game threshold; for φ=0 it reduces to the known π>1/3 threshold, so temperature tuning can nominally cross the critical line within a single architecture.
- Temperature does not universally slow consensus; for some architectures it accelerates consolidation by reducing inventory diversity, so treating temperature simply as 'noise' is misleading.
- The effective finite-size exponent β(T) can exceed the canonical 3/2 under strong repaint noise, suggesting either a different universality class or a slow crossover — both left as open questions.
Where Pith is reading between the lines
- Editorial inference: if the two-rate parametrization is as general as claimed, the same (π,φ) diagnostic should transfer to any binary LLM decision with a ground-truth state, such as conformity experiments, where the false-positive rate plays the role of φ; measuring it offline could predict group-level yielding.
- A testable extension: measure π and φ on a held-out architecture, then run the multi-agent game; the sign of R=3π−2φ−1 should predict the ordering/disordering balance, and the temperature sensitivity α should be derivable from the rate trajectories — a prediction the current paper only illustrates on three models.
- The inventory-size closure in the appendix suggests the conservative regime may exhibit fragmentation at system sizes beyond N=150, where the mean-field transition sharpens; locating that crossover would be a sharper test than the current finite-size data.
- The paper's drift proxy uses λ=1 as a phenomenological choice; a mean-field derivation of λ from the microscopic rates could refine the boundary between regimes without changing the critical line.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies a minimal LLM-based Naming Game in which the deterministic inventory check is replaced by a single-token LLM call at decoding temperature T. It proposes a microscopic parametrization by two conditional rates, π(T) = P(YES|w∈P_j) and φ(T) = P(YES|w∉P_j), and claims that these rates, together with a drift proxy Δ(t)=mπ−λ(1−m)φ, organize the macroscopic dynamics into three architecture-dependent regimes: permissive (llama3.1:8b), near-deterministic (mistral:7b), and conservative (phi3:14b). A mean-field two-word calculation yields the ordering condition R = 3π − 2φ − 1 > 0, which generalizes the stochastic-NG threshold. The paper further reports finite-size scaling exponents β(T) and temperature-response exponents α, and proposes temperature as a control parameter for decentralized LLM populations.
Significance. If the two-rate parametrization were validated as a closed microscopic description, this would be a useful bridge between LLM stochasticity and the statistical mechanics of convention formation, with practical implications for multi-agent LLM deployment. The direct measurement of π and φ from agent interactions, the explicit mean-field derivation of the critical line, and the honest discussion of limitations are strengths. However, the central claim that the (π,φ) pair captures leading-order macroscopic phenomenology across all three regimes is not yet secured, especially for the conservative phi3 regime where inventory size demonstrably co-determines the dynamics. The paper's own concessions in Secs. IV B and VII indicate that the two-rate reduction and the derived critical line lack demonstrated validity in exactly the regime where the paper is most novel. The study is therefore significant but currently overclaims its predictive scope.
major comments (4)
- [Sec. VII, Eq. (7)] The effective two-rate rule treats π and φ as constants depending only on inventory membership, but the paper's own data contradict this in the conservative regime. Sec. IV B reports that phi3 at T=0.05 accumulates an average inventory of ≈34 words per agent and stalls without consensus, and Sec. VII concedes that "the two-word reduction does not capture the inventory-size effects that dominate phi3 at low T." Since the LLM's decision is conditioned on the full prompt, which includes the entire inventory, π and φ can depend on inventory size, composition, and even word order. This means Eq. (11) has no demonstrated validity in the large-inventory conservative regime, where the paper nevertheless uses R to predict a fragmentation crossing. Please either measure π and φ conditioned on inventory size and show that the two-rate closure is adequate, or substantially restrict the claim that th
- [Sec. IV B / Fig. 5c / Fig. 7c vs. Sec. VIII] There is an internal contradiction about consensus. The Abstract and Sec. VIII state that "consensus is always reached" and "strict 1-consensus is always reached on the fully connected graph," but Sec. IV B reports that phi3 at T=0.05 plateaus at N_d≈7 without reaching consensus through 9×10^4 steps, and Fig. 7c uses upward arrows to indicate seeds that did not reach consensus within the 1.75×10^5-step horizon. These are censored observations, not reached consensus states. The claim "always reached" is unsupported as written. Revise to describe the fraction of seeds reaching consensus, use survival analysis or censoring-aware estimators, or clearly separate 'reached within horizon' from 'would reach at infinite time'.
- [Sec. VII, Eq. (11)] The predictive content of the critical line R>0 is not verified in the most relevant regime. For phi3 at T=2.0, the measured plateau values give π≈0.30 and φ≈0, hence R≈−0.1, nominally below the critical line; nevertheless strict consensus is observed at all explored sizes. The paper attributes this to finite-N rounding, but no finite-size collapse, crossing analysis, or divergence of consensus time near R=0 is provided. Without such evidence, Eq. (11) remains a suggestive mean-field result rather than a validated ordering condition for the LLM-NG. Please add finite-size scaling of the order parameter or survival probability for phi3 near R=0, or clearly label the R-based prediction as an untested asymptotic conjecture.
- [Sec. III, Eq. (5)] The drift proxy Δ(t)=mπ−λ(1−m)φ uses λ=1 as a free, ad hoc choice. The paper acknowledges that a rigorous mean-field treatment would yield λ≠1, possibly state-dependent. Since the sign of Δ is used to label ordering versus disordering and to rationalize the negative-drift transients in Fig. 4, the classification is not robust unless λ is derived from the microscopic rates or its value is shown not to change any qualitative conclusion. Please provide a derivation of λ or a sensitivity analysis over a plausible range of λ values.
minor comments (4)
- [Fig. 7 / Fig. 8 captions] The upward arrows indicating censored seeds are essential but easy to overlook. Consider stating explicitly in the caption that censored seeds are treated as lower bounds and give the fraction of censored seeds per temperature for phi3.
- [Sec. V] The statement that phi3's dynamics 'coincides' with the stochastic negotiation model of Ref. [7] under β↔π(T) should be softened: the equivalence holds only for the ϕ=0 slice and under the constant-rate assumption, which the paper itself questions. Suggest 'reduces to the ϕ=0 slice of the (π,ϕ) plane in the plateau approximation'.
- [General] No data or code availability statement is provided. Given the compute budget and the focus on reproducibility of LLM-based simulations, a statement on data/code availability or at least seed/config details would strengthen the paper.
- [Sec. II] The prompt variation caveat is appropriately noted, but the claim that alternative phrasings 'could shift the numerical values' should be tied to a concrete sensitivity estimate if possible; otherwise it remains a loose caveat.
Circularity Check
No significant circularity; minor non-load-bearing self-citation (Ref. [22]) and an acknowledged scope limitation for phi3 low-T.
full rationale
The paper's derivation chain is largely self-contained. The conditional rates pi(T) and phi(T) are measured directly from interaction outcomes (Eqs. 1-2), not fitted to reproduce consensus times or the regime classification. The mean-field critical line R=3*pi-2*phi-1>0 follows from an explicit two-word sector calculation (Eqs. 8-11) under the stated two-rate rule (Eq. 7); this is an analytical derivation, not a parameter fit. The finite-size exponents beta and the temperature-response exponents alpha are presented as measurements, and the paper does not claim they are derived from pi/phi. The only self-citation is Ref. [22] (first author C. De Nobili), used as a supporting observation about prompt rewordings and 'temperature blindness' in a related lattice study; it is not load-bearing for the central critical-line result. The most serious caveat is explicit: Sec. VII concedes 'By construction, the two-word reduction does not capture the inventory-size effects that dominate phi3 at low T (Sec. IV B)', so the theoretical mechanism does not cover the conservative large-inventory regime. This is a scope limitation and an honest admission of incompleteness, not a circular step in which an input is renamed as a prediction. The drift proxy Delta in Eq. (5) is by construction defined so that positive drift means net ordering, but the paper labels it a phenomenological proxy and anchors the theoretical claim in the independently derived mean-field condition. Overall, no 'prediction' reduces to a fitted input by construction; score 2 reflects the minor non-load-bearing self-citation.
Axiom & Free-Parameter Ledger
free parameters (3)
- lambda =
1
- beta(T) =
llama approx 1.3-2.0; mistral avg 1.28 +/- 0.13; phi3 avg 1.59 +/- 0.47
- alpha =
llama 0.67 +/- 0.14; mistral 0.01 +/- 0.02; phi3 0.43 +/- 0.31
axioms (4)
- domain assumption Complete-graph mean-field: Eq. (A1) assumes homogeneous mixing and N goes to infinity so that pair correlations are negligible.
- ad hoc to paper Two-word-sector reduction: ordering is analyzed by restricting dynamics to two competing words A and B.
- domain assumption LLM listener behavior is summarized by two constants pi and phi independent of inventory size, word identity, and agent heterogeneity (Eq. 7).
- ad hoc to paper Drift proxy Delta(t) = m*pi - lambda*(1-m)*phi with lambda=1 has a sign that reflects ordering vs disordering.
read the original abstract
Decentralized populations of Large Language Model (LLM) agents can spontaneously reach consensus on shared conventions, yet the microscopic mechanisms by which their internal stochasticity shapes macroscopic ordering remain unexplored. We study a minimal LLM Naming Game in which the listener's decision is a single-token LLM call at decoding temperature $T$, replacing the inventory check of the deterministic Naming Game. Each interaction decomposes into an in-inventory and an out-inventory channel with conditional rates $\pi(T)\!\equiv\!P(\text{YES}\mid w\in P_j)$ and $\phi(T)\!\equiv\!P(\text{YES}\mid w\notin P_j)$, whose balance controls an ordering-disordering drift. A mean-field theory of the two-rate dynamics yields an analytical ordering condition that generalizes the consensus threshold of the stochastic Naming Game to a critical line in the $(\pi,\phi)$ plane. Across three open-weight architectures, consensus is always reached, but through three distinct listener regimes: permissive (repaint-noise dominated), near-deterministic, and conservative (missed-collapse dominated). The effective finite-size exponent $\beta(T)$ in $t_{\rm conv}\!\sim\!N^{\beta}$ shifts with temperature, and the temperature-sensitivity $\alpha$ in $t_c\!\sim\!e^{\alpha T}$ ranges from ${\approx}\,0.67$ to ${\approx}\,0$ across architectures. Decoding temperature thus emerges as an architecture-dependent control parameter for decentralized LLM populations, quantitatively characterized by the statistical-physics toolkit.
Figures
Reference graph
Works this paper leans on
-
[1]
Castellano, S
C. Castellano, S. Fortunato, and V. Loreto, Rev. Mod. Phys.81, 591 (2009)
2009
-
[2]
Steels, Artificial Life2, 319–332 (1995)
L. Steels, Artificial Life2, 319–332 (1995). 11
1995
-
[3]
Baronchelli, M
A. Baronchelli, M. Felici, V. Loreto, E. Caglioti, and L. Steels, J. Stat. Mech. , P06014 (2006)
2006
-
[4]
temperature blindness
This is the macroscopic fingerprint of the repaint- dominated regime: asTrises,ϕ(T) grows and the dis- ordering channel becomes increasingly active; the con- sensus time inherits this growth at an exponential rate. The exponential, rather than linear, dependence onTis itself informative; it suggests thatt c is sensitive to thecu- mulativeeffect of many re...
-
[5]
Loreto, A
V. Loreto, A. Baronchelli, A. Mukherjee, A. Puglisi, and F. Tria, J. Stat. Mech. , P04006 (2011)
2011
-
[6]
Baronchelli, L
A. Baronchelli, L. Dall’Asta, A. Barrat, and V. Loreto, Phys. Rev. E73, 015102(R) (2006)
2006
-
[7]
Dall’Asta, A
L. Dall’Asta, A. Baronchelli, A. Barrat, and V. Loreto, Phys. Rev. E74, 036105 (2006)
2006
-
[8]
Baronchelli, L
A. Baronchelli, L. Dall’Asta, A. Barrat, and V. Loreto, Phys. Rev. E76, 051102 (2007)
2007
-
[9]
T. Guoet al., Large language model based multi- agents: A survey of progress and challenges (2024), arXiv:2402.01680
Pith/arXiv arXiv 2024
-
[10]
S. Honget al., MetaGPT: Meta programming for a multi- agent collaborative framework (2024), arXiv:2308.00352
Pith/arXiv arXiv 2024
-
[11]
N. F. Johnson, Increasing intelligence in AI agents can worsen collective outcomes (2026), arXiv:2603.12129
arXiv 2026
-
[12]
A. F. Ashery, L. M. Aiello, and A. Baronchelli, Science Advances11, eadu9368 (2025)
2025
-
[13]
Centola and A
D. Centola and A. Baronchelli, Proceedings of the Na- tional Academy of Sciences112, 1989–1994 (2015)
1989
-
[14]
G. De Marzo, C. Castellano, and D. Garcia, AI agents can coordinate beyond human scale (2025), arXiv:2409.02822 [physics.soc-ph]
Pith/arXiv arXiv 2025
- [15]
-
[16]
G. De Marzo, A. Bellina, C. Castellano, V. Priesemann, and D. Garcia, Conformity generates collective misalign- ment in AI agents societies (2026), arXiv:2605.10721 [physics.soc-ph]
Pith/arXiv arXiv 2026
-
[17]
A. Bellina, G. D. Marzo, and D. Garcia, Conformity and social impact on ai agents (2026), arXiv:2601.05384 [cs.AI]
arXiv 2026
-
[18]
Z.-Y. Song, Q.-H. Cao, M.-X. Luo, and H. X. Zhu, Detailed balance in large language model-driven agents (2025), arXiv:2512.10047
arXiv 2025
- [19]
-
[20]
Chuanget al., Simulating opinion dynamics with networks of LLM-based agents (2024), arXiv:2311.09618
Y.-S. Chuanget al., Simulating opinion dynamics with networks of LLM-based agents (2024), arXiv:2311.09618
Pith/arXiv arXiv 2024
-
[21]
G. Piatti, Z. Hu, and K. Cho, Cooperate or collapse: Emergence of sustainable cooperation in a society of LLM agents (2024), arXiv:2404.16698
Pith/arXiv arXiv 2024
-
[22]
C. Ruan, Y. Wang, Z. Shi, and J. Li, Reaching agreement among reasoning LLM agents (2025), arXiv:2512.20184 [cs.DC]
arXiv 2025
-
[23]
C. De Nobili, Collective alignment in LLM multi-agent systems: Disentangling bias from cooperation via sta- tistical physics (2026), arXiv:2605.10528 [cond-mat.stat- mech]
Pith/arXiv arXiv 2026
-
[24]
Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, and J. Wang, inProc. ICML 2018, PMLR, Vol. 80 (2018) p. 5571
2018
- [25]
-
[26]
A. Holtzman, J. Buys, L. Du, M. Forbes, and Y. Choi, The curious case of neural text degeneration (2020), arXiv:1904.09751 [cs.CL]
Pith/arXiv arXiv 2020
-
[27]
Ollama, Ollama (2024)
2024
-
[28]
A. Mehdizadeh and M. Hilbert, Exploring the topol- ogy and memory of consensus: How llm agents agree, fragment, or settle when forming conventions (2026), arXiv:2606.04197 [cs.MA]
Pith/arXiv arXiv 2026
-
[29]
R. Zhang and H. Woisetschl¨ ager, Sign: Schema-induced games for naming (2025), arXiv:2510.21855 [cs.AI]
arXiv 2025
-
[30]
J. Xie, S. Sreenivasan, G. Korniss, W. Zhang, C. Lim, and B. K. Szymanski, Phys. Rev. E84, 011130 (2011)
2011
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.