Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that under some conditions, large language models make near-deterministic maximum-likelihood decisions despite nonzero sampling temperature, so simulated Gibbs sampling can converge to a false prior rather than the…

desk verdict The §3.1 MLE chain analysis is correct and new, but the initial-condition diagnostic is unvalidated and the experimental support is thin; send to review conditionally. read the letter →

arxiv 2506.10268 v1 pith:5JYQYDZY submitted 2025-06-12 cs.CL cs.AIcs.LG

Andrea Yaoyun Cui , Pengfei Yu This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords languagemodelpriorsiteratedlearningGibbssamplingmaximumlikelihoodestimationdeterministicdecision-makingstochasticpriorelicitationMarkovchain
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are usually treated as stochastic samplers over token distributions, and prior work has used simulated Gibbs sampling to infer their priors on quantities like coin bias. This paper argues that this treatment can be wrong: under certain conditions, models make near-deterministic decisions that match maximum likelihood estimation (MLE), even when the sampling temperature is nonzero. If so, an iterated-learning chain based on such a model does not converge to a prior at all; it converges to a polarization artifact, which can look like a reasonable bimodal prior. The paper derives the stationary distribution of an MLE-driven chain, proposes a simple diagnostic that varies the initial value of the chain, and shows experimentally that some large language models behave deterministically on a coin-bias task while behaving stochastically on a life-expectancy task. The practical stake is that prior-elicitation results from this line of work may be artifacts of the procedure rather than genuine model beliefs.

What carries the argument

The central object is the Markov chain formed by alternating a model's point estimate with an observation step: in the coin task, the model outputs $\Omega_i$, the chain updates $\omega_i \sim \mathrm{Binomial}(N, \omega_{i-1}/N)$, and the estimated bias is $\theta_i = \omega_{i-1}/N$. Under an MLE decision rule this chain is absorbed at $0$ or $N$, with the stationary weight at $N$ equal to the expected initial count divided by $N$; this formula is what lets a deterministic process produce an apparent bimodal prior. The paper's diagnostic machinery is the invariance principle: the stationary distribution of a well-mixed stochastic Gibbs sampler is the same for different starting values $\omega_0$, so comparing stationary distributions across initial values separates stochastic from deterministic decision patterns.

What would settle it

Run the proposed initial-value test on a controlled stochastic baseline, for instance a synthetic Gibbs sampler targeting a known bimodal distribution, or a language model whose responses are verified to vary across repeated identical prompts at temperature 1.0. If a genuinely stochastic process nevertheless shows starting-value-dependent stationary distributions (due to finite chains, slow mixing, or multi-modality), the diagnostic would misclassify it as deterministic; conversely, if a model flagged as deterministic produces multiple distinct estimates on repeated identical prompts, the deterministic label is wrong.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that deterministic decision-making can masquerade as Bayesian sampling in iterated-learning protocols. For the proportion-estimation task, if a language model replaces posterior sampling with the MLE rule $\theta_i = \omega_{i-1}/N$, the induced Markov chain $\omega_i \sim \mathrm{Binomial}(N, \omega_{i-1}/N)$ has absorbing states at $\omega = 0$ and $\omega = N$, and its stationary distribution satisfies $\hat{p}_{\mathrm{MLE}}(0) = 1 - \mathbb{E}[\omega_0]/N$ and $\hat{p}_{\mathrm{MLE}}(N) = \mathbb{E}[\omega_0]/N$, concentrated on the extremes. Because this reproduces the bimodal coin-bias prior reported by prior work, the paper argues that the reported prior may be an artifact of deterministic MLE-style behavior rather than evidence of a genuine bimodal belief. The paper's proposed distinction is that a genuine stochastic Gibbs sampler has a stationary distribution invariant to the initial value $\omega_0$, whereas deterministic processes can show dependence on it; experiments that vary $\omega_0$ across models and tasks reveal both patterns.

Load-bearing premise

The diagnostic relies on the premise that a genuinely stochastic Gibbs-sampling process always yields a stationary distribution that is independent of the starting value, so any observed dependence on the starting value can only mean deterministic behavior.

Editorial extensions

If this is right

  • Prior-elicitation studies that use simulated Gibbs sampling on large language models should treat reported priors as provisional until they check whether the chain's stationary distribution depends on initialization.
  • A deterministic MLE-style model can produce the same extreme, bimodal-looking output as a supposedly Bayesian model, so observing a bimodal distribution is not evidence for a bimodal prior.
  • Models can switch decision patterns by task: the paper finds deterministic behavior in coin-bias estimation and stochastic behavior in life-expectancy estimation, so a single 'stochastic vs deterministic' label for a model is too coarse.
  • The proposed initial-value test is a sufficient condition for detecting non-Gibbs processes: if distributions diverge across starting values, the process is not posterior Gibbs sampling.
  • Behavioral conclusions about language model priors drawn from iterated learning should be re-examined, because the apparent prior may instead reflect the iterative prompting dynamics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper does not develop: the same initial-value test could diagnose deterministic behavior in other Markov-chain-with-LLM protocols, such as chain-of-thought or MCMC-based mental-representation recovery, whenever the stationary distribution would be initialization-dependent.
  • Because the paper tests each model with a single temperature of 1.0, a natural next experiment is to sweep temperature and seed; if deterministic patterns persist at high temperature, that would strengthen the claim that temperature alone does not restore stochastic decision-making.
  • A calibration baseline using a known stochastic Gibbs sampler (for example, a synthetic posterior with a symmetric bimodal target) would quantify how often the test mislabels slow-mixing chains as deterministic; the paper does not provide that baseline.
  • The deterministic classification could be checked directly by inspecting repeated responses or logprob distributions on identical prompts, which would offer a complementary within-chain test rather than a between-chain test.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper revisits the question of whether LLMs make stochastic decisions during iterated-learning prior elicitation. It analyzes a deterministic MLE decision rule for a coin-flip proportion-estimation task, derives the stationary distribution of the induced Markov chain (Eq. 4), and shows that this distribution is bimodal and depends on the initial value. It then proposes that varying the initial value ω0 can distinguish deterministic from stochastic decision processes, reports experiments on six LLMs, and concludes that some models (notably gpt-4o-mini) behave as MLE decision makers while others behave stochastically. The theoretical derivation in Section 3.1 is mathematically correct, but the proposed diagnostic and the empirical support have substantial gaps, and the final limitation section overclaims the method's sufficiency.

Significance. The paper's main strength is the clean martingale derivation in Section 3.1: the MLE decision rule combined with binomial resampling yields a Markov chain whose absorption probabilities are given exactly by Eq. (4), producing a bimodal 'false prior' that can mimic Bayesian prior-elicitation results. This is a credible, concrete challenge to prior work that infers LLM priors from iterated in-context learning, and the paper explicitly identifies a failure mode that should be taken seriously. If the empirical classification were validated, the paper would be a useful cautionary result for the behavioral ML community. However, the significance is currently conditional: the diagnostic used to classify models is not validated against known stochastic processes, the experimental evidence is largely qualitative, and the manuscript does not provide code or full experimental protocol. The paper deserves credit for stating its limitations explicitly, but the limitation section's claim of 'universal applicability' and sufficiency is not established.

major comments (4)
  1. [3.2, 6] The proposed initial-condition test is not a valid discriminator between stochastic and deterministic decision processes as stated. Invariance of the stationary distribution to the initial value is an asymptotic property of an irreducible, aperiodic finite chain; for the finite chains used in the experiments (N=10, no convergence diagnostics or burn-in reported), a stochastic Gibbs sampler can exhibit initial-value dependence due to slow mixing or multimodality. Conversely, Section 3.2 concedes that some deterministic processes converge to the same distribution across initial values, and this caveat is dismissed only by an appeal to 'empirical evidence' without reporting a test. The manuscript needs a positive control on a known stochastic process to measure the false-positive rate before any model can be classified as deterministic on the basis of this test.
  2. [4.2, Figures 1-3] The empirical classifications are made by visual inspection. The manuscript does not report the number of chains, iterations per chain, burn-in, repeated seeds, or any quantitative comparison of the empirical distributions across initial ω0. For example, the claim that claude-3-haiku, claude-3-5-sonnet and LLaMA-3.1-70B-Instruct produce 'consistent distributions regardless of the initial values' is not backed by a test statistic, divergence measure, or error bars. Without this, the main empirical contrast between deterministic and stochastic models is not established, and the paper is not reproducible.
  3. [6] The final limitation section claims that 'our proposed identification approach is a sufficient condition to detect non-Gibbs-sampling processes, which further indicates non-stochastic decision processes.' This is a non sequitur: observing initial-value dependence can indicate non-convergence or non-stationarity of a stochastic process, not necessarily non-stochasticity. In addition, non-Gibbs is not equivalent to non-stochastic, because many stochastic decision rules are not Gibbs samplers. This claim should be either removed or proved under explicit assumptions about the chain and the number of iterations.
  4. [4.2, Eq. (4)] The evidence that gpt-4o-mini is an MLE decision maker rests on 'the black dotted line ... which we fit featuring p(θ=1|Ω0) with respect to Ω0' coinciding with Eq. (4). It is unclear what was fitted: if the line is the theoretical curve from Eq. (4), it has no free parameters and the word 'fit' is misleading; if it is a fitted line, the fitted parameters and goodness of fit should be reported. Relatedly, the interpretation of gpt-4o and Gemma-2-2b-it as 'near-deterministic, except under certain conditions' is presented without a testable model or statistical comparison to an alternative stochastic prior.
minor comments (5)
  1. [3.1, Eq. (2)] The displayed computation of E[ω_i] is difficult to read; it should be written cleanly as E[ω_i] = E[E[ω_i | ω_{i-1}]] = E[ω_{i-1}].
  2. [4.2] The paper uses 'stationary distribution' to describe the empirical distribution of finite simulation runs; please distinguish the asymptotic stationary distribution from the empirical distribution over the finite number of iterations actually run.
  3. [Figures 1-3] The figure captions appear garbled (the text contains unicode token sequences such as '/uni00000013/...') and do not explain the axes, plotted curves, or the dotted line; the figures are consequently not interpretable from the captions.
  4. [4.1] The experimental protocol is underspecified: please report the number of chains per initial value, the number of iterations per chain, any burn-in, temperature and decoding parameters, and how the 'single value' response was extracted from each model.
  5. [4.2, LifeExpectancy] The LifeExpectancy MLE comparison (Li = Ai-1) is degenerate and thus not a strong baseline; a sentence explaining why this is still informative would help the reader interpret the stochasticity claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the MLE stationary distribution is derived from the stated deterministic decision rule, and the only empirical comparison is an explicit fit to a parameter-free theoretical curve.

full rationale

The paper's central derivation in Section 3.1 begins from the explicit assumption of an MLE decision rule (Omega_i = M * omega_{i-1}/N) and derives the Markov transition, the absorbing states, and the stationary distribution in Equations (1)-(4) by direct algebra; Equation (4) is a consequence of the assumed rule, not an input to it. The Section 3.2 initial-value criterion is a proposed heuristic with an acknowledged caveat that some deterministic processes may also converge to a common distribution, and Section 6's claim that the criterion detects non-Gibbs-sampling processes 'which further indicates non-stochastic decision processes' is logically unsupported because non-Gibbs stochastic processes exist; however, that is an assumption-validity or correctness concern, not a circular reduction. The black dotted line in Figure 1a is described as a fit of p(theta=1|Omega_0) to the data, and its agreement with the parameter-free MLE expression in Equation (4) is an external comparison rather than a fitted parameter being relabeled as a prediction. No load-bearing argument rests on a self-citation or an imported uniqueness theorem. The derivation chain is self-contained, so no step reduces by construction to its own inputs.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central derivation relies on standard probability theory (Markov chains, martingales). The domain assumptions concern the validity of the Gibbs-sampling analogy and the decoding setup. No new physical or algorithmic entities are introduced.

free parameters (1)
  • Consistency threshold for classifying models as deterministic vs stochastic = not specified
    The paper does not define a quantitative criterion for whether stationary distributions across initial values are 'consistent' or 'diverging'. The classification of gpt-4o-mini as deterministic and claude-3-5-sonnet as stochastic appears to be by visual inspection of figures.
assumptions (4)
  • domain assumption The iterated learning process with a stochastic decision maker is a Gibbs sampler whose stationary distribution is the model's prior and is invariant to initial conditions.
    Section 3.2 uses this to infer that initial-value dependence indicates deterministic behavior. Standard MCMC theory supports this for ergodic chains, but the paper does not verify mixing or address multi-modality.
  • ad hoc to paper The language model's decisions can be modeled as a deterministic function of the input, specifically MLE for the CoinFlip task, for at least some models.
    Section 3.1 defines the MLE rule Ω_i = M ω_{i-1}/N as an alternative to posterior sampling. This is the paper's proposed model, not independently verified.
  • domain assumption Sampling with temperature 1.0 provides the stochasticity expected of a Gibbs sampler.
    The experiments set temperature 1.0, but the paper does not verify that the model's output distribution matches the intended next-token distribution; decoding details are unspecified.
  • ad hoc to paper When a model avoids predicting 0 or N heads, it is 'near-deterministic except under certain conditions' rather than stochastic with a particular prior.
    Section 4.2 interprets gpt-4o and Gemma's one-sided extreme behavior in this way, but the interpretation is post hoc.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models." pith.science (2026). https://pith.science/paper/5JYQYDZY

@misc{pith2026250610268,
  author       = {Pith},
  title        = {Pith review of: Do Language Models Have Bayesian Brains? Distinguishing Stochastic and Deterministic Decision Patterns within Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JYQYDZY}},
  note         = {Machine review of arXiv:2506.10268}
}
read the original abstract

Language models are essentially probability distributions over token sequences. Auto-regressive models generate sentences by iteratively computing and sampling from the distribution of the next token. This iterative sampling introduces stochasticity, leading to the assumption that language models make probabilistic decisions, similar to sampling from unknown distributions. Building on this assumption, prior research has used simulated Gibbs sampling, inspired by experiments designed to elicit human priors, to infer the priors of language models. In this paper, we revisit a critical question: Do language models possess Bayesian brains? Our findings show that under certain conditions, language models can exhibit near-deterministic decision-making, such as producing maximum likelihood estimations, even with a non-zero sampling temperature. This challenges the sampling assumption and undermines previous methods for eliciting human-like priors. Furthermore, we demonstrate that without proper scrutiny, a system with deterministic behavior undergoing simulated Gibbs sampling can converge to a "false prior." To address this, we propose a straightforward approach to distinguish between stochastic and deterministic decision patterns in Gibbs sampling, helping to prevent the inference of misleading language model priors. We experiment on a variety of large language models to identify their decision patterns under various circumstances. Our results provide key insights in understanding decision making of large language models.

Figures

Figures reproduced from arXiv: 2506.10268 by the authors.

Figure 1
Figure 1. CoinFlip results on all models. 20 40 60 80 100 Initial Value 0 20 40 60 80 100 Life Expectancy 0.0 0.1 0.2 0.3 0.4 0.5 0.6 Frequency (a) gpt-4o-mini 20 40 60 80 100 Initial Value 0 20 40 60 80 100 Life Expectancy 0.0 0.2 0.4 0.6 0.8 Frequency (b) Gemma-2-2b-it 20 40 60 80 100 Initial Value 0 20 40 60 80 100 Life Expectancy 0.0 0.2 0.4 0.6 0.8 Frequency (c) claude-3-haiku [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. LifeExpectancy results on selected models. Results of other models in Appendix. Notably, gpt-4o consistently converges to a distribution focused at θ = 1. Further analysis reveals that gpt-4o actively avoids predicting 0 heads, which prevents Ω = 0 from becoming an absorb￾ing state in the Markov process discussed in Section 3.1. This behavior suggests that the model incorporates a prior that avoids θ = 0. However, i… view at source ↗
Figure 3
Figure 3. Additional LifeExpectancy results. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Tractable Asymmetric Verification for Large Language Models via Deterministic Replicability

    cs.AI 2025-09 conditional novelty 3.0 of 10

    An LLM output can be verified by regenerating a few randomly chosen segments under identical hardware, with a tunable detection probability and 12.4x speedup over full regeneration.

Reference graph

Works this paper leans on

19 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    The claude 3 model family: Opus, sonnet, haiku, 2024

    Anthropic. The claude 3 model family: Opus, sonnet, haiku, 2024. Accessed: 13-Sep-2024

    work page 2024

  3. [3]

    Claude 3.5 sonnet, 2024

    Anthropic. Claude 3.5 sonnet, 2024. Accessed: 13-Sep-2024

    work page 2024

  4. [4]

    Revealing human inductive biases for category learning by simulating cultural transmission.Psychonomic Bulletin & Review, 21:785–793, 2014

    Kevin R Canini, Thomas L Griffiths, Wolf Vanpaemel, and Michael L Kalish. Revealing human inductive biases for category learning by simulating cultural transmission.Psychonomic Bulletin & Review, 21:785–793, 2014

    work page 2014

  5. [5]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    arXiv 2024

  6. [6]

    Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.IEEE Trans

    Stuart Geman and Donald Geman. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images.IEEE Trans. Pattern Anal. Mach. Intell., 6(6):721–741, 1984

    work page 1984

  7. [7]

    Using category structures to test iterated learning as a method for identifying inductive biases.Cognitive Science, 32(1):68–107, 2008

    Thomas L Griffiths, Brian R Christian, and Michael L Kalish. Using category structures to test iterated learning as a method for identifying inductive biases.Cognitive Science, 32(1):68–107, 2008

    work page 2008

  8. [8]

    Language evolution by iterated learning with bayesian agents.Cognitive science, 31(3):441–480, 2007

    Thomas L Griffiths and Michael L Kalish. Language evolution by iterated learning with bayesian agents.Cognitive science, 31(3):441–480, 2007

    work page 2007

Show all 19 references
  1. [9]

    Iterated learning: Intergen- erational knowledge transmission reveals inductive biases.Psychonomic Bulletin & Review, 14(2):288–294, 2007

    Michael L Kalish, Thomas L Griffiths, and Stephan Lewandowsky. Iterated learning: Intergen- erational knowledge transmission reveals inductive biases.Psychonomic Bulletin & Review, 14(2):288–294, 2007

    2007

  2. [10]

    The wisdom of individuals: Exploring people’s knowledge about everyday events using iterated learning.Cognitive science, 33(6):969–998, 2009

    Stephan Lewandowsky, Thomas L Griffiths, and Michael L Kalish. The wisdom of individuals: Exploring people’s knowledge about everyday events using iterated learning.Cognitive science, 33(6):969–998, 2009

    2009

  3. [11]

    Gpt-4o mini: advancing cost-efficient intelligence, 2024

    OpenAI. Gpt-4o mini: advancing cost-efficient intelligence, 2024. Accessed: 13-Sep-2024

    2024

  4. [12]

    Hello gpt-4o, 2024

    OpenAI. Hello gpt-4o, 2024. Accessed: 13-Sep-2024

    2024

  5. [13]

    The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning.Cognition, 111(3):317–328, 2009

    Florencia Reali and Thomas L Griffiths. The evolution of frequency distributions: Relating regularization to inductive biases through iterated learning.Cognition, 111(3):317–328, 2009

    2009

  6. [14]

    Markov chain monte carlo with people.Advances in neural information processing systems, 20, 2007

    Adam Sanborn and Thomas Griffiths. Markov chain monte carlo with people.Advances in neural information processing systems, 20, 2007

    2007

  7. [15]

    Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024

    Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at a practical size.arXiv preprint arXiv:2408.00118, 2024. 5

    2024 arXiv

  8. [16]

    Probing BERT’s priors with serial reproduction chains

    Takateru Yamakoshi, Thomas Griffiths, and Robert Hawkins. Probing BERT’s priors with serial reproduction chains. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, pages 3977–3992, Dublin, ...

    2022

  9. [17]

    Identifying expectations about the strength of causal relationships.Cognitive psychology, 76:1–29, 2015

    Saiwing Yeung and Thomas L Griffiths. Identifying expectations about the strength of causal relationships.Cognitive psychology, 76:1–29, 2015

    2015

  10. [18]

    Eliciting the priors of large language models using iterated in-context learning.arXiv preprint arXiv:2406.01860, 2024

    Jian-Qiao Zhu and Thomas L Griffiths. Eliciting the priors of large language models using iterated in-context learning.arXiv preprint arXiv:2406.01860, 2024

    2024 arXiv

  11. [19]

    Recovering mental representations from large language models with markov chain monte carlo.arXiv preprint arXiv:2401.16657, 2024

    Jian-Qiao Zhu, Haijiang Yan, and Thomas L Griffiths. Recovering mental representations from large language models with markov chain monte carlo.arXiv preprint arXiv:2401.16657, 2024. A Model V ersions We use following snapshots for closed-source models: gpt-4o-mini-2024-07-18,...

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.