Pith. sign in

REVIEW 4 major objections 5 minor 15 references

Exploring Next Token Prediction in Theory of Mind (ToM) Tasks: Comparative Experiments with GPT-2 and LLaMA-2 AI Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that adding more context sentences lowers next-token prediction confidence in Theory of Mind prompts, and that Llama-2-7b handles the added complexity better than GPT-2, especially on second-order belief questions.

desk verdict A small confidence-measurement study whose headline claims about Theory of Mind accuracy do not survive contact with its own definition of 'correct token.' read the letter →

arxiv 2504.15604 v1 pith:COZEYJL4 submitted 2025-04-22 cs.CL cs.AI

Pavan Yadav , Nikhil Khandalkar , Krishna Shinde , Lokesh B. Ramegowda , Rajarshi Das This is my paper · ORCID
classification cs.CLcs.AI
keywords next-tokenpredictionTheoryofMindGPT-2LLaMA-2infillcontexttemperaturebeliefreasoningtokenprobability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tests whether two language models, GPT-2 and Llama-2-7b-chat-hf, can track characters' beliefs and false beliefs by predicting the next token in short narratives. It builds story variants with 0, 1, 4, 16, or 64 extra sentences inserted by GPT-4, then measures the probability the models assign to their top token across three reasoning levels: factual state tracking, first-order belief reasoning, and second-order recursive belief reasoning. The central claim is that more context reduces top-token probability for both models, but Llama-2 degrades less and achieves higher top-token probability than GPT-2, particularly on second-order questions. A sympathetic reader would care because next-token probability is offered as a window into whether language models implicitly represent who knows what in a conversation.

What carries the argument

The measuring instrument is top-token probability: for each prompt the model produces a ranked token list, the highest-probability token is labeled the correct token (CT), and the paper averages CT probabilities over three predicted tokens as its accuracy metric. This machinery lets the authors study context, temperature, and reasoning level without labeling ground truth, but it also means the reported 'accuracy' is really the model's self-assigned confidence.

What would settle it

Take the same prompts and compare each top token against the actual completion string from the ExploreToM dataset; if the top token is often not the expected answer while its probability stays high, the reported 'prediction accuracy' collapses to confidence rather than correctness.

Watch

Extended reading notes

Core claim

On the ten ExploreToM stories and their infilled variants, increasing infill density reduces the average probability of the model's most likely next token, and Llama-2-7b-chat-hf is more resilient to this degradation than GPT-2. Second-order questions, which require reasoning about one character's belief about another character's belief, produce the largest probability drops and the greatest response diversity in both models. Higher temperature settings flatten the token distribution and spread probability across alternatives, with GPT-2 showing more variability and Llama-2 retaining more consistent top-token confidence.

Load-bearing premise

The whole accuracy story rests on assuming the model's own highest-probability token is the correct answer, so no ground-truth comparison is needed.

Editorial extensions

If this is right

  • Longer narrative context, at least when it is programmatically inserted, tends to lower a model's confidence in its next-token choice on Theory of Mind-style prompts.
  • Larger models with longer context windows, such as Llama-2-7b-chat, can handle more injected context without as much confidence loss as GPT-2.
  • Second-order belief reasoning is harder for both models than zero-order state tracking, as measured by top-token probability and response diversity.
  • Higher sampling temperatures reduce top-token probability for both models and increase the spread of alternative tokens, so temperature is a real control knob for prediction confidence.
  • The 1024-token input limit of GPT-2 caused some 64-infill questions to be skipped, so cross-model comparisons at the highest context level are incomplete.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's 'prediction accuracy' is better read as 'model confidence', since the top token is never checked against the ExploreToM ground-truth answer; an extension would compare CT tokens to the expected completion string.
  • The infill sentences are generated by GPT-4 and may not match the style or factual structure of the original stories, so the observed decline could partly reflect distribution shift rather than reasoning failure.
  • A testable follow-up is to insert matched distractors that are semantically relevant versus irrelevant to the belief question, which would separate genuine context-tracking from superficial priming effects.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper constructs a dataset from 10 ExploreToM stories by programmatically inserting 0, 1, 4, 16, or 64 GPT-4-generated infill sentences, then prompts GPT-2 and Llama-2-7b-chat-hf to predict the next three tokens for zero-, first-, and second-order theory-of-mind questions under several temperature settings. The authors define the 'correct token' as the model's own highest-probability token, average the probabilities of these top tokens, and report that increasing infill density reduces prediction 'accuracy' for both models, that higher temperatures reduce 'accuracy', and that Llama-2 'outperforms' GPT-2, particularly on second-order questions. The central results are presented as claims about prediction accuracy and reasoning performance, but the evaluation never compares any prediction to an external ground-truth continuation or to the ExploreToM gold answers.

Significance. If the claims were valid, the study would offer useful evidence on how context length, temperature, and reasoning complexity affect next-token prediction in theory-of-mind tasks. The experimental matrix is broad (two models, five infill levels, five temperatures, three question orders, ten stories), and the authors provide a code repository and a reproducible prompting protocol, which are strengths. However, the validity of every headline claim depends on an external standard of correctness, and that standard is absent. The reported quantities are self-confidence scores, not accuracies, so the paper's main conclusions do not follow from the measurements. The work is better framed as an exploratory study of model confidence and distributional spread, but as written the accuracy-based conclusions are unsupported.

major comments (4)
  1. [Section 5.1, 5.2, 6, 7] The central evaluation is self-referential. Section 5.1 states that 'The token with the highest probability in the list is referred to as the correct token (CT)', and Section 5.2 averages the probabilities of these CTs. Section 6 then relabels these averages as 'prediction probability' and 'accuracy', and Section 7 concludes that 'increasing infill density reduces prediction accuracy for both models'. Because CT is defined as the model's own argmax, the reported 'accuracy' is a measure of self-confidence, not correctness. No comparison is made to the actual next token of a canonical continuation or to any ExploreToM gold answer. The displayed tokens make the problem concrete: for the second-order prompt of Story 1 (Figure 1), GPT-2's top-token sequence at temperature 0.01 is 'leather, brief, case' at infill 0 but 'hotel, lobby, .' at infill 1, both with average probability 1.00; both cannot be correct, yet both are scored as fully correct by the paper's metric.
  2. [Section 6, 7] The temperature effects on 'accuracy' are also not measured. The observation that higher temperatures lower average top-token probability is a near-definitional consequence of the softmax temperature sharpening the distribution, not evidence about whether the predicted token is semantically correct. The claim in Section 6 that 'higher temperatures also led to greater response diversity ... this also increased the likelihood of incorrect or less coherent outputs' is not supported by any ground-truth scoring or human evaluation. To support such a claim, the authors would need to sample completions at each temperature and compare them against expected answers, or at least report top-1 agreement with the original story continuation.
  3. [Section 6, 7, Figures 1-2] The cross-model comparison is not meaningful as reported. The conclusion that 'Llama-2 consistently outperforms GPT-2 in accuracy' and 'outperforms GPT-2 in higher-order reasoning tasks' rests on comparing average top-token probabilities across models whose tokenizers and top-token strings differ completely. For example, in Figure 2 Llama-2's top tokens are often subword fragments such as 'le, ather, brief', while GPT-2's are full words in Figure 1; there is no common output space over which the comparison is defined. Additionally, no statistical tests, confidence intervals, or repeated trials are reported, and some conditions are skipped for GPT-2 because of its 1024-token limit, as acknowledged in Section 6. With 10 stories and unmatched conditions, the comparative accuracy claims are not established.
  4. [Section 4.2, 5] A ground-truth evaluation was available but not used. The prompts constructed in Section 5 continue actual story sentences (e.g., 'Liam moved the silver letter opener to the'), and the ExploreToM dataset used in Section 4 contains narrative facts and question answers that could serve as a reference. The paper's own research questions in Section 5.3 ask 'does the probability of correct token prediction decrease?', but no operational definition of 'correct' beyond the model's highest-probability token is ever given. This is a load-bearing gap: without an external reference standard, the paper cannot distinguish between a model becoming less confident and a model becoming less accurate.
minor comments (5)
  1. [Section 5.2] The formula 'Average Probability = P P N' is unclear; please define P and N explicitly and clarify whether the average is taken over the three predicted tokens within a single completion, over stories, or over both.
  2. [Section 5, 6.2, Appendix] The abstract and Section 5 list temperature settings as 0.01, 0.5, 1.0, and 2.0, but Section 6.2 and the appendix also report results at 1.5; this inconsistency should be reconciled.
  3. [Figures 1-45] Several figures contain tokens rendered as '#', '!', '$', '□', and '<unk>'; these special tokens and truncated subwords should be explained in the caption or in Section 5 so readers can interpret them.
  4. [Section 4] The tolerance of ±3 sentences for higher infill conditions is mentioned but not quantified in the analysis; please describe how the actual infill counts vary and whether they were controlled for in the comparison.
  5. [Section 7] The conclusion states that 'GPT-2 remains a faster alternative' but no runtime or computational efficiency measurements appear anywhere in the paper.

Circularity Check

2 steps flagged · score 7.0 of 10

Central 'accuracy' metric is self-referential: CT is defined as the model's top token, so the claimed predictions reduce to confidence.

  1. self definitional [Section 5.1 'Token Prediction Process'; Section 5.2 'Computation of Average Probabilities'; Section 7 Conclusion]
    "The token with the highest probability in the list is referred to as the correct token (CT), while the next four tokens are considered alternative predictions (1AP, 2AP, 3AP, 4AP)."

    The paper's central 'accuracy' metric is the average probability of the CT, computed as Average Probability = sum P / N, and CT is defined as the model's own argmax token. No prediction is ever compared to the ExploreToM gold answer or to a canonical continuation. Consequently, the Conclusion's claims that 'increasing infill density reduces prediction accuracy for both models' and 'Llama-2 outperforms GPT-2 in higher-order reasoning tasks' are claims about the model's self-assigned top-token confidence, not about correctness. Because the word 'correct' is defined into the measurement, 'accuracy' reduces by construction to the model's confidence.

  2. renaming known result [Section 5.3 'Analysis of Probability Distributions and Model Performance'; Section 6 'Results']
    "Lower temperatures (0.01) should result in highly confident, deterministic predictions. Higher temperatures (2.0) introduce more randomness, leading to a flatter probability distribution across multiple tokens."

    The observed 'decline in prediction probability' with temperature is the definitional behavior of temperature-scaled softmax: raising the temperature flattens the distribution, so the mass of the top token necessarily falls. The paper then relabels this mathematical fact as an empirical accuracy finding, stating that 'as the temperature increased, both GPT-2 and LLaMA-2 exhibited a decline in prediction probability, indicating reduced confidence' and later describing this as 'a greater probability of errors.' This is a property of the input sampling function presented as a measured result, not a correctness finding against any ground truth.

full rationale

The paper's headline conclusions are built on a self-referential metric. Section 5.1 defines the 'correct token' as the token with the highest model probability, Section 5.2 averages the probabilities of these top tokens, and Section 6/7 call those averages 'prediction accuracy' and 'accuracy.' Nothing in the paper compares a prediction to the ExploreToM gold answer or to the actual next token of a canonical continuation. Therefore the central claims about accuracy and about Llama-2's superiority in higher-order reasoning reduce to claims about the models' top-token confidence. The temperature finding is likewise a renamed mathematical consequence of softmax temperature scaling rather than an emergent empirical result. I did not score higher than 7 because the infill-density trend is not itself forced by the metric: the displayed Story 1 second-order rows show CT probability 1.00 across infill levels at temperature 0.01, so that specific directional claim is unsupported rather than definitionally guaranteed. I found no load-bearing self-citation, no imported uniqueness theorem, and no ansatz smuggled in via citation; the paper's circularity is entirely in the naming of its own confidence score as accuracy.

Assumptions & free parameters 1 free parameters · 5 assumptions · 0 invented entities

No model parameters are fitted in this paper. The only hand-chosen numeric condition affecting the dataset is the infill sentence tolerance. The load-bearing assumptions are the identification of the top token with correctness, the preservation of ground truth by GPT-4 infills, the representativeness of single forward passes, and the asserted link between next-token probability and Theory of Mind ability.

free parameters (1)
  • Infill sentence count tolerance = ±3 sentences
    Section 4 allows a tolerance of ±3 sentences in higher infill conditions because GPT-4 generation is non-deterministic. Actual sentence counts are not logged, so the five context levels are not exact and the dataset is not fully specified.
assumptions (5)
  • domain assumption The token with the highest probability from the model is the correct token; no external ground-truth label is needed to score accuracy.
    Section 5.1 defines CT as the top-probability token and all headline claims about prediction accuracy rely on this identification.
  • domain assumption GPT-4-generated infill sentences add contextual complexity without changing the narrative ground truth or the intended answer.
    Section 4 creates infill variants with GPT-4 but never verifies that inserted sentences preserve the story facts used by the questions.
  • domain assumption A single forward pass at each temperature is representative of model behavior, so probability averages need no variance or repeated sampling.
    Sections 5.2 and 6 report averaged probabilities without repeated sampling or error bars, although temperature greater than zero makes sampling stochastic.
  • domain assumption Next-token probability on short probe completions is a valid measure of Theory of Mind reasoning ability.
    Section 3 argues that prioritizing contextually relevant words indicates latent intent modeling; this is asserted, not established, and no comparison to ToM accuracy is made.
  • standard math Temperature scaling flattens the softmax output distribution.
    Used implicitly in Section 6 when attributing lower top-token probability at higher temperature to reduced confidence; this property follows from the softmax scaling formula.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Exploring Next Token Prediction in Theory of Mind (ToM) Tasks: Comparative Experiments with GPT-2 and LLaMA-2 AI Models." pith.science (2026). https://pith.science/paper/COZEYJL4

@misc{pith2026250415604,
  author       = {Pith},
  title        = {Pith review of: Exploring Next Token Prediction in Theory of Mind (ToM) Tasks: Comparative Experiments with GPT-2 and LLaMA-2 AI Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/COZEYJL4}},
  note         = {Machine review of arXiv:2504.15604}
}
read the original abstract

Language models have made significant progress in generating coherent text and predicting next tokens based on input prompts. This study compares the next-token prediction performance of two well-known models: OpenAI's GPT-2 and Meta's Llama-2-7b-chat-hf on Theory of Mind (ToM) tasks. To evaluate their capabilities, we built a dataset from 10 short stories sourced from the Explore ToM Dataset. We enhanced these stories by programmatically inserting additional sentences (infills) using GPT-4, creating variations that introduce different levels of contextual complexity. This setup enables analysis of how increasing context affects model performance. We tested both models under four temperature settings (0.01, 0.5, 1.0, 2.0) and evaluated their ability to predict the next token across three reasoning levels. Zero-order reasoning involves tracking the state, either current (ground truth) or past (memory). First-order reasoning concerns understanding another's mental state (e.g., "Does Anne know the apple is salted?"). Second-order reasoning adds recursion (e.g., "Does Anne think that Charles knows the apple is salted?"). Our results show that adding more infill sentences slightly reduces prediction accuracy, as added context increases complexity and ambiguity. Llama-2 consistently outperforms GPT-2 in prediction accuracy, especially at lower temperatures, demonstrating greater confidence in selecting the most probable token. As reasoning complexity rises, model responses diverge more. Notably, GPT-2 and Llama-2 display greater variability in predictions during first- and second-order reasoning tasks. These findings illustrate how model architecture, temperature, and contextual complexity influence next-token prediction, contributing to a better understanding of the strengths and limitations of current language models.

Figures

Figures reproduced from arXiv: 2504.15604 by the authors.

Figure 1
Figure 1. Prediction results of the GPT-2 model for different temperature settings and [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗
Figure 2
Figure 2. Prediction results of the Llama-2 model for different temperature settings and [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 9 canonical work pages

  1. [1]

    Sclar, M., Yu, J., Fazel-Zarandi, M., Tsvetkov, Y., Bisk, Y., Choi, Y., & Celikyilmaz, A. (2024). Explore Theory-of-Mind: Program-Guided Adversarial Data 13 Generation for Theory of Mind Reasoning . arXiv preprint. https://doi. org/10.48550/arXiv.2412.12175

    arXiv doi:10.48550/arxiv.2412.12175 2024

  2. [2]

    Strachan, M., et al. (2024). Testing the Theory of Mind Capabilities of Large Language Models . Nature, 627, 2024. https://www.nature.com/articles/ s41562-024-01882-z

    work page 2024

  3. [3]

    Emu3 Team. (2024). Emu3: Next-Token Prediction is All You Need . arXiv preprint. https://arxiv.org/pdf/2409.18869

    arXiv 2024

  4. [5]

    Meta’s Llama-2 Model: meta-llama/Llama-2-7b-chat-hf

    Meta. Meta’s Llama-2 Model: meta-llama/Llama-2-7b-chat-hf . Hugging Face. https://huggingface.co/meta-llama/Llama-2-7b-chat-hf

    work page

  5. [6]

    OpenAI’s GPT-2 Model: GPT-2 on Hugging Face

    OpenAI. OpenAI’s GPT-2 Model: GPT-2 on Hugging Face . Hugging Face. https://huggingface.co/gpt-2

    work page

  6. [7]

    Saved Plots and Supplementary Material , https://github.com/ Enkefalos-Technologies/next-token-prediction

    work page

  7. [8]

    E., & Das, R

    Das, R. E., & Das, R. (2024). Iterative Theory of Mind Assay of Multimodal AI Models. In ICML 2024 Workshop on LLMs and Cognition . https://openreview. net/forum?id=PsGVVQJZGk

    work page 2024

  8. [9]

    Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2024). Training Large Language Models to Reason in a Continuous Latent Space. arXiv preprint. https://doi.org/10.48550/arXiv.2412.06769

    arXiv doi:10.48550/arxiv.2412.06769 2024

Show all 15 references
  1. [10]

    Mitchell, M., & Krakauer, D. C. (2023). The debate over understanding in AI’s large language models . Proceedings of the National Academy of Sciences, 120(13). https://doi.org/10.1073/pnas.2215907120

    2023 doi

  2. [11]

    Wang, Q., Walsh, S., Si, M., Kephart, J., Weisz, J., & Noel, A. (2024). Theory of mind in human–AI interaction . In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613905. 3636308

    2024 doi

  3. [12]

    Melanie Mitchell, AI Guide. (2024). LLMs and World Models - Part 1. Substack. https://aiguide.substack.com/p/llms-and-world-models-part-1

    2024

  4. [13]

    Melanie Mitchell, AI Guide. (2024). LLMs and World Models - Part 2. Substack. https://aiguide.substack.com/p/llms-and-world-models-part-2

    2024

  5. [14]

    Nanda, N., et al. (2024). TransformerLens: Interpretability for Transformer Models. GitHub. https://transformerlensorg.github.io/TransformerLens/

    2024

  6. [15]

    Yaniv Nikankin. (2024). Arithmetic Without Algorithms: Language Models Solve Math with a Bag of Heuristics . arXiv preprint. https://arxiv.org/ pdf/2410.21272 14

    2024 arXiv

  7. [16]

    , $, $ 4AP: !,

    Emmanuel A., et al. (2025). Circuit Tracing: Revealing Computational Graphs in Language Models . https://transformer-circuits.pub/2025/ attribution-graphs/methods.html A Appendix In this appendix, we present a series of supplementary materials related to the experi- ments and ...

    2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.