Pith. sign in

REVIEW 2 major objections 5 minor 24 references

Supervised fine-tuning of LLMs pushes action diversity below the minimum that accuracy requires, and training on every optimal action per state preserves that diversity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:30 UTC pith:XXUUTPXD

load-bearing objection Solid empirical study of SFT-induced diversity collapse; the qualitative findings hold, but the headline quantitative claim is undercut by 8-sample entropy estimates. the 2 major comments →

arxiv 2607.19523 v1 pith:XXUUTPXD submitted 2026-07-21 cs.CL

When Reasoning Narrows the Move: Diversity Collapse in LLM Game Play

Junyi Sha , Renfei Tan , David Simchi-Levi This is my paper
classification cs.CL
keywords diversity collapsesupervised fine-tuningaction entropyiso-accuracy entropy ceilingLLM decision-makingaction augmentationreasoning modetic-tac-toe variants
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that standard supervised fine-tuning (SFT) improves LLM accuracy in decision-making tasks but at the cost of a measurable, avoidable collapse in action diversity — a collapse that goes beyond what accuracy gains require. Using four deterministic tic-tac-toe-like games where optimal moves are exactly known, the authors show that reasoning-mode generation also suppresses diversity without consistent accuracy improvements. They introduce an 'iso-accuracy entropy ceiling' to separate necessary concentration from extra collapse and find that plain SFT pushes policies below that ceiling. A data-level fix — action augmentation, training on all optimal actions per state — partially restores diversity while keeping accuracy competitive. If right, this identifies narrow-support imitation as a real failure mode in LLM decision-making and points to concrete data-design choices to fix it.

Core claim

The paper's central claim is that SFT and reasoning-mode generation cause LLM policies to concentrate on single moves beyond what optimal play demands. The authors measure action entropy per state in four solved board games and compare model trajectories against an analytic upper bound — the iso-accuracy entropy ceiling of Eq. 2 — which is the maximum entropy any policy of a given accuracy can achieve. They find that plain SFT often loses more entropy than needed to reach its accuracy, a condition they call premature diversity collapse; reasoning mode amplifies this without reliably improving accuracy; and training on all optimal actions per state (action augmentation) keeps the policy much

What carries the argument

The load-bearing object is the iso-accuracy entropy ceiling — the per-state maximum entropy formula H^max_s(p_s) = h_2(p_s) + p_s log2 k_s + (1-p_s) log2(n_s - k_s), where p_s is the probability mass on optimal actions, k_s the number of optimal actions, n_s legal actions, and h_2 the binary entropy. It gives the highest mean entropy attainable at any mean accuracy on a state pool, so the gap between a model's observed trajectory and this curve isolates diversity loss beyond what accuracy mechanically requires. The second mechanism is action augmentation: including every optimal action per state in SFT data, rather than a single demonstrated action, which directly widens the support the mode

Load-bearing premise

The claim that diversity loss 'exceeds what is minimally required' depends on the measured per-state action entropy from only 8 samples per state faithfully reflecting the model's true entropy — with 8 samples the estimate is biased downward and capped at 3 bits, so some of the measured collapse could be an artifact of the sampling budget.

What would settle it

Recompute the empirical action entropy with a much larger sample budget per state (say 64 or 256) or with a bias-corrected estimator (e.g., Chao-Shen or Miller-Madow) on the same evaluation pools, and compare the resulting trajectories against the iso-accuracy ceiling. If the gap between plain SFT and the ceiling largely disappears, or the early-epoch collapse no longer appears, the 'premature collapse' result fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Standard SFT can quietly replace exploratory, multi-move behavior with near-deterministic play even on simple solved games.
  • Reasoning traces, when trained as single demonstrations, concentrate the policy further; reasoning alone is not a substitute for diverse training data.
  • Training on all optimal actions per state preserves substantially more action entropy at comparable accuracy, offering a concrete data-level intervention.
  • Evaluations of LLM decision-making should measure action diversity alongside accuracy, since the two can decouple.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The entropy measurement at N=8 samples per state caps estimated diversity at log2(8)=3 bits; bias-corrected or larger-N estimates could shrink the reported gap to the iso-accuracy ceiling, so the magnitude (not necessarily the ordering) of the collapse may be overstated.
  • The iso-accuracy ceiling presupposes that accuracy is the only driver of optimal concentration; in games with strategic value beyond win/loss (e.g., depth-to-mate), the minimum entropy could be even lower, which would make the ceiling conservative — a direction the authors did not explore.
  • The action-augmentation intervention could be tested as a general recipe for other sequential decision tasks (e.g., coding or tool use) where multiple correct actions exist per state; if it transfers, it offers a simple alternative to loss-function modifications for preserving diversity.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. This paper studies how supervised fine-tuning (SFT) affects action diversity in LLM decision-making, using four deterministic tic-tac-toe variants with exactly computable optimal actions. The authors evaluate Qwen3-8B and LoRA fine-tuned variants in direct and reasoning modes, with either single-action ('plain') or multi-action ('augmented') supervision. Across state-level accuracy/entropy evaluations, arena Elo/trajectory-entropy measurements, and checkpoint trajectories, they report that reasoning compresses diversity without consistent accuracy gains, that plain SFT induces an early diversity collapse that they claim exceeds the accuracy–diversity tradeoff (by comparison to an analytic 'iso-accuracy entropy ceiling', Eq. 2), and that action-augmented SFT partially mitigates the collapse. The paper interprets these results as evidence of narrow-support imitation as a failure mode.

Significance. If the central quantitative claim is established, the paper would provide a clean, controlled demonstration that standard SFT can push LLM policies below the accuracy–diversity frontier, with a simple data-level intervention (multi-action augmentation) that partially restores diversity. The study's strengths are its exact game solvers, three independent measurement modes (state-level, arena, checkpoints), bootstrap uncertainty quantification, and an analytic ceiling derived from first principles rather than fitted to data. These features make the qualitative ordering of conditions (plain < augmented; reasoning < direct) credible. The quantitative 'exceeds the tradeoff' claim, however, is load-bearing and currently depends on measurement and ceiling-construction details that need to be addressed before the headline conclusion can be accepted.

major comments (2)
  1. [§4.1, §5, Figure 4] The central claim that plain SFT 'exceeds what is minimally required' is read off the gap between measured checkpoint entropies and the Eq. (2) ceiling. All state-level entropies are plug-in estimates from N=8 action samples per state. This estimator is downward-biased (Miller–Madow bias is roughly (m−1)/(2N ln2), i.e., several tenths of a bit for policies with entropy around 1–2 bits) and hard-capped at log2 8 = 3 bits. Figure 4 shows the base model already at the cap, and mid-training checkpoints are in the bias-prone region. The magnitude and, in borderline cases, the existence of the 'premature collapse' gap are therefore not established at the reported precision. Please re-estimate entropies with larger N or a bias-corrected estimator and report confidence intervals on the gap. The qualitative ordering is likely robust.
  2. [§5, Eq. (2)] The statement 'No policy with the same mean accuracy can achieve higher mean entropy on that state pool' is not justified by the derivation as written. H_max_s(p_s) is a state-dependent concave function of the per-state accuracy p_s. If the dashed curve is obtained by averaging H_max_s(p) at a common p, it is not a valid upper envelope for policies that are allowed to allocate their correct actions unevenly across states: with state-dependent (n_s,k_s), a policy with mean accuracy p̄ can achieve strictly higher average entropy than E_s[H_max_s(p̄)] by allocating its correct decisions unevenly across states. The comparison in Figure 4 can therefore overstate how far SFT goes below the frontier. Either define the ceiling as the solution of the constrained optimization over per-state accuracies, or explicitly restrict the comparison to policies with the same per-state accuracy profile, and
minor comments (5)
  1. [Abstract] Typo: 'would partially mitigates this effect' → 'would partially mitigate this effect.'
  2. [§3] 'P fis the transition kernel' appears to be a typo for 'P is the transition kernel.'
  3. [§4.2] The Elo expected-reward formula introduces τ without defining it; state the scaling constant used.
  4. [Figures 2 and 3] Legend labels ('Base SFT-Plain SFT-Aug Direct Reasoning') are visually crowded and make it hard to associate arrows with conditions; use separate subpanels or clearer glyphs.
  5. [§5] The description of checkpoint selection says 'three intermediate checkpoints' but lists {1,6,12,18}; clarifying which are intermediate vs final would help reproducibility.

Circularity Check

0 steps flagged

No significant circularity: the headline diversity-collapse claim is benchmarked against an externally derived analytic ceiling, not against a fitted curve or self-citation chain.

full rationale

The paper's central derivation chain is empirical comparison against an externally derived analytic ceiling. Equation (2) defines the iso-accuracy entropy ceiling from per-state numbers of optimal actions k_s, legal actions n_s, and the policy's accuracy p_s; it is a mathematical upper envelope, not a quantity fitted to the measured entropy data. Accuracy and diversity are measured independently: accuracy is the frequency of solver-verified optimal actions, and diversity is the plug-in action entropy from N=8 samples per state. The claim that plain SFT 'exceeds what is minimally required' is the gap between these measured trajectories and the ceiling, which is a benchmark rather than a fitted prediction. No parameter is fitted to the data and then reported as a prediction. There are no load-bearing self-citations by the authors; the cited game environments, baselines, and methods are external. The N=8 plug-in entropy estimator may be biased and capped at log2(8)=3 bits, and the ceiling construction could be debated on concavity or state-level aggregation grounds, but these are measurement/correctness concerns, not circularity: the comparison is not true by construction, as shown by the aug-reason condition retaining higher diversity at comparable accuracy. The stated limitation to small deterministic games and a single model family is a scope limitation, not a circular step. Therefore the analysis is self-contained with respect to circularity.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The central analysis is a controlled measurement rather than a derivation, so the ledger consists mostly of domain assumptions and measurement choices, not fitted constants. No parameter is fitted to data to produce the headline results: the iso-accuracy ceiling (Eq. 2) is derived, Elo and entropy are measurements, and the SFT interventions change training data rather than tune free parameters. Two hand-chosen quantities do load on the central claim: N=8 samples per state (3-bit entropy cap, downward bias) and the depth-4 cutoff of the sTTT solver (approximate 'optimal' labels for the largest game). The augmentation result rests on the assumption that training over all optimal actions is learnable and that the measured diversity is not merely an artifact of the label distribution — partly by construction, as the paper's framing implies. No invented entities are introduced.

free parameters (2)
  • Evaluation samples per state N = 8
    Hand-chosen in §4.1; caps measurable entropy at log2(8)=3 bits and biases plug-in entropy downward, directly loading on the quantitative 'premature collapse' claim.
  • sTTT solver search depth cutoff = d=4
    For Squares Tic-Tac-Toe (~1e10 states), 'optimal' labels and the minimax baseline come from depth-limited negamax with a positional heuristic (Appendix A.1), not exact search; the abstract's 'exactly computable' claim is thereby overstated for this game.
axioms (5)
  • domain assumption Solver-computed game-theoretic values for oTTT/cTTT/dTTT define the true 'optimal action sets' used as SFT labels and accuracy ground truth.
    Invoked in §3 and §A.3 for all labels; relies on the exhaustive-search implementation being correct and value-optimality being the right criterion.
  • domain assumption The empirical entropy of 8 decoded samples approximates the policy's true action entropy per state.
    §4.1: N=8 independent action predictions per state; violated at high entropy (3-bit cap, downward bias), which is the weakest structural premise of the quantitative claim.
  • standard math For fixed per-state accuracy p_s, maximum entropy is attained by uniform mass over optimal actions and uniform mass over suboptimal actions (Eq. 2).
    §5, the iso-accuracy entropy ceiling; follows from a standard convexity/majorization argument but is asserted without proof in the main text.
  • standard math Trajectory entropy decomposes as the sum of per-step action entropies under the induced state distribution (Eq. 1).
    §4.2; requires Markovian policies and deterministic transitions, which hold for these games.
  • ad hoc to paper The curated 200-state evaluation pools (tactical-category balanced, excluding states where every action is optimal) are representative of realistic play.
    §4.1; excluding fully-symmetric forced-draw states and balancing categories could amplify measured accuracy/diversity differences relative to uncurated play.

pith-pipeline@v1.3.0-alltime-deepseek · 14176 in / 20549 out tokens · 181996 ms · 2026-08-01T12:30:01.560566+00:00 · methodology

0 comments
read the original abstract

Supervised fine-tuning (SFT) is widely used to adapt large language models to downstream tasks, but its effect on behavioral diversity in sequential decision-making remains under-explored. We study this question in a controlled suite of deterministic board games based on tic-tac-toe variants, where optimal actions are exactly computable and diversity can be measured directly. Across state-level evaluation, arena gameplay, and training trajectories, we find that reasoning-mode generation frequently suppresses action diversity without uniformly improving action accuracy. Furthermore, standard SFT improves accuracy but often induces premature diversity collapse, which exceeds what is minimally required by the accuracy-diversity tradeoff. We then show that action augmentation, which trains on all optimal actions per state rather than a single demonstrated action, would partially mitigates this effect. Our results identify narrow-support imitation as a source of policy collapse in LLM decision-making and suggest that preserving action support during SFT is important for maintaining exploratory behavior.

Figures

Figures reproduced from arXiv: 2607.19523 by David Simchi-Levi, Junyi Sha, Renfei Tan.

Figure 1
Figure 1. Figure 1: Overview of game environments. et al., 2024]. A loss of diversity may therefore limit performance and hinder subsequent post-training methods such as reinforcement learning [Kirk et al., 2023, Murthy et al., 2025]. Understanding how SFT impacts action diversity is therefore crucial for developing more effective training pipelines and improving LLM’s decision-making abilities. Games offer a clean and contro… view at source ↗
Figure 2
Figure 2. Figure 2: State-level accuracy-diversity profiles per game on [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Arena-level Elo-diversity profiles per game. For each game, we visualize the joint tradeoff [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Per-game accuracy-diversity trajectories across SFT epochs on [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Elo rating versus trajectory entropy HMM(τ ) for three instruction-tuned LLMs (Qwen3-8B, Gemma-3-12B, Llama-3.1-8B, all at temperature 0.6) compared against deterministic baselines across four game variants. B.2 Temperature Study A natural first response to trajectory collapse is to raise the temperature of the model at inference time: higher temperature increases sampling randomness and, in principle, sho… view at source ↗
Figure 6
Figure 6. Figure 6: Elo versus Diversity plot for base and plain_sft Qwen3-8B model [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Elo rating versus diversity for Qwen3-8B before and after GEPA prompt optimization. GEPA (Genetic-Pareto) is a reflective prompt optimizer introduced by Agrawal et al. [2025] that learns high-level behavioral rules from natural language feedback rather than from scalar reward gradients. Given an AI system and a task metric, GEPA samples rollouts, reflects on them in natural language to diagnose failures an… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

24 extracted references · 17 linked inside Pith

  1. [1]

    Gepa: Reflective prompt evolution can outperform reinforcement learning.arXiv preprint arXiv:2507.19457,

    Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, et al. Gepa: Reflective prompt evolution can outperform reinforcement learning.arXiv preprint arXiv:2507.19457,

    Pith/arXiv arXiv

  2. [6]

    D4rl: Datasets for deep data-driven reinforcement learning.arXiv preprint arXiv:2004.07219,

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning.arXiv preprint arXiv:2004.07219,

    Pith/arXiv arXiv 2004

  3. [7]

    Marble: A hard benchmark for multimodal spatial reasoning and planning.arXiv preprint arXiv:2506.22992,

    Yulun Jiang, Yekun Chai, Maria Brbi´c, and Michael Moor. Marble: A hard benchmark for multimodal spatial reasoning and planning.arXiv preprint arXiv:2506.22992,

    Pith/arXiv arXiv

  4. [8]

    Understanding the effects of rlhf on llm generalisation and diversity.arXiv preprint arXiv:2310.06452,

    Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on llm generalisation and diversity.arXiv preprint arXiv:2310.06452,

    Pith/arXiv arXiv

  5. [10]

    Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,

    Pith/arXiv arXiv 2005

  6. [13]

    One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity

    Sonia Krishna Murthy, Tomer Ullman, and Jennifer Hu. One fish, two fish, but not the whole sea: Alignment reduces language models’ conceptual diversity. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11241–11258,

    2025

  7. [15]

    Do llm agents have regret? a case study in online learning and games.arXiv preprint arXiv:2403.16843,

    Chanwoo Park, Xiangyu Liu, Asuman Ozdaglar, and Kaiqing Zhang. Do llm agents have regret? a case study in online learning and games.arXiv preprint arXiv:2403.16843,

    arXiv

  8. [16]

    Offline learning of controllable diverse behaviors.arXiv preprint arXiv:2504.18160,

    Mathieu Petitbois, Rémy Portelas, Sylvain Lamprier, and Ludovic Denoyer. Offline learning of controllable diverse behaviors.arXiv preprint arXiv:2504.18160,

    Pith/arXiv arXiv

  9. [17]

    Evaluating large language models with grid-based game competitions: An extensible LLM benchmark and leaderboard.arXiv preprint arXiv:2407.07796,

    Oguzhan Topsakal, Colby Jacob Edell, and Jackson Bailey Harper. Evaluating large language models with grid-based game competitions: An extensible LLM benchmark and leaderboard.arXiv preprint arXiv:2407.07796,

    Pith/arXiv arXiv

  10. [18]

    Chessqa: Evaluating large language models for chess understanding.arXiv preprint arXiv:2510.23948,

    Qianfeng Wen, Zhenwei Tang, and Ashton Anderson. Chessqa: Evaluating large language models for chess understanding.arXiv preprint arXiv:2510.23948,

    arXiv

  11. [19]

    Agentgym-rl: Training llm agents for long-horizon decision making through multi-turn reinforcement learning.arXiv preprint arXiv:2509.08755,

    Zhiheng Xi, Jixuan Huang, Chenyang Liao, Baodai Huang, Honglin Guo, Jiaqi Liu, Rui Zheng, Junjie Ye, Jiazheng Zhang, Wenxiang Chen, et al. Agentgym-rl: Training llm agents for long-horizon decision making through multi-turn reinforcement learning.arXiv preprint arXiv:2509.08755,

    Pith/arXiv arXiv

  12. [20]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388,

    Pith/arXiv arXiv

  13. [21]

    How to leverage diverse demonstrations in offline imitation learning.arXiv preprint arXiv:2405.17476,

    Sheng Yue, Jiani Liu, Xingyuan Hua, Ju Ren, Sen Lin, Junshan Zhang, and Yaoxue Zhang. How to leverage diverse demonstrations in offline imitation learning.arXiv preprint arXiv:2405.17476,

    Pith/arXiv arXiv

  14. [22]

    The price of format: Diversity collapse in llms.arXiv preprint arXiv:2505.18949,

    Longfei Yun, Chenyang An, Zilong Wang, Letian Peng, and Jingbo Shang. The price of format: Diversity collapse in llms.arXiv preprint arXiv:2505.18949,

    Pith/arXiv arXiv

  15. [23]

    Good sft optimizes for sft, better sft prepares for reinforcement learning.arXiv preprint arXiv:2602.01058,

    Dylan Zhang, Yufeng Xu, Haojin Wang, Qingzhi Chen, and Hao Peng. Good sft optimizes for sft, better sft prepares for reinforcement learning.arXiv preprint arXiv:2602.01058,

    Pith/arXiv arXiv

  16. [24]

    move": <action_label>}</action> The JSON key must be

    11 Appendix A Implementation Details A.1 Arena Players Table 1 summarizes the players introduced in our arena: four deterministic policies and three open- source language models. We select the deterministic policies to provide interpretable anchors that span both skill and behavioral diversity, yielding broad coverage of the Elo-versus-diversity 2D space....

    2024

  17. [1978]

    Rvs: What is essential for offline rl via supervised learning?arXiv preprint arXiv:2112.10751,

    Scott Emmons, Benjamin Eysenbach, Ilya Kostrikov, and Sergey Levine. Rvs: What is essential for offline rl via supervised learning?arXiv preprint arXiv:2112.10751,

    Pith/arXiv arXiv

  18. [2020]

    Preserv- ing diversity in supervised fine-tuning of large language models.arXiv preprint arXiv:2408.16673,

    Ziniu Li, Congliang Chen, Tian Xu, Zeyu Qin, Jiancong Xiao, Zhi-Quan Luo, and Ruoyu Sun. Preserv- ing diversity in supervised fine-tuning of large language models.arXiv preprint arXiv:2408.16673,

    Pith/arXiv arXiv

  19. [2021]

    Sed-sft: Selectively encouraging diversity in supervised fine-tuning.arXiv preprint arXiv:2602.07464,

    Yijie Chen, Yijin Liu, and Fandong Meng. Sed-sft: Selectively encouraging diversity in supervised fine-tuning.arXiv preprint arXiv:2602.07464,

    arXiv

  20. [2022]

    Attributing mode collapse in the fine-tuning of large language models

    Laura O’Mahony, Leo Grinsztajn, Hailey Schoelkopf, and Stella Biderman. Attributing mode collapse in the fine-tuning of large language models. InICLR 2024 Workshop on Mathematical and Empirical Understanding of Foundation Models, volume 2, page 2,

    2024

  21. [2023]

    Llm chess: Benchmarking reasoning and instruction- following in llms through chess.arXiv preprint arXiv:2512.01992,

    Sai Kolasani, Maxim Saplin, Nicholas Crispino, Kyle Montgomery, Jared Quincy Davis, Matei Zaharia, Chi Wang, and Chenguang Wang. Llm chess: Benchmarking reasoning and instruction- following in llms through chess.arXiv preprint arXiv:2512.01992,

    arXiv

  22. [2024]

    Ttt-bench: A benchmark for evaluating reasoning ability with simple and novel tic-tac-toe-style games.arXiv preprint arXiv:2506.10209,

    Prakamya Mishra, Jiang Liu, Jialian Wu, Xiaodong Yu, Zicheng Liu, and Emad Barsoum. Ttt-bench: A benchmark for evaluating reasoning ability with simple and novel tic-tac-toe-style games.arXiv preprint arXiv:2506.10209,

    Pith/arXiv arXiv

  23. [2025]

    Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862,

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback.arXiv preprint arXiv:2204.05862,

    Pith/arXiv arXiv

  24. [2026]

    Game reasoning arena: A framework and benchmark for assessing reasoning capabilities of large language models via game play.arXiv preprint arXiv:2508.03368,

    Lucia Cipolina-Kun, Marianna Nezhurina, and Jenia Jitsev. Game reasoning arena: A framework and benchmark for assessing reasoning capabilities of large language models via game play.arXiv preprint arXiv:2508.03368,

    Pith/arXiv arXiv