Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy

Alexander Duffy; Baptiste Alloui-Cros; Elizabeth Karpinski; Ishana Shastri; Matthew Lyle Olson; Samuel J Paech; Tyler Marques

REVIEW 4 major objections 4 minor 1 cited by

Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Any local LLM can now play full-press Diplomacy without fine-tuning

desk verdict A useful local-evaluation harness for Diplomacy, genuinely new as an artifact, but the 'any LLM' claim rides on a tuning loop that the visible text does not yet control for. read the letter →

arxiv 2508.07485 v1 pith:KWXIMKAM submitted 2025-08-10 cs.AI cs.CLcs.CYcs.LG

Alexander Duffy , Samuel J Paech , Ishana Shastri , Elizabeth Karpinski , Baptiste Alloui-Cros , Tyler Marques , Matthew Lyle Olson This is my paper

classification cs.AIcs.CLcs.CYcs.LG

keywords DiplomacyLLMevaluationfull-pressstrategicreasoningnegotiationmulti-agenttextrepresentationcriticalstateanalysis

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that with the right textual representation of the game state, any out-of-the-box large language model can play a full match of Diplomacy's negotiation-heavy 'full-press' variant, without additional training. The authors built an evaluation harness and used data-driven iteration to compress the board, orders, and messages into a format that even a 24B model can follow reliably. If true, this turns Diplomacy into a cheap, accessible testbed for strategic reasoning, negotiation, and deception in LLMs, lowering the barrier that previously required frontier models or fine-tuning.

What carries the argument

The central mechanism is the optimized textual game-state representation: a structured, text-only encoding of the board, unit positions, orders, and message history that is iteratively refined to maximize legal play and match completion for untuned models. This representation carries the argument, since it converts a visually dense, multi-channel game state into a format that an ordinary LLM can parse and act on without specialized training or vision.

What would settle it

Run the harness on a held-out 24B model that was not seen during representation tuning, such as a newly released open-weight model, and measure whether it completes matches with a legal-order rate comparable to the paper's reported models; a large drop in completion or legality would indicate overfitting to the tuning set.

Watch

Extended reading notes

Core claim

The paper's central claim is that full-press Diplomacy, previously accessible only to frontier LLMs or fine-tuned models, can be played by any sufficiently capable local LLM when the game state is presented in an optimized text-only form. The authors develop a harness that converts the board and all relevant information into a compact textual representation, tuned through data-driven iteration until a 24B model can complete matches legally and competitively. They report experiments across popular LLMs showing larger models perform best but smaller models still play adequately, and introduce Critical State Analysis, a protocol for probing key decision points in depth. The contribution is an e

Load-bearing premise

The data-driven iteration that tuned the text representation was not overfit to the specific models and matches it was tested on; if the representation generalizes to unseen models and games, the 'any local LLM' claim holds, but if it is tuned to the tested models' quirks, the claim collapses.

Editorial extensions

If this is right

Diplomacy becomes a practical low-cost benchmark for evaluating strategic reasoning in local LLMs, since no fine-tuning or frontier API is required.
A 24B model can complete full-press matches, implying strategic negotiation capability is already present in mid-size open-weight models rather than exclusive to frontier systems.
Researchers can run controlled multi-agent experiments on persuasion, deception, and playstyle using the harness's tooling for hypothesis testing and statistical analysis.
Critical State Analysis offers a fast way to probe how models behave at pivotal game moments, potentially generalizing to other long-horizon tasks.
The performance gradation across model sizes suggests a smooth emergence of strategic capability, making Diplomacy a useful scaling probe.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The representation optimization may implicitly encode prior knowledge about which information is salient, meaning the harness could overfit to particular model families; a held-out model test would separate representation quality from model capability.
The same text-representation approach could transfer to other negotiation-heavy games or multi-agent settings, turning any information-dense board game into an LLM evaluation task.
Because the harness works without fine-tuning, it could serve as a standardized probe for cross-model comparisons of social reasoning, complementing static benchmarks with dynamic adversarial interaction.
The finding that smaller models still 'play adequately' suggests a floor of strategic competence in pretrained models, which might be exploited in game-playing agents for low-resource deployment.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The useful thing here is the harness: a text-only game-state representation plus the surrounding tooling that gets a 24B parameter model through full-press Diplomacy games without fine-tuning. If that works outside the authors' own setup, it turns an expensive and high-variance benchmark into something you can run locally on mid-size models, and the Critical State Analysis protocol is a reasonable idea for digging into decisive moments. The code is promised open source, which is the right way to ship this kind of claim.

What I want to check before believing the headline: the phrase "data-driven iteration" is doing a lot of work. They optimized the representation until a 24B model could reliably complete matches, and then they interpret the resulting play as "natural" capability. That is a tuning loop, and it can inflate apparent competence. The abstract does not describe any holdout protocol—no models or game sets set aside during the iteration, no sensitivity analysis. The "any LLM" claim is therefore stronger than what is shown so far. The "smaller models still play adequately" line is ambiguous about whether those smaller models were inside the loop too.

Also, the "first" claim is not yet checked against prior prompt-only attempts. There are other efforts at getting LLMs to play Diplomacy without fine-tuning, and I'd want the related work section to engage with them explicitly rather than leaving "first" as an assertion.

The paper may well be solid. The authors are upfront about high match variance, and they say they built statistical tooling. But none of the actual numbers, error bars, or legal-order completion rates are in the visible text—just the claim. I'm not calling that a flaw so much as an absence; the preprint is incomplete in that respect.

My take: this is a credible contribution to the multi-agent LLM evaluation subfield, and it deserves a serious referee. I'd send it to review, but with explicit instructions to look at the released code, verify the completion-rate numbers, and ask whether the state-representation optimization generalizes to models outside the tuning set. The central claim is plausible but not yet demonstrated.

Recommendation: engage with it, but treat the headline claims as hypotheses until the code and the holdout analysis are available.

Referee Report

4 major / 4 minor

Summary. The paper proposes an evaluation harness that allows out-of-the-box, local LLMs to play full-press Diplomacy without fine-tuning. The core technical contribution is a text-based game-state representation that was optimized through data-driven iteration so that a 24B model can complete matches reliably. The paper also describes statistical tooling, case studies on persuasion and aggressive playstyles, and a new Critical State Analysis protocol. Experiments across several popular LLMs are claimed, with larger models performing best and smaller models still playing adequately. The version of the manuscript provided to me contains only the abstract and part of the introduction; no quantitative results, experimental setup, or detailed protocol are visible, which severely limits verification of the central claims.

Significance. If the harness works as claimed, it would substantially lower the cost of studying strategic reasoning, negotiation, and multi-agent interaction in LLMs, enabling broad model comparisons without fine-tuning. The commitment to open-source code and the emphasis on reproducible statistical tooling are commendable. However, the scientific significance rests on whether the optimized representation generalizes across model families and sizes rather than being tuned to the specific models used in the iteration loop. The current manuscript provides no evidence on this point, so the significance is potential rather than established.

major comments (4)

[Abstract and Introduction] The central claim that 'any out-of-the-box, local LLM' can play full-press Diplomacy is not supported by any quantitative results in the visible text. There are no legal-move completion rates, match completion rates, error bars, numbers of games, or statistical tests. The abstract states that experiments were run across many LLMs, but the body as provided stops after the introduction. Please include the missing experimental sections, with precise metrics and confidence intervals, before the claims can be assessed.
[Abstract ('data-driven iteration')] The abstract says the text representation was optimized so that a 24B model can reliably complete matches. This is a tuning process, and without a description of the protocol it is impossible to rule out overfitting to the particular model(s) used during iteration. The later statement that capabilities 'emerge naturally' is therefore potentially circular: the representation itself was shaped by the target model's strengths. Please report how many models/iterations were used, at what point evaluation models were frozen, and whether any models or model sizes were held out from the optimization loop.
[Generalization to 'any LLM'] The load-bearing claim is that the optimized representation transfers to unseen models. No held-out model validation is described. A concrete test would be to finalize the representation using only a subset of model families/sizes and then evaluate on a disjoint set of models, reporting performance per model. Without this, the 'any local LLM' claim is unsupported. The use of multiple models in the final evaluation mitigates, but does not eliminate, the concern that the design choices were correlated with models seen during iteration.
[Statistical tooling and variance] The introduction says 'high variance of matches' is a known difficulty, and the paper promises tooling 'to facilitate hypothesis testing and statistical analysis.' The visible text gives no example of such analysis: no effect sizes, no variance decomposition, no power analysis. If the full paper contains these, please point to them explicitly; otherwise, the statistical tooling appears to be only a promise.

minor comments (4)

[Figure 1] The caption states 'The visual representation of the board and how it gets converted into a text-only representation for the LLMs.' The figure itself is not visible in the provided manuscript; please ensure the conversion is legible and that the text representation is clearly explained in the caption or body.
[Terminology] The phrase 'out-of-the-box, local' could be read as implying no prompt engineering at all, yet the harness necessarily uses a structured text representation and likely a prompt template. Consider wording such as 'without fine-tuning' to avoid overpromising.
[Related Work] The introduction references prior work on LLM diplomacy but the visible text lacks a dedicated related-work section. In particular, comparisons to Cicero and other specialized systems should be summarized, with a clear statement of what the harness adds beyond existing approaches.
[Consistency] The abstract uses both 'fine tuning' and 'fine-tuning.' Please standardize the spelling.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the representation optimization is transparent engineering, and cross-model evaluation supports the generalization claim.

full rationale

The paper's derivation chain is: (1) design a text representation of the Diplomacy game state, (2) optimize that representation through data-driven iteration so a 24B model can complete matches, (3) evaluate the fixed representation across many popular LLMs, and (4) draw conclusions about emergent strategic capabilities. The optimization step is explicitly disclosed as a design goal in the abstract: 'we used data-driven iteration to optimize a textual game state representation such that a 24B model can reliably complete matches.' This is not circular because the reported '24B model can reliably complete matches' is presented as the outcome of that optimization, not as a prediction derived independently of it. The broader 'any out-of-the-box, local LLM' claim rests on the subsequent multi-model evaluation, which is an inductive generalization rather than a tautology. The only self-citation with author overlap is 'Payne and Alloui-Cros 2025,' used to motivate that off-the-shelf LLMs possess strategic capabilities; this is not load-bearing because the paper provides its own experiments as evidence. The skeptical concern about possible overfitting of the representation to the models used during iteration is a legitimate external-validity limitation, but it is not a circularity: no equation or conclusion reduces by construction to its own inputs, and no fitted parameter is renamed as a prediction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on a tuned state representation, the assumption that a text-only board state is sufficient for strategic play, and the assumption that LLM-vs-LLM self-play in this harness is a valid measure of Diplomacy skill. The representation tuning is the most important free parameter: it was fit by data-driven iteration, so it is not an external benchmark.

free parameters (2)

Textual game state representation design = optimized via data-driven iteration; exact configuration not shown in abstract
The harness's central contribution is this representation; it was tuned so a 24B model could reliably complete matches, so it is a fitted design choice rather than a fixed, theory-derived input.
Prompt and parsing hyperparameters = not specified
Prompt formatting, state ordering, and order-parsing rules likely affect completion rates; no values or templates appear in the abstract.

assumptions (3)

domain assumption A text-only serialization of the board retains enough information for competent full-press Diplomacy play.
The whole harness is built on converting the visual board to text (Figure 1); if information is lost or formatting makes it unusable, the central claim fails. Invoked in the abstract's description of the textual representation.
domain assumption Off-the-shelf LLM outputs can be parsed into legal Diplomacy orders without fine-tuning.
'Reliably complete matches' requires reliable parsing of free-form press and orders; no parsing error rates are reported in the abstract.
domain assumption Performance against other harness-driven LLMs is a valid measure of strategic ability.
The experiments compare models against each other within the harness; this measures relative performance in this pool, not absolute Diplomacy strength against human or fine-tuned opponents.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy." pith.science (2026). https://pith.science/paper/KWXIMKAM

@misc{pith2026250807485,
  author       = {Pith},
  title        = {Pith review of: Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KWXIMKAM}},
  note         = {Machine review of arXiv:2508.07485}
}

read the original abstract

We present the first evaluation harness that enables any out-of-the-box, local, Large Language Models (LLMs) to play full-press Diplomacy without fine-tuning or specialized training. Previous work required frontier LLMs, or fine-tuning, due to the high complexity and information density of Diplomacy's game state. Combined with the high variance of matches, these factors made Diplomacy prohibitive for study. In this work, we used data-driven iteration to optimize a textual game state representation such that a 24B model can reliably complete matches without any fine tuning. We develop tooling to facilitate hypothesis testing and statistical analysis, and we present case studies on persuasion, aggressive playstyles, and performance across a range of models. We conduct a variety of experiments across many popular LLMs, finding the larger models perform the best, but the smaller models still play adequately. We also introduce Critical State Analysis: an experimental protocol for rapidly iterating and analyzing key moments in a game at depth. Our harness democratizes the evaluation of strategic reasoning in LLMs by eliminating the need for fine-tuning, and it provides insights into how these capabilities emerge naturally from widely used LLMs. Our code is available in the supplement and will be open sourced.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

The Foreign Policy AI Evaluation Gap
cs.CY 2026-07 conditional novelty 6.0 of 10

Public technical AI governance almost never evaluates real foreign-policy AI workflows; the paper maps that gap and proposes task-scoped, human-recombined evaluation instead of model leaderboards.

Reference graph

Works this paper leans on

39 extracted references · 19 canonical work pages · cited by 1 Pith paper

[1]

L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023
[2]

J.; Bethge, M.; and Schulz, E

Akata, E.; Schulz, L.; Coda-Forno, J.; Oh, S. J.; Bethge, M.; and Schulz, E. 2025. Playing repeated games with large language models. Nature Human Behaviour, 1--17

work page 2025
[3]

Anthropic. 2025 a . Claude 3.7 Sonnet and Claude Code. Technical Blog

work page 2025
[4]

Anthropic. 2025 b . Introducing Claude 4. Technical Blog

work page 2025
[5]

P.; et al

Bakhtin, A.; Brown, N.; Dinan, E.; Farina, G.; Flaherty, C.; Fried, D.; Goff, A.; Gray, J.; Hu, H.; Jacob, A. P.; et al. 2022. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science, 378(6624): 1067--1074

2022
[6]

E.; and Wang, W

Belle, N.; Barnes, D.; Amayuelas, A.; Bercovich, I.; Wang, X. E.; and Wang, W. 2025. Agents of Change: Self-Evolving LLM Agents for Strategic Planning. arXiv preprint arXiv:2506.04651

arXiv 2025
[7]

N.; Li, T.; Li, D.; Zhu, B.; Zhang, H.; Jordan, M.; Gonzalez, J

Chiang, W.-L.; Zheng, L.; Sheng, Y.; Angelopoulos, A. N.; Li, T.; Li, D.; Zhu, B.; Zhang, H.; Jordan, M.; Gonzalez, J. E.; and Stoica, I. 2024. Chatbot Arena: An Open Platform for Evaluating LLM s by Human Preference. In Salakhutdinov, R.; Kolter, Z.; Heller, K.; Weller, A.; Oliver, N.; Scarlett, J.; and Berkenkamp, F., eds., Proceedings of the 41st Inter...

work page 2024
[8]

A.; Cao, C.; Cao, K.; Castagné, R.; Cendrero, J.; Currie, L

Cohere, T.; Aakanksha; Ahmadian, A.; Ahmed, M.; Alammar, J.; Alnumay, Y.; Althammer, S.; Arkhangorodsky, A.; Aryabumi, V.; Aumiller, D.; Avalos, R.; Aviv, Z.; Bae, S.; Baji, S.; Barbet, A.; Bartolo, M.; Bebensee, B.; Beladia, N.; Beller-Morales, W.; Bérard, A.; Berneshawi, A.; Bialas, A.; Blunsom, P.; Bobkin, M.; Bongale, A.; Braun, S.; Brunet, M.; Cahyaw...

arXiv 2025

Show all 39 references

[9]

Comanici, G.; Bieber, E.; Schaekermann, M.; Pasupat, I.; Sachdeva, N.; Dhillon, I.; Blistein, M.; Ram, O.; Zhang, D.; Rosen, E.; et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv ...

2025 arXiv
[10]

J.; et al

Costarelli, A.; Vyas, R.; Bamford, M.; Ho, G.; Lin, J.; Weihs, F.; Choi, J.; Strange, J.; Cannesson, M.; Cho, S. J.; et al. 2024. GameBench : Evaluating Strategic Reasoning Abilities of LLM Agents. arXiv preprint arXiv:2406.06613

2024 arXiv
[11]

de Wynter, A.; and Yuan, T. 2025. The Thin Line Between Comprehension and Persuasion in LLMs. arXiv preprint arXiv:2507.01936

2025 arXiv
[13]

C.; Rajani, A.; and Suhr, A

Gandhi, K.; Lee, D.; Grand, G.; Liu, M.; Weng, W. C.; Rajani, A.; and Suhr, A. 2023. Strategic Reasoning with Language Models. arXiv preprint arXiv:2305.19165

2023 arXiv
[14]

GLM, T.; Zeng, A.; Xu, B.; Wang, B.; Zhang, C.; Yin, D.; Rojas, D.; Feng, G.; Zhao, H.; Lai, H.; Yu, H.; Wang, H.; Sun, J.; Zhang, J.; Cheng, J.; Gui, J.; Tang, J.; Zhang, J.; Li, J.; Zhao, L.; Wu, L.; Zhong, L.; Liu, M.; Huang, M.; Zhang, P.; Zheng, Q.; Lu, R.; Duan, S.; Zhan...

2024 arXiv
[15]

Grattafiori, A.; Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Vaughan, A.; et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

2024 arXiv
[16]

Guan, Z.; Liu, X.; Su, W.; Zhang, Y.; Li, B.; and Xie, Y. 2024. Richelieu: Self-Evolving LLM -Based Agents for AI Diplomacy . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

2024
[17]

Guo, D.; Yang, D.; Zhang, H.; Song, J.; Zhang, R.; Xu, R.; Zhu, Q.; Ma, S.; Wang, P.; Bi, X.; et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

2025 arXiv
[18]

H.; Bhatia, K.; Abbeel, P.; and Dragan, A

Huang, S. H.; Bhatia, K.; Abbeel, P.; and Dragan, A. D. 2018. Establishing appropriate trust via critical states. In 2018 IEEE/RSJ international conference on intelligent robots and systems (IROS), 3929--3936. IEEE

2018
[19]

Huang, Y.; Xie, X.; Chen, Y.; Liao, D.; and Wu, F. 2024. DipLLM : Fine-Tuning LLM for Strategic Decision-making in Diplomacy . arXiv preprint arXiv:2506.09655

2024 arXiv
[20]

Kang, J.; Tong, Q.; Cai, J.-J.; He, T.; Liang, Y.; de Rijke, M.; Mei, Y.; Wen, Y.; and Liu, Y. 2024. GTBench : Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations. arXiv preprint arXiv:2402.12348

2024 arXiv
[21]

B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D

Kaplan, J.; McCandlish, S.; Henighan, T.; Brown, T. B.; Chess, B.; Child, R.; Gray, S.; Radford, A.; Wu, J.; and Amodei, D. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361

2020 arXiv
[22]

Kimi; Bai, Y.; Bao, Y.; Chen, G.; Chen, J.; Chen, N.; Chen, R.; Chen, Y.; Chen, Y.; Chen, Y.; et al. 2025. Kimi K2: Open Agentic Intelligence. arXiv preprint arXiv:2507.20534

2025 arXiv
[23]

Light, J.; Cai, M.; Shen, S.; and Hu, Z. 2023. AvalonBench : Evaluating LLMs Playing the Game of Avalon . In Advances in Neural Information Processing Systems, volume 36

2023
[24]

Lor \`e , N.; and Heydari, B. 2024. Strategic behavior of large language models and the role of game structure versus contextual framing. Scientific Reports, 14(1): 18492

2024
[25]

Malmqvist, L. 2024. Sycophancy in large language models: Causes and mitigations. arXiv preprint arXiv:2411.15287

2024 arXiv
[26]

Meta AI . 2025. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation. https://ai. meta. com/blog/llama-4-multimodal-intelligence/, checked on, 4(7): 2025

2025
[27]

Mistral AI . 2025 a . Devstral. Technical Blog

2025
[28]

Mistral AI . 2025 b . Medium is the new large. Technical Blog

2025
[29]

Mistral AI . 2025 c . Mistral Small 3.1. Technical Blog

2025
[30]

OpenAI. 2025 a . Introducing GPT-4.1 in the API. Technical Blog

2025
[31]

OpenAI. 2025 b . Introducing o3 and o4-mini. Technical Blog

2025
[32]

Paquette, P. 2020. Diplomacy : DATC -Compliant Game Engine with Web Interface. https://github.com/diplomacy/diplomacy. Version 1.1.2, accessed 1 August 2025

2020
[33]

Payne, K.; and Alloui-Cros, B. 2025. Strategic Intelligence in Large Language Models: Evidence from evolutionary Game Theory. arXiv preprint arXiv:2507.02618

2025 arXiv
[34]

Qwen Team . 2025. QwQ-32B: Embracing the Power of Reinforcement Learning

2025
[35]

Savani, B. 2021. DistilBERT model fine-tuned for emotion classification (distilbert-base-uncased-emotion). https://huggingface.co/bhadresh-savani/distilbert-base-uncased-emotion

2021
[36]

D.; Fan, Y.; Zhang, Y.; Mukobi, G.; and Fong, N

Wongkamjan, W.; Akter, S. D.; Fan, Y.; Zhang, Y.; Mukobi, G.; and Fong, N. N. 2024. More Victories, Less Cooperation: Assessing Cicero's Diplomacy Play. arXiv preprint arXiv:2406.04643

2024 arXiv
[37]

xAI. 2025. Grok 4. Technical Blog

2025
[38]

Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

2025 arXiv
[39]

, " * write output.state after.block = add.period write newline

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...
[40]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

arXiv 2023

[2] [2]

J.; Bethge, M.; and Schulz, E

Akata, E.; Schulz, L.; Coda-Forno, J.; Oh, S. J.; Bethge, M.; and Schulz, E. 2025. Playing repeated games with large language models. Nature Human Behaviour, 1--17

work page 2025

[3] [3]

Anthropic. 2025 a . Claude 3.7 Sonnet and Claude Code. Technical Blog

work page 2025

[4] [4]

Anthropic. 2025 b . Introducing Claude 4. Technical Blog

work page 2025

[5] [5]

P.; et al

Bakhtin, A.; Brown, N.; Dinan, E.; Farina, G.; Flaherty, C.; Fried, D.; Goff, A.; Gray, J.; Hu, H.; Jacob, A. P.; et al. 2022. Human-level play in the game of Diplomacy by combining language models with strategic reasoning. Science, 378(6624): 1067--1074

2022

[6] [6]

E.; and Wang, W

Belle, N.; Barnes, D.; Amayuelas, A.; Bercovich, I.; Wang, X. E.; and Wang, W. 2025. Agents of Change: Self-Evolving LLM Agents for Strategic Planning. arXiv preprint arXiv:2506.04651

arXiv 2025

[7] [7]

N.; Li, T.; Li, D.; Zhu, B.; Zhang, H.; Jordan, M.; Gonzalez, J

Chiang, W.-L.; Zheng, L.; Sheng, Y.; Angelopoulos, A. N.; Li, T.; Li, D.; Zhu, B.; Zhang, H.; Jordan, M.; Gonzalez, J. E.; and Stoica, I. 2024. Chatbot Arena: An Open Platform for Evaluating LLM s by Human Preference. In Salakhutdinov, R.; Kolter, Z.; Heller, K.; Weller, A.; Oliver, N.; Scarlett, J.; and Berkenkamp, F., eds., Proceedings of the 41st Inter...

work page 2024

[8] [8]

A.; Cao, C.; Cao, K.; Castagné, R.; Cendrero, J.; Currie, L

Cohere, T.; Aakanksha; Ahmadian, A.; Ahmed, M.; Alammar, J.; Alnumay, Y.; Althammer, S.; Arkhangorodsky, A.; Aryabumi, V.; Aumiller, D.; Avalos, R.; Aviv, Z.; Bae, S.; Baji, S.; Barbet, A.; Bartolo, M.; Bebensee, B.; Beladia, N.; Beller-Morales, W.; Bérard, A.; Berneshawi, A.; Bialas, A.; Blunsom, P.; Bobkin, M.; Bongale, A.; Braun, S.; Brunet, M.; Cahyaw...

arXiv 2025