OpenTable-R1: A Reinforcement Learning Augmented Tool Agent for Open-Domain Table Question Answering

Zipeng Qiu

REVIEW 5 major objections 6 minor 19 references

OpenTable-R1: A Reinforcement Learning Augmented Tool Agent for Open-Domain Table Question Answering

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that open-domain table QA can be solved end-to-end by an LLM that searches tables and runs SQL, with a cold-start-plus-async-RL recipe that lifts a 4B model from 12.0% to 86.2% exact match.

desk verdict A plausible and useful two-stage recipe for turning a 4B model into a strong table-QA tool agent, but the headline 86.2% claim needs baselines, a defined reward, and seeds before I would call it state of the art. read the letter →

arxiv 2507.03018 v1 pith:NI3MCMLX submitted 2025-07-02 cs.CL

Zipeng Qiu This is my paper

classification cs.CL

keywords tablequestionansweringopen-domainretrievaltool-augmentedlanguagemodelsreinforcementlearningGRPOSQLexecutionBM25+

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that open-domain table question answering does not need a separate retrieval stage and a separate reader. Instead, a single language model can do the whole job by calling two tools—a BM25+ table search API and a SQLite SQL executor—in multi-turn dialogue, and a compact 4-billion-parameter model can learn this behavior in two stages: supervised fine-tuning on 'simple' questions followed by an asynchronous reinforcement-learning pass, called Async GRPO, on the harder leftovers. The claimed payoff is a jump in exact match from 12.0% to 86.2% on a 1,024-question held-out sample of Open WikiTable, with fewer tokens and turns than the cold-start model used. If the claim holds, it shows that tool-use policy, not the retrieval algorithm, is what small models lack in table QA.

What carries the argument

The load-bearing mechanism is the Async GRPO training loop: an off-policy policy-gradient objective with importance sampling, clipping, and batch-normalized advantages, applied over multi-turn trajectories of search and SQL tool calls. LoRA adapters keep the updated policy cheap to swap, and a small rollout buffer collects samples from the previous adapter while new adapters train, masking the straggler delays that come from high-variance multi-turn generation. The cold-start SFT stage supplies a stable starting policy; the RL stage then optimizes when and how to call the tools, which is what produces the second accuracy jump and the drop in token and turn counts.

What would settle it

Recompute the simple/difficult split with a different teacher model and rerun the exact same cold-start plus 1,504-step Async GRPO training with a formally defined exact-match reward, then evaluate on the same 1,024 questions; if exact match no longer beats the 72.1% cold-start baseline by the reported margin, the central claim depends on the specific split and reward rather than on the training recipe.

Watch

Extended reading notes

Core claim

The central claim is that a two-stage fine-tuning recipe turns a small, tool-calling language model into a competitive open-domain table QA system. Starting from Qwen3-4B at 12.0% exact match, the paper performs supervised cold-start SFT on the 31,959 training questions that Qwen3-32B answers correctly, reaching 72.1% EM, then applies Async GRPO to the remaining 21,860 'difficult' questions, reaching 86.2% EM on the same 1,024-question held-out set. The trained model uses 2,690 tokens and 6.8 turns per question on average, beating zero-shot Qwen3-8B (41.0%, 5,205 tokens, 16.6 turns) and Qwen3-32B (58.2%, 3,918 tokens, 9.0 turns). The paper's message is that end-to-end agentic tool use, hardened by RL, is a viable alternative to static retrieval-then-answer pipelines.

Load-bearing premise

The load-bearing assumption is that the Async GRPO reward is exact match against the ground-truth answers of the 21,860 'difficult' training questions, and that optimizing this reward on that split generalizes to the held-out 1,024-question test set; the paper never formally defines the reward, so a different reward definition could change the result.

Editorial extensions

If this is right

A 4-billion-parameter agent trained with cold-start SFT plus Async GRPO reaches 86.2% exact match on the held-out 1,024-question sample, outperforming zero-shot Qwen3-8B and Qwen3-32B under identical tool settings.
The supervised cold start on easy questions is alone responsible for the largest gain, taking the 4B model from 12.0% to 72.1% EM while cutting average tokens from 11,997 to 3,906 and average turns from 52.9 to 8.8.
Adding Async GRPO on the difficult questions adds about 14 absolute EM points and lowers token consumption and turn count by roughly 30% relative to the cold-start model.
Because the search backend is only BM25+, the results imply that retrieval sophistication is not the main bottleneck; the model's learned planning over tool feedback is what drives accuracy.
The rollout-buffer design makes RL training over multi-turn tool calls practical under GPU constraints, so the same recipe could be reapplied to other tool-augmented tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension the paper leaves implicit is ablating the reward: since no reward formula is given, defining it explicitly (for example, exact match with a token penalty) and re-running Async GRPO would show how much of the 14-point gain and efficiency improvement comes from reward design.
The simple/difficult split made by a stronger teacher's exact-match answers acts as a curriculum; a testable extension is whether a learned difficulty scorer or a human-annotated split preserves the same 14-point RL gain.
The same cold-start-plus-async-RL recipe should transfer to other open-domain tool tasks, such as code execution over heterogeneous data sources, whenever a supervised cold-start policy can be obtained.
Swapping BM25+ for a dense retriever would test whether retrieval quality still matters once the policy has learned to search; the paper's modular design predicts the model can adapt, but that is an extrapolation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

This is a compact empirical paper with a genuinely interesting internal result: a 4B model goes from 12% exact match zero-shot to 72.1% after cold-start SFT on the "simple" split, then to 86.2% after Async GRPO on the "difficult" split. That 12→72.1→86.2 trajectory is the real news, and it is clearly presented. The two-stage idea—SFT on examples a larger model can already answer, RL on the rest—is sensible and worth stealing.

What the paper does well: it reports token consumption and turn counts alongside accuracy, which is the right way to evaluate tool agents. The Async GRPO design with a rollout buffer is a practical engineering choice that addresses a real bottleneck in multi-turn rollout. The result that a 4B model can rival a 32B zero-shot model on this benchmark is meaningful for efficient tool-use training.

The soft spots are real but mostly correctable reporting gaps, not evidence of a false result. First, the RL reward is never defined. Equation (1) refers to cumulative reward Ri, but the paper never says what Ri is. The natural reading is exact match against the ground-truth answers of the 21,860 "difficult" training questions, but the text should say so explicitly. Without that, the 14-point GRPO gain cannot be independently interpreted. Second, there are no external baselines—no comparison to prior open-domain table QA systems, no comparison to static retriever + reader pipelines, and no ablation of Async GRPO against plain GRPO. The "state of the art" claim in the introduction is therefore unsupported. Third, no seeds or confidence intervals are reported, and the 1,024-question test subset is randomly sampled without a stated seed, so the evaluation is not reproducible. Finally, the abstract says "single-digit zero-shot performance" while the body reports 12% for Qwen3-4B, which is double-digit; that is a minor but sloppy inconsistency.

None of these issues sink the central empirical claim. The internal trajectory is plausible and the training progression makes sense. The paper deserves a serious referee, but only with the expectation that the author adds baselines, defines the reward, and reports seeds or repeated evaluations.

For a reading group, this is a reasonable "engineering result" paper to skim, especially if you care about RL for tool agents, but it is not foundational. I would not cite it as a baseline until the missing details are provided, though the code release is promising.

Referee Report

5 major / 6 minor

Summary. The manuscript presents OpenTable-R1, an agentic pipeline for open-domain table QA built on Qwen3-4B with two tools: a BM25+ table search API and a SQLite SQL executor. The method uses a two-stage adaptation: SFT cold start on 31,959 'simple' training questions (labeled by exact-match agreement with Qwen3-32B) followed by an 'Async GRPO' RL stage on the 21,860 'difficult' questions with LoRA adapters and a rollout buffer. On a randomly sampled 1,024-question subset of the Open WikiTable test split, the paper reports exact-match accuracy rising from 12.0% (zero-shot Qwen3-4B) to 72.1% after cold start to 86.2% after RL, with average token use falling from 11,997 to 2,690 and turns from 52.9 to 6.8; this exceeds the reported zero-shot Qwen3-8B (41.0%) and Qwen3-32B (58.2%). The paper claims state-of-the-art results on Open WikiTable and attributes the gain to the two-stage RL pipeline.

Significance. If the reported numbers are reproducible, the paper makes a useful empirical contribution: a compact 4B model trained with tool calls and RL outperforms much larger zero-shot models (58.2% for Qwen3-32B) on the same 1,024-question set, while cutting token and turn costs substantially. The internal trajectory 12.0% to 72.1% to 86.2% is coherent and the tool and prompt design is described in enough detail to replicate the API layer. The code release further supports reproducibility. However, the paper's significance is currently capped by three gaps: the RL reward is not specified, no seed or confidence information is given, and the 'state-of-the-art' claim is not backed by any external baseline on the Open WikiTable benchmark. These are fixable reporting issues rather than evidence of a false result.

major comments (5)

[Section 3.2, Eqs. (1)-(2)] Section 3.2, Eqs. (1)-(2): the cumulative reward Ri used in the Async GRPO loss is never defined. The introduction's second contribution says the RL framework 'trains the LLM to balance tool-calling cost against QA reward,' but the training description in Section 3.2 only states that RL is applied to the remaining difficult questions. The reader must infer that Ri is exact match against ground-truth answers; if instead Ri includes token, turn, or SQL-execution penalties, the reported 14-point RL gain and the token/turn reductions would be partly artifacts of reward shaping. Please state Ri explicitly, including how exact match is computed on the <answer> format and whether any cost terms are included, and report the reward values during training.
[Section 3, evaluation subset] Section 3, 'we randomly sample 1,024 examples from the 6,602 test questions': no random seed is given, so the exact evaluation set is not reproducible, and no multiple-seed or bootstrap confidence intervals are reported for any of the 12.0%, 72.1%, or 86.2% numbers. Because RL training and sampling are stochastic, a single unseeded run cannot establish the stability of the reported 14.1-point GRPO improvement. Please provide the seed used for the test-subset selection and report confidence intervals or repeated-run statistics for the main accuracy and cost metrics.
[Introduction and Related Work, 'State-of-the-art' claim] Introduction and Related Work: the contribution bullet 'State-of-the-art results on open WikiTable QA' and the Related Work claim of advancing the state of the art are not supported by any external comparison. Table 5 contains only Qwen3 variants evaluated by the authors; there is no comparison with existing retrieval+QA pipelines or published results on the Open WikiTable benchmark (e.g., the baselines in Kweon et al. or any later work). Either add such baselines or temper the claim to 'strong results on our 1,024-question evaluation subset.'
[Section 2.2, Async GRPO validation] Section 2.2: Async GRPO is motivated as an efficiency improvement with Figure 1 illustrating scheduling, but no experiment reports training wall-clock time, GPU utilization, or accuracy relative to standard GRPO. As written, the paper does not demonstrate that the asynchronous rollout buffer contributes to the 86.2% accuracy or to training efficiency; the central pipeline claim requires at least one ablation comparing Async GRPO with synchronous GRPO under matched compute and data.
[Section 3.2, data partitioning and SFT labels] Section 3.2, Table 4: the simple/difficult partition is made by having Qwen3-32B generate answers and checking exact match against ground truth, but it is not stated what the SFT labels are for the simple subset: the model-generated answers or the ground-truth answers. If the SFT target is Qwen3-32B's generated output, the cold start is trained on model-generated text and may inherit Qwen3-32B's error patterns; if the target is ground truth, this should be stated explicitly. Please clarify the label source and report the stability of the partition (e.g., the exact-match rate of Qwen3-32B on the training split).

minor comments (6)

[Abstract and Section 3.1/Table 3] Abstract says 'single-digit zero-shot performance' and 'over 86% exact match,' but Section 3.1 reports Qwen3-4B zero-shot at 12.0% and Section 3.2 reports 86.2%. Please make these numbers consistent.
[Section 2.1, prompt schema] Section 2.1: the tool table lists the argument for `code_interpreter` as `sql query`, but the example prompt wraps code in `<code>...</code>` tags inside an otherwise empty `arguments` object; please clarify how the tool parses the SQL argument from the XML block.
[Related Work, reference [16]] Related Work: the sentence 'Recent studies combine sparse and dense signals in hybrid indexes [16]' cites SparTA, which is a model sparsity system, not a retrieval paper; this citation appears to be a factual error.
[Eq. (2)] Eq. (2): the denominator std(R1,...,RG) is undefined when all rewards in a group are equal; please add a numerical safeguard or state the convention used in that case.
[Section 2.2, batch ratio] Section 2.2: 'We assume a 6:1 ratio between rollout and training batch sizes' is stated as a design assumption; please indicate whether this ratio was tuned and how sensitive the results are to it.
[Throughout] There are several typographical and encoding issues, e.g., 'na¨ıvely' in Section 1 and nonstandard spacing in the prompt listing; a careful proofread would improve presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the evaluation is held out from both training stages, and no load-bearing self-citation or fitted-as-prediction step appears.

full rationale

No circular step is exhibited. The paper's central claim is an empirical accuracy measurement on a 1,024-example held-out subsample of the Open WikiTable test split. The cold-start SFT uses a simple/difficult partition generated by Qwen3-32B answers that match ground truth, but this partition only selects training data; it does not define the evaluation metric, the test labels, or the reported EM. The Async GRPO stage trains on the difficult training subset, and the natural reading of the reward is exact match against the training ground-truth answers; this reward is not the held-out test evaluation, so the reported 86.2% is not a by-construction consequence of the training objective. The lack of an explicitly stated reward function and the unseeded random sampling of the evaluation subset are reproducibility and reporting gaps, not circularity. The cited GRPO, LoRA, Qwen3, and Open WikiTable works are external background references, and no self-citation chain is load-bearing. The derivation chain is therefore self-contained relative to the inputs, and the honest finding is no significant circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The paper's central claim rests on standard RL machinery plus two domain assumptions: the Qwen3-32B-based curriculum split and the transfer of an unnamed exact-match reward from training to held-out test. The strongest load-bearing premise is that the training reward and curriculum generalize to the 1,024-question test sample.

free parameters (3)

LoRA rank = 12
Chosen by hand for the RL stage; affects how much the 4B model can adapt, reported in Section 3.2.
Clipping parameter epsilon = 0.2
Fixed for the GRPO loss in Eq. 1; standard value, but chosen by hand.
Simple/difficult data split via Qwen3-32B threshold = 31,959 simple / 21,860 difficult
The split of training data into SFT and RL stages is determined by Qwen3-32B exact-match correctness; this data-derived rule shapes the curriculum.

assumptions (4)

ad hoc to paper Qwen3-32B exact-match labeling of training questions into simple and difficult sets is a valid curriculum for SFT and RL.
Section 3.2; no analysis of label stability or alternative splits is given, yet the full pipeline depends on it.
domain assumption The RL reward is exact match against ground-truth answers on the difficult training subset, and optimizing it transfers to the held-out test subset.
Section 3.2; the reward is not explicitly defined in the paper.
standard math The GRPO loss with importance sampling, baseline, and clipping (Eq. 1) provides stable policy-gradient updates for multi-turn tool use.
Section 2.2; standard RL result from the PPO and GRPO literature.
domain assumption BM25+ retrieval on the Open WikiTable corpus returns the tables needed to answer the test questions within the tool-call loop.
Section 2.1; no recall analysis is provided.

how reviews work

0 comments

Cite this review

Pith. "Pith review of OpenTable-R1: A Reinforcement Learning Augmented Tool Agent for Open-Domain Table Question Answering." pith.science (2026). https://pith.science/paper/NI3MCMLX

@misc{pith2026250703018,
  author       = {Pith},
  title        = {Pith review of: OpenTable-R1: A Reinforcement Learning Augmented Tool Agent for Open-Domain Table Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NI3MCMLX}},
  note         = {Machine review of arXiv:2507.03018}
}

read the original abstract

Open-domain table question answering traditionally relies on a two-stage pipeline: static table retrieval followed by a closed-domain answer. In contrast, we propose an end-to-end agentic framework that embeds multi-turn tool calls-using a BM25+-based search API and a SQLite SQL executor-directly into a large language model. To further adapt a compact 4B-parameter model, we introduce a two-stage fine-tuning process: supervised cold-start on easy questions, then Async GRPO reinforcement learning on harder cases with LoRA adapters and a rollout buffer. This unified approach enables the model to jointly retrieve, reason, and execute queries, yielding a dramatic accuracy improvement from single-digit zero-shot performance to over 0.86 exact match on a held-out test set. Our results underscore the effectiveness of integrating structured tool calls with targeted RL fine-tuning for scalable, accurate table QA. The code is available at https://github.com/TabibitoQZP/OpenTableR1.

Figures

Figures reproduced from arXiv: 2507.03018 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

19 extracted references · 5 canonical work pages

[1]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey , Abhinav Jauhri, Abhinav Pandey , Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024
[2]

Qwen2.5 technical report, 2025

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

2025
[3]

Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

arXiv 2025
[4]

Proximal policy optimization algorithms

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. 8

arXiv 2017
[5]

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

2023
[6]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

arXiv 2024
[7]

Nousresearch/hermes-function-calling

NousResearch. Nousresearch/hermes-function-calling

work page
[8]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

arXiv 2025

Show all 19 references

[9]

Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.ICLR, 1(2):3, 2022

2022
[10]

Open-wikitable: Dataset for open domain question answering with complex reasoning over table.arXiv preprint arXiv:2305.07288, 2023

Sunjun Kweon, Yeonsu Kwon, Seonhee Cho, Yohan Jo, and Edward Choi. Open-wikitable: Dataset for open domain question answering with complex reasoning over table.arXiv preprint arXiv:2305.07288, 2023

2023 arXiv
[11]

Tapas: Weakly supervised table parsing via pre-training.arXiv preprint arXiv:2004.02349, 2020

Jonathan Herzig, Paweł Krzysztof Nowak, Thomas M¨uller, Francesco Piccinno, and Julian Martin Eisen- schlos. Tapas: Weakly supervised table parsing via pre-training.arXiv preprint arXiv:2004.02349, 2020

2004 arXiv
[12]

Tapex: Table pre-training via learning a neural sql executor.arXiv preprint arXiv:2107.07653, 2021

Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou. Tapex: Table pre-training via learning a neural sql executor.arXiv preprint arXiv:2107.07653, 2021

2021 arXiv
[13]

Toolformer: Language models can teach themselves to use tools

Timo Schick, Jane Dwivedi-Yu, Roberto Dess`ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539–68551, 2023

2023
[14]

The probabilistic relevance framework: Bm25 and beyond

Stephen Robertson, Hugo Zaragoza, et al. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389, 2009

2009
[15]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InEMNLP (1), pages 6769–6781, 2020

2020
[16]

{SparTA}:{Deep-Learning} model sparsity via {Tensor-with-Sparsity-Attribute}

Ningxin Zheng, Bin Lin, Quanlu Zhang, Lingxiao Ma, Yuqing Yang, Fan Yang, Yang Wang, Mao Yang, and Lidong Zhou. {SparTA}:{Deep-Learning} model sparsity via {Tensor-with-Sparsity-Attribute}. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), page...

2022
[17]

Training language models to follow instructions with human feedback

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray , et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:277...

2022
[18]

Adalora: Adaptive budget allocation for parameter-efficient fine-tuning

Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adalora: Adaptive budget allocation for parameter-efficient fine-tuning. arXiv preprint arXiv:2303.10512, 2023

2023 arXiv
[19]

Asynchronous methods for deep reinforcement learning

Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley , David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In International conference on machine learning, pages 1928–1937. PmLR, 2016. 9

1928

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey , Abhinav Jauhri, Abhinav Pandey , Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024

[2] [2]

Qwen2.5 technical report, 2025

Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

2025

[3] [3]

Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

arXiv 2025

[4] [4]

Proximal policy optimization algorithms

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. 8

arXiv 2017

[5] [5]

Direct preference optimization: Your language model is secretly a reward model

Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728–53741, 2023

2023

[6] [6]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024

arXiv 2024

[7] [7]

Nousresearch/hermes-function-calling

NousResearch. Nousresearch/hermes-function-calling

work page

[8] [8]

Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

arXiv 2025