Artificial Intelligence · arXiv:2610.06347 v1 · submitted 5 Oct 2026 · 41 references
ImproveAnyTask: An Autonomous Post-Training Harness for Iterative Model Self-Improvement
The paper says
“Adapting general-purpose large language models to specific tasks requires substantial human effort in designing data and training strategies. Sustaining improvement is especially challenging because model updates change the error distribution, requiring strategies to be continually refined.”
more of the abstract
We introduce ImproveAnyTask, an autonomous post-training harness that improves task performance under a limited compute budget. Drawing inspiration from gradient-based parameter optimization, the harness organizes adaptation into error attribution, update-direction selection, and executable model updates. It combines metric-level and case-level analysis to identify a focal problem, then investigates research-backed strategies and compares their reported gains and reproduction difficulty. The selected strategy is translated into training data and a training configuration, with small-scale execution checks preceding full post-training. Subsequent evaluation guides model selection and further adaptation, while validated strategies and scripts are retained for reuse. Across 11 tasks, ImproveAnyTask achieves mean gains of 18.29 and 11.97 percentage points on the Base and Instruct models, respectively, with a maximum gain of 41.96 points, under a 24-hour budget with resources equivalent to eight H20 GPUs.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 6 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. An honest, well-run empirical paper on agentic post-training whose headline numbers are plausible but whose small-set task gains rest on fragile validation-based checkpoint selection — and the withheld code/traces are the real reproducibility gap. This line is wrong
3 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 41 references say.
The paper says, at §4.1, Table 3, Appendix A.1
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.67):
“Exact-overlap auditing does not rule out semantic contamination in external data, and repeated use of validation feedback can still lead to validation-set overfitting.”
Pith says: Objection 1 of 3 · major
The per-task claims are not supported for the tasks with tiny validation splits. Checkpoint selection is the argmax over at most three validation-scored candidates, and one changed validation response moves a candidate's validation score by 20 percentage points on AIME (n=5), about 7.1 on Terminal-Bench (n=14), about 3.3 on GPQA-Diamond (n=30), about 3.8 on TAU2 (n=26), about 1.2 on IFEval (n=82), and about 2.2 on IFBench (n=45).
more of objection 1
A single response can therefore reverse the selected checkpoint, making the reported test score on these tasks partly a noise-driven draw; the paper's own numbers show the AIME Base gain (+4.00) equals one test response on the 25-question test. The ethics statement's acknowledgment that repeated validation feedback can overfit validation is not quantified for these splits, and the problem is compounded because validation scores guide the strategy trajectory (Section 3.4), not just the final argmax. I recommend reporting the per-run validation-selected checkpoints and their test scores for at least the tasks with fewer than about 100 validation examples, and softening the 'improves all eleven tasks' phrasing to claims restricted to tasks with adequate validation sizes.
The paper says, at §4.1, Table 1, §4.4
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
All main and ablation results are means over two independent runs, with no per-run values, standard deviations, or ranges. Because run-to-run variation stems from acknowledged sources (API behavior, external resource availability, training randomness, as stated in the reproducibility statement), the headline mean gains of 18.29 and 11.97 points have no stated uncertainty, and the reported differences over the coding-agent baselines are not testable.
more of objection 2
On the smallest tasks the baseline deltas are comparable to the selection noise: Base Terminal-Bench shows ours 12.00 versus Claude Code 5.33 against roughly 7.1-point selection granularity, and Base AIME shows ours 60.00 versus Claude Code 56.00 against 4-point test granularity. Reporting the two per-run values, even in an appendix, or a sensitivity analysis of the selected checkpoint to one or two changed validation responses, would let a reader judge whether the observed gains exceed run-to-run noise.
The paper says, at §4.1, Appendix A.2, ethics statement
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.88):
“Exact-overlap auditing does not rule out semantic contamination in external data, and repeated use of validation feedback can still lead to validation-set overfitting.”
Pith says: Objection 3 of 3 · major
Contamination risk is material for the small-validation tasks and is only acknowledged, not assessed. The harness deliberately selects among released instruction-following and reasoning datasets (Tulu 3 Persona-IF, RubricHub best-of-six responses, Nemotron, and teacher-generated reasoning traces for IFEval, MATH, GPQA and others) and optimizes explicitly for the target-task metric.
more of objection 3
The authors acknowledge that exact-overlap auditing does not rule out semantic contamination, but no quantitative or qualitative risk assessment is provided, which matters because for the small-validation tasks a handful of near-duplicate training examples can move the test score substantially. A concrete, cheap check would be to (i) inspect whether improved checkpoints reproduce, verbatim or near-verbatim, answer strings present in the training sources on a sample of test items, and (ii) re-run the validation-selection on a date-split (future) portion of at least one task per domain to separate genuine capability gains from exposure to near-duplicate content.
5 minor objections
The paper says, at Abstract and §4.2
Pith says: minor objection 1 of 5
The headline mean gains of 18.29 and 11.97 include the renormalized BFCL no-Web metric, which the authors themselves state is not directly comparable to the official Overall (Appendix A.4); one sentence of caveat in the abstract would prevent over-reading of the aggregate.
The paper says, at §4.3, Figure 3
Pith says: minor objection 2 of 5
The 100-hour IFEval result is a single trajectory; the text says 'in this run,' but the figure's C7 value (85.18) should be labeled as a single-run outcome so it is not mistaken for an expected value or a 24-hour result.
show 3 more
The paper says, at Appendix A.1
Pith says: minor objection 3 of 5
TAU2's validation split is 8 Airline plus 18 Telecom examples; with only 8 examples in the Airline domain, the per-domain selection noise is larger than the combined-size estimate suggests, and per-domain validation-selected candidates should be reported.
The paper says, at §4.4, Table 2
Pith says: minor objection 4 of 5
The ablation variants are each run twice and differ from the full harness by up to roughly 10 points, but the TAU2 (n=26) and IFEval (n=82) rows rest on the same fragile validation sizes as the main results; the TAU2 ablation row in particular should note the selection-noise level.
The paper says, at §4.1, Related Work
Pith says: minor objection 5 of 5
The claim that 'no related specialized system provides a complete open-source implementation for our full 11-task suite' is hard to verify as written; listing which cited systems were actually attempted as baselines, and which were ruled out for lack of release, would strengthen the baseline comparison.
What would settle it
Use the saved per-checkpoint validation predictions from the existing AIME runs to bootstrap the selection rule: resample the 5 validation responses with replacement, pick the argmax validation checkpoint, and record that checkpoint's fixed 25-question test score. If the bootstrap distribution of selected test gains includes zero or negative gains with substantial mass, the reported AIME figures are not robust to validation noise; repeat the same procedure for Terminal-Bench and TAU2. If per-question predictions were not saved, rerun AIME under six-fold cross-validation over all 30 problems and compare the selected checkpoint's held-out score across folds.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 6 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Read it. This is an honest, unusually well-executed empirical piece on an agentic post-training harness, and the central claim is plausible. The stress-test concern is real but proportionate: it undermines specific point estimates, not the paper.
What's actually new is the integrated three-module loop — dual-level error attribution, research-backed strategy comparison with reproduction-difficulty ratings, execution checks before full training — applied across 11 tasks and two initializations with outcome-aware reuse. The cited predecessors (FT-Agent, AutoTrainess, PostTrainBench, TREX) share the iterative loop, but not this package at this scale. The baseline comparison is the rare fair one: Codex and Claude Code get the same budget, tool permissions, and teacher access.
Credit where earned. They report the one regression (SuperGPQA-Instruct, −2.00) with three failed attempts and decline to pin a single cause. The 100-hour IFEval run shows a genuine trajectory, including a checkpoint that drops and recovery by restarting from C3. The ablation replaces modules with basic alternatives rather than deleting them, which keeps the loop intact and makes the comparison meaningful. The ethics statement is candid about validation overfitting and semantic contamination. Scores computed before truncation — small detail, but it signals care.
The soft spots, in proportion. The tiny validation splits are statistically fragile: AIME validates on 5 questions, and one changed response moves the score 20 points. With two or three checkpoints per run, the argmax can flip on a single response. TAU2-Base (+30.87) makes me most nervous — small splits on both sides and a huge gain. But the headline averages are carried by the larger tasks (MMLU-Pro, SuperGPQA, MATH-500, IFBench, LiveCodeBench), which are sturdier; the small-set entries should just be down-weighted. Two runs with no error bars compounds this everywhere. The bigger problem for an agentic system is that source code and execution traces come only upon acceptance — for this kind of work, the traces are the actual evidence that the loop does what the prose claims. And the harness relies on a proprietary teacher for both strategy selection and distillation, so gains are bounded by teacher capability; they acknowledge this, and the shared-teacher setup is fair.
Worth a serious referee. Send it, and ask for code and traces, more runs or bootstrap variance, and a stability check on the small-set tasks. If they deliver, that's an accept.
The strongest objection
The central claim is that the harness delivers genuine test-set gains under a fixed budget. Section 4.1 states that each run selects the candidate with the highest validation score among those validated before the deadline, and Appendix A.1 gives the validation splits: AIME 2025 has 5 validation examples, Terminal-Bench 14, GPQA-Diamond 30, IFEval 82, and TAU2-Bench 8+18. One changed validation response consequently moves a candidate's validation score by 20 percentage points on AIME, about 7.1 on Terminal-Bench, and 3.3 on GPQA. With at most three checkpoints per run, the argmax over such quantized estimates can be reversed by one or two responses, making the selected checkpoint's test score partly a noise-driven draw. The ethics statement acknowledges repeated validation feedback can overfit validation, but the paper never quantifies this risk for the tiny splits; two-run averaging does not estimate variance. If these entries are noise-led, the per-task point estimates (AIME +4/+24, Terminal-Bench +12/+10.7, GPQA +22.6/+4.8, TAU2 +30.9/+8.3) lose support, and the headline averages shrink, even though larger-validation tasks may remain credible.
The reader's weakest assumption identifies the same load-bearing concern: checkpoint selection by validation score on very small validation sets. I agree with that prioritization. The reported protocol is explicitly max-validation (Section 4.1), and Appendix A.1 shows AIME at 5 validation questions, so selection is quantized at 20 percentage points per item; Terminal-Bench and GPQA are similarly coarse. The authors state in the ethics statement that repeated validation feedback can overfit validation, but they do not quantify selection bias or report variance across runs, and the two-run average cannot estimate it. This is not a claim that the gains are impossible: MMLU-Pro, SuperGPQA, LiveCodeBench, and the extended IFEval trajectory have larger or more stable validation support. But the headline averages include the small-set tasks, and without a bootstrap or cross-validation check the reported AIME, Terminal-Bench, GPQA, and TAU2 gains are not distinguishable from selection noise. Since the reader's CONDITIONAL verdict already hinges on this weakness, my stress-test does not change the verdict. The remaining demands (released code and execution traces, more runs) would also resolve much of this uncertainty, but the statistical check on validation noise is the most direct test of the central quantitative claim.
The referee's report
The paper's contribution is a complete, documented autonomous post-training loop rather than a single new training technique, and the evaluation design is among the more transparent in this emerging literature: fixed validation/test splits with reported sizes (Appendix A.1), full checkpoint ancestry and per-trial data counts (A.2-A.3), honest reporting of the SuperGPQA-Instruct regression and of declining checkpoints C4 and C8, and an explicit disclosure that the BFCL score is a renormalized no-Web variant not comparable to the official Overall. If the reported gains are genuine, this is a significant result: an agent-driven loop that selects strategies from literature and execution feedback, substantially improving a 4B model's task performance under bounded compute and exceeding the gains of strong general-purpose agents under the same budget. The main risk to the headline claims is statistical rather than methodological: checkpoint selection operates on validation splits as small as 5 examples, and the paper reports no variance across its two runs.
Pith's reading of the paper (a machine's summary, not the authors')
The paper claims that adapting a general-purpose language model to a specific task can be fully automated — deciding what to fix, finding a research-backed fix, building the training data, and retraining — within a single 24-hour budget. The harness, ImproveAnyTask, organizes this as an optimization loop that mimics gradient descent: evaluate the current model, attribute its errors, pick one focal problem, select a strategy, and execute a post-training update. Across 11 tasks in five domains on a 4B model, the paper reports mean gains of 18.29 percentage points (Base) and 11.97 (Instruct), with a maximum of 41.96 points on LiveCodeBench, all under compute equivalent to eight H20 GPUs. These gains exceed those of the Codex and Claude Code coding agents run under the same budget and resources, which is the comparison the paper treats as decisive.
What carries the argument. The load-bearing object is a three-module optimization loop that treats each post-training trial as one outer 'parameter update' in a search over strategies. Error Attribution combines metric-level and case-level failure analysis to fix one focal problem per round, deliberately narrowing scope like a regularizer. Update Direction forces the agent to compare several research-backed candidate strategies against their reported gains and reproduction difficulty on a three-level scale — released training-ready data, data requiring processing or distillation, and complex or incomplete procedures — before committing resources. Executable Model Updates runs small-scale checks on a few real samples and a short training run before full post-training, then re-evaluates the candidate and retains validated strategies and scripts in an asset library for reuse in later rounds.
Core claim. The central claim is that a structured, autonomous loop — error attribution, update-direction selection, and executable model updates — can sustain task improvement that naive agents do not reach. Repetition matters: each round re-diagnoses the model's new errors, so the chosen strategy evolves as the model changes. The paper reports improvements on all 11 Base-model tasks and ten of eleven Instruct-model tasks, with the single decline on SuperGPQA-Instruct (−2.00). The largest gain, +41.96 points on LiveCodeBench v6, takes a 4B Base model from 19.64% to 61.60%. An extended 100-hour IFEval run shows continued gains to 85.18% at the seventh checkpoint, evidence that longer budgets can keep paying off, though the eighth checkpoint falls back to 83.00%.
The premise it rests on. The harness selects its final model by the highest validation score, and several validation sets are so small — AIME 2025 has 5 examples and Terminal-Bench 14 — that one response can reverse the chosen checkpoint, so part of the measured gain may be validation noise rather than genuine improvement.
If the paper is right (Pith's inferences, not the authors')
If correct, task-specific adaptation of small models no longer needs per-task human engineering: a bounded, autonomous loop can pick data and training strategies competitive with manual ones.
The improvements span five domains — knowledge, reasoning, coding, instruction following, and tool use — so the mechanism is not specialized to a single skill.
Optimized Base models beat optimized Instruct models on MATH-500 and LiveCodeBench, implying the harness can recover or exceed what general post-training already provides.
The 100-hour IFEval run shows the loop still gains after 24 hours, suggesting practical deployments could trade longer budgets for more improvement.
Beating general-purpose coding agents under the same budget supports the claim that an integrated diagnose-research-execute structure, rather than raw agent capability, drives the gains.
Because checkpoint selection follows the validation score and several validation sets are tiny — AIME 2025 has just 5 examples, Terminal-Bench 14 — a single response can flip the chosen checkpoint, so the reported gains on those tasks may partly reflect validation noise rather than genuine strategy value.
A natural testable extension is raising the validation-split fraction on small tasks and checking whether the same strategies are still selected; the harness's reliability likely scales with validation set size.
The explicit comparison of candidate strategies by reported gain and reproduction difficulty is the least automatable part of manual post-training, and if it transfers it may be reusable as a recipe beyond this harness.
The method restricts executable updates to supervised post-training, so extending to reinforcement objectives is the obvious next axis, and gains from RL-based strategies remain untested here.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Adapting general-purpose large language models to specific tasks requires substantial human effort in designing data and training strategies. Sustaining improvement is especially challenging because model updates change the error distribution, requiring strategies to be continually refined. We introduce ImproveAnyTask, an autonomous post-training harness that improves task performance under a limited compute budget. Drawing inspiration from gradient-based parameter optimization, the harness organizes adaptation into error attribution, update-direction selection, and executable model updates. It combines metric-level and case-level analysis to identify a focal problem, then investigates research-backed strategies and compares their reported gains and reproduction difficulty. The selected strategy is translated into training data and a training configuration, with small-scale execution checks preceding full post-training. Subsequent evaluation guides model selection and further adaptation, while validated strategies and scripts are retained for reuse. Across 11 tasks, ImproveAnyTask achieves mean gains of 18.29 and 11.97 percentage points on the Base and Instruct models, respectively, with a maximum gain of 41.96 points, under a 24-hour budget with resources equivalent to eight H20 GPUs.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-06
The authors' main claim, as the classifier picked it out 0.93
“Across 11 tasks, ImproveAnyTask achieves mean gains of 18.29 and 11.97 percentage points on the Base and Instruct models, respectively, with a maximum gain of 41.96 points, under a 24-hour budget with resources equivalent to eight H20 GPUs.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.60 · human participants 0.06 · controlled comparison 0.67 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.75 · mentions code or data availability 0.18
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Adapting general-purpose large language models to specific tasks requires substantial human effort in designing data and training strategies. | background 0.99 | 0.06 | 0.02 | asserted 1.00 |
| Sustaining improvement is especially challenging because model updates change the error distribution, requiring strategies to be continually refined. | background 0.91 | 0.10 | 0.02 | asserted 1.00 |
| We introduce ImproveAnyTask, an autonomous post-training harness that improves task performance under a limited compute budget. | contribution 1.00 | 0.87 | 0.26 | asserted 1.00 |
| Drawing inspiration from gradient-based parameter optimization, the harness organizes adaptation into error attribution, update-direction selection, and executable model updates. | method 0.99 | 0.76 | 0.04 | asserted 1.00 |
| It combines metric-level and case-level analysis to identify a focal problem, then investigates research-backed strategies and compares their reported gains and reproduction difficulty. | method 0.99 | 0.83 | 0.04 | asserted 0.98 |
| The selected strategy is translated into training data and a training configuration, with small-scale execution checks preceding full post-training. | method 1.00 | 0.82 | 0.03 | asserted 1.00 |
| Subsequent evaluation guides model selection and further adaptation, while validated strategies and scripts are retained for reuse. | method 1.00 | 0.81 | 0.05 | asserted 0.99 |
| Across 11 tasks, ImproveAnyTask achieves mean gains of 18.29 and 11.97 percentage points on the Base and Instruct models, respectively, with a maximum gain of 41.96 points, under a 24-hour budget with resources equivalent to eight H20 GPUs. | result 1.00 | 0.96 | 0.76 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.92
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-06: 1 request, 6,241 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (4)
Figures are the authors', reproduced from the paper.
References (41)
- [2] Variant Step 1 Step 2 Step 3 MMLU-Pro Full Train on 20K long mathemati- cal reasoning examples. Train on 20K concise final- answer examples; repair end- of-sequence handling and pre- mature termination. Introduce cross-domain super- vision using complete, veri- fied teacher re... on Pith
- [3] URLhttps://arxiv. org/abs/2506.07982. Berkeley Function Calling Leaderboard Team. The Berkeley Function Calling Leaderboard: From Tool Use to Agentic Evaluation of Large Language Models. OpenReview, on PithPith's reviewarXiv
- [4] URLhttps://arxiv. org/abs/2605.10906. Haoyang Fang, Wei Zhu, Boran Han, Alex Zhang, Zhenyu Pan, Shuo Yang, Shuai Zhang, Jiad- ing Gai, Peng Tang, Cuixiong Hu, Xuan Zhu, Huzefa Rangwala, George Karypis, and Bernie Wang. LLMZero: Discovering Adaptive Training Strategies for RL P... on PithPith's reviewarXiv
- [5] URLhttps://arxiv.org/abs/2606.18388. 10 Preprint GLM-5 Team. GLM-5: From Vibe Coding to Agentic Engineering, on PithPith's reviewarXiv
- [6] URLhttps: //arxiv.org/abs/2602.15763. Dan Greenstein, Zohar Karnin, Chen Amiraz, and Oren Somekh. Linguistic and Argument Diver- sity in Synthetic Data for Function-Calling Agents, on PithPith's reviewarXiv
- [7] URLhttps://arxiv.org/abs/ 2601.17829. Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, Ashima Suvarna, Benjamin Feuer, Liangyu Chen, Zaid Khan, Eric Frankel, Sachin Grover, Caroline Ch... on PitharXiv
- [8] URLhttps://arxiv.org/abs/2506.04178. Hanxu Hu, Yuxuan Wang, Maggie Huan, Jannis Vamvas, Yinya Huang, Zhijiang Guo, and Rico Sennrich. DeReason: A Difficulty-Aware Curriculum Improves Decoupled SFT-then-RL Training for General Reasoning, on PithPith's reviewarXiv
- [9] URLhttps://arxiv.org/abs/2603.11193. Jiahao Huang, Fei Cheng, Junfeng Jiang, and Akiko Aizawa. Tailoring the Curriculum: Student- Centered Reasoning Distillation via Dynamic Data-Model Compatibility, on PitharXiv
- [10] URLhttps: //arxiv.org/abs/2605.29229. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code, on PithPith's reviewarXiv
- [12] URLhttps://arxiv. org/abs/2606.04261. Kimi Team. Kimi K3: Open Frontier Intelligence, on PithPith's reviewarXiv
- [13] URLhttps://arxiv.org/abs/ 2607.24653. Qizheng Li, Yifei Zhang, Xiao Yang, Xu Yang, Zhuo Wang, Weiqing Liu, and Jiang Bian. FT- Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents, on PitharXiv
- [14] URLhttps: //arxiv.org/abs/2603.01712. Yang Li, Youssef Emad, Karthik Padthe, Jack Lanchantin, Weizhe Yuan, Thao Nguyen, Jason We- ston, Shang-Wen Li, Dong Wang, Ilia Kulikov, and Xian Li. NaturalThoughts: Selecting and Distilling Reasoning Traces for General Reasoning Tasks, on PithPith's reviewarXiv
- [15] URLhttps://arxiv.org/ abs/2507.01921. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s Verify Step by Step, on PithPith's reviewarXiv
- [17] URL https://arxiv.org/abs/2608.19072. M-A-P Team. SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines, on PithPith's reviewarXiv
- [18] URL https://arxiv.org/abs/2502.14739. Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, and Wenhu Chen. General-Reasoner: Advancing LLM Reasoning Across All Domains, on PithPith's reviewarXiv
- [19] URLhttps://arxiv.org/abs/ 2505.14652. 11 Preprint Zerun Ma, Guoqiang Wang, Xinchen Xie, Yicheng Chen, He Du, Bowen Li, Yanan Sun, Wenran Liu, Kai Chen, and Yining Li. TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration, on PithPith's reviewarXiv
- [20] URLhttps://arxiv.org/abs/2604.14116. Huanzhi Mao, Raymond Tsao, Jingzhuo Zhou, Shishir G. Patil, and Joseph E. Gonzalez. BFCL V4: Web Search. Berkeley Function Calling Leaderboard official blog, on PithPith's reviewarXiv
- [21] URLhttps: //gorilla.cs.berkeley.edu/blogs/15_bfcl_v4_web_search.html. Mathematical Association of America. American Invitational Mathematics Examination 2025, on Pith
- [22] URLhttps://arxiv. org/abs/2601.11868. Jaehyun Nam, Jinsung Yoon, Jiefeng Chen, Jinwoo Shin, Sercan ¨O. Arık, and Tomas Pfister. MLE- STAR: Machine Learning Engineering Agent via Search and Targeted Refinement, on PithPith's reviewarXiv
- [23] URL https://arxiv.org/abs/2506.15692. OpenAI. Introducing Codex. OpenAI, on PithPith's reviewarXiv
show 21 more
- [24] URLhttps://arxiv.org/abs/ 2507.02833. Qwen Team. Qwen3.5-4B and Qwen3.5-4B-Base Model Cards. Hugging Face, on PithPith's reviewarXiv
- [25] URLhttps://arxiv.org/abs/2603.08640. David Rein et al. GPQA: A Graduate-Level Google-Proof Q&A Benchmark, on PitharXiv
- [26] URLhttps: //arxiv.org/abs/2311.12022. Jiabin Tang, Lianghao Xia, Zhonghang Li, and Chao Huang. AI-Researcher: Autonomous Scientific Innovation, on PithPith's reviewarXiv
- [27] URLhttps://arxiv.org/abs/2505.18705. Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, and Wanxiang Che. CurateEvo: Data- Curation Evolving for Agentic Post-Training, 2026a. URLhttps://arxiv.org/abs/ 2607.06140. Haoqing Wang, Xiang Long, Ziheng Li, Yilong Xu, Tingguang Li... on PithPith's reviewarXiv
- [28] URLhttps://arxiv.org/abs/2406.01574. Changyi Xiao, Caijun Xu, and Yixin Cao. Reinforcement Learning with Conditional Expectation Reward, on PithPith's reviewarXiv
- [29] URLhttps://arxiv.org/abs/2603.10624. Mingyu Xu, Cheng Fang, Keyue Jiang, Yuqian Zheng, Yanghua Xiao, Baojian Zhou, Qifang Zhao, Suhang Zheng, Xiuwen Zhu, Jiyang Tang, Yongchi Zhao, Yijia Luo, Zhiqi Bai, Yuchi Xu, Wenbo Su, Wei Wang, Bing Zhao, Lin Qu, and Xiaoxiao Xu. Logics-S... on PitharXiv
- [30] URL https://arxiv.org/abs/2601.01562. Yutaro Yamada, Robert Tjarko Lange, Cong Lu, Shengran Hu, Chris Lu, Jakob Foerster, Jeff Clune, and David Ha. The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search, on PitharXiv
- [31] URLhttps://arxiv.org/abs/2504.08066. 12 Preprint John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. SWE-smith: Scaling Data for Software Engineering Agents, on PithPith's reviewarXiv
- [32] URLhttps://arxiv.org/abs/2504.21798. Shidong Yang, Ziyu Ma, Tongwen Huang, Yiming Hu, Yong Wang, and Xiangxiang Chu. CoEvolve: Training LLM Agents via Agent-Data Mutual Evolution, on PithPith's reviewarXiv
- [33] URLhttps://arxiv.org/ abs/2604.15840. Taro Yano, Yoichi Ishibashi, and Masafumi Oyamada. LaMDAgent: An Autonomous Frame- work for Post-Training Pipeline Optimization via LLM Agents. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 3... on PithPith's reviewarXiv
- [34] doi: 10.18653/v1/2025.emnlp-main.1529. URL https://aclanthology.org/2025.emnlp-main.1529/. Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. LIMO: Less is More for Reasoning, on Pithdoi:10.18653/v1/2025.emnlp-main.1529
- [35] URLhttps://arxiv.org/abs/2502.03387. Jiachen Yu, Zhihao Xu, Junjie Wang, and Yujiu Yang. Think-with-Rubrics: From External Evaluator to Internal Reasoning Guidance, 2026a. URLhttps://arxiv.org/abs/2605.07461. Zhaojian Yu, Penghao Yin, Shuzheng Gao, Shilin He, Kai Cai, and Xiao... on PithPith's reviewarXiv
- [36] URLhttps://arxiv.org/abs/ 2511.10395. Xueliang Zhao, Wei Wu, Jian Guan, Zhuocheng Gong, and Lingpeng Kong. PromptCoT 2.0: Scal- ing Prompt Synthesis for Large Language Model Reasoning, on PitharXiv
- [37] URLhttps://arxiv. org/abs/2509.19894. Zhengyang Zhao, Shengjie Ye, Lu Ma, Hao Liang, Hengyi Feng, and Wentao Zhang. ANDES: Agent Native Data Evolving Synthesis Tool for Autonomous Instruction Alignment, on PitharXiv
- [38] URL https://arxiv.org/abs/2606.01279. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-Following Evaluation for Large Language Models, on PithPith's reviewarXiv
- [39] URLhttps: //arxiv.org/abs/2311.07911. Adam Zweiger, Jyothish Pari, Han Guo, Ekin Aky ¨urek, Yoon Kim, and Pulkit Agrawal. Self- Adapting Language Models, on PithPith's reviewarXiv
- [40] URLhttps://arxiv.org/abs/2506.10943. 13 Preprint A APPENDIX A.1 EVALUATIONSPLITS ANDMETRICS Table 3 gives the fixed validation and test sizes. We allocate approximately 85% of each task to testing, rounding the test size down; TAU2 is split and rounded by domain, and BFCL by c... on PitharXiv
- [2023] URL https://arxiv.org/abs/2305.20050. Joy Jia Yin Lim, Xin Huang, Hao Peng, Yaxi Lu, Xin Cong, Zhong Zhang, Maosong Sun, and Yankai Lin. What is Missing from AI Post-Training AI: An Empirical Analysis, on PithPith's reviewarXiv
- [2024] URLhttps://arxiv.org/abs/ 2403.07974. Feiyang Kang, Hanze Li, Adam Nguyen, Mahavir Dabas, Jiaqi W. Ma, Frederic Sala, Dawn Song, and Ruoxi Jia. Can Generalist Agents Automate Data Curation?, on PithPith's reviewarXiv
- [2025] URLhttps://arxiv.org/abs/2504.01943. Anthropic. Claude 3.7 Sonnet and Claude Code. Anthropic, on PithPith's reviewarXiv
- [2026] URLhttps://arxiv.org/abs/2604.09791. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan.τ 2-Bench: Eval- uating Conversational Agents in a Dual-Control Environment, on PithPith's reviewarXiv
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- RSI-Prime: Training Models to Train Models cs.AI · Nov 2026
- AutoRL: A Multi-Agent System towards Autonomous Reinforcement Learning cs.AI · Nov 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment