Machine Learning · arXiv:2611.14515 v1 · submitted 10 Nov 2026 · 111 references
AutoScaffold: AI-Guided Scaffolding for Reinforcement Learning of Language Models
The paper says
“Reinforcement learning with verifiable rewards (RLVR) produces rich training experience, yet outcome rewards reduce each attempt to a single number and leave most of this information unused.”
more of the abstract
Training-time scaffolds, such as strategies or hints added to the prompt during training, offer a natural way to turn this information into guidance, but their design and adaptation are largely left to humans and tailored to specific domains. We introduce AutoScaffold, a unified AI-for-AI framework that turns a student's training experience into adaptive scaffolds, with minimal domain-specific information and no human intervention during training. A frozen teacher periodically examines this experience, diagnoses bottlenecks, and designs scaffolds in two forms: (i) strategies shared across a class of tasks and (ii) hints for individual hard tasks, adjusting the content and amount of support as the student learns. The scaffolds are removed at evaluation. We apply the same framework to ALFWorld, search-based QA, mathematical reasoning, and GPU kernel generation. When evaluated without scaffolds, students trained with AutoScaffold outperform those trained with the corresponding RL baselines in all four domains, with gains of 7.2 points in ALFWorld success and 4.0 points in average QA exact match. On GPU kernel generation, AutoScaffold increases the fraction of tasks with a correct kernel that runs faster than the PyTorch reference by up to 5.0 points. On math, our student surpasses QuestA's reported accuracy using approximately one-third as many student training steps. Analyses show that the teacher identifies a different failure in each task class and shifts its scaffolds from step-by-step workflows to targeted fixes as the student improves, and that its revised scaffolds help the student learn more than keeping the previous scaffold or using no scaffold.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 11 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A well-ablated and honest empirical paper — the scaffolding framework genuinely moves past prior work and the headline gains are credible — but every result uses one proprietary teacher, so the AI-for-AI scope is narrower than the framing implies. This line is wrong
3 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 111 references say.
The paper says, at §4.1, §4.2, §5
The review did not quote this passage.
Pith says: Objection 1 of 3 · major
All four headline gains are generated by one frozen teacher, GPT-5.5, and no experiment varies teacher identity, strength, or instruction quality. The held-out validation in Algorithm 2 can filter proposals from a given teacher, but it cannot supply domain knowledge that a weaker teacher never proposes, so the current evidence does not separate 'AutoScaffold improves the student' from 'GPT-5.5's latent competence, routed through AutoScaffold, improves the student.'
more of objection 1
Since the conclusion (sentence b16) and the 'unified AI-for-AI framework' framing are stated generally, I recommend either adding a teacher-strength ablation on at least one domain (for example, an open-weight teacher or degraded teacher instructions) or explicitly scoping the central claim to an existence proof for a frontier teacher.
The paper says, at Tables 5, 6, 7
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
The math and KernelBench results are the weakest legs of the all-four-domains claim. The math gains are 1.4 and 2.0 points over GRPO and DAPO, and the KernelBench gains are 3 to 5 points over StitchCUDA. Section 4.1 states that results are averaged over 3 independent runs, but Table 5 reports standard deviations only for AutoScaffold, Table 6 reports none, and Table 7 reports none for either method.
more of objection 2
Baseline variance is not reported anywhere. Please provide per-seed results and baseline variance or a significance test; otherwise the headline claim that AutoScaffold improves over the RL baseline in every domain is not fully supported in these two domains.
The paper says, at §3.2, §4.4, Figure 7
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The paper says the teacher 'decides on its own what scaffold to give, to which tasks, and how much' (sentence b59), but the teacher's instructions include a fixed 50% withdrawal threshold and a fixed injection cap q_max, and the math instance-level hint form is initialized from QuestA's prefix data.
more of objection 3
These are acknowledged in Section 3.2, and they are domain-agnostic rather than domain-specific, but the 'teacher autonomy' language in Section 4.4 is stronger than the procedure supports. Please reconcile the wording, or state explicitly that the 50% threshold and prefix initialization are part of the method rather than teacher decisions.
5 minor objections
The paper says, at §4.1
Pith says: minor objection 1 of 5
Please add a reproducibility statement covering code, teacher decision logs, per-seed checkpoints, and the exact GPT-5.5 configuration; the paper currently relies on a proprietary teacher without specifying a reproducibility path.
The paper says, at §3.2
Pith says: minor objection 2 of 5
The sentence that instance-level scaffolds are not validated is welcome, but since math is the only domain using them, please add one sentence explaining why prefix-ratio adjustments were not subjected to the same held-out validation, or quantify their effect in an ablation.
show 3 more
The paper says, at Appendix B.2
Pith says: minor objection 3 of 5
In the step-100 math update, the two proposed class-level notes are rejected because the eight-note cap is reached; this outcome appears only in the Outcome paragraph, so the Decision paragraph should state it directly to avoid the impression that the notes were deployed.
The paper says, at Table 4
Pith says: minor objection 4 of 5
The claim that AutoScaffold is best on six of seven benchmarks is correct among the 3B models shown, but StepSearch is listed with dashes for single-hop benchmarks; please clarify in the caption that the ranking excludes benchmarks for which a baseline has no reported value.
The paper says, at Appendix A
Pith says: minor objection 5 of 5
Propositions 1 and 2 are attributed to Liao et al. in their statements, and the uniform-shift Assumption 1 is acknowledged as an idealization; this is appropriate. A brief sentence noting that the analysis is motivational and does not by itself predict scaffold-free evaluation gains would further reduce the risk of overreading the theory.
What would settle it
Replace GPT-5.5 with an open-weight teacher of clearly lower capability, e.g., Qwen3-32B, in the full four-domain protocol of Section 4.2, keeping Algorithm 2, the teacher instructions, query budget B=15, K=10, q_max=0.5, students, RL algorithms, and validation protocol identical; report the same three-seed scaffold-free metrics against the same cited baselines. If the gain over the corresponding RL baseline vanishes or reverses in any domain, teacher competence is a confound and the Section 4.2 claim must be scoped to GPT-5.5; if all four gains persist within seed noise, the teacher-dependence concern is resolved.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 11 Nov 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Read AutoScaffold (2611.14515). Short version: this is a real step beyond the existing scaffolding literature, the experiments are coherent and reported honestly, and the central empirical claim — the same teacher-driven procedure improves scaffold-free RL performance across four domains — holds up. The thing to watch is the unvaried teacher: every headline gain uses GPT-5.5, with no ablation on teacher identity or strength.
What's actually new: prior methods use a fixed sample of failures (SkillRL), fixed rules (SAGE), or a hand-designed schedule (QuestA). AutoScaffold lets the teacher choose which evidence to inspect via query tools, write both class- and instance-level scaffolds, validate class-level proposals on held-out tasks before deployment, and withdraw support as the student improves. It is unified across ALFWorld, search QA, math, and KernelBench under four different RL algorithms, and evaluation is scaffold-free on separate test sets, so the results are not constructed from the quantities the teacher optimizes. The ablations are the best part: removing validation drops ALFWorld by 14.1 points, fixed skill banks and always-on injection both hurt, and the branch-run comparison (updated vs. old scaffold vs. vanilla RL at several checkpoints) is a genuine causal check. The Appendix B decision logs are concrete and make the teacher's behavior legible rather than hand-waved.
Soft spots, in proportion. First, teacher strength is unsecured. Validation can reject bad proposals but cannot supply domain knowledge a weaker teacher never proposes, and instance-level math hints are not validated at all (stated in Section 3.2). So the results are an existence proof for one frontier teacher, not a demonstration of teacher-generality. The authors do not overclaim in precise language — the conclusion says "a teacher," not "any teacher" — but the abstract's framing invites the broader reading. Second, several baselines (GiGPO, SAGE, DAPO, QuestA, StitchCUDA) are quoted from prior papers rather than re-run with matched seeds and environments. Standard for this literature, but the deltas should be read as approximate. Third, no code, data, or exact teacher prompts are provided, so exact replication is impossible. Fourth, the math result starts from QuestA's hand-designed prefixes, and the teacher's instructions encode two human-chosen principles (focus on low-success classes; withdraw above 50% success), so "no human intervention" is bounded by construction — disclosed, but worth remembering when citing. The theory is motivational rather than load-bearing; the authors say plainly it does not model transfer (b422–b423), which is fine for this paper.
Who it's for: anyone working on LLM post-training or scaffolding methods. It deserves a serious referee. I'd send it to review with a request for teacher ablations and released prompts in the revision.
The strongest objection
The framework's central claim (Section 4.2) is that the same AutoScaffold procedure, with teacher instructions differing only in domain description and class list, improves scaffold-free performance over the corresponding RL baseline in all four domains. Every one of those results is produced by the same proprietary frozen teacher, GPT-5.5 (Section 4.1 Implementation), and no experiment varies teacher identity or strength. The only objective filter in the design, held-out validation of class-level scaffolds (Section 3.2, Algorithm 2), rejects proposals by comparing scaffolded success on held-out tasks; it can filter bad proposals from a given teacher, but it cannot supply domain knowledge that a weaker teacher never proposes. Instance-level math prefix adjustments are explicitly not validated at all (Section 3.2: 'Instance-level scaffolds are not validated...'). A plausible alternative reading of the results is therefore that the measured gains are a property of GPT-5.5's latent competence in ALFWorld action semantics, search-QA strategies, math heuristics, and CUDA optimization, rather than of the AutoScaffold update loop. Under that reading, the paper establishes an existence proof for one frontier teacher, not a teacher-general framework. The authors do not claim teacher-independence in so many words, but the conclusion (sentence b16: a teacher can turn training experience into guidance) and the abstract's 'frozen teacher' framing are general, so the unvaried teacher is the load-bearing gap. This is a scope/correctness risk, not an internal inconsistency: the reported numbers are coherent and the ablations are informative.
The reader's weakest_assumption identifies the same condition I find least secured: the teacher is never varied, and the validation loop only guards proposal quality for the teacher that is present, not the dependence of the entire result on that teacher. I considered two other candidate concerns and do not think they are load-bearing. First, the Appendix A theory is limited: the authors state themselves that it characterizes per-step signal under the scaffolded prompt and does not model transfer to the unscaffolded policy (sentences b422-b423), and Assumption 1 (uniform shift) is explicitly an idealization. Since the paper's main claim is empirical, this is an acknowledged scope limit, not a flaw. Second, baseline numbers quoted from GiGPO, SAGE, QuestA, and StitchCUDA come from cited published sources under settings the paper says it follows; under the reviewing rules those are supported, and I have no evidence the settings are mismatched. The unvaried teacher is different in kind: no amount of re-derivation can rule out teacher-specificity, because the teacher is both the source of all scaffold content and a single proprietary model. The branch experiments in Section 4.4 are genuinely strong evidence for adaptivity, and the q_max=1 and static-scaffold ablations show the update loop matters, so I am not proposing rejection. The concern moves the accepted scope of the claim, not the verdict: the paper should remain conditional until a teacher-strength ablation (or a released teacher-agnostic protocol) shows the framework, not GPT-5.5's latent expertise, carries the gains. Hence verdict_should_be is UNCHANGED relative to the reader's CONDITIONAL.
The referee's report
If the results hold, the paper makes a useful empirical contribution: it demonstrates that a single automated loop, driven by a strong frozen teacher, can convert student rollouts into temporary training-time guidance that improves the student's independent capability, across four structurally different domains. The scaffold-free evaluation protocol is the right choice for measuring independent capability, and the ALFWorld ablations are informative, particularly the 14.1-point drop when validation is removed and the 8.6-point drop when interactive query tools and memory are replaced by a fixed failure sample. The paper is also transparent about limitations: instance-level hints are not validated, the theoretical analysis does not model transfer to the unscaffolded policy, and the teacher's withdrawal rule is partly instructed. The main gap is that all four headline results are produced by a single proprietary teacher, GPT-5.5, with no teacher variation, so the framework-level generality of the result is not yet established. The math and KernelBench margins are also small enough that per-seed and baseline variance reporting matters for the central all-four-domains claim.
Pith's reading of the paper (a machine's summary, not the authors')
AutoScaffold claims that a frozen teacher LLM can read a student LLM's reinforcement-learning rollouts and turn them into temporary training-time guidance — strategies shared by a class of tasks plus hints for individual hard tasks — that leaves the student better on its own when the guidance is removed. The same procedure, with teacher instructions differing only in a domain description and class list, is run on ALFWorld, search-based QA, mathematical reasoning, and GPU kernel generation, and in all four domains the scaffold-free student beats the corresponding RL baseline: +7.2 points in ALFWorld success, +4.0 in average QA exact match, up to +2.0 in math accuracy, and +3.0 in KernelBench correctness. The point of the claim is that designing training-time scaffolding — hint tiers, skill libraries, withdrawal schedules — has been a human job done separately for each domain, and this framework proposes that an automatic diagnostic loop can replace it. If true, the rich experience that outcome rewards discard can be recovered and used to make the student independently more capable.
What carries the argument. The load-bearing object is the periodic teacher-update loop. Every $K=10$ training steps, a frozen teacher LLM gathers evidence from the training log through three query tools — class statistics, filtered trajectories, and full rollout groups of a single task — plus a memory of its own past diagnoses; it then proposes edits to two scaffold levels, a class-level strategy $S_{\mathrm{class}}[C]$ injected with probability $q[C]$ and an instance-level hint $S_{\mathrm{inst}}[x]$ such as a fraction of a reference solution; and it validates every class-level edit against no-scaffold and current-scaffold success on held-out tasks before deployment. The theoretical anchor is the mixed-outcome probability $u(p)=1-(1-p)^{G}-p^{G}$ for a group of $G$ rollouts, which is symmetric, concave, and maximized at $p=1/2$: scaffolds help by lifting all-failure classes toward that point, and must be withdrawn once scaffold-free success passes one half. That analysis is what lets one procedure run in any domain with no domain-specific rules.
Core claim. The paper's central claim, stated in its main results, is that the same framework — a frozen teacher (GPT-5.5) whose instructions differ across domains only in the task description and class list — improves over the corresponding RL baseline under four RL algorithms (GiGPO, rubric-based GRPO, GRPO, and DAPO) and five students, with the student always evaluated with scaffolds removed. Concretely, AutoScaffold reaches 93.9% versus 86.7% ALFWorld success, 46.1% versus 42.1% average QA exact match, 70.1% versus 68.7% math accuracy with a 4B student and 62.4% versus 60.4% with a 1.5B student (the latter surpassing QuestA's reported 61.1% in about one-third as many training steps), and Level 1 KernelBench correctness of 63% versus 59%. The underlying mechanism is the group-relative learning signal: under binary rewards a rollout group whose members all succeed or all fail produces zero standardized advantage, so a task contributes no gradient when the student's success probability is near 0 or 1, while a scaffold that moves success toward $1/2$, where the mixed-outcome probability $u(p)=1-(1-p)^{G}-p^{G}$ peaks, restores the signal. The paper further shows, through ablations, that the gain comes from the teacher's interactive evidence-gathering, held-out validation of class-level edits, scaffold revision, and capped injection probability — not from extra guidance alone.
The premise it rests on. The whole result rests on one frozen teacher model, GPT-5.5, being able to read the student's noisy rollouts and judge what help the student needs and how much; the paper never varies the teacher, and it states in Section 3.2 that instance-level math hints are deployed without validation, so nothing in the framework would catch a teacher that writes plausible but useless guidance.
If the paper is right (Pith's inferences, not the authors')
If the central claim holds, training-time scaffolding for RL post-training can be fully automated: no human-written hint tiers, skill banks, or withdrawal schedules are needed, and a single procedure covers agentic, QA, math, and code domains.
The scaffold-learned skills transfer beyond the training data: in search-based QA the gain over GiGPO is as large on five out-of-domain benchmarks (+4.1 average) as on the two in-domain ones (+3.8).
The teacher's observed behavior — broad step-by-step workflows early, targeted fix-it reminders later, support withdrawn once scaffold-free success clears 50% — offers a general template for automatically scheduled curricula in RL.
The mechanism is not tied to binary rewards: under rubric-based rewards for GPU kernel generation, AutoScaffold improves both correctness and speed, raising Level 2 Fast1 from 20% to 25%.
Scaffold-based training can be much more efficient: the 1.5B math student surpasses QuestA's reported accuracy using roughly one-third of QuestA's training steps, starting from the same partial solutions but without its fixed schedule.
The paper never varies the teacher, so it does not separate what the framework contributes from what this particular teacher model contributes; a direct test would be to run the same loop with a much weaker teacher and see whether the evidence-gathering and validation machinery still carries the gain.
The theory only characterizes per-step learning signal, not transfer to the unscaffolded policy, which the paper acknowledges; an extension would check whether the teacher's $p=1/2$ target is actually optimal for final scaffold-free performance, or whether deliberately keeping success lower early in training would generalize better.
Instance-level math hints are prefixes of verified reference solutions and are never validated, so a natural follow-up is to let the teacher write and validate hints exactly as it writes class-level strategies, and measure whether the remaining gap to fully teacher-generated scaffolding closes.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Reinforcement learning with verifiable rewards (RLVR) produces rich training experience, yet outcome rewards reduce each attempt to a single number and leave most of this information unused. Training-time scaffolds, such as strategies or hints added to the prompt during training, offer a natural way to turn this information into guidance, but their design and adaptation are largely left to humans and tailored to specific domains. We introduce AutoScaffold, a unified AI-for-AI framework that turns a student's training experience into adaptive scaffolds, with minimal domain-specific information and no human intervention during training. A frozen teacher periodically examines this experience, diagnoses bottlenecks, and designs scaffolds in two forms: (i) strategies shared across a class of tasks and (ii) hints for individual hard tasks, adjusting the content and amount of support as the student learns. The scaffolds are removed at evaluation. We apply the same framework to ALFWorld, search-based QA, mathematical reasoning, and GPU kernel generation. When evaluated without scaffolds, students trained with AutoScaffold outperform those trained with the corresponding RL baselines in all four domains, with gains of 7.2 points in ALFWorld success and 4.0 points in average QA exact match. On GPU kernel generation, AutoScaffold increases the fraction of tasks with a correct kernel that runs faster than the PyTorch reference by up to 5.0 points. On math, our student surpasses QuestA's reported accuracy using approximately one-third as many student training steps. Analyses show that the teacher identifies a different failure in each task class and shifts its scaffolds from step-by-step workflows to targeted fixes as the student improves, and that its revised scaffolds help the student learn more than keeping the previous scaffold or using no scaffold.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-11
The authors' main claim, as the classifier picked it out 0.97
“When evaluated without scaffolds, students trained with AutoScaffold outperform those trained with the corresponding RL baselines in all four domains, with gains of 7.2 points in ALFWorld success and 4.0 points in average QA exact match.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.23 · human participants 0.03 · controlled comparison 0.97 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.98 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Reinforcement learning with verifiable rewards (RLVR) produces rich training experience, yet outcome rewards reduce each attempt to a single number and leave most of this information unused. | background 0.76 | 0.12 | 0.03 | asserted 1.00 |
| Training-time scaffolds, such as strategies or hints added to the prompt during training, offer a natural way to turn this information into guidance, but their design and adaptation are largely left to humans and tailored to specific domains. | gap 0.61 | 0.05 | 0.02 | asserted 0.97 |
| We introduce AutoScaffold, a unified AI-for-AI framework that turns a student's training experience into adaptive scaffolds, with minimal domain-specific information and no human intervention during training. | contribution 1.00 | 0.90 | 0.14 | asserted 1.00 |
| A frozen teacher periodically examines this experience, diagnoses bottlenecks, and designs scaffolds in two forms: (i) strategies shared across a class of tasks and (ii) hints for individual hard tasks, adjusting the content and amount of support as the student learns. | method 0.99 | 0.86 | 0.06 | asserted 1.00 |
| The scaffolds are removed at evaluation. | method 0.99 | 0.82 | 0.03 | asserted 1.00 |
| We apply the same framework to ALFWorld, search-based QA, mathematical reasoning, and GPU kernel generation. | method 0.57 | 0.94 | 0.03 | asserted 0.98 |
| When evaluated without scaffolds, students trained with AutoScaffold outperform those trained with the corresponding RL baselines in all four domains, with gains of 7.2 points in ALFWorld success and 4.0 points in average QA exact match. | result 1.00 | 0.96 | 0.46 | asserted 1.00 |
| On GPU kernel generation, AutoScaffold increases the fraction of tasks with a correct kernel that runs faster than the PyTorch reference by up to 5.0 points. | result 1.00 | 0.95 | 0.07 | asserted 0.99 |
| On math, our student surpasses QuestA's reported accuracy using approximately one-third as many student training steps. | result 1.00 | 0.90 | 0.10 | asserted 1.00 |
| Analyses show that the teacher identifies a different failure in each task class and shifts its scaffolds from step-by-step workflows to targeted fixes as the student improves, and that its revised scaffolds help the student learn more than keeping the previous scaffold or using no scaffold. | interpretation 0.58 | 0.95 | 0.18 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.93
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-11: 1 request, 7,814 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (7)
Figures are the authors', reproduced from the paper.
References (111)
- [1] Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards on Pith
- [2] Hinton, Geoffrey E. and Osindero, Simon and Teh, Yee Whye , journal =. A Fast Learning Algorithm for Deep Belief Nets , volume = on Pith
- [3] Deep learning , author=. 2016 , publisher= on Pith
- [4] Feng, Lang and Xue, Zhenghai and Liu, Tingcong and An, Bo , booktitle=. Group-in-Group Policy Optimization for on Pith
- [5] Jin, Bowen and Zeng, Hansi and Yue, Zhenrui and Yoon, Jinsung and Arik, Sercan and Wang, Dong and Zamani, Hamed and Han, Jiawei , journal=. Search- on Pith
- [6] Hu, Jinchao and Zhong, Meizhi and Chen, Kehai and Zhang, Min , journal=. SearchSkill: Teaching on Pith
- [7] Reflexion: Language Agents with Verbal Reinforcement Learning , author=. 2023 , eprint= on Pith
- [8] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models , author=. 2026 , eprint= on Pith
- [9] Meta-Harness: End-to-End Optimization of Model Harnesses , author=. 2026 , eprint= on Pith
- [10] Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses , author=. 2026 , eprint= on Pith
- [11] SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering , author=. 2024 , eprint= on Pith
- [12] Building Effective Agents , author=. 2024 , url= on Pith
- [13] RULER: What's the Real Context Size of Your Long-Context Language Models? , author=. 2024 , eprint= on Pith
- [14] Lost in the Middle: How Language Models Use Long Contexts , author=. 2023 , eprint= on Pith
- [15] AgentNoiseBench: Benchmarking Robustness of Tool-Using LLM Agents Under Noisy Condition , author=. 2026 , eprint= on Pith
- [16] DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author=. 2024 , eprint= on Pith
- [17] DAPO: An Open-Source LLM Reinforcement Learning System at Scale , author=. 2025 , eprint= on Pith
- [18] The role of tutoring in problem solving , author=. Journal of child psychology and psychiatry , volume=. 1976 , publisher= on Pith
- [19] Mind in society: The development of higher psychological processes , author=. 1978 , publisher= on Pith
- [20] Self-Hinting Language Models Enhance Reinforcement Learning , author=. 2026 , eprint= on Pith
show 80 more
- [21] EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning , author=. 2026 , eprint= on Pith
- [22] Privileged Information Dropout in Reinforcement Learning , author=. 2020 , eprint= on Pith
- [23] ReAct: Synergizing Reasoning and Acting in Language Models , author=. 2023 , eprint= on Pith
- [24] SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning , author=. 2026 , eprint= on Pith
- [25] SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization , author=. 2026 , eprint= on Pith
- [26] SKILLC: Learning Autonomous Skill Internalization in LLM Agents via Contrastive Credit Assignment , author=. 2026 , eprint= on Pith
- [27] Learning to Reason under Off-Policy Guidance , author=. 2025 , eprint= on Pith
- [28] Scaf-GRPO: Scaffolded Group Relative Policy Optimization for Enhancing LLM Reasoning , author=. 2026 , eprint= on Pith
- [29] QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation , author=. 2026 , eprint= on Pith
- [30] Proximal Policy Optimization Algorithms , author=. 2017 , eprint= on Pith
- [31] Back to Basics: Revisiting REINFORCE Style Optimization for Learning from Human Feedback in LLMs , author=. 2024 , eprint= on Pith
- [32] ZeroSearch: Incentivize the Search Capability of LLMs without Searching , author=. 2026 , eprint= on Pith
- [33] StepSearch: Igniting LLMs Search Ability via Step-Wise Proximal Policy Optimization , author=. 2025 , eprint= on Pith
- [34] StitchCUDA: An Automated Multi-Agents End-to-End GPU Programing Framework with Rubric-based Agentic Reinforcement Learning , author=. 2026 , eprint= on Pith
- [35] ALFWorld: Aligning Text and Embodied Environments for Interactive Learning , author=. 2021 , eprint= on Pith
- [36] Kwiatkowski, Tom and Palomaki, Jennimaria and Redfield, Olivia and Collins, Michael and Parikh, Ankur and Alberti, Chris and Epstein, Danielle and Polosukhin, Illia and Devlin, Jacob and Lee, Kenton and Toutanova, Kristina and Jones, Llion and Kelcey, Matthew and Chang, Ming-W... on PithPith's reviewdoi:10.1162/tacl
- [37] TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension , author=. 2017 , eprint= on Pith
- [38] When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories , author=. 2023 , eprint= on Pith
- [39] HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , author=. 2018 , eprint= on Pith
- [40] Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps , author=. 2020 , eprint= on Pith
- [41] MuSiQue: Multihop Questions via Single-hop Question Composition , author=. 2022 , eprint= on Pith
- [42] Measuring and Narrowing the Compositionality Gap in Language Models , author=. 2023 , eprint= on Pith
- [43] KernelBench: Can LLMs Write Efficient GPU Kernels? , author=. 2025 , eprint= on Pith
- [44] MathArena: Evaluating LLMs on Uncontaminated Math Competitions , author=. 2026 , eprint= on Pith
- [45] Let's Verify Step by Step , author=. 2023 , eprint= on Pith
- [46] Solving Quantitative Reasoning Problems with Language Models , author=. 2022 , eprint= on Pith
- [47] OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problems , author=. 2024 , eprint= on Pith
- [48] AI4AI at Test-Time: Strong-to-Weak Capability Transfer via Harnesses , author=. 2026 , eprint= on Pith
- [49] Kai Wu and Hao Lyu and Zhen Luo and Chaofan Wang and Siyu Ye and Jinghao Lin and Xiaozhong Ji and Boyuan Jiang and Shengzhi Wang and Zihan Wang and Yiwen Ye and Hao Wang and Zimu Wang and Wenzhe Liu and Ruobing Wang and Kai Cai and Mingliang Xiong and Wen Fang and Mingqing Liu... on Pithdoi:10.20944/preprints202608.2108.v1
- [50] From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning , author=. 2026 , eprint= on Pith
- [51] 2025 , howpublished = on Pith
- [52] 2024 , howpublished = on Pith
- [53] 2023 , howpublished = on Pith
- [54] LLMZero: Discovering Adaptive Training Strategies for RL Post-Training via LLM Agents , author=. 2026 , eprint= on Pith
- [55] Andrej Karpathy , title =. 2026 , howpublished = on Pith
- [56] ReSkill: Reconciling Skill Creation with Policy Optimization in Agentic RL , author=. 2026 , eprint= on Pith
- [57] Dr. Kernel: Reinforcement Learning Done Right for Triton Kernel Generations , author=. 2026 , eprint= on Pith
- [58] PostTrainBench: Can LLM Agents Automate LLM Post-Training? , author=. 2026 , eprint= on Pith
- [59] The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery , author=. 2024 , eprint= on Pith
- [60] Large Language Model Agent for Hyper-Parameter Optimization , author=. 2025 , eprint= on Pith
- [61] Eureka: Human-Level Reward Design via Coding Large Language Models , author=. 2024 , eprint= on Pith
- [62] CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization , author=. 2025 , eprint= on Pith
- [63] The AI Scientist-v2: Workshop-Level Automated Scientific Discovery via Agentic Tree Search , author=. 2025 , eprint= on Pith
- [64] Liu, Zichen and Chen, Changyu and Li, Wenjun and Qi, Penghui and Pang, Tianyu and Du, Chao and Lee, Wee Sun and Lin, Min , journal =. Understanding on Pith
- [65] Adaptive Guidance Accelerates Reinforcement Learning of Reasoning Models , author =. 2025 , eprint = on Pith
- [66] Zhang, Kaiyi and Lv, Ang and Li, Jinpeng and Wang, Yongbo and Wang, Feng and Hu, Haoyuan and Yan, Rui , year =. 2507.02841 , archivePrefix = on PithPith's reviewarXiv
- [67] Liu, Ziru and Gong, Cheng and Fu, Xinyu and Liu, Yaofang and Chen, Ran and Hu, Shoubo and Zhang, Suiyun and Liu, Rui and Zhang, Qingfu and Tu, Dandan , year =. 2507.10628 , archivePrefix = on PithPith's reviewarXiv
- [68] Zhang, Xuechen and Huang, Zijian and Li, Yingcong and Ni, Chenshun and Chen, Jiasi and Oymak, Samet , year =. 2506.17211 , archivePrefix = on PithPith's reviewarXiv
- [69] Qu, Yuxiao and Setlur, Amrith and Smith, Virginia and Salakhutdinov, Ruslan and Kumar, Aviral , year =. 2601.18779 , archivePrefix = on PitharXiv
- [70] Chen, Justin Chih-Yao and Peng, Becky Xiangyu and Choubey, Prafulla Kumar and Huang, Kung-Hsiang and Zhang, Jiaxin and Bansal, Mohit and Wu, Chien-Sheng , year =. Nudging the Boundaries of. 2509.25666 , archivePrefix = on PitharXiv
- [71] Wu, Jinyang and Liao, Chonghua and Feng, Mingkuan and Zhang, Shuai and Wen, Zhengqi and Luo, Haoran and Yang, Ling and Xu, Huazhe and Tao, Jianhua , year =. 2505.15692 , archivePrefix = on PithPith's reviewarXiv
- [72] Learning to Hint for Reinforcement Learning , author =. 2026 , eprint = on Pith
- [73] Jin, Xiaolong and Wang, Dingmin and Lingam, Vijay and Kumar, Varun , year =. 2608.24114 , archivePrefix = on PithPith's reviewarXiv
- [74] Xie, Can and Zhou, Yuyi and Yang, Wen and Zhang, Ziyi and Song, Siyao and Deng, Yingzhuo and Ren, Shuo and Zhang, Jiajun , year =. 2608.21946 , archivePrefix = on PithPith's reviewarXiv
- [75] Dynamic Skill Lifecycle Management for Agentic Reinforcement Learning , author =. 2026 , eprint = on Pith
- [76] Blending Supervised and Reinforcement Fine-Tuning with Prefix Sampling , author =. 2025 , eprint = on Pith
- [77] Liu, Mingyang and Farina, Gabriele and Ozdaglar, Asuman , year =. 2505.16984 , archivePrefix = on PitharXiv
- [78] Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questions , author =. 2025 , eprint = on Pith
- [79] More Than One Teacher: Adaptive Multi-Guidance Policy Optimization for Diverse Exploration , author =. 2025 , eprint = on Pith
- [80] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author =. 2026 , eprint = on Pith
- [81] Reinforcement Learning via Self-Distillation , author =. 2026 , eprint = on Pith
- [82] Zhang, Xiaoying and Zhang, Yipeng and Sun, Hao and Feng, Kaituo and Lu, Chaochao and Yang, Chao and Meng, Helen , year =. 2506.03106 , archivePrefix = on PithPith's reviewarXiv
- [83] Expanding the Capabilities of Reinforcement Learning via Text Feedback , author =. 2026 , eprint = on Pith
- [84] Zhang, Junjie and Ma, Guozheng and Liu, Shunyu and Hu, Zetian and Jing, Yongcheng and Lin, Ting-En and Li, Yongbin and Tao, Dacheng , year =. 2605.18851 , archivePrefix = on PithPith's reviewarXiv
- [85] Wang, Hanyang and Wang, Lu and Zhang, Chaoyun and Mao, Tianjun and Qin, Si and Lin, Qingwei and Rajmohan, Saravan and Zhang, Dongmei , year =. 2505.22338 , archivePrefix = on PitharXiv
- [86] Da, Jeff and Wang, Clinton and Deng, Xiang and Ma, Yuntao and Barhate, Nikhil and Hendryx, Sean , year =. 2506.11425 , archivePrefix = on PithPith's reviewarXiv
- [87] Zhao, Andrew and Huang, Daniel and Xu, Quentin and Lin, Matthieu and Liu, Yong-Jin and Huang, Gao , year =. 2308.10144 , archivePrefix = on PithPith's reviewarXiv
- [88] Agent Workflow Memory , author =. 2024 , eprint = on Pith
- [89] Ouyang, Siru and Yan, Jun and Hsu, I-Hung and Chen, Yanfei and Jiang, Ke and Wang, Zifeng and Han, Rujun and Le, Long T. and Daruki, Samira and Tang, Xiangru and Tirumalashetty, Vishy and Lee, George and Rofouei, Mahsan and Lin, Hangfei and Han, Jiawei and Lee, Chen-Yu and Pfi... on PithPith's reviewarXiv
- [90] Tang, Xiangru and Qin, Tianrui and Peng, Tianhao and Zhou, Ziyang and Shao, Daniel and Du, Tingting and Wei, Xinming and Xia, Peng and Wu, Fang and Zhu, He and Zhang, Ge and Liu, Jiaheng and Wang, Xingyao and Hong, Sirui and Wu, Chenglin and Cheng, Hao and Wang, Chi and Zhou, ... on PitharXiv
- [91] Zhou, Huichi and Chen, Yihang and Guo, Siyuan and Yan, Xue and Lee, Kin Hei and Wang, Zihan and Lee, Ka Yiu and Zhang, Guchun and Shao, Kun and Yang, Linyi and Wang, Jun , year =. 2508.16153 , archivePrefix = on PithPith's reviewarXiv
- [92] Zhang, Shengtao and Wang, Jiaqian and Zhou, Ruiwen and Liao, Junwei and Feng, Yuchen and Li, Zhuo and Zheng, Yujie and Zhang, Weinan and Wen, Ying and Li, Zhiyu and Xiong, Feiyu and Qi, Yutao and Tang, Bo and Wen, Muning , year =. 2601.03192 , archivePrefix = on PithPith's reviewarXiv
- [93] Wu, Rong and Wang, Xiaoman and Mei, Jianbiao and Cai, Pinlong and Fu, Daocheng and Yang, Cheng and Wen, Licheng and Yang, Xuemeng and Shen, Yufan and Wang, Yuxin and Shi, Botian , year =. 2510.16079 , archivePrefix = on PithPith's reviewarXiv
- [94] Zhang, Xiaoying and Liu, Zichen and Zhang, Yipeng and Hu, Xia and Shao, Wenqi , year =. 2603.08561 , archivePrefix = on PithPith's reviewarXiv
- [95] Self-Distilled Agentic Reinforcement Learning , author =. 2026 , eprint = on Pith
- [96] Wang, Guanzhi and Xie, Yuqi and Jiang, Yunfan and Mandlekar, Ajay and Xiao, Chaowei and Zhu, Yuke and Fan, Linxi and Anandkumar, Anima , year =. 2305.16291 , archivePrefix = on PithPith's reviewarXiv
- [97] Online Difficulty Filtering for Reasoning Oriented Reinforcement Learning , author =. 2025 , eprint = on Pith
- [98] Learning to Reason at the Frontier of Learnability , author =. 2025 , eprint = on Pith
- [99] Chen, Xiaoyin and Lu, Jiarui and Kim, Minsu and Zhang, Dinghuai and Tang, Jian and Pich. Self-Evolving Curriculum for. 2025 , eprint = on Pith
- [100] Teacher-Student Curriculum Learning , author =. 2017 , eprint = on Pith
The first 100 of 111 references are listed here. All of them are in this paper's citation record.
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment