Machine Learning · arXiv:2611.16304 v1 · submitted 12 Nov 2026 · 56 references
StateAlign: Internalizing State-Aware Decision-Making for Reliable Multi-Turn Tool Agents via Reinforcement Learning
The paper says
“Reinforcement learning has improved the tool-use capabilities of large language models. However, better task performance can coincide with increased tool hallucination. Reliable tool use requires decisions to follow the current task state: available information, tool capabilities, and task progress.”
more of the abstract
Our analysis shows that agents sometimes execute when they should ask for information, retrieve it, or decline an infeasible request, even when reasoning identifies missing information. We introduce StateAlign, combining task-state construction with state-contrastive reinforcement learning. The data cover five task conditions and pair states with different action requirements. Outcome rewards guide task completion; token-level contrasts compare the same generated reasoning and actions under paired states to train outputs that fit the current state. Experiments across five base models from three families show improvements on BFCL v4 multi-turn, ToolSandbox, and When2Call. Ablations show that state construction improves performance and reliability under different reward designs. The complete method further reduces tool hallucination across three Qwen3 sizes. On Qwen3-8B, StateAlign improves BFCL v4 multi-turn accuracy from 45.5% to 55.2% and reduces no-tool hallucination on SimpleToolHalluBench from 62.2% under outcome-only training to 56.1%. Behavioral analyses show more consistent decisions, more complete multi-call execution, and stronger recovery after tool errors.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 13 Nov 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. StateAlign is a solid, well-documented paper that introduces a genuinely new training signal—paired-state contrastive RL—for reducing tool hallucination in LLM agents. The main soft spot is the 70.4% raw agreement on condition labels, which the stress test correctly flags but the paper itself acknowledges only obliquely. This line is wrong
4 major objections · 8 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 56 references say.
The paper says, at §3.3, Eq. (2)-(3)
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The state-contrastive objective uses the base model π0 as a fixed scorer to compute token-level log-ratio weights r_{i,t}. This design assumes that π0's likelihoods under the paired contexts (h_i, z_i) and (h̃_i, z̃_i) provide a reliable signal for state compatibility.
more of objection 1
However, π0 is the same model that the paper diagnoses as failing to act appropriately in many cases (Section 2). If π0 itself has poor state awareness, the contrastive signal may be noisy or biased. The authors should provide analysis of the quality of π0's scoring, e.g., correlation with human judgments or with outcome rewards, to justify this design choice.
The paper says, at §4.2, Table 1
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
Most headline results are reported as single means without variance. Only BFCL in Table 2 includes standard deviations (from six evaluations). For ToolSandbox, When2Call, and SimpleToolHalluBench, the paper reports means over three evaluations but no variance. Given the modest improvements in some cases (e.g., Qwen3-14B BFCL 54.8 vs AWM-14B 55.5, within the standard deviation of ~1.5), it is unclear whether the claimed improvements are statistically significant.
more of objection 2
The authors should report variance or confidence intervals for all benchmarks, or at minimum state which differences exceed the measurement noise.
The paper says, at §4.3, Table 2B
The review did not quote this passage.
Pith says: Objection 3 of 4 · major
The ablation comparing 'Outcome only' vs 'StateAlign' on constructed states shows that adding the contrastive objective reduces hallucination. However, the 'Outcome only' baseline already uses the constructed states and the same outcome reward (Eq. 1). The improvement from StateAlign could come from the contrastive objective itself or from the fact that StateAlign uses a different training configuration (e.g., the 'successful-group share' of 20% reserved for paired successful trajectories, as noted in Appendix B).
more of objection 3
The authors should clarify whether the 'Outcome only' baseline also uses this group reservation or whether it is a pure GRPO without any pairing. If the latter, the comparison conflates the effect of the contrastive objective with the effect of the group selection strategy.
The paper says, at §5.1, Figure 4
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
The qualitative example in Figure 4 is illustrative but the paper does not provide a quantitative analysis of how often the contrastive signal actually provides differentiated supervision when outcome rewards are tied. The authors mention that 'Mean first-response scores differ within every group despite identical outcome rewards' based on 50 groups, but no aggregate statistics (e.g., mean absolute deviation, proportion of tokens with non-zero scores) are reported.
more of objection 4
A more systematic analysis would strengthen the claim that the contrastive objective provides useful signal beyond outcome rewards.
8 minor objections
The paper says, at Abstract
Pith says: minor objection 1 of 8
The abstract states 'no-tool hallucination on SimpleToolHalluBench falls from 62.2% under outcome-only training to 56.1%'. It would be clearer to specify that this is the NTA (no-tool-available) setting, as the benchmark also has a DT (distractor-tool) setting.
The paper says, at §3.1, Table 5
Pith says: minor objection 2 of 8
Table 5 reports the distribution of task states in the training data. The 'Other' column for StateAlign is 0.0%, which is expected given the construction process. However, the caption says '— indicates that no such links are provided' for the Paired column; this is clear but could be stated more explicitly.
show 6 more
The paper says, at §4.1, Implementation Details
Pith says: minor objection 3 of 8
The paper states 'We train with LoRA on eight NVIDIA H100 80GB GPUs with λ=0.1'. The choice of λ=0.1 is not justified; a sensitivity analysis on this hyperparameter would be informative.
The paper says, at Appendix B, Table 6
Pith says: minor objection 4 of 8
The hyperparameter 'Successful-group share' is set to 20%. The paper should explain how this value was chosen and whether results are sensitive to it.
The paper says, at §5.2, Table 3
Pith says: minor objection 5 of 8
Table 3 reports 'Both correct' as the mean product of per-state action accuracies within each pair. This metric is not standard; the authors should clarify its interpretation and why it is preferred over simpler metrics like joint accuracy.
The paper says, at Appendix D.2
Pith says: minor objection 6 of 8
The annotation validation reports 70.4% raw agreement for condition judgments. The authors should discuss whether this level of agreement is sufficient for the claims made, especially given that the construction process uses a single model (Qwen3.8-27B) without human verification.
The paper says, at References
Pith says: minor objection 7 of 8
Several references are to 'Preprint' or 'arXiv' without indication of publication status. While this is acceptable for recent work, the authors should update references where possible (e.g., [Zhong et al., 2026] is marked as arXiv; [Zhao et al., 2026] is marked as ICML 2025 but the year in the citation is 2026).
The paper says, at Figure 1
Pith says: minor objection 8 of 8
Figure 1 plots BFCL v4 multi-turn accuracy against hallucination rates. The axes are not labeled with units (though percentages are implied). The figure would benefit from error bars or confidence regions.
What would settle it
Release the 30,433 constructed states with their condition and decision labels, then have three independent annotators re-label a stratified sample (at least 500 instances per condition, without re-weighting) using the Appendix F codebooks, and report raw per-condition agreement for INFEASIBLE and STOP. If agreement is high, retrain Qwen3-8B outcome-only and StateAlign on the subset of states whose labels are unanimously confirmed and re-run BFCL v4 multi-turn and SimpleToolHalluBench NTA/DT. If the NTA gap (62.2 vs 56.1) and the BFCL difference (53.0 vs 55.2) survive, the concern is resolved; if they shrink to noise, the central claim is not separable from construction-label noise.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 13 Nov 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 8 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The paper's central idea—constructing paired task states and using a token-level contrastive reward to score the same generated tokens under two states with different valid actions—is genuinely new relative to the cited baselines. The five-condition state construction pipeline applied to ToolMind and LoopTool is also a real contribution. The empirical evaluation is broad: five base models from three families, multiple benchmarks, and controlled ablations that isolate state construction and the contrastive objective. The behavioral analyses (multi-call execution, error recovery) are a nice touch and give the reader confidence that the method changes behavior in the intended direction.
The load-bearing premise is that the 30k constructed instances faithfully encode the five task conditions. The paper reports 70.4% raw agreement with Claude Opus on condition judgments, and the stress test correctly notes that per-condition reliability for INFEASIBLE (6.5% of the pool) and STOP (11.8%) is unreported despite driving the hallucination claim. The authors acknowledge limitations on trajectory coverage but not this per-condition label reliability issue. That said, the 97.2% population-weighted figure suggests the low raw agreement is concentrated in rare classes, and the paper does report an additional 83.3% agreement check on 60 trained-model judgments. This is a soft spot, not a fatal flaw.
Other concerns: headline tables lack variance estimates for most benchmarks (only BFCL has standard deviations in Table 2), and the code and data are linked but not pinned to a commit. These are reporting issues, not demonstrated flaws.
Who this is for: researchers working on tool-use reliability for LLM agents, especially those interested in RL-based training signals beyond outcome-only rewards. The paper deserves a serious referee. I would recommend acceptance with requests for per-condition label reliability analysis, variance estimates for all benchmarks, and a pinned code release.
The strongest objection
The load-bearing premise is that the 30,433 constructed instances in Section 3.1 faithfully encode the five task conditions. This premise feeds every training signal: Eq. (1) gives zero reward for premature execution in NEED_USER/NEED_FETCH and terminal checks in INFEASIBLE/STOP, and Eqs. (2)-(3) weight tokens by log-ratios against reference paths z, z̃ that Qwen3.8-27B synthesizes. A wrong condition label can therefore contaminate both R_out and J_pair. The only independent check, Appendix A/D.2, reports 70.4% raw agreement with Claude Opus on condition judgments; the 97.2% population-weighted figure is concentrated in frequent classes and does not reveal whether INFEASIBLE (6.5% of the pool) and STOP (11.8%) are reliably labeled. These are exactly the classes behind SimpleToolHalluBench's NTA/DT metrics and When2Call infeasibility, so the claimed 62.2 -> 56.1 NTA reduction and the large Llama NTA drop could in part be artifacts of generator-specific label noise rather than state awareness. The paper acknowledges limitations on trajectory coverage, but not this per-condition label reliability issue.
I read the paper in good faith as a method paper whose central claim is conditional on constructed states faithfully representing real task states. The evaluation is broad, the ablations are well designed, and the overlap check against BFCL and When2Call is a genuine effort to control contamination. The reader's weakest assumption already targets construction quality, and I agree with that choice. I sharpen it in one respect: the aggregate 70.4% raw / 97.2% weighted agreement gap is not decisive by itself, but the absence of per-condition agreement for the rare conditions is important because the hallucination benchmarks test exactly those rare conditions. If INFEASIBLE and STOP labels are noisy, both the outcome reward and the state-contrast reward inherit that noise, so the central reduction in no-tool hallucination could reflect fitting the generator's label distribution rather than learning state awareness. This is a robustness concern, not a demonstrated flaw, and it can be settled by a stratified annotation audit plus a retraining check on validated states. I therefore keep the reader's CONDITIONAL verdict unchanged.
The referee's report
If the results hold, the paper makes a useful contribution by demonstrating that explicitly training on constructed task states with a contrastive objective can improve both task performance and reliability in multi-turn tool use. The approach is relatively simple and the empirical evaluation is broad across models and benchmarks. The behavioral analyses (Section 5) provide some insight into how the method changes agent behavior. However, the significance is somewhat tempered by the reliance on a single generator (Qwen3.8-27B) for state construction and the modest absolute gains on some metrics.
Pith's reading of the paper (a machine's summary, not the authors')
The paper tries to establish that training a tool-using language model with constructed task states and a token-level state-contrastive reinforcement-learning objective reduces the gap between what the model reasons it should do and what it actually does when information is missing, tools are unavailable, or clarification is needed. The authors argue that outcome-only RL can improve task completion while simultaneously increasing tool hallucination — the model calls a tool with fabricated arguments even when its own reasoning identifies the missing information. They propose StateAlign, which first converts tool-use trajectories into paired training states that differ in one condition (e.g. a required value present vs. missing), then combines a standard outcome reward for task completion with a token-level contrastive reward that compares the same generated reasoning and actions under both states. The contrastive reward penalizes tokens that assume availability of information or tools that the current state does not provide. Experiments on five base models from three families show consistent improvements on BFCL v4 multi-turn accuracy, ToolSandbox similarity, When2Call accuracy, and reductions in no-tool hallucination on SimpleToolHalluBench. On Qwen3-8B, BFCL v4 multi-turn accuracy rises from 45.5% to 55.2% and no-tool hallucination falls from 62.2% under outcome-only training to 56.1%. Behavioral analyses show more consistent decisions across task states, more complete multi-call execution, and stronger recovery after tool errors. If the paper is right, then a relatively lightweight data-construction and training procedure can make tool-using LLMs substantially more reliable without sacrificing task performance — a finding that matters for deploying agents in settings where hallucinated tool calls waste resources or cause harm.
What carries the argument. The key machinery is the combination of (1) a task-state construction pipeline that converts tool-use trajectories into five types of states (EXEC, NEED_USER, NEED_FETCH, INFEASIBLE, STOP) with explicit pairs differing in one condition, and (2) a token-level state-contrastive reinforcement-learning objective J_pair that compares the same generated reasoning and actions under paired states using a fixed base model as scorer. The contrastive score for each token is the clipped log-probability ratio rt = clip(log π0(yt | h, z, p) - log π0(yt | h̃, z̃, p), -1, 1), where π0 is the frozen base model, h and h̃ are the paired contexts, z and z̃ are reference paths describing appropriate actions, and p is the shared prefix. Positive scores favor tokens that fit the current state; negative scores favor tokens that fit the paired state. J_pair weights the policy's token log-likelihoods by these scores, providing fine-grained supervision beyond outcome rewards. The outcome objective J_out is a token-normalized GRPO objective on interaction outcomes, with rewards that penalize extra calls and require appropriate continuation after state changes. The combined objective J = J_out + λ J_pair is maximized with λ = 0.1.
Core claim. The paper claims that a reinforcement-learning agent trained on tool-use tasks can simultaneously improve task completion and reduce tool hallucination if it is trained with explicit awareness of the current task state — what information is available, what tools are present, and what remains to be done. The central discovery is that outcome-only RL (rewarding only final task success) is insufficient because it can reward outputs that happen to succeed while containing reasoning or actions that assume information the state does not provide. StateAlign addresses this by constructing, from each source trajectory, five types of task states (EXEC, NEED_USER, NEED_FETCH, INFEASIBLE, STOP) that differ in a single condition, and pairing each non-EXEC state with an EXEC counterpart. The model is then trained with a combined objective J = J_out + λ J_pair, where J_out is a token-normalized GRPO objective based on interaction outcomes, and J_pair is a token-level contrastive objective that uses a fixed base model to score each generated token under both the current state and its paired state, giving positive weight to tokens that fit the current state and negative weight to tokens that fit the paired state. This contrastive signal provides supervision even when outcome rewards are tied (e.g. all rollouts in a group succeed or all fail). The paper reports that across five base models (Qwen3-4B/8B/14B, Granite-4.2-8B, Llama-3.1-8B-Instruct), StateAlign improves every BFCL v4 category and When2Call accuracy and reduces no-tool hallucination, with the largest gains on Llama-3.1-8B-Instruct (BFCL multi-turn from 8.0% to 26.1%, NTA hallucination from 72.6% to 15.7%).
The premise it rests on. The load-bearing premise is that the Qwen3.8-27B-generated state construction produces training states that faithfully represent real task states, with no leaked values, no unrealistic retrieval tools, and correct action labels — the paper acknowledges this by reporting validation checks (70.4% agreement with Claude Opus on condition judgments, 90.0% on decisions) but if construction quality is lower in distribution than the validation sample, the outcome and contrastive rewards would encode artifacts rather than state awareness, weakening all downstream benchmark claims.
If the paper is right (Pith's inferences, not the authors')
If StateAlign generalizes, then training tool-using LLMs with state-contrastive rewards could become a standard component of agentic RL pipelines, replacing or supplementing outcome-only rewards to reduce hallucination without sacrificing task performance.
The method's reliance on a fixed base model as scorer means that the contrastive signal is available from the start of training, potentially accelerating learning compared with methods that require a learned reward model or environment dynamics.
The explicit construction of five task states (EXEC, NEED_USER, NEED_FETCH, INFEASIBLE, STOP) provides a template for data augmentation in tool-use training that other researchers can adopt and extend to new conditions (e.g. ambiguous requests, multiple missing values, tool errors).
The behavioral improvements — more consistent decisions, more complete multi-call execution, and stronger recovery after tool errors — suggest that state-aware training may reduce the need for hand-crafted fallback logic in deployed agents.
The paper's finding that outcome-only RL can increase hallucination while improving task performance implies that benchmark scores alone are insufficient for evaluating tool-use agents; reliability metrics (e.g. hallucination rates, clarification rates) should be reported alongside accuracy.
StateAlign's approach of contrasting paired states could be extended to other agentic domains beyond tool use, such as web navigation or robotics, where the distinction between a state that permits execution and one that requires information-gathering or declining is equally critical.
The paper's use of a fixed base model as scorer (rather than a learned reward model) suggests a family of methods where a weaker or frozen model provides contrastive supervision to a stronger policy — a form of self-distillation that may be applicable to other RL-from-human-feedback settings.
The five task-state categories are defined by the paper's construction pipeline, but the underlying principle — that an agent should be trained to distinguish states that differ in a single relevant condition — could be formalized as a general inductive bias for reliable agentic behavior.
The paper's observation that models can reason correctly but act incorrectly (e.g. state in reasoning that a value is missing, then call a tool with a fabricated value) points to a decoupling between reasoning and action that state-contrastive training directly addresses; this decoupling may be a general phenomenon in LLM agents that warrants further study.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Reinforcement learning has improved the tool-use capabilities of large language models. However, better task performance can coincide with increased tool hallucination. Reliable tool use requires decisions to follow the current task state: available information, tool capabilities, and task progress. Our analysis shows that agents sometimes execute when they should ask for information, retrieve it, or decline an infeasible request, even when reasoning identifies missing information. We introduce StateAlign, combining task-state construction with state-contrastive reinforcement learning. The data cover five task conditions and pair states with different action requirements. Outcome rewards guide task completion; token-level contrasts compare the same generated reasoning and actions under paired states to train outputs that fit the current state. Experiments across five base models from three families show improvements on BFCL v4 multi-turn, ToolSandbox, and When2Call. Ablations show that state construction improves performance and reliability under different reward designs. The complete method further reduces tool hallucination across three Qwen3 sizes. On Qwen3-8B, StateAlign improves BFCL v4 multi-turn accuracy from 45.5% to 55.2% and reduces no-tool hallucination on SimpleToolHalluBench from 62.2% under outcome-only training to 56.1%. Behavioral analyses show more consistent decisions, more complete multi-call execution, and stronger recovery after tool errors.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-13
The authors' main claim, as the classifier picked it out 0.34
“Experiments across five base models from three families show improvements on BFCL v4 multi-turn, ToolSandbox, and When2Call.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.17 · human participants 0.03 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.95 · mentions code or data availability 0.03
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Reinforcement learning has improved the tool-use capabilities of large language models. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| However, better task performance can coincide with increased tool hallucination. | background 0.79 | 0.14 | 0.03 | asserted 0.50 |
| Reliable tool use requires decisions to follow the current task state: available information, tool capabilities, and task progress. | background 0.99 | 0.07 | 0.08 | asserted 1.00 |
| Our analysis shows that agents sometimes execute when they should ask for information, retrieve it, or decline an infeasible request, even when reasoning identifies missing information. | result 0.63 | 0.91 | 0.07 | asserted 0.97 |
| We introduce StateAlign, combining task-state construction with state-contrastive reinforcement learning. | contribution 0.83 | 0.87 | 0.09 | asserted 0.98 |
| The data cover five task conditions and pair states with different action requirements. | method 0.99 | 0.84 | 0.02 | asserted 0.99 |
| Outcome rewards guide task completion; token-level contrasts compare the same generated reasoning and actions under paired states to train outputs that fit the current state. | method 1.00 | 0.87 | 0.05 | asserted 0.96 |
| Experiments across five base models from three families show improvements on BFCL v4 multi-turn, ToolSandbox, and When2Call. | result 1.00 | 0.95 | 0.20 | asserted 1.00 |
| Ablations show that state construction improves performance and reliability under different reward designs. | result 0.99 | 0.95 | 0.08 | asserted 1.00 |
| The complete method further reduces tool hallucination across three Qwen3 sizes. | result 1.00 | 0.94 | 0.25 | asserted 1.00 |
| On Qwen3-8B, StateAlign improves BFCL v4 multi-turn accuracy from 45.5% to 55.2% and reduces no-tool hallucination on SimpleToolHalluBench from 62.2% under outcome-only training to 56.1%. | result 1.00 | 0.95 | 0.18 | asserted 1.00 |
| Behavioral analyses show more consistent decisions, more complete multi-call execution, and stronger recovery after tool errors. | result 0.91 | 0.94 | 0.19 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-13: 1 request, 8,734 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (10)
Figures are the authors', reproduced from the paper.
References (56)
- [1] Zhou, Yifei and Zanette, Andrea and Pan, Jiayi and Levine, Sergey and Kumar, Aviral , booktitle =. 2024 , editor = on Pith
- [3] Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle =. 2023 , url = on Pith
- [7] Qin, Yujia and Liang, Shihao and Ye, Yining and Zhu, Kunlun and Yan, Lan and Lu, Yaxi and Lin, Yankai and Cong, Xin and Tang, Xiangru and Qian, Bill and Zhao, Sihan and Hong, Lauren and Tian, Runchu and Xie, Ruobing and Zhou, Jie and Gerstein, Mark and Li, Dahai and Liu, Zhiyu... on Pith
- [9] Liu, Weiwen and Huang, Xu and Zeng, Xingshan and Hao, Xinlong and Yu, Shuai and Li, Dexun and Wang, Shuai and Gan, Weinan and Liu, Zhengying and Yu, Yuanqing and Wang, Zezhong and Wang, Yuxian and Ning, Wu and Hou, Yutai and Wang, Bin and Wu, Chuhan and Wang, Xinzhi and Liu, Y... on Pith
- [11] Shaokun Zhang and Yi Dong and Jieyu Zhang and Jan Kautz and Bryan Catanzaro and Andrew Tao and Qingyun Wu and Zhiding Yu and Guilin Liu , booktitle = on Pith
- [12] Feng, Jiazhan and Huang, Shijue and Qu, Xingwei and Zhang, Ge and Qin, Yujia and Zhong, Baoquan and Jiang, Chengquan and Chi, Jinxin and Zhong, Wanjun , booktitle = on Pith
- [19] Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following , author =. 2026 , eprint = on Pith
- [20] Kong, Fanqi and Zhang, Jiayi and Deng, Mingyi and Wu, Chenglin and Luo, Yuyu and Liu, Bang , booktitle =. 2026 , editor = on Pith
- [21] Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author =. Proceedings of the 43rd International Conference on Machine Learning , pages =. 2026 , editor = on Pith
- [22] Agent World Model: Infinity Synthetic Environments for Agentic Reinforcement Learning , author=. Proceedings of the 43rd International Conference on Machine Learning , year= on Pith
- [23] Patil, Shishir G and Mao, Huanzhi and Yan, Fanjia and Ji, Charlie Cheng-Jie and Suresh, Vishnu and Stoica, Ion and Gonzalez, Joseph E. , booktitle=. The Berkeley Function Calling Leaderboard (. 2025 , volume= on Pith
- [26] Zeng, Xingshan and Liu, Weiwen and Wang, Lingzhi and Li, Liangyou and Mi, Fei and Wang, Yasheng and Shang, Lifeng and Jiang, Xin and Liu, Qun , booktitle =. 2026 , url = on Pith
- [27] Reducing Tool Hallucination via Reliability Alignment , author =. Proceedings of the 42nd International Conference on Machine Learning , pages =. 2025 , volume = on Pith
- [28] An Yang and Anfeng Li and Baosong Yang and Beichen Zhang and Binyuan Hui and Bo Zheng and Bowen Yu and Chang Gao and Chengen Huang and Chenxu Lv and Chujie Zheng and Dayiheng Liu and Fan Zhou and Fei Huang and Feng Hu and Hao Ge and Haoran Wei and Huan Lin and Jialong Tang and... on Pith
- [29] Dubey, Abhimanyu and Jauhri, Abhinav and Pandey, Abhinav and Kadian, Abhishek and Al-Dahle, Ahmad and Letman, Aiesha and Mathur, Akhil and Schelten, Alan and Yang, Amy and Fan, Angela and Goyal, Anirudh and Hartshorn, Anthony and Yang, Aobo and Mitra, Archi and Sravankumar, Ar... on Pith
- [30] Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning , author =. 2026 , eprint = on Pith
- [35] Hu, Edward J. and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle =. 2022 , url = on Pith
- [36] Peng, Bowen and Quesnelle, Jeffrey and Fan, Honglu and Shippole, Enrico , booktitle =. 2024 , url = on Pith
- [38] Emre Can Acikgoz, Jeremiah Greer, Akul Datta, Ze Yang, William Zeng, Oussama Elachqar, Emmanouil Koukoumidis, Dilek Hakkani-T \"u r, and Gokhan Tur. 2025. https://doi.org/10.18653/v1/2025.acl-long.605 Can a single model master both multi-turn conversations and tool use? C o AL... on Pithdoi:10.18653/v1/2025.acl-long.605
- [39] Yun-Shiuan Chuang, Chaitanya Kulkarni, Alec M. Chiu, Avinash Thangali, Zijie Pan, Shivani Shekhar, Yirou Ge, Yixi Li, Uma Kona, Linsey Pang, and Prakhar Mehrotra. 2026. https://doi.org/10.18653/v1/2026.acl-industry.87 Toward scalable verifiable reward: Proxy state-based evalua... on Pithdoi:10.18653/v1/2026.acl-industry.87
show 36 more
- [40] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zha... on PithPith's reviewarXiv
- [41] Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2026. https://proceedings.iclr.cc/paper_files/paper/2026/file/4038c9208dfc22644c60ad39c24e5c53-Paper-Conference.pdf ReTool : Reinforcement learning for st... on Pith
- [42] Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. 2025. https://doi.org/10.52202/085713-1544 Group-in-group policy optimization for LLM agent training . In Advances in Neural Information Processing Systems, volume 38, Main Conference, pages 46375--46408. Curran Associates, Inc on Pithdoi:10.52202/085713-1544
- [43] Rongfeng Guo, Yinxuan Huang, Yusen Wu, Maoqing Zhong, Yunlu Chen, Meng Tang, Teng Long, and Vincent Tao Hu. 2026. https://arxiv.org/abs/2608.24368 From state to action: OODA-Tool for reliable multi-turn tool use . Preprint, arXiv:2608.24368 on PithPith's reviewarXiv
- [44] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 LoRA : Low-rank adaptation of large language models . In International Conference on Learning Representations on Pith
- [45] IBM Research . 2026. https://huggingface.co/blog/ibm-granite/granite-4-2 Granite 4.2 language models . Accessed: 2026-10-18 on Pith
- [46] Fanqi Kong, Jiayi Zhang, Mingyi Deng, Chenglin Wu, Yuyu Luo, and Bang Liu. 2026. https://proceedings.mlr.press/v306/kong26g.html I nfo PO : Information-driven policy optimization for user-centric agents . In Proceedings of the 43rd International Conference on Machine Learning,... on Pith
- [47] Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, Zezhong Wang, Yuxian Wang, Wu Ning, Yutai Hou, Bin Wang, Chuhan Wu, Xinzhi Wang, Yong Liu, Yasheng Wang, and 8 others. 2025. https://proceedings.iclr.cc/pa... on Pith
- [48] Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, and Caiming Xiong. 2024. https://doi.org/10.52202/079017-1... on Pithdoi:10.52202/079017-1725
- [49] Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haoping Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, and Ruoming Pang. 2025. https://doi.org/10.18653/v1/2025.findings-naacl.65 ToolSandbox : A stateful, conversational, interactive evaluation be... on Pithdoi:10.18653/v1/2025.findings-naacl.65
- [50] Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. https://proceedings.mlr.press/v267/patil25a.html The berkeley function calling leaderboard ( BFCL ): From tool use to agentic evaluation of large language m... on Pith
- [51] Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2024. https://openreview.net/forum?id=wHBfxhZu1u YaRN : Efficient context window extension of large language models . In The Twelfth International Conference on Learning Representations on Pith
- [52] Akshara Prabhakar, Zuxin Liu, Ming Zhu, Jianguo Zhang, Tulika Manoj Awalgaonkar, Shiyu Wang, Zhiwei Liu, Haolin Chen, Thai Hoang, Juan Carlos Niebles, Shelby Heinecke, Weiran Yao, Huan Wang, Silvio Savarese, and Caiming Xiong. 2025. https://doi.org/10.52202/085713-2184 APIGen-... on Pithdoi:10.52202/085713-2184
- [53] Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-Tur, Gokhan Tur, and Heng Ji. 2025. https://doi.org/10.52202/085713-3524 ToolRL : Reward is all tool learning needs . In Advances in Neural Information Processing Systems, volume 38, Main Conference, p... on Pithdoi:10.52202/085713-3524
- [54] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Lauren Hong, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. https://proceedings.iclr.cc/paper_files/pa... on Pith
- [55] Hayley Ross, Ameya Sunil Mahabaleshwarkar, and Yoshi Suhara. 2025. https://doi.org/10.18653/v1/2025.naacl-long.174 When2Call : When (not) to call tools . In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguisti... on Pithdoi:10.18653/v1/2025.naacl-long.174
- [56] Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. https://doi.org/10.52202/075280-2997 Toolformer: Language models can teach themselves to use tools . In Advances in Neural Inf... on Pithdoi:10.52202/075280-2997
- [57] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 DeepSeekMath : Pushing the limits of mathematical reasoning in open language models . Preprint, arXiv:2402.03300 on PithPith's reviewarXiv
- [58] Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. 2025. https://arxiv.org/abs/2505.01441 Agentic reasoning and tool integration for LLMs via reinforcement learning . Preprint, arXiv:2505.01441 on PithPith's reviewarXiv
- [59] Zhaoyang Wang, Canwen Xu, Boyi Liu, Yite Wang, Siwei Han, Zhewei Yao, Huaxiu Yao, and Yuxiong He. 2026 a . https://proceedings.mlr.press/v306/wang26jh.html Agent world model: Infinity synthetic environments for agentic reinforcement learning . In Proceedings of the 43rd Intern... on Pith
- [60] Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, and Dakuo Wang. 2026 b . https://doi.org/10.18653/v1/2026.acl-long.2037 Trajectory2Task : Training robust tool-calling... on Pithdoi:10.18653/v1/2026.acl-long.2037
- [61] Hongshen Xu, Zichen Zhu, Lei Pan, Zihan Wang, Su Zhu, Da Ma, Ruisheng Cao, Lu Chen, and Kai Yu. 2025. https://proceedings.mlr.press/v267/xu25ap.html Reducing tool hallucination via reliability alignment . In Proceedings of the 42nd International Conference on Machine Learning,... on Pith
- [62] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025 a . https://arxiv.org/abs/2505.09388... on PithPith's reviewarXiv
- [63] Chen Yang, Ran Le, Yun Xing, Zhenwei An, Zongchao Chen, Wayne Xin Zhao, Yang Song, and Tao Zhang. 2025 b . https://arxiv.org/abs/2511.15718 ToolMind technical report: A large-scale, reasoning-enhanced tool-use dataset . Preprint, arXiv:2511.15718 on PitharXiv
- [64] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE_vluYUL-X ReAct : Synergizing reasoning and acting in language models . In International Conference on Learning Representations on Pith
- [65] Chenlong Yin, Zeyang Sha, Shiwen Cui, Changhua Meng, and Zechao Li. 2026. https://doi.org/10.18653/v1/2026.acl-long.376 The reasoning trap: How enhancing LLM reasoning amplifies tool hallucination . In Proceedings of the 64th Annual Meeting of the Association for Computational... on Pithdoi:10.18653/v1/2026.acl-long.376
- [66] Zhenrui Yue, Huimin Zeng, Yueqi Wang, Yaokun Liu, Fengran Mo, Jinghan Zhang, Mung Yao Jia, Gyuseok Lee, Yang Zhang, Na Wei, and Dong Wang. 2026. https://arxiv.org/abs/2609.36742 SIPO : Unifying reinforcement learning with on-policy self-distillation . Preprint, arXiv:2609.36742 on PithPith's reviewarXiv
- [67] Xingshan Zeng, Weiwen Liu, Lingzhi Wang, Liangyou Li, Fei Mi, Yasheng Wang, Lifeng Shang, Xin Jiang, and Qun Liu. 2026. https://openreview.net/forum?id=KznJt9Fhjc ToolACE-MT : Non-autoregressive generation for agentic multi-turn interaction . In International Conference on Lea... on Pith
- [68] Kangning Zhang, Wenxiang Jiao, Kounianhua Du, Yong Yu, Yuan Lu, Weiwen Liu, and Weinan Zhang. 2026 a . https://doi.org/10.18653/v1/2026.acl-long.968 L oop T ool: Closing the data -- training loop for robust LLM tool calls . In Proceedings of the 64th Annual Meeting of the A ss... on Pithdoi:10.18653/v1/2026.acl-long.968
- [69] Shaokun Zhang, Yi Dong, Jieyu Zhang, Jan Kautz, Bryan Catanzaro, Andrew Tao, Qingyun Wu, Zhiding Yu, and Guilin Liu. 2026 b . https://arxiv.org/abs/2505.00024 Nemotron-Research-Tool-N1 : Exploring tool-using language models with reinforced reasoning . In International Conferen... on PithPith's reviewarXiv
- [70] Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. 2026. https://proceedings.mlr.press/v306/zhao26be.html Self-distilled reasoner: On-policy self-distillation for large language models . In Proceedings of the 43rd International Conferen... on Pith
- [71] Weikang Zhao, Xili Wang, Chengdi Ma, Lingbin Kong, Zhaohua Yang, Mingxiang Tuo, Xiaowei Shi, Yitao Zhai, and Xunliang Cai. 2025. https://arxiv.org/abs/2508.18669 MUA-RL : Multi-turn user-interacting agent reinforcement learning for agentic tool use . Preprint, arXiv:2508.18669 on PithPith's reviewarXiv
- [72] Yanzhao Zheng, Yuanqiang Yu, Tianze Xu, Chao Ma, Zhentao Zhang, Jihuai Zhu, Baohua Dong, Hangcheng Zhu, and Ruohui Huang. 2026. https://arxiv.org/abs/2609.27421 Counterfactual constraint-conditioned on-policy distillation for multi-constraint instruction following . Preprint, ... on PithPith's reviewarXiv
- [73] Haitian Zhong, Jixiu Zhai, Lei Song, Jiang Bian, Qiang Liu, and Tieniu Tan. 2026. https://arxiv.org/abs/2602.03025 RC-GRPO : Reward-conditioned group relative policy optimization for multi-turn tool calling agents . Preprint, arXiv:2602.03025 on PitharXiv
- [74] Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. 2024. https://proceedings.mlr.press/v235/zhou24t.html A r CH er: Training language model agents via hierarchical multi-turn RL . In Proceedings of the 41st International Conference on Machine Learning, vol... on Pith
- [75] Yijin Zhou, Linqian Zeng, Xiaoya Lu, Wenyuan Xie, Dongrui Liu, Junchi Yan, and Jing Shao. 2026. https://arxiv.org/abs/2606.06976 Exploring agentic tool-calling decisions via uncertainty-aligned reinforcement learning . Preprint, arXiv:2606.06976. Accepted to EMNLP 2026 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment