{"id":"ba0fed68-5c9b-4df8-a27f-7b29179256ce","arxiv_id":"2412.04141","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A reliability alignment framework, Relign, that expands the LLM tool-use action space with indecisive actions reduces tool hallucination rates and improves task success on the new RelyToolBench benchmark.","lead":"This paper defines a taxonomy of tool hallucinations in LLMs, builds a benchmark and metrics to measure them, and trains models to ask for clarification or switch tools instead of calling the wrong tool. If the results hold, it could make AI agents that use external tools significantly more reliable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline hallucination reduction may partly reflect alignment to the GPT-4o judge that both labels the training preference data and computes the Table 2 metrics; Appendix A human validation covers only 150 cases and omits the answer-correlation step used by RePR.","rationale":"The reader's weakest assumption identifies the central risk: the GPT-4o judge is both the training labeler and the final evaluator, so the reported hallucination reduction could partly reflect overfitting to that judge. I agree with this concern. The paper's own Appendix A partially mitigates it, but the human check is small and does not cover the answer-correlation step that RePR depends on, nor the exact DPO labeling step. The apparent utility metric sign ambiguity and the absence of error bars are real secondary issues, but they do not threaten the hallucination-reduction claim as directly as judge dependence. The concrete human re-annotation test would settle whether the concern lands. Because this is a validation gap rather than a demonstrated contradiction, the reader's CONDITIONAL verdict remains appropriate; no change is needed.","tokens_in":17900,"tokens_out":4984,"duration_ms":53176,"concrete_test":"Independently re-annotate a stratified sample of 300 RelyToolBench trajectories, 75 from each subset, for LLaMA-3.1-8B baseline and +Relign, using three human annotators and the paper's four hallucination subtypes plus the Appendix D answer-correlation rule. Compute hallucination rate and RePR from the human labels only. If the human-measured Relign-minus-baseline reduction is more than 10 percentage points smaller than the GPT-4o-measured reduction in Table 2, or is not statistically significant at the task level, the headline claim is not established independently of the judge.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2.2 builds DPO preference pairs using GPT-4o as a hallucination evaluator, and Section 4.1 states that 'evaluations regarding tool hallucinations and task success were performed using the GPT-4O model.' The central claim in Table 2, that on LLaMA-3.1-8B the hallucination rate falls from 50.8 to 14.6 and RePR rises from 65.3 to 77.2, is therefore measured by the same model used to create the training signal. If GPT-4o has systematic biases, such as treating certain legitimate tool calls as hallucinated or accepting some fabricated parameters as grounded, Relign can be rewarded for matching those biases rather than for reducing true hallucinations. The reported human check in Appendix A validates only three categories on 150 cases, namely no hallucination, parameter value hallucination, and tool relevance hallucination; it does not validate timing or format hallucination judgments, and it does not validate the Appendix D answer-correlation step that converts tool hallucinations into the task hallucination rate subtracted in RePR. Because RePR is the headline reliability metric, this missing validation is load-bearing. This is a validation gap, not evidence of misconduct; the multi-model consistency and public code are real supporting evidence, but they do not break the dependence on a single judge.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper addresses tool hallucination in LLM tool-use by (1) proposing a taxonomy (tool selection vs. tool usage, with four subtypes), (2) introducing RelyToolBench, a benchmark with missing-parameter and unmatched-tools subsets, along with two new metrics—Reliable Pass Rate (RePR) and Benefit-Cost Utility, and (3) presenting Relign, a reliability alignment framework that augments the tool-use action space with indecisive actions (ChangeTools, TalkToUser) and trains via SFT and DPO. Experiments on ToolLLaMA, LLaMA-3.1-8B, and Qwen2.5-7B report large reductions in tool hallucination rate (e.g., LLaMA-3.1: 50.8% to 14.6%) and improvements in RePR (65.3% to 77.2%), plus an OOD evaluation on APIBench.","tokens_in":18189,"tokens_out":6344,"duration_ms":61261,"significance":"If the results hold, the paper makes a useful contribution: it provides a systematic taxonomy of tool hallucinations, a benchmark that stresses failure modes not covered by existing tool benchmarks, and a practical training approach that yields substantial improvements across multiple base models. The release of code and data, the multi-model consistency, and the OOD generalization experiment are concrete strengths. The significance is tempered, however, by the fact that the headline claims rest on a single LLM evaluator (GPT-4o) that is also used to construct the DPO training signal, and the human validation of that evaluator is narrow. The metric definitions also need correction. These issues are addressable but currently leave the central quantitative claims less certain than the paper suggests.","major_comments":[{"comment":"The same GPT-4o model is used both to label hallucination types for DPO preference construction (§3.2.2) and to compute the reported tool hallucination rate and RePR (§4.1). The human validation in Appendix A covers only 150 cases across three categories (no hallucination, parameter value hallucination, tool relevance hallucination) and does not validate timing or format hallucinations, nor does it validate the Appendix D answer-correlation step that converts tool hallucinations into the task hallucination rate subtracted in RePR. Since RePR and the hallucination-rate reductions are the paper's central claims, this evaluator circularity is load-bearing. I recommend validating the evaluator with a second independent LLM or a larger human sample covering all four subtypes and the answer-correlation step, and reporting agreement statistics separately for each component.","section":"§3.2.2, §4.1, Appendix A"},{"comment":"The definition of P_hallucination is internally inconsistent. The text states 'P_hallucination is the penalty for hallucinations in the task (-10 for hallucination)' but the formula subtracts P_hallucination. If P_hallucination is negative, subtracting it adds to the utility, which would reward hallucinations; if the intended penalty is a nonnegative magnitude of 10, the parenthetical is misleading. Please clarify the sign convention and recompute the Utility values in Table 2 under a consistent definition, or explain why the current convention is correct.","section":"§2.2.2, Eq. (Utility)"},{"comment":"The table caption reads 'Utility: Benifit-cost utility↓' while the column header and the text indicate that higher Utility is better (GPT-4o scores 13.4 vs. 5.2 for GPT-3.5-turbo, and the text describes utility as a measure to be maximized). This direct contradiction in the direction of the arrow makes the utility results ambiguous. Please correct the caption and ensure the arrow direction matches the text and the reported numbers.","section":"§4.2, Table 2 caption"}],"minor_comments":[{"comment":"The word 'Benifit' is misspelled; it should be 'Benefit'.","section":"§2.2.2"},{"comment":"The term 'Acoreect' appears to be a typo for 'Acorrect'.","section":"§3.2.2"},{"comment":"The phrase 'For computing both benefit-cost utility' is ungrammatical; it should read 'For computing the benefit-cost utility'.","section":"§4.1"},{"comment":"The sentence 'the training data we selected involves only a single tool' is ambiguous—does it mean each training example contains only one tool call, or that the examples are restricted to a single-tool task? Please clarify.","section":"§3.2.1"},{"comment":"The human evaluation is described as validating 'outputs generated by GPT-4, GPT-3.5, and ToolLLaMA3.1,' but the paper's evaluator is GPT-4o; please specify which model's outputs were judged and whether the human agreement applies directly to the GPT-4o-based pipeline.","section":"Appendix A"},{"comment":"The column header 'Utility↑' is inconsistent with the caption's 'Benifit-cost utility↓' (see major comment); beyond the direction, consider using a consistent notation such as 'Utility ↑' and 'Tool Hallu ↓' in both the header and caption.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea—adding indecisive actions and training with preference data to reduce tool hallucination—is interesting and the experimental setup is largely standard, but the evaluator circularity is a real concern for the headline numbers. The human validation currently covers only a small subset of the evaluation pipeline, and the utility metric definition has a sign ambiguity. These are fixable with additional experiments or clear corrections, so I do not recommend rejection, but the authors should be required to address them before the claims are accepted as stated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a solid, practical paper that does something real. It defines a two-level tool hallucination taxonomy, builds RelyToolBench with two adversarial subsets (missing parameters, unmatched tools), and proposes Relign, which lets the model abstain or ask the user rather than forcing a tool call. The reported gains are large: on LLaMA-3.1-8B, tool hallucination rate drops from 50.8 to 14.6, and RePR goes from 65.3 to 77.2. I've read the setup and the consistency across three open models plus a small OOD test on APIBench makes me believe the method actually does something.\n\nWhat's genuinely new: the indecisive action space is a good idea, and prior work like StepTool and ToolLLM doesn't have it. The two stress-test subsets are a reasonable way to probe failure modes. The paper also ships code and data, and the Discussion is unusually honest — it admits that more training data doesn't monotonically reduce hallucinations, which suggests they're not hiding the messy bits.\n\nSoft spots, in proportion. The load-bearing one is the evaluator. GPT-4o is used both to label the DPO preference data and to compute the reported hallucination rates and RePR in Table 2. The human check in Appendix A covers only 150 cases, tests only three of the four hallucination categories, and does not validate the answer-correlation step (Appendix D) that converts tool hallucination into task hallucination for RePR. That step matters because RePR is the headline metric. This doesn't mean the results are fake — the multi-model consistency is real evidence — but it is a genuine validation gap, and it should be fixed before I'd trust the numbers as firm. Also, the Benefit-Cost Utility formula is confusing: Phallucination is described as '-10 for hallucination' but then subtracted, which suggests either the sign is flipped or the score bounds are different from what the text implies. Minor: no error bars or significance tests anywhere; with a handful of runs per condition that's not disqualifying, but it would make the claim stronger.\n\nWho this is for: anyone working on tool-use reliability, agent evaluation, or hallucination mitigation. It deserves a serious referee — not a desk reject — and my own verdict would be conditional accept rather than clean accept. The fixes are manageable: validate the answer-correlation step, run a second judge or a blind human sample, clarify the utility formula, and report variance. I'd cite it.","headline":"Solid, practical contribution on tool hallucination that deserves engagement; the main caveat is a real evaluator-dependence gap that revision should close.","tokens_in":18755,"tokens_out":2328,"would_cite":true,"duration_ms":23404,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Giving LLMs an explicit option to defer or switch tools cuts tool-call hallucinations by roughly 70 percent and raises reliable task completion.","keywords":["tool hallucination","reliability alignment","tool learning","indecisive action space","preference optimization","RelyToolBench","Reliable Pass Rate","LLM agents"],"falsifier":"Have human annotators independently label a random sample of RelyToolBench trajectories, including the final-answer correlation step, for the LLaMA-3.1 baseline and LLaMA-3.1 plus Relign; if the human labels do not reproduce the reported 50.8% to 14.6% hallucination drop and 65.3 to 77.2 RePR gain within reasonable error bars, the claim that Relign reduces hallucinations is not established.","tokens_in":17690,"feed_emoji":"🛠️","tokens_out":9081,"duration_ms":77706,"temperature":0.7,"pith_summary":"The paper claims that a large share of LLM tool-use failures are avoidable hallucinations, and that models can be trained out of them by letting the model decline to call a tool when preconditions are unmet. It defines two hallucination families—selection (wrong or mistimed tool) and usage (wrong format or fabricated parameter content)—and builds RelyToolBench, a benchmark with missing-parameter and unmatched-tool cases, on top of StableToolBench. Its Relign method adds 'change tool' and 'talk to user' to the action space, trains with supervised fine-tuning plus direct preference optimization, and reports that on LLaMA-3.1-8B the tool hallucination rate drops from 50.8% to 14.6% while the reliable pass rate rises from 65.3% to 77.2%. The payoff, if true, is that reliability can be trained into tool-using LLMs at modest data scale rather than engineered around after the fact.","feed_headline":"An 'ask or switch' escape hatch cuts tool hallucinations 50.8%→14.6%","feed_subtitle":"A reliability-alignment training recipe lifts reliable pass rate from 65.3 to 77.2 percent on LLaMA-3.1-8B.","key_machinery":"The load-bearing mechanism is the indecisive action space: two deliberately non-decisive actions, ChangeTools and TalkToUser, that the model may output when tool preconditions are unmet. The paper also introduces RePR, the Reliable Pass Rate, defined as pass rate minus task hallucination rate, which discounts tasks whose final answers are contaminated by hallucinated tool calls. The preference hierarchy used to build DPO pairs—correct call preferred over indecisive action preferred over hallucinated call—is what turns the expanded space into a training signal.","core_discovery":"The paper's central claim is that tool hallucination is better modeled as a decision under missing preconditions than as a generic text-generation error, and that the remedy is to enlarge the model's action space so that abstention is a legitimate move. In Relign the model may still call the tool correctly, but may also invoke ChangeTools or TalkToUser, and the alignment objective orders trajectories as success > failure > hallucination, and actions as correct call > indecisive action > hallucinated call. Using SFT to teach the two new actions and DPO with preference pairs synthesized by a hallucination detector, the trained models reduce hallucination rates and tool-call counts across RelyToolBench, and the improvement transfers to APIBench without retraining on that benchmark.","pith_inferences":["Inference: the abstention hierarchy is not tool-specific; any sequential decision task with detectable missing preconditions, such as web browsing, database queries, or robot commands, could use the same correct > indecisive > hallucinated preference ordering with domain-appropriate fallback actions.","Inference: because GPT-4o labels the preference pairs and scores the final metrics, the reported improvement could partly reflect overfitting to one judge; the paper's human validation covers only 150 cases and not the answer-correlation step that RePR relies on.","Inference: a natural testable extension is to re-run Relign's DPO pipeline with an open-weight judge or human preference labels; if the hallucination drop persists, the method is robust to judge choice, and if not, the measured gains are partially judge-fitting.","Inference: RePR treats hallucinated-call-to-answer correlation as a binary discount; future work could relax this to partial credit and estimate error bars on the discounted pass rate."],"forward_implications":["A model trained with Relign on LLaMA-3.1-8B cuts tool hallucination from 50.8% to 14.6% overall, and the reliable pass rate rises from 65.3% to 77.2%.","Relign lowers average tool calls per task (for example, LLaMA-3.1 goes from 2.2 to 1.5), so reliability gains also reduce compute and latency.","RePR is consistently below the original pass rate, showing that ordinary task success metrics overstate how often a task was completed without hallucination-tainted steps.","The gains transfer out of distribution: on APIBench, Relign reduces hallucination and improves tool-calling accuracy across HuggingFace, TorchHub, and TensorHub retrievals.","The paper's scaling analysis finds that larger models reduce hallucination under fixed data, while simply adding more ToolBench data does not, because that data lacks failure-case examples."],"supporting_citations":[{"why":"Supplies the stable evaluation environment and solvable subsets from which RelyToolBench is derived, and provides the StableToolBench evaluation setup reused in Section 4.1.","marker":"(Guo et al., 2024)"},{"why":"Provides ToolBench training data and the ToolLLaMA-7B base model, plus the pass-rate evaluation method that RePR refines.","marker":"(Qin et al., 2023b)"},{"why":"Supplies DPO, the preference-optimization objective that Relign uses after SFT to order correct, indecisive, and hallucinated actions.","marker":"(Rafailov et al., 2024)"},{"why":"StepTool is the main baseline; the RLHF and StepTool rows in Table 2 come from this work's step-level reward setup.","marker":"(Yu et al., 2024)"},{"why":"LLaMA-3.1-8B-Instruct is one of the three base models on which Relign is evaluated.","marker":"(Dubey et al., 2024)"},{"why":"Qwen2.5-7B-Instruct is another base model in the experiments and in the scaling analysis.","marker":"(Yang et al., 2024)"}],"fun_headline_variants":["Asking or switching tools cuts LLM hallucination from 50.8 to 14.6%","Relign: reliability alignment that cuts tool misuse to 14.6%","Letting LLMs defer or ask cuts tool errors by 71%","Tool hallucination down 71% via enlarged action space","New alignment method reduces tool hallucination to 14.6%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything hinges on GPT-4o judging hallucinations correctly, both when labeling training data and when scoring results, and the human check only covers 150 cases and not the step that decides whether a hallucinated call also corrupted the final answer.","fun_headline_variants_meta":{"raw":{"variants":["Asking or switching tools cuts LLM hallucination from 50.8 to 14.6%","Relign: reliability alignment that cuts tool misuse to 14.6%","Letting LLMs defer or ask cuts tool errors by 71%","Tool hallucination down 71% via enlarged action space","New alignment method reduces tool hallucination to 14.6%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000786,"raw_usage":{"total_tokens":3431,"prompt_tokens":872,"completion_tokens":2559,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":2459}},"tokens_in":488,"tokens_out":2559,"duration_ms":18915,"temperature":1.0,"reasoning_tokens":2459,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T21:42:40.994899+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have human annotators independently label a random sample of RelyToolBench trajectories, including the final-answer correlation step, for the LLaMA-3.1 baseline and LLaMA-3.1 plus Relign; if the human labels do not reproduce the reported 50.8% to 14.6% hallucination drop and 65.3 to 77.2 RePR gain within reasonable error bars, the claim that Relign reduces hallucinations is not established.","supporting_citations":[],"review_version":1}