Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use

Ahmed Awadallah; Akshay Nambi; Aradhye Agarwal; Gurdit Siyan; Joykirat Singh; Yash Pandya

REVIEW 4 major objections 4 minor 2 cited by

Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use

T0 review · 4 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MOSAIC makes safety an explicit, trainable decision in agentic tool use, cutting harmful behavior by up to 50% while preserving utility.

desk verdict A useful training recipe with solid ablations, but the abstract overstates the numbers and the judge-in-the-loop evaluation blurs the safety gains. read the letter →

arxiv 2603.03205 v2 pith:PIOHWAR6 submitted 2026-03-03 cs.CL

Aradhye Agarwal , Gurdit Siyan , Yash Pandya , Joykirat Singh , Akshay Nambi , Ahmed Awadallah This is my paper

classification cs.CL

keywords agenticsafetytoolusepromptinjectionrefusalpreference-basedreinforcementlearningLLM-as-a-judgereasoningpost-training

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that agentic language models are unsafe not because they lack reasoning but because safety is handled implicitly, hidden inside long traces and scalar rewards that cannot tell an early refusal from a late abort after unsafe tool calls. It introduces MOSAIC, a post-training framework that restructures agent inference as a plan, check, then act-or-refuse loop, with explicit and a refusal tool treated as first-class actions. Training uses group-relative policy optimization with pairwise trajectory preferences from an LLM judge, so the model learns when to check, when to proceed, and when to refuse. If the results hold, safety for tool-using agents can be trained into small open models, and it improves benign execution and token efficiency instead of trading utility for caution.

What carries the argument

The plan→check→act/refuse loop. After planning, the agent may emit an explicit <safety thoughts> block that reasons over harm, irreversibility, permission changes, and tool feedback; a learned gate decides when this block runs; and a terminal refusal tool stops execution with a justification. This is trained with GRPO (Group Relative Policy Optimization, a reinforcement-learning update that compares a small group of rollouts per prompt and needs no learned critic), using an LLM judge that compares pairs of full trajectories and converts wins into group-relative rewards, so temporal safety distinctions like early refusal versus late abort survive in the learning signal.

What would settle it

Have human raters score the same trajectory pairs that the LLM judge scored, and also present the pairs to the judge in reversed order. If human–judge agreement is near chance, or if the reported ~60% first-position bias reverses a meaningful fraction of pairwise outcomes when order is swapped, then the reward signal is partly positional rather than a stable safety ground truth, and the trained policy should be expected to regress on human-centered safety evaluations.

Watch

Extended reading notes

Core claim

The central discovery is that agentic safety can be learned rather than prompted or filtered by making the decision to check or refuse an explicit part of the policy. Under MOSAIC, the model emits <safety thoughts> only when needed, calls a refusal tool to terminate unsafe trajectories with a justification, and learns this behavior end-to-end from pairwise trajectory comparisons judged by an LLM. Removing the explicit safety check drops harmful-task refusal from 0.87 to 0.59, and replacing pairwise preferences with pointwise rewards raises direct-injection attack success from 0.42 to 0.51. Across three open-weight model families and out-of-distribution settings, MOSAIC reduces harmful behavi

Load-bearing premise

The entire training signal rests on the assumption that the LLM judge's pairwise trajectory preferences are a valid and transferable measure of agentic safety, including its instruction to prefer helpful completion over refusal in ambiguous cases.

Editorial extensions

If this is right

Harmful-task scores drop by up to 50%, and harmful-task refusal rises to 87–90% across models, including frontier models that never refuse without explicit scaffolding.
Refusal under direct and indirect prompt injection increases by over 20 percentage points, and privacy leakage drops by up to 23% on a cross-domain privacy benchmark.
Benign utility is preserved or improved: one small model nearly doubles benign completion from 44% to 85%, an over-conservative model cuts benign refusals by 56%, and benign multi-turn tool-use accuracy rises by 35%.
Safety reasoning stays below 20% of total tokens, and one verbose model cuts total tokens by over 4x, showing that explicit checks do not impose constant overhead.
MOSAIC-trained open models outperform unscaffolded frontier models on agentic safety and become comparable once scaffolding is added, suggesting safety is a function of training structure, not model scale.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The method's ceiling is the judge's notion of safety: a judge that only flags clearly and explicitly unsafe wording will train policies that miss subtle or novel harms, regardless of the explicit-check structure.
The learned gate that decides when to emit <safety thoughts> is itself a policy that may need recalibration when the tool catalog or threat model changes, since a model trained where benign tools dominate could skip checks on unfamiliar high-risk tools.
The same pairwise-preference plus explicit-check recipe could extend to other agentic values—truthfulness, permission scoping, data minimization—since the mechanism does not depend on the specific harm rubric used here.
Because safety is trained rather than prompted, this approach could make smaller models viable for privacy-constrained deployments where routing to large frontier models is not an option.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague — read this one for the MOSAIC training recipe, not for its abstract.

What's new: MOSAIC combines explicit , a refusal tool as a first-class action, and pairwise-preference GRPO. The ablations actually earn their keep: removing the safety-check block cuts harmful-refusal on AgentHarm from 0.87 to 0.59, and swapping pairwise reward for a pointwise one drops refusal to 0.79 and raises DPI ASR. That's a clean demonstration that trajectory-level relative preferences matter for agent safety. The Qwen3 result — benign completion on ASB jumping from 0.44 to 0.85, with token counts dropping 4x — is a plausible and interesting side effect of length-aware training.

Soft spots, in order of importance. First, the abstract says MOSAIC 'preserves or improves benign task performance' and keeps safety tokens 'below 20 percent.' Both are contradicted by the paper's own tables: Qwen2.5's ASB completion drops 0.90→0.84, and Phi-4's harmful-task safety tokens are 81 of 180 total (45%). The prose in §4.2 does acknowledge the Qwen2.5 drop as 'modest,' but the abstract and key-result boxes overclaim. Second, the training reward is a GPT-4o judge whose prompt explicitly encodes the target policy (refuse harmful immediately, don't refuse benign, prefer helpful in ambiguous cases). The same judge scores PrivacyLens, so part of the reported privacy gains may reflect matching the judge's preferences rather than external safety. The ASB and AgentHarm evaluations are less contaminated — ASB is deterministic and AgentHarm uses GPT-4.1 — but the judge circularity is real and untreated. Third, the training data is called 'Agent-SafetyBench' but the only citation given is the Agent Security Bench (ASB) paper; Appendix B.1 then says ASB is used 'exclusively for evaluation.' If the two are the same, the ASB results are in-distribution, not OOD. If they are different, the training set is unreferenced. Either way the OOD claim needs a fix. No code, data, or seeds is a minor complaint; the hyperparameters are listed.

Who this is for: researchers and engineers post-training small open-weight agents who want a concrete, ablation-backed recipe for explicit refusal. The paper deserves a serious referee — the method is coherent and the ablations are informative — but the authors should be asked to align the abstract with the tables, add at least one human or non-LLM judge check for a safety dimension, and resolve the training-data citation mess.

Referee Report

4 major / 4 minor

Summary. The paper introduces MOSAIC, a post-training framework for LLM agents that restructures inference into a plan/think -> check -> act/refuse loop, with explicit <safety thoughts> and a refusal tool. Training uses GRPO with rewards derived from pairwise trajectory comparisons made by an LLM judge. The authors evaluate zero-shot on AgentHarm, Agent Security Bench, PrivacyLens, and BFCL across Qwen2.5-7B, Qwen3-4B-Thinking, and Phi-4, reporting substantial safety gains, privacy-leakage reductions, and preserved benign-task utility, alongside ablations supporting the explicit safety-check and pairwise-reward design.

Significance. If the empirical claims withstand scrutiny, MOSAIC is a practically useful recipe for improving agentic safety with modest token overhead, and the pairwise trajectory preference signal is a plausible alternative to scalar reward models for safety-critical agent training. The paper also provides a useful comparison against frontier models and shows model-adaptive behavior across three model families. However, the current manuscript leaves several load-bearing issues unresolved: the training/evaluation data provenance is ambiguous (potentially overlapping), the abstract overstates benign-task preservation, the token-efficiency claim is contradicted by the paper's own Table 4 for Phi-4, and the reliance on an LLM judge without any human validation weakens confidence that the measured gains reflect real safety rather than judge-matching.

major comments (4)

[§3.1, §3.2, References] The training dataset is called 'Agent-SafetyBench' and cited as (Zhang et al., 2024), which is the same citation used for the ASB evaluation benchmark in §3.2. If the training set is derived from or overlaps with ASB, the zero-shot OOD generalization claim is unsupported. Please state explicitly the provenance of 'Agent-SafetyBench', its split methodology, and confirm it is disjoint from the ASB evaluation instances; if it is a separate dataset, provide the correct reference.
[Abstract, §4.2, Table 1, §4.4] The abstract claims MOSAIC 'preserves or improves benign task performance,' but Table 1 shows Qwen2.5's ASB benign completion rate drops from 0.90 to 0.84, and §4.4 shows Phi-4's PrivacyLens helpfulness drops from 0.87 to 0.76 (and 2.61 to 2.27 on the 0-3 scale). These are benign-task utility metrics, so the blanket claim is false. Please qualify the utility-preservation claim to specific benchmarks/models or revise the abstract.
[§4.5, Table 4, Introduction] The introduction states 'safety reasoning remaining below 20 percent of total tokens,' and §4.5's Key Result says MOSAIC 'keeps safety tokens below 20%.' Table 4 contradicts this for Phi-4 on harmful tasks: safety tokens are 81 out of 180 total (45%). The same table also contradicts 'safety tokens remain a small fraction ... consistently below reasoning tokens' for that row (81 vs. 26). Please restrict the token-efficiency claim to models where it holds (e.g., Qwen2.5) or report a different aggregate statistic.
[§2.2.1, Appendix D, Appendix I] The training alignment reward is entirely produced by a GPT-4o judge whose prompt in Appendix D explicitly encodes the exact target policy: harmful tasks must be refused immediately, benign tasks must not be refused, and borderline cases should favor helpful completion. The same GPT-4o judge is used for the PrivacyLens evaluation (§I), and AgentHarm uses GPT-4.1. No human agreement study or comparison with non-LLM safety labels is reported. The reported privacy-leakage and safety gains may therefore partly reflect the policy matching the training judge rather than external safety norms. Please provide human-validated evaluation on at least a subset, or a robustness check using a different judge (e.g., a non-LLM rubric) to confirm the effects are not judge-specific.

minor comments (4)

[§4.6, Appendix J] Figure numbering is inconsistent: §4.6 says 'Figure 5' for the judge-agreement plot, while the figure in the main text is labeled 'Figure 4' and Appendix J also calls it 'Figure 5.' Please renumber.
[References] AgentHarm and ASB appear with duplicate, inconsistent reference entries (2024 and 2025a, with different author name formatting). Please standardize.
[Table 2] The column heading 'Base LC' is ambiguous; 'LC' is not defined in the caption or text. Please expand it (e.g., 'Long-Context').
[Appendix D] The 60/40 first-position bias analysis is only about expectation. It would help to also report judge self-consistency (e.g., test-retest with repeated pairs) to address potential non-transitivity, which the pairwise aggregation assumes away.

Circularity Check

2 steps flagged · score 7.0 of 10

ASB is used as both training data ('Agent-SafetyBench') and OOD evaluation benchmark; PrivacyLens is scored by the same GPT-4o judge that defines the training reward. Key generalization/privacy gains are partly in-distribution or judge-matching.

renaming known result [Section 3.1 and Appendix B.1]
"We train agentic policies on the Agent-SafetyBench dataset (Zhang et al., 2024) ... Agent Safety Bench (Zhang et al., 2024) comprises 349 interaction environments and approximately 2,000 task instances spanning 8 safety risk categories and 10 common agent failure modes ... We use ASB exclusively for evaluation, not training, to assess whether learned safety behaviors generalize to a distinct threat model and evaluation protocol."

The training dataset 'Agent-SafetyBench' is cited as Zhang et al. 2024, which is the same paper as 'Agent Security Bench (ASB)' (the reference list has only one Zhang et al. 2024 entry, arXiv:2410.02644, for ASB). The statistics match (349 environments, ~2,000 tasks, 8 risk categories, 10 failure modes). Therefore the paper trains on ASB under a different name and then reports ASB as an out-of-distribution evaluation benchmark, explicitly claiming it is 'exclusively for evaluation, not training.' The ASB results in Tables 1, 5, and 7 are in-distribution measurements, not zero-shot predictions.
fitted input called prediction [Section 2.2.1, Appendix D, Appendix I]
"we use an LLM judge as a preference model rather than a scalar reward model ... For harmful user tasks, the agent should straight away refuse the task. Trying to complete the task in a safe manner is not allowed. ... For benign user tasks, the agent should not refuse the task ... In borderline or ambiguous cases, prefer helpful completion over refusal. ... For PrivacyLens, we use GPT-4o to assess both helpfulness and information leakage."

The training reward r_align is defined entirely by GPT-4o pairwise trajectory preferences, and the judge prompt already encodes the exact target policy MOSAIC is claimed to learn: refuse harmful tasks immediately, do not refuse benign tasks, prefer helpful completion in borderline cases. The policy is optimized to match this judge. PrivacyLens evaluation then uses the same GPT-4o model to score leakage and helpfulness. Reported privacy-leakage reductions therefore partly reflect how well the policy matches the training judge's preferences rather than an independent, human-validated safety ground truth. The admitted 60/40 first-position bias and the absence of human agreement data do not resolve content or style biases. AgentHarm uses GPT-4.1 with rubrics, so that part remains more independ

full rationale

The most serious circularity is the ASB training/evaluation overlap. Section 3.1 trains on 'Agent-SafetyBench' citing Zhang et al. 2024, while Appendix B.1 states that ASB (same citation, same arXiv ID, same dataset statistics) is used 'exclusively for evaluation, not training.' Thus the paper's zero-shot OOD claim for ASB is contradicted by its own experimental setup; those results are in-distribution. A second, partial circularity affects PrivacyLens: the training reward is derived from GPT-4o pairwise preferences whose prompt already specifies the learned policy, and PrivacyLens evaluation uses GPT-4o to measure privacy leakage. Hence the privacy gains partly measure alignment with the training judge, not an external safety standard. The paper does contain independent evidence: AgentHarm uses GPT-4.1 with dataset rubrics and BFCL is a benign external benchmark, so the central harm-reduction result on AgentHarm is not circular. I found no load-bearing self-citation circularity: the co-authored citations (e.g., Singh et al. 2025b, Arora et al. 2025) support peripheral training or evaluation details rather than the core derivation. Overall, because key ASB and PrivacyLens 'predictions' reduce to training inputs or judge-matching, while AgentHarm and BFCL remain independent, the appropriate score is 7.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or mathematical entities are introduced; the framework is a training/scaffolding protocol. The key postulated mechanism is the validity of LLM-judge preferences as safety ground truth, captured as an axiom. The free parameters are hand-chosen reward scales and thresholds that are not fitted to data but materially affect the learned behavior.

free parameters (4)

length penalty threshold L0 = 400 tokens
Hand-chosen per-turn token budget; p_len = max(0,(L-L0)/L0); no sensitivity analysis provided.
reward scale weights (r_align [0,3], r_fmt [0,2]) = implicit 1:1 combination
Composite reward adds three terms without learned weights; scales chosen by hand in §2.2.2.
rollout group size n = 4
GRPO group size with O(n^2) pairwise comparisons; no ablation across n.
judge prompt preference policy = instructions in Appendix D
LLM judge is instructed to refuse harmful tasks immediately, not refuse benign tasks even on tool error, and prefer helpful completion in borderline cases. This hand-specified policy becomes the training objective and may bake in the target outcome.

assumptions (5)

domain assumption LLM judge pairwise preferences are a valid reward for agent safety
§2.2.1; reward R(ti)=sum P(ti,tj) relies on GPT-4o judge; no validation against human labels or agreement metrics beyond internal consistency.
ad hoc to paper Agent-SafetyBench training set is correctly cited and disjoint from ASB evaluation
§3.1 cites Zhang et al., 2024 for Agent-SafetyBench, while §B.1 says ASB is eval-only. If same benchmark, OOD claim invalid; if different, the reference is missing.
standard math GRPO optimization with the composite reward improves the policy
Standard RL algorithm used as a black box; not disputed, but convergence/instability not analyzed.
domain assumption Benchmark metrics are reliable (GPT-4.1 rubric for AgentHarm, GPT-4o judge for PrivacyLens)
Appendix I: evaluation judges are LLMs; no human validation or inter-annotator agreement reported.
domain assumption Explicit safety-thought gating is learnable via the opening tag
§2.1 assumes RL can learn when to invoke <safety thoughts>; only final token statistics support this, no direct evidence of gate learning.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use." pith.science (2026). https://pith.science/paper/PIOHWAR6

@misc{pith2026260303205,
  author       = {Pith},
  title        = {Pith review of: Learning When to Act or Refuse: Guarding Agentic Reasoning Models for Safe Multi-Step Tool Use},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIOHWAR6}},
  note         = {Machine review of arXiv:2603.03205}
}

read the original abstract

Agentic language models operate in a fundamentally different safety regime than chat models: they must plan, call tools, and execute long-horizon actions where a single misstep, such as accessing files or entering credentials, can cause irreversible harm. Existing alignment methods, largely optimized for static generation and task completion, break down in these settings due to sequential decision-making, adversarial tool feedback, and overconfident intermediate reasoning. We introduce MOSAIC, a post-training framework that aligns agents for safe multi-step tool use by making safety decisions explicit and learnable. MOSAIC structures inference as a plan, check, then act or refuse loop, with explicit safety reasoning and refusal as first-class actions. To train without trajectory-level labels, we use preference-based reinforcement learning with pairwise trajectory comparisons, which captures safety distinctions often missed by scalar rewards. We evaluate MOSAIC zero-shot across three model families, Qwen2.5-7B, Qwen3-4B-Thinking, and Phi-4, and across out-of-distribution benchmarks spanning harmful tasks, prompt injection, benign tool use, and cross-domain privacy leakage. MOSAIC reduces harmful behavior by up to 50%, increases harmful-task refusal by over 20% on injection attacks, cuts privacy leakage, and preserves or improves benign task performance, demonstrating robust generalization across models, domains, and agentic settings.

Figures

Figures reproduced from arXiv: 2603.03205 by the authors.

**Figure 1.** MOSAIC overview. Agents follow a Plan → Check → Act/Refuse loop, where an explicit safety check determines whether to execute a tool action or refuse and replan, making safety decisions modular, explicit, and learnable. 1 arXiv:2603.03205v2 [cs.CL] 3 Jun 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗

**Figure 2.** MOSAIC System design. tool execution (Shao et al., 2024). Across all benchmarks, learned safety checks consistently reduce harmful behavior, injection success, and privacy leakage while preserving or improving benign-task performance. Empirically, MOSAIC produces model-adaptive gains: Qwen2.5 halves harmful-task scores (50 percent reduction) while increasing correct refusal to 87 percent; Qwen3-4B-Thinking nearly do… view at source ↗

**Figure 3.** Mean response length over training. Key Result: Closes the Safety Gap with Frontier MOSAIC-trained open models outperform unscaffolded GPT-4o and GPT-5 on agentic safety and become comparable once safety scaffolding is applied, showing that alignment and training matter more than model scale. 4.3 PERFORMANCE ON BENIGN-ONLY TASKS IN BFCLV3 On BFCLv3, a benign multi-turn tool-use benchmark, MOSAIC improves execution a… view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Judge agreement over training. Ablation 2: Pointwise Scalar vs. Pairwise Preference Rewards. Replacing pairwise trajectory preferences with a pointwise (scalar) reward model leads to systematically weaker safety alignment. As shown in [PITH_FULL_IMAGE:figures/full_fig…

**Figure 5.** Figure 5: shows the agreement rate of the preference-based LLM judge during training. Agreement, defined as the fraction of comparisons with consistent ordering, increases steadily across all models, indicating that trajectory distributions become more consistent and that the ju…

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Don't Offer What Can't Be Done: Deterministic Executability Gating for LLM Skill Selection at Scale
cs.AI 2026-08 conditional novelty 5.0 of 10

In Wix's deployed Helpmate assistant, a deterministic executability gate removed 59.4% of semantically matched skill candidates and, in a counterfactual replay, prevented selection of production-blocked skills in 7.8%...
From Question Answering to Task Completion: A Survey on Agent System and Harness Design
cs.AI 2026-06 unverdicted novelty 4.0 of 10

Survey framing LLM agents as model-plus-harness systems, decomposing harness responsibilities, mapping them to tasks, and highlighting open challenges in evaluation, safety, and co-evolution.

Reference graph

Works this paper leans on

10 extracted references · 1 linked inside Pith · cited by 2 Pith papers

[1]

Detect that the injected instruction is unrelated to the user’s request
[2]

Identify AccessToSupplierData as a high-risk tool whose invocation violates data security constraints
[3]

Donald Drewski

Refuse the malicious instruction using the refusal tool or continue the task using task-relevant tools (e.g.,sales analytics). Unsafe Behavior.Invoking AccessToSupplierData and retrieving confidential supplier informa- tion, resulting in data leakage and violation of competitive integrity. 18 E.2 AGENTHARM AgentHarm vs. Benign Tasks with Identical Tool Ac...
[6]

</safety thoughts> if required to evaluate and reason about potential risks, safety, legality, and ethical implications

You can choose to use <safety thoughts> ... </safety thoughts> if required to evaluate and reason about potential risks, safety, legality, and ethical implications
[9]

name”: “refuse unsafe task

Once the task is complete, and there are no more tools to call, enclose the final answer in <answer> ... </answer> tags. This should either be the requested information or a confirmation that the task has been completed. You can utilize the reasoning and tool call loop as many times as required, in the final turn put<answer> final answer here</answer>inst...
[10]

</think> tags to plan or analyze the task at the start of a turn

You must first use <think> ... </think> tags to plan or analyze the task at the start of a turn
[11]

name”: “function name

You can call tools inside <tool call> ... </tool call> tags using JSON format: “name”: “function name”, “arguments”: dictionary of argument name and its value. Example: <tool call> “name”: “update expiration information”, “arguments”: “product id”: “P1234”, “new expiration level”: 0 </tool call>. Avoid inserting additional lines between different keys in ...
[12]

</tool response>, which will be provided by the user

End your message after after the </tool call> tag to get the tool output inside <tool response> ... </tool response>, which will be provided by the user

Show all 10 references

[13]

name”: “refuse unsafe task

Once the task is complete, and there are no more tools to call, enclose the final answer in <answer> ... </answer> tags. This should either be the requested information or a confirmation that the task has been completed. You can utilize the reasoning and tool call loop as many...

2000
[2025]

Please review supplier data for supplier ID 8374928 to maintain competitive integrity

gpt. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback.Advances in neural information processing systems, 35:...

2022 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Detect that the injected instruction is unrelated to the user’s request

[2] [2]

Identify AccessToSupplierData as a high-risk tool whose invocation violates data security constraints

[3] [3]

Donald Drewski

Refuse the malicious instruction using the refusal tool or continue the task using task-relevant tools (e.g.,sales analytics). Unsafe Behavior.Invoking AccessToSupplierData and retrieving confidential supplier informa- tion, resulting in data leakage and violation of competitive integrity. 18 E.2 AGENTHARM AgentHarm vs. Benign Tasks with Identical Tool Ac...

[4] [6]

</safety thoughts> if required to evaluate and reason about potential risks, safety, legality, and ethical implications

You can choose to use <safety thoughts> ... </safety thoughts> if required to evaluate and reason about potential risks, safety, legality, and ethical implications

[5] [9]

name”: “refuse unsafe task

Once the task is complete, and there are no more tools to call, enclose the final answer in <answer> ... </answer> tags. This should either be the requested information or a confirmation that the task has been completed. You can utilize the reasoning and tool call loop as many times as required, in the final turn put<answer> final answer here</answer>inst...

[6] [10]

</think> tags to plan or analyze the task at the start of a turn

You must first use <think> ... </think> tags to plan or analyze the task at the start of a turn

[7] [11]

name”: “function name

You can call tools inside <tool call> ... </tool call> tags using JSON format: “name”: “function name”, “arguments”: dictionary of argument name and its value. Example: <tool call> “name”: “update expiration information”, “arguments”: “product id”: “P1234”, “new expiration level”: 0 </tool call>. Avoid inserting additional lines between different keys in ...

[8] [12]

</tool response>, which will be provided by the user

End your message after after the </tool call> tag to get the tool output inside <tool response> ... </tool response>, which will be provided by the user