Pith. sign in

REVIEW 2 major objections 4 minor 16 references

A model that holds a system rule against direct user attacks can still follow instructions smuggled into tool outputs; the benchmark's two conflict tracks behave as separate abilities.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 00:54 UTC pith:7MT4QQSR

load-bearing objection A genuinely useful two-track hierarchy benchmark whose dissociation headline needs one correlation coefficient before it fully convinces. the 2 major comments →

arxiv 2607.25987 v2 pith:7MT4QQSR submitted 2026-07-28 cs.CR cs.SE

IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications

Conor McCauley , Zeliang Kan , Jason Martin This is my paper
classification cs.CR cs.SE
keywords instruction hierarchytool-output injectionprompt injectionLLM agentsbenchmarkconstraint familiesbinary pass/fail evaluationLLM safety
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that instruction-hierarchy robustness in language models is not one skill but several. To show it, the authors built a conflict-centered benchmark with 2,336 scenarios that stage priority conflicts on two surfaces: a system instruction versus a user request (S≻U) and a user request versus a tool output (U≻T). Across 37 models, compliance ranges from 98.2% to 20.5%, and the key dissociation is that high S≻U compliance does not predict U≻T resistance: some models are nearly perfect when a user directly attacks a system rule yet fail badly when the same conflict arrives inside a tool response. The paper also finds that stronger warning language fixes failures for some models but not others, and that subtle manipulations—disclaimers, small factual edits—are harder to resist than overtly destructive actions. If this is right, evaluations that use only system-prompt jailbreak tests will overstate agentic safety.

Core claim

Central claim: strong S≻U (system-over-user) constraint following is not a reliable proxy for resistance to tool-output injection. One model-class example scores 96.9% on direct system-user conflicts but 65.6% when the conflict arrives in tool outputs; the best U≻T (user-over-tool) over S≻U margin among all models is 11.3 points. Failures also split by type: stronger warnings lift some weak models from about 20% to about 90% on generic U≻T scenarios while others stay near 10% at every strictness level, and the hardest families are low-stakes ones—appended disclaimers and minor factual distortions, around 50%—rather than unauthorized purchases or bulk ticket closure, around 88–97%. The author

What carries the argument

The carrying mechanism is the benchmark itself: two tracks built from 44 human-authored constraint families (19 for S≻U, 25 for U≻T) instantiated across generic, health, finance, retail, and coding domains, with three stateful agent simulators—coding, retail, health support—where tool-call side effects persist across steps. Three generation axes vary how each conflict is presented: constraint strictness (L1–L3, the strength of the higher-priority instruction or the system's warning about untrusted tool output), user prompt phrasing (P1/P2, explicit versus implicit), and delivery variant (D1–D4: plain, breakout-format, acknowledge-then-inject, switch-task) for how the payload is embedded in t

Load-bearing premise

The paper treats a model's score on direct system-vs-user conflicts as directly comparable to its score on user-vs-tool conflicts, even though the two test sets differ sharply in size and average difficulty; if the tool-output set is simply harder, the claimed separation may be an artifact.

What would settle it

Run a matched-difficulty control: select a subset of U≻T scenarios whose per-model pass rates equal those of the S≻U track across the same model panel; if the model ranking then correlates strongly with S≻U, the claimed dissociation is an artifact of track difficulty. Equivalently, report the correlation between the two track scores and show it is not driven by scenario-count and difficulty imbalance.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Deployment evaluations that rely only on S≻U jailbreak tests will miss tool-injection vulnerabilities: the paper reports models with near-perfect S≻U scores degrading sharply on U≻T.
  • Stronger warning language is not a universal mitigation: hardening separates models into those whose failures are mostly fixed by stricter instructions and those that stay near floor compliance regardless of strictness.
  • The failure taxonomy puts subtle manipulations—disclaimers, minor factual distortions, scripted refusals—at the top of the risk list because they are the least resisted.
  • Overall compliance numbers are misleading: a single aggregate score can hide the dissociation, so reporting should be per conflict surface and per constraint group.
  • High-severity resistance may partly reflect safety alignment rather than hierarchy reasoning, meaning benchmarks should not conflate refusing obviously harmful actions with correctly prioritizing instructions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If this pattern extends, model providers' safety leaderboards that use system-prompt-only tests will overstate agentic robustness; a U≻T tool-injection suite looks like a necessary second axis.
  • A testable extension: vary which tool carries the injection within each simulator to see whether the U≻T gap is a general property of tool outputs or specific to certain tools; the paper fixes one designated injection-bearing tool per domain.
  • The halt-and-warn-is-a-pass rule for U≻T could let a model that refuses every tool-mediated task score well on conflict scenarios; because U≻T non-conflict scenarios number only 66 against 1,536 conflict scenarios, the benchmark may under-detect over-refusal.
  • If the dissociation holds during training, curriculum data should be built per conflict surface: improving a model's S≻U compliance should not be assumed to transfer to U≻T.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper introduces IH-Benchmark, a new benchmark for instruction-hierarchy robustness in LLM applications, covering two tracks: system-vs-user (S≻U) conflicts and user-vs-tool (U≻T) conflicts. The benchmark contains 2,336 executable scenarios built from 44 human-authored constraint families across five domains, with a uniform binary pass/fail evaluation using a predicate DSL and selectively applied LLM judges. The authors evaluate 37 model variants and report that compliance ranges from 98.2% to 20.5%. Their central findings are that strong S≻U compliance is not a reliable proxy for U≻T robustness, that constraint-hardening helps some models but not others, and that subtle failures (e.g., injected disclaimers, minor factual distortions) are more common than overtly dangerous ones. The paper releases the corpus, simulators, and harness.

Significance. If the findings hold, IH-Benchmark fills a genuine gap: existing benchmarks typically cover one hierarchy edge or repurpose public datasets, whereas this work jointly evaluates direct and tool-mediated conflicts in an executable, controlled setting. The human-authored taxonomy and procedural generation are strengths, as is the use of stateful agent simulators for U≻T. The paper also ships reproducible evaluation infrastructure and reports run-to-run standard deviations (≤0.9 pp overall), which is commendable. The main conceptual contribution—that hierarchy robustness is not a single capability—is plausible and practically important, but the empirical support for the headline dissociation is incomplete, as detailed below.

major comments (2)
  1. [§4, 'Overall results'] The claim that strong S≻U compliance is not a reliable proxy for U≻T robustness is not quantified. The paper cites one outlier (Grok 4.20 (R)) and the observation that five models have higher U≻T than S≻U, but no correlation is reported. From Table 16, the Spearman rank correlation between track-average conflict compliance across the 37 models is approximately 0.62—moderate, not negligible. Moreover, the tracks have different base rates (85.4% vs 68.6%) and different domain compositions (finance only in S≻U, coding only in U≻T), so a single-capability model could produce such a pattern. The authors should report the rank correlation, provide a difficulty-matched comparison on overlapping domains, and test whether the U≻T shortfall is larger than expected from a common-mechanism model.
  2. [§3.4, U≻T pass criterion] For U≻T conflict scenarios, pass is defined as not following the tool-output injection; completing the user's task is not required, and halting with a warning is acceptable. This means a model that over-refuses or refuses to engage at all can pass a conflict scenario without demonstrating hierarchy robustness. Because the main results are conflict-only compliance rates, this could inflate U≻T scores for models with high refusal behavior. The paper should distinguish pass modes (task completion vs. halt/warn vs. unsupported refusal) and check whether U≻T conflict passes correlate with over-refusal, e.g., by comparing with non-conflict compliance (Table 17). If needed, the pass criterion should require task completion or an explicit user-visible warning about the injection.
minor comments (4)
  1. [§4, 'Effect of constraint strictness'] The phrase 'gap from L1 to the better of L2 and L3' is a post-hoc selection; report the L2 and L3 gains separately (Table 2 already provides the data) to avoid cherry-picking and to give readers the full distribution.
  2. [§4, 'Experimental setup'] The LLM judge used in 189 scenarios was validated on only 50 human-labeled outputs. This is a small sample; report per-category accuracy and a confidence interval, and consider a larger validation set, especially since the judge is in the same model family as some evaluated models.
  3. [Figure 2] The predicate example 'is_language_conf(‘chinese’, ‘<’, 0.2)' is shown without explaining the threshold semantics. Add a sentence in §3.4 defining how such thresholds are set and whether they were tuned on held-out data.
  4. [§4, 'Overall results'] The phrase 'only loosely correlated' is imprecise; use a statistical measure (e.g., Spearman ρ, which we estimate at about 0.62 from Table 16) and interpret it in context.

Circularity Check

0 steps flagged

No circularity: IH-Benchmark is an empirical measurement study with direct pass/fail evaluation and no fitted parameters or load-bearing self-citations.

full rationale

The paper is an empirical benchmark study: IH-B compliance rates are direct pass/fail measurements produced by a predicate DSL and category-scoped LLM judges, with no fitted parameters, no derivation from the benchmark's own outputs, and no load-bearing self-citations. The central claim that S≻U compliance does not reliably transfer to U≻T robustness is an empirical observation across 37 models, not a consequence of how the benchmark was defined. The only evaluative component that depends on an LLM is the judge used for 189 of 2,336 scenarios, but the paper validates the judge configuration against 50 human-labeled outputs with 100% agreement across two runs, which is external evidence rather than circular reasoning. The claimed dissociation is statistically under-supported (no cross-track correlation is reported and track difficulties are not matched), but that is a correctness/evidence concern, not a circularity concern. No step in the paper reduces by construction to its inputs, so no significant circularity is present.

Axiom & Free-Parameter Ledger

1 free parameters · 4 axioms · 0 invented entities

The benchmark rests on the Wallace et al. priority hierarchy, the validity of LLM-as-a-judge for subjective categories, and an asserted-but-not-proven representativeness of its hand-authored constraint families. These are domain assumptions, not vetted external facts; the paper discloses the hand-authored origin in its Limitations.

free parameters (1)
  • Predicate/judge thresholds (e.g., is_language_conf threshold 0.2 in Figure 2) = 0.2 (example)
    Hand-set thresholds in the evaluation DSL determine pass/fail boundaries; they are design choices, not fitted to data, but they directly affect measured compliance rates.
axioms (4)
  • domain assumption LLMs' behavior can be meaningfully decomposed into a fixed priority order: system > user > tool outputs.
    The benchmark's pass/fail design encodes this hierarchy from Wallace et al. (2024); if this is not a real behavioral dimension, the measurements lose their target (§1, §3).
  • domain assumption LLM-as-a-judge (GPT-5-mini) judgments match human judgments for brand/topic constraint satisfaction.
    Used for 189 scenarios; validated on 50 outputs only, with 100% reported accuracy; this unproven-on-larger-scale assumption directly affects reported compliance rates (§3.4).
  • domain assumption The 44 hand-authored constraint families and the five domains are representative of real-world deployment conflicts.
    Stated explicitly in the Limitations: 35/44 families are author-written, 9 from GPT-5.3 with manual filtering; representativeness is asserted, not demonstrated (§5).
  • domain assumption The DSL predicate checks and the 4-tier rubric are a correct operationalization of the pass/fail criterion.
    The pass/fail protocol is defined by the authors; different predicate thresholds would change results, and this definition is loaded into every verdict (§3.4, Figure 2).

pith-pipeline@v1.3.0-alltime-deepseek · 34760 in / 14498 out tokens · 134452 ms · 2026-08-01T00:54:30.332377+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications." pith.science (2026). https://pith.science/paper/7MT4QQSR

@misc{pith2026260725987,
  author       = {Pith},
  title        = {Pith review of: IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7MT4QQSR}},
  note         = {Machine review of arXiv:2607.25987}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

When a language model receives conflicting instructions from different priority levels, which one does it actually follow? This question lies at the heart of reliable LLM deployment. Existing benchmarks answer this only partially, often focusing on a single hierarchy edge or adapting public datasets with limited tool-use coverage. We present IH-Benchmark, a conflict-centered benchmark for instruction-hierarchy robustness across direct system-user conflicts (S>U) and tool-mediated user-tool (U>T) conflicts. IH-Benchmark is built from a human-authored taxonomy of 44 constraint families across generic, health, finance, retail, and coding settings, and evaluates scenarios with a uniform binary pass/fail protocol combining a predicate DSL with category-scoped LLM judges. Across 37 evaluated models, hierarchy compliance ranges from 98.2% to 20.5%. We find that strong S>U compliance is not a reliable proxy for U>T robustness: several models preserve system constraints under direct user conflict but degrade sharply when conflicting instructions appear in tool outputs. Constraint hardening also reveals a split between models: some failures are largely fixed by stronger warnings, while others persist across all strictness levels. Finally, the most revealing failures are often subtle rather than overtly dangerous; models resist unauthorized purchases or bulk ticket closure more reliably than injected disclaimers or small factual distortions. These results suggest that instruction-hierarchy robustness is not a single capability, but a set of behaviors that must be evaluated across conflict surfaces, constraint types, and attack presentations.

Figures

Figures reproduced from arXiv: 2607.25987 by Conor McCauley, Jason Martin, Zeliang Kan.

Figure 1
Figure 1. Figure 1: Construction and evaluation pipeline for IH-B. The two top-level columns correspond to the S ≻ U and U ≻ T tracks. Rows show the message components and the generation axes varied within each track, the resulting model response, and the binary pass/fail evaluation. [C] denotes the higher-priority constraint and [K] denotes the conflict. L, P, and D index constraint strictness level, user prompt phrasing, an… view at source ↗
Figure 2
Figure 2. Figure 2: Representative examples from IH-B. Each column illustrates one benchmark track as a concrete conversation-level conflict. Left: an S ≻ U example, where the system prompt is composed of the base persona and the constraint, and the user request conflicts with that constraint. Right: a U ≻ T example, where the system prompt is composed of the base persona and the constraint, the user requests their latest ema… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

16 extracted references · 6 linked inside Pith

  1. [1]

    Amazon Bedrock User Guide

    Amazon Web Services. Amazon Bedrock User Guide. https://docs.aws.amazon.com/bedrock/latest/ userguide/what-is-bedrock.html, a. Accessed: 2026-05-04. Amazon Web Services. Amazon Elastic Compute Cloud Documentation. https://docs.aws.amazon.com/ ec2/, b. Accessed: 2026-05-04. Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin...

    Pith/arXiv arXiv 2026

  2. [6]

    Identifying the risks of lm agents with an lm-emulated sandbox.arXiv preprint arXiv:2309.15817,

    Yangjun Ruan, Honghua Dong, Andrew Wang, Silviu Pitis, Yongchao Zhou, Jimmy Ba, Yann Dubois, Chris J Maddison, and Tatsunori Hashimoto. Identifying the risks of lm agents with an lm-emulated sandbox.arXiv preprint arXiv:2309.15817,

    Pith/arXiv arXiv

  3. [7]

    Together AI Documentation

    Together AI. Together AI Documentation. https://docs.together.ai/docs/quickstart. Accessed: 2026-05-04. Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions.arXiv preprint arXiv:2404.13208,

    Pith/arXiv arXiv 2026

  4. [8]

    Instructional segment embedding: Improving llm safety with instruction hierarchy.arXiv preprint arXiv:2410.09102,

    Tong Wu, Shujian Zhang, Kaiqiang Song, Silei Xu, Sanqiang Zhao, Ravi Agrawal, Sathish Reddy Indurthi, Chong Xiang, Prateek Mittal, and Wenxuan Zhou. Instructional segment embedding: Improving llm safety with instruction hierarchy.arXiv preprint arXiv:2410.09102,

    Pith/arXiv arXiv

  5. [9]

    xAI Inference REST API Overview

    xAI. xAI Inference REST API Overview. https://docs.x.ai/developers/rest-api-reference/ inference. Accessed: 2026-05-04. Jingwei Yi, Yueqi Xie, Bin Zhu, Emre Kiciman, Guangzhong Sun, Xing Xie, and Fangzhao Wu. Benchmarking and defending against indirect prompt injection attacks on large language models. InProceedings of the 31st ACM SIGKDD Conference on Kn...

    2026

  6. [10]

    Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents

    Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. Injecagent: Benchmarking indirect prompt injections in tool-integrated large language model agents. InFindings of the Association for Computational Linguistics: ACL 2024, pages 10471–10506,

    2024

  7. [11]

    Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents.arXiv preprint arXiv:2410.02644,

    Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang. Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents.arXiv preprint arXiv:2410.02644,

    Pith/arXiv arXiv

  8. [12]

    Iheval: Evaluating language models on following the instruction hierarchy

    Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, et al. Iheval: Evaluating language models on following the instruction hierarchy. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Techno...

    2025

  9. [13]

    Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911,

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXiv preprint arXiv:2311.07911,

    Pith/arXiv arXiv

  10. [14]

    Y denotes primary coverage, P denotes partial or adjacent coverage, and – denotes not a primary focus

    A Coverage Comparison Table 6: Coverage comparison with representative instruction-hierarchy and agent-security bench- marks. Y denotes primary coverage, P denotes partial or adjacent coverage, and – denotes not a primary focus. Controls denotes aligned, benign, reference, or non-conflict settings used to distin- guish hierarchy robustness from ordinary t...

    2024

  11. [15]

    11 Table 7: Model providers and exact LiteLLM identifiers for all 37 model variants

    As of the time of writing (May 2026), Qwen 3 235B-A22B is no longer available for serverless inference on TogetherAI but remains accessible via on-demand dedicated endpoints. 11 Table 7: Model providers and exact LiteLLM identifiers for all 37 model variants. Where reasoning effort or thinking modes apply, they are shown in parentheses:Rindicates a reason...

    2026

  12. [2023]

    doi: 10.1145/3600006.3613165. OpenAI. OpenAI API Reference. https://developers.openai.com/api/reference/overview. Ac- cessed: 2026-05-04. Yunjia Qi, Hao Peng, Xiaozhi Wang, Amy Xin, Youfeng Liu, Bin Xu, Lei Hou, and Juanzi Li. Agentif: Bench- marking instruction following of large language models in agentic scenarios.arXiv preprint arXiv:2505.16944,

    arXiv 2026

  13. [2024]

    LiteLLM: Open-source library and ai gateway for calling llm providers

    BerriAI. LiteLLM: Open-source library and ai gateway for calling llm providers. https://docs.litellm. ai/docs/. Accessed: 2026-05-04. Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents.Advances in Neural I...

    2026

  14. [2025]

    Infobench: Evaluating instruction following ability in large language models

    Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. Infobench: Evaluating instruction following ability in large language models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13025–13048,

    2024

  15. [2026]

    Ih-challenge: A training dataset to improve instruction hierarchy on frontier llms.arXiv preprint arXiv:2603.10521,

    Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu, Christopher A Choquette-Choo, Steph Lin, Nikhil Kandpal, Milad Nasr, Sam Toyer, Miles Wang, Yaodong Yu, et al. Ih-challenge: A training dataset to improve instruction hierarchy on frontier llms.arXiv preprint arXiv:2603.10521,

    arXiv

  16. [8192]

    no action required

    Reasoning effort and thinking mode were configured where applicable. API requests were retried up to five times; scenarios for which all retries failed were excluded. Across the 37 model variants, 86,423 scenarios completed successfully and 9 were excluded under this policy. B.3 Cost The cost of evaluating a single model on the full benchmark varies acros...

    2029