Pith.

Artificial Intelligence · arXiv:2610.12292 v1 · submitted 8 Oct 2026 · 47 references

One Word Opens the Gate: The Option-Channel Attack on Typed Decision Models as Agent Guardrails

Seyedarmin Azizi, Erfan Baghaei Potraghloo, Massoud Pedram

The paper says

“A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text.”

more of the abstract

Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. Code is available at https://github.com/ArminAzizi98/option-channel-attack.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 9 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. Clean, careful measurement of a real guardrail vulnerability; the option-channel attack holds up and deserves peer review. This line is wrong

7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 47 references say.

7 minor objections

The paper says, at §6.3, Table 4

Pith says: minor objection 1 of 7

RLCD is described as immune by construction to the option channel, but the construction is shown only for VON through the option_marker_backend.py excerpt; please add the RLCD rendering path or reword Table 4 and the §6.3 text to say that RLCD is empirically invariant across the 198-label sweep.

The paper says, at §5, Eqs. (1)-(2)

Pith says: minor objection 2 of 7

RLCD can return an explicit insufficient-evidence output, yet the metrics in Eqs. (1)-(2) are defined over a binary decision; state how this third output is mapped into the binary rates (or that it never occurs in these runs), since RLCD appears in most of the main tables.

show 5 more

The paper says, at §2, §6.3

Pith says: minor objection 3 of 7

For Qwen2.5-7B the option label is the name whose log-probability the readout scores rather than a token rendered into the input context; a sentence making this mechanism difference explicit would prevent the typed-model label channel from being conflated with the readout channel.

The paper says, at §5, Tables 3-5

Pith says: minor objection 4 of 7

Bootstrap intervals are defined in §5 but reported only for the toxic-content closing statistic; please add intervals to the headline quantities (Table 3 unauth irrev row, Table 4 worst-label range, Table 5 best and k=10 columns) so claims like 93-100% and 0 to 63% carry stated uncertainty.

The paper says, at Code availability

Pith says: minor objection 5 of 7

The authors promise that every number is reproducible without commercial APIs, but the repository is not pinned; provide a commit hash, package versions, and model-weight identifiers so the copy on record matches the runs in the paper.

The paper says, at Table 3

Pith says: minor objection 6 of 7

The dash for LAYA-EN on escape or exfil is unexplained in the caption; if it means there are no correctly blocked items in that cell, say so there, as is done for Table 12.

The paper says, at §7, Table 9

Pith says: minor objection 7 of 7

The text says the label-based defenses lose two cells all on the same policy but does not name the policy; naming it would let a reader recover the losing configuration from the table.

What would settle it

Reproduce Table 5 for LAYA-TD on unauth irrev and Qwen2.5-7B on secret from the released repository, instrumenting the tokenized encoder input to verify that the option label token is present for these models and absent for VON/RLCD. An independent run that reproduces the 0-to-~100% fail-open shift and the flat label-invariance control settles the Reader's weakest assumption.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 9 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees 3 min 20 s · 19,717 tokens read, 26,989 written
  3. Editor wrote the verdict above 17 s · 20,505 tokens read, 2,380 written
  4. Referee minor revision: 7 minor 2 min 2 s · 19,992 tokens read, 18,994 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Short version: this is a careful evaluation of typed decision models as guardrails, and the option-channel result holds up. The paper separates fail-open from fail-closed, conditions attack success on clean correct decisions, and includes label-invariant models as controls. That last piece is what makes the headline credible: VON and RLCD stay flat across a 198-label sweep while LAYA/Qwen move sharply, so the label is genuinely in the input pathway, not a prompt-order artifact. I'd send this to review.

What's new: prior work did report option-name sensitivity (Sun et al.), but nobody had isolated the label channel with definitions and judged text fixed, measured asymmetric fail-open, shown transfer across models, and tested defenses against an adaptive attacker. The matched baseline for defenses—biasing toward block until equal fail-closed—is exactly the right comparison. The deterministic-rule result is correctly labeled a property of GuardBench's construction, not a general claim. The limitations section is unusually honest.

Soft spots are minor relative to the claims. The code repository is referenced but has no commit hash and I can't verify it; that's reproducibility friction, not evidence against direction. A few later tables lack confidence intervals; precision issue only. The commercial Jev model isn't measured, so claims are properly scoped to open-weight models, but the deployment argument would be stronger with any evidence from the closed API—the paper leans on another group's results there. GuardBench is template-generated, which the authors state; it buys exact ground truth at the cost of realism, and the public datasets cover that gap well enough.

I disagree with the reader's weakest assumption: the label-rendering condition is not hidden. The authors state it in §2 and §6.3, show library code for VON that excludes labels, and the empirical contrast between flat controls and sharp label sensitivity is the control the experiment needed. If a deployment strips labels before encoding, yes, the channel closes—that is the same condition the paper gives, not a flaw.

Who benefits: anyone deploying these models as allow/block gates, and reviewers of the guardrail literature. This deserves a serious referee; I'd recommend accept with requests for repo hardening and confidence intervals on remaining tables.

The strongest objection

Reading in good faith, the central claim—typed decision models used as final allow/block decisions can be opened by attacker-controlled text and, under an explicit configuration condition, by attacker-chosen option labels—is supported by the separated error metrics, the label-invariance controls, and the matched defense comparison. The condition the Reader flagged (labels actually rendered into the input) is stated by the paper rather than hidden, and it is checked empirically: VON and RLCD are flat across the 198-label sweep, while the label-sensitive models move sharply when only the label changes, so the label is in the input pathway. The clean-door demonstrations (Qwen2.5-7B; LAYA-TD on unauth irrev) show the effect on gates that otherwise decide correctly, so the headline is not an artifact of already-broken gates. Remaining caveats—unverified repository, no commit hash, sparse confidence intervals in later tables—affect precision and reproducibility rather than the direction of the central claim. I do not see a load-bearing flaw.

The Reader's weakest assumption is a real condition but not a hidden one: the paper explicitly scopes the option-channel result to the four models that place the label in their input, and the empirical design includes two label-invariant controls. The VON/RLCD flatness across the label sweep and the sharp label sensitivity of the other models is direct evidence that the label is in the input pathway for the affected models. The clean-door cases show that the attack matters precisely where the gate is otherwise worth deploying. The deterministic-rule limitation is acknowledged as a property of GuardBench's construction, and the public screening datasets carry the argument where no parsed rule is available. The remaining gaps are reproducibility and interval reporting, which affect precision, not the direction or load-bearing status of the central conclusion. I therefore agree with ACCEPT and see no basis for changing the verdict.

The referee's report

If the measurements hold, this is a timely and important negative result for a fast-growing deployment practice, and the evaluation is unusually careful. Attack success is conditioned on the decisions a gate made correctly before the attack (Eq. 2); GuardBench ground truth is computed from generation attributes rather than from model outputs; the option-channel claim rests on a 198-label sweep with random-order query budgets and cross-model transfer, with VON and RLCD acting as natural label-invariance controls, including a library-code demonstration for VON; and the defenses are compared against a matched blocking-biased baseline and then attacked adaptively, following Carlini et al. (2019) and Tramèr et al. (2020). The deterministic-rule result and the synthetic nature of GuardBench are explicitly scoped by the authors rather than hidden. The remaining gaps, which concern sparse confidence intervals, an unpinned code release, and a few underspecified metric details, affect precision and reproducibility rather than the direction of the central claims.

Pith's reading of the paper (a machine's summary, not the authors')

Typed decision models—small classifiers that read text and return a probability over caller-defined options rather than generating text—are being adopted as the allow-or-block gate in agent systems, and this paper evaluates whether they can hold that role as a security control. Across seven open-weight models and three public screening tasks, accuracy never exceeds 72% against a 50% chance level, and a low error rate in one direction usually just reflects which answer the model defaults to. The central result is the option-channel attack: renaming only the permissive option, with its definition and the judged text untouched, raises the rate at which prohibited actions are allowed to between 93% and 100% on the four models whose calling libraries place the label in the input. Every defense the paper tests is defeated by an attacker who knows it, and escalating low-confidence decisions fails because a decision an attack reversed is no less confident than the one it replaced. If the paper is right, these models can still reduce how many cases reach a human reviewer, but they should not be the component that decides.

What carries the argument. The typed question is the load-bearing object: an instruction plus a set of options, each carrying a short label and a written definition, from which the model emits one probability per option and no text. Whether the label reaches the network is a property of the calling library, and that difference is what separates the four vulnerable models from the two that are immune by construction. The margin $m(x) = \lvert P(\text{block}\mid x) - \tfrac{1}{2}\rvert$ carries the explanation for the text attacks: correct block decisions cluster just above 0.5, so a few hundredths of probability movement reverses them while a reversed decision stays as confident as the one it replaced, which is why confidence-based escalation cannot detect the attack. The defense comparison rests on a matched baseline that charges each defense for the legitimate traffic it blocks before any improvement in the attacked fail-open rate counts.

Core claim. On its own terms, the paper's discovery is that an option label is not a neutral name but an input. In four of the seven models studied, the short label attached to each option is encoded into the sequence the model reads, and in deployments where the question is assembled from a configuration file, a fetched policy document, or a tool-supplied schema, that label is attacker-reachable. An attacker who changes only the permissive option's name, leaving both the bound definition and the judged state untouched, can drive the gate to permit actions the policy prohibits: the fail-open rate reaches 93% to 100% on the susceptible models, a sweep of 198 labels shows the effect is cheap to find, and labels found on one ModernBERT model transfer to another. The paper also shows why such attacks succeed: correct block decisions sit close to the 0.5 boundary, with a median $P( ext{block})$ of 0.57, so a small move in probability reverses them, and the returned probabilities cannot be repurposed as a tunable risk threshold. Finally, every tested defense—canonical renaming, ensembling, disagreement escalation, structured fields, and instruction rewrites—is defeated by an adaptive attacker, except that passing parsed field values removes one attack completely while making the model unnecessary, because a deterministic rule over the same parsed values reaches 100% accuracy on all six synthetic policies.

The premise it rests on. The headline attack assumes the short option label is actually rendered into the model's input sequence; the paper demonstrates the opposite for VON with a library code snippet and relies on the statement that calling libraries differ for the other models, so a deployment whose encoder never includes the label would close the channel entirely.

If the paper is right (Pith's inferences, not the authors')

Deployments must treat option labels as data an attacker can write: rewriting labels to canonical neutral names at the call site closed the option channel completely in the paper's tests, at a modest cost in blocked legitimate traffic.

The returned probability cannot serve as an operating-point threshold, because no setting between 0.30 and 0.40 makes the gate both usable and resistant, and the clean margins that predict attackability are not available at decision time.

Escalating low-confidence decisions to a human or stronger model does not catch these attacks, since a reversed decision is as confident as the one it replaced; escalation reduces reviewer workload, not risk.

Where a tool call's fields can be parsed into typed values, a deterministic rule over those values reached 100% accuracy on all six policies, making the model unnecessary in exactly the settings where the strongest defense is available.

Typed decision models may still serve as pre-filters that shrink the traffic a reviewer must examine, but the paper's evidence says they should not make the final allow-or-block decision.

Editorial: the transfer result, in which labels found on one ModernBERT model open a sibling model, suggests label sensitivity is a family-level property, so a deployment auditing a single open-weight model should sweep label vocabularies across the whole family before trusting any member of it.

Editorial: the stated mechanism implies a cheap deployment test—inspect the encoded sequence once, and if the option-label token appears, assume the option channel is open and either strip labels at the call site or replace the decision with a deterministic rule.

Editorial: because neutral tokens and unrelated nouns also open the gate, the vulnerability looks like positional or format sensitivity of the decision head rather than semantic persuasion; an analogous test on the other closed-form question types, such as rubric positions or proposition probabilities, would show whether the channel generalizes beyond option labels.

Editorial: the clean deterministic-rule result depends on GuardBench's exact attributes being visible in the trace, and the paper's own limitation note concedes this; the open question the paper leaves is whether typed-value parsing and rule-based decisions can be extended to unstructured real-world traces, where the public screening results suggest the models' signal is weak anyway.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. Code is available at https://github.com/ArminAzizi98/option-channel-attack.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-09

The authors' main claim, as the classifier picked it out 1.00

“These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides.”

How the main claim is stated: asserted 0.65

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.69 · human participants 0.04 · controlled comparison 0.83 · randomized assignment 0.02 · states quantitative result 0.99 · compares with prior results 0.86 · mentions code or data availability 0.98

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
A typed decision model reads a piece of text and returns a probability over caller-defined options, each with a short written definition, generating no text. background 0.97 0.07 0.02 asserted 1.00
Recent work places these models in agent systems as guardrails: the component that reads a proposed tool call or incoming message and decides whether to allow it. background 1.00 0.05 0.02 asserted 0.99
We evaluate seven open-weight models in that role and report the two error directions separately: a fail-open error allows a prohibited action and is a vulnerability; a fail-closed error blocks a permitted one and is only a cost. method 0.55 0.78 0.04 asserted 0.99
On prompt-injection, jailbreak and toxic-content screening, accuracy at the allow-or-block decision ranges from 36% to 72% against a chance level of 50%. result 1.00 0.90 0.06 asserted 1.00
A low error rate in one direction only reflects which answer a model defaults to: one allows nearly everything, another blocks nearly everything. interpretation 0.89 0.71 0.05 asserted 0.98
On a synthetic suite of agent tool calls, six lines of server log text that say nothing about the policy raise a gate's fail-open rate from 0% to 63% on a policy it otherwise decides correctly. result 1.00 0.94 0.05 asserted 1.00
Giving the permissive option a misleading name, with its definition and the judged text untouched, raises that rate to between 93% and 100% on the four models that place the label in their input. result 0.98 0.94 0.06 asserted 1.00
Every defense we tested is defeated, either by an attacker who targets its mechanism or by attacker-controlled text. result 0.98 0.87 0.19 asserted 1.00
Escalating the least confident decisions does not help either: a decision an attack has reversed is no less confident than the one it replaced. result 0.99 0.91 0.04 asserted 1.00
Parsing each policy field into a typed value does eliminate one attack, but it also makes the model unnecessary: a deterministic rule over those values reaches 100% accuracy on all six policies. result 0.90 0.88 0.10 asserted 1.00
These models can reduce how many cases reach a reviewer, but on this evidence they should not be the component that decides. interpretation 0.99 0.71 0.89 asserted 0.65
Code is available at https://github.com/ArminAzizi98/option-channel-attack. contribution 0.73 0.52 0.02 asserted 0.65

What kind of paper the classifier reports this is: empirical study 0.99 · evidence from observation or experiment 0.99

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-09: 1 request, 9,028 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (7)
Figure 1 of the paper
Figure 1. A typed decision model takes a state and a typed question and returns one probability per
Figure 2 of the paper
Figure 2. The same results plotted as the two error rates against each other. A useful gate would sit
Figure 3 of the paper
Figure 3. Fail-open rate against the amount of attacker-controlled text appended, on
Figure 4 of the paper
Figure 4. On jailbreak screening, the fraction of correct block decisions that an attack reverses,
Figure 5 of the paper
Figure 5. The option channel over a vocabulary of 198 labels for the permissive option. (a) Mean
Figure 6 of the paper
Figure 6. Why confidence-based escalation does not catch these attacks. (a) Each point is one model,
Figure 7 of the paper
Figure 7. The matched comparison for one model and policy (

Figures are the authors', reproduced from the paper.

References (47)
  1. [1] Typed Decision Models: An Early Evidence Audit and Evaluation Checklist , author=. 2026 , eprint= on Pith
  2. [2] Type-Safe Is Not Error-Free: A Constrained Decision Head Follows the Option Name, Not the Rubric Bound to It , author=. 2026 , eprint= on Pith
  3. [3] JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models , author=. 2026 , eprint= on Pith
  4. [4] JevOut: Natural Context Can Flip Decision Models , author=. 2026 , eprint= on Pith
  5. [5] Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions , author=. 2026 , eprint= on Pith
  6. [6] JEV-as-a-Judge: Accept When Confident, Escalate When Unsure , author=. 2026 , eprint= on Pith
  7. [7] Evaluating System One Models for Agent Security Decisions: Reliability, Calibration, and Selective Automation , author=. 2026 , eprint= on Pith
  8. [8] JEV as a Judge for Agent Trace Security: An Empirical Comparison with Generative LLM Judges , author=. 2026 , eprint= on Pith
  9. [9] Calibrated Decision Models for Autonomous Penetration-Testing Harnesses , author=. 2026 , eprint= on Pith
  10. [10] Jev in the Wild: A Data-Driven Analysis of the Jev Model's Functionality, Applications and Ecosystem , year=. 2609.30216 , archivePrefix= on PithPith's reviewarXiv
  11. [11] Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder , author=. ACL , year= on Pith
  12. [12] ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation , author=. EMNLP Findings , year= on Pith
  13. [13] Prompt Injections , author =. 2023 , howpublished = on Pith
  14. [14] Jailbreak Classification , author =. 2023 , howpublished = on Pith
  15. [15] Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations , author=. 2023 , eprint= on Pith
  16. [16] Rebedea, Traian and Dinu, Razvan and Sreedhar, Makesh Narsimhan and Parisien, Christopher and Cohen, Jonathan , booktitle= on Pith
  17. [17] Greshake, Kai and Abdelnabi, Sahar and Mishra, Shailesh and Endres, Christoph and Holz, Thorsten and Fritz, Mario , year=. Not What You've Signed Up For: Compromising Real-World. 2302.12173 , archivePrefix= on PithPith's reviewarXiv
  18. [18] Debenedetti, Edoardo and Zhang, Jie and Balunovi\'c, Mislav and Beurer-Kellner, Luca and Fischer, Marc and Tram\`er, Florian , booktitle= on Pith
  19. [19] Wallace, Eric and Feng, Shi and Kandpal, Nikhil and Gardner, Matt and Singh, Sameer , booktitle=. Universal Adversarial Triggers for Attacking and Analyzing on Pith
  20. [20] Universal and Transferable Adversarial Attacks on Aligned Language Models , author=. 2023 , eprint= on Pith
show 27 more
  1. [21] Calibrate Before Use: Improving Few-Shot Performance of Language Models , author=. ICML , year= on Pith
  2. [22] Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity , author=. ACL , pages= on Pith
  3. [23] On Evaluating Adversarial Robustness , author=. 2019 , eprint= on Pith
  4. [24] On Adaptive Attacks to Adversarial Example Defenses , author=. NeurIPS , year= on Pith
  5. [25] Nicholas Carlini, Anish Athalye, Nicolas Papernot, Wieland Brendel, Jonas Rauber, Dimitris Tsipras, Ian Goodfellow, Aleksander Madry, and Alexey Kurakin. On evaluating adversarial robustness, 2019 on Pith
  6. [26] Edoardo Debenedetti, Jie Zhang, Mislav Balunovi\'c, Luca Beurer-Kellner, Marc Fischer, and Florian Tram\`er. AgentDojo : A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents. In NeurIPS Datasets and Benchmarks, 2024 on Pith
  7. [27] deepset . Prompt injections. Hugging Face dataset deepset/prompt-injections , 2023. https://huggingface.co/datasets/deepset/prompt-injections on Pith
  8. [28] Joas Antonio dos Santos Barbosa. Calibrated decision models for autonomous penetration-testing harnesses, 2026 on Pith
  9. [29] Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. Not what you've signed up for: Compromising real-world LLM -integrated applications with indirect prompt injection, 2023 on Pith
  10. [30] Jack Hao. Jailbreak classification. Hugging Face dataset jackhhao/jailbreak-classification , 2023. https://huggingface.co/datasets/jackhhao/jailbreak-classification on Pith
  11. [31] Jianyi Hu, Hangtao Zhang, Yi Liu, et al. Jevadvbench: A benchmark and black-box attacks for reinforcement learning for calibrated decisions models, 2026 on Pith
  12. [32] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. Llama guard: Llm-based input-output safeguard for human-ai conversations, 2023 on Pith
  13. [33] Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman. Jev-as-a-judge: Accept when confident, escalate when unsure, 2026 on Pith
  14. [34] Zi Lin, Zihan Wang, Yongqi Tong, et al. Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation. In EMNLP Findings, 2023 on Pith
  15. [35] Yixuan Liu. Evaluating system one models for agent security decisions: Reliability, calibration, and selective automation, 2026 on Pith
  16. [36] Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In ACL, pp.\ 8086--8098, 2022 on Pith
  17. [37] Traian Rebedea, Razvan Dinu, Makesh Narsimhan Sreedhar, Christopher Parisien, and Jonathan Cohen. NeMo guardrails: A toolkit for controllable and safe LLM applications with programmable rails. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proce... on Pith
  18. [38] Yu Sun, Junhao Xu, Jiajia Shi, and Zijin Yang. Type-safe is not error-free: A constrained decision head follows the option name, not the rubric bound to it, 2026 on Pith
  19. [39] Lijuan Tang and Yuemeng Zheng. Typed decision models: An early evidence audit and evaluation checklist, 2026 on Pith
  20. [40] Florian Tram\`er, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. In NeurIPS, 2020 on Pith
  21. [41] Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP . In EMNLP, 2019 on Pith
  22. [42] Zhiqiang Wang and Yichao Gao. Jev as a judge for agent trace security: An empirical comparison with generative llm judges, 2026 on Pith
  23. [43] Benjamin Warner, Antoine Chaffin, Benjamin Clavi\'e, et al. Smarter, better, faster, longer: A modern bidirectional encoder. In ACL, 2025 on Pith
  24. [44] Tiantong Wu and Wei Yang Bryan Lim. Decision hijacking: Prompt injection attacks on jev's typed probabilistic decisions, 2026 on Pith
  25. [45] Zixiang Xu. Jevout: Natural context can flip decision models, 2026 on Pith
  26. [46] Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In ICML, 2021 on Pith
  27. [47] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023 on Pith
Cited by 4 Pith papers

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment