Pith.

Artificial Intelligence · arXiv:2610.34826 v1 · submitted 26 Oct 2026 · 51 references

TrustJev: Separating Policy Authority from Untrusted Evidence in LLM-Based Decisions

Zhiwei Xu, Xuyang Liu, Xiaolin Peng, Weixian Deng, Jiabao Gao, Zirui Li, Tian Qiu, Boya Zhou

show 3 more authors

Wei Jiao, Hai Wan, Xibin Zhao

The paper says

“Decision models enable software to select from predefined actions using natural language inputs. However, the external evidence that supplies facts for these decisions may also contain instructions that attempt to override the application's policy.”

more of the abstract

We propose TrustJev, a framework combining protected policy scoring with bounded evidence corrections. A single Transformer backbone computes policy scores and a correction budget from trusted policy and candidates, then bounds evidence corrections by that budget. Paired supervision encourages stable predictions under instruction insertions that preserve task facts and changed preferences when edits change the correct answer. Authority supervision accommodates evidence when the policy alone cannot determine the decision. We evaluate a 0.6B implementation against 14 baselines on three datasets covering authentication, instruction hierarchy, and executable authorization rules. TrustJev achieves the highest mean Balance, the harmonic mean of accuracy on non-attack and attack inputs, exceeding the strongest baseline on each by 0.57-1.05 percentage points. In the authorization benchmark, it correctly answers all 32 attacks per source in a fixed pool for 95.77% of sources, compared with 80.98% for the strongest comparator. TrustJev maintains comparable inference memory and median latency, with additional overhead primarily during training.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 27 Oct 2026 · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up. A principled architecture for prompt injection defense in decision models, with consistent but modest gains and thorough evaluation. This line is wrong

6 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the 51 references say.

6 minor objections

The paper says, at Table VII

Pith says: minor objection 1 of 6

The latency for the 'Extraction + rules' baseline is reported as 0.0075 ms. This appears implausibly fast (7.5 microseconds for what seems to be a non-trivial pipeline including extraction and rule matching). Please verify the unit or measurement and correct if it is a typo (e.g., 0.075 ms or 0.75 ms).

The paper says, at §III-A, Equation (3)

Pith says: minor objection 2 of 6

The budget ceiling B(K) = 8 + 12 log K is introduced as a 'preset' design choice. While the ablation in Table IV shows that removing the bound (A2) hurts performance, the specific functional form and constants are not justified or ablated. Consider adding a brief justification or a sensitivity analysis (e.g., varying the constants) to clarify that the result does not depend critically on this particular parameterization.

show 4 more

The paper says, at §IV-A, Table II

Pith says: minor objection 3 of 6

The dataset statistics table reports 'Training groups / views' etc. It would be helpful to explicitly state what constitutes a 'group' (e.g., a base case plus its variants) and how views within a group are generated, especially for reproducibility.

The paper says, at §IV-E, Figure 5

Pith says: minor objection 4 of 6

In the latency scaling experiments, the paper reports that TrustJev's latency increases more with evidence length than baselines (4.3 ms vs 1.7–1.9 ms from 128 to 1024 tokens). While this is acknowledged, the practical impact could be discussed more concretely—e.g., whether this growth is sub-linear and what fraction of real-world inputs fall into the longer evidence range.

The paper says, at §IV-F, Table VII

Pith says: minor objection 5 of 6

The industrial case study uses 178 source cases. With this modest size, the 95% confidence intervals around the reported accuracies could be wide. The paper acknowledges small differences in general (Threats to Validity) but a short note on the statistical precision for this specific study would strengthen the presentation.

The paper says, at General

Pith says: minor objection 6 of 6

Several acronyms (AEB, P50, P95, etc.) are not defined in their first occurrence in the evaluation section for readers who might skip the earlier notation. Consider adding a brief definition at the start of §IV.

What would settle it

Re-run the AEB evaluation with the released code and fixed random seeds to confirm that TrustJev achieves ≥99.40% Balance and ≥95.77% cumulative robust accuracy at 32 attempts; any discrepancy greater than 0.5 percentage points would warrant investigation.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 27 Oct 2026

  1. Reader holds up confidence high
  2. Skeptic partly agrees 42 s · 21,470 tokens read, 3,877 written
  3. Editor wrote the verdict above 18 s · 22,172 tokens read, 1,489 written
  4. Referee minor revision: 6 minor 27 s · 21,669 tokens read, 2,425 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

The core idea here is solid: separate policy scoring from evidence, bound the influence of evidence with a learned budget, and train with paired/authority supervision to distinguish legitimate fact changes from injection. The architectural design — attention masks, position assignments, the two-branch scoring — is novel relative to prior work like StruQ or SecAlign, which do not explicitly bound evidence influence. The evaluation is the paper's real strength: three datasets including a new executable-rule benchmark (AEB), 14 baselines, ablations, robustness to attack variants and multiple attempts, an industrial case study, and latency/memory measurements. The ablations in Table IV and the paired diagnostics in Figure 3 convincingly show that each component contributes, and the stability analysis in Table V confirms that policy scores behave as intended.

The gains are real but incremental — 0.57 to 1.05 percentage points in Balance over the strongest baseline on each dataset. That's consistent across all three datasets, which counts for something, but the improvements are modest. The budget ceiling B(K) = 8 + 12 log K is a design choice that lacks a strong theoretical motivation; the paper calls it a “preset” but doesn't justify the functional form or constants. Also, the evaluation uses only one model scale (0.6B) and at most three candidates, so generalization to larger models or more options is untested. The latency overhead at longer evidence lengths is noticeable (Figure 5b). The authors do acknowledge most of these limitations — they cite response-adaptive attacks, ambiguous policies, and unequal error costs as open issues — so the letter is honest.

Overall: this is a well-motivated, clearly described, and honestly evaluated piece of work on an important problem. The improvement is small but the architecture is principled and the evidence is strong enough to take seriously. I'd send it to peer review. The main thing I'd ask the authors to address in revision is motivation and sensitivity analysis of the budget ceiling, and possibly an additional experiment at a larger scale or with more candidates to test generality. But the paper as is is a clear accept at a good venue.

The strongest objection

The paper's central claim is well-supported by the evaluation, including ablations, robustness tests, and industrial case study. The concern about attention mask sufficiency is partially addressed by empirical evidence (Table V shows policy-sufficiency coverage 100% and policy accuracy 100%, indicating that policy scores are indeed unaffected by evidence for policy-sufficient inputs). The budget ceiling generalization is a limitation acknowledged by the authors. The improvements over strong baselines are consistent and likely statistically significant. I find no single load-bearing concern that would undermine the conclusion.

The paper is thorough, with clear architecture, extensive evaluation, and explicit limitations. The attention mask and position assignment mechanism is empirically validated (Table V), and the budget ceiling is a tunable hyperparameter acknowledged in limitations. The modest but consistent improvements across datasets and the robustness generalization results support the central claim. The reader's weakest assumption is valid but not load-bearing given the empirical evidence. Therefore, I recommend no change to the reader's ACCEPT verdict.

The referee's report

If the reported results hold, TrustJev offers a principled and effective approach to prompt-injection resistance in decision models, a growing practical concern as these models are deployed in software workflows. The architectural guarantee (protected policy scores with bounded evidence corrections) is cleanly motivated, and the behavioral supervision strategy for distinguishing harmless fact changes from policy violations is well-designed. The evaluation is unusually thorough: three diverse datasets, 14 baselines, multiple seeds, component ablations, robustness to attack variants and composition, cumulative robustness across multiple attempts, latency and memory profiling, and an industrial case study. The authors also release code and the AEB benchmark, supporting reproducibility and future comparisons. The limitations are explicitly discussed, including the absence of response-adaptive attacks and coverage of only one model scale, which are appropriate caveats.

Pith's reading of the paper (a machine's summary, not the authors')

The paper proposes TrustJev, a framework for LLM-based decision models that prevents prompt injection attacks by separating the computation of policy scores from evidence corrections. Policy scores are derived solely from the trusted policy and candidate options, and an evidence correction budget is also computed from these trusted inputs. Evidence corrections are bounded by this budget before being added to policy scores, ensuring that external evidence can inform decisions but cannot override the policy. Paired supervision and authority supervision train the model to distinguish between fact-preserving injections (which should not change the decision) and legitimate fact or policy changes (which should). On three datasets covering authentication, instruction hierarchy, and authorization rules, a 0.6B implementation of TrustJev achieves the highest mean Balance (harmonic mean of non-attack and attack accuracy), exceeding the strongest baseline on each dataset by 0.57–1.05 percentage points. In the authorization benchmark, it correctly answers all 32 attacks per source for 95.77% of sources, compared with 80.98% for the strongest comparator.

What carries the argument. The key machinery is the combination of attention-masked dual-candidate scoring and bounded evidence corrections. In the input sequence, each candidate appears twice: once without access to evidence (C_A) and once with access (C_D). Attention permissions ensure C_A tokens can only read the policy and themselves, while C_D tokens can read policy, evidence, and themselves. The policy scores a_k and the budget β are computed from the C_A representations, so they are entirely independent of the evidence. The budget is $\beta = B(K) \sigma(G([p, \bar{u}]))$ where $B(K) = 8 + 12\log K$ is a preset ceiling. Evidence scores are computed from C_D and then centered and transformed via $\beta \tanh(e_k - \bar{e})$ to produce bounded corrections. Paired supervision and authority supervision coordinate when decisions should change or stay stable.

Core claim. TrustJev's central discovery is that policy authority in decision models can be preserved by architecturally separating the computation of policy scores from evidence-derived corrections, and bounding the magnitude of those corrections by a budget that depends only on the trusted policy and candidates. This is achieved through a single Transformer backbone with carefully designed attention masks and position assignments: two copies of each candidate text (C_A and C_D) are included, where C_A can only attend to policy and itself, while C_D can also attend to evidence. The policy scores a_k and the budget β are computed from the C_A branch alone, ensuring they are independent of evidence. Evidence scores e_k are computed from the C_D branch, centered, scaled by tanh, and multiplied by β to obtain bounded corrections r_k. The final score is z_k = a_k + r_k. A sufficient condition for the policy-preferred candidate to remain selected is that its policy margin exceeds 2β. Paired supervision enforces stable predictions when injected instructions preserve task facts, and changed predictions when facts or authorized policies change. Authority supervision trains the policy scores to be confident when the policy suffices and uniform when facts are needed. The approach is evaluated on three datasets (TensorTrust, IHEval, and AEB) and achieves the highest mean Balance on each.

The premise it rests on. The central assumption is that the application-controlled attention masks and position assignments are sufficient to prevent any leakage of evidence influence into policy scores and budget computation, and that the budget ceiling $B(K)=8+12\log K$ generalizes across task complexities.

If the paper is right (Pith's inferences, not the authors')

LLM decision models built with TrustJev can be deployed in environments where external content may contain malicious instructions, without sacrificing accuracy on legitimate inputs.

The separation of policy scoring from evidence correction provides a clear sufficient condition for stable decisions: if the policy margin exceeds 2β, the evidence cannot change the winner.

The paired and authority supervision objectives ensure that the model learns to respect policy authority while remaining responsive to genuine fact changes, avoiding evidence neglect.

TrustJev's inference cost is comparable to baselines, making it feasible for real-time decision systems.

The budget ceiling formula $B(K)=8+12\log K$ was empirically chosen; future work could learn the budget end-to-end or make it task-dependent for tighter control.

The evaluation uses a 0.6B model; it remains an open question whether the same approach scales to larger LLMs where attention patterns might leak information across the mandated boundaries.

TrustJev defends against a fixed pool of attacks; it remains untested against adaptive adversaries who can craft attacks after observing the model's behavior, a limitation the authors acknowledge.

The framework assumes that the application can reliably separate policy from evidence and assign source boundaries; in practice, some workflows may have blurred boundaries that this approach cannot handle.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Decision models enable software to select from predefined actions using natural language inputs. However, the external evidence that supplies facts for these decisions may also contain instructions that attempt to override the application's policy. We propose TrustJev, a framework combining protected policy scoring with bounded evidence corrections. A single Transformer backbone computes policy scores and a correction budget from trusted policy and candidates, then bounds evidence corrections by that budget. Paired supervision encourages stable predictions under instruction insertions that preserve task facts and changed preferences when edits change the correct answer. Authority supervision accommodates evidence when the policy alone cannot determine the decision. We evaluate a 0.6B implementation against 14 baselines on three datasets covering authentication, instruction hierarchy, and executable authorization rules. TrustJev achieves the highest mean Balance, the harmonic mean of accuracy on non-attack and attack inputs, exceeding the strongest baseline on each by 0.57-1.05 percentage points. In the authorization benchmark, it correctly answers all 32 attacks per source in a fixed pool for 95.77% of sources, compared with 80.98% for the strongest comparator. TrustJev maintains comparable inference memory and median latency, with additional overhead primarily during training.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-10-27

The authors' main claim, as the classifier picked it out 0.99

“TrustJev achieves the highest mean Balance, the harmonic mean of accuracy on non-attack and attack inputs, exceeding the strongest baseline on each by 0.57-1.05 percentage points.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.33 · human participants 0.04 · controlled comparison 0.95 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.99 · mentions code or data availability 0.04

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Decision models enable software to select from predefined actions using natural language inputs. background 1.00 0.04 0.02 asserted 1.00
However, the external evidence that supplies facts for these decisions may also contain instructions that attempt to override the application's policy. background 0.85 0.05 0.03 hedged possibility 0.94
We propose TrustJev, a framework combining protected policy scoring with bounded evidence corrections. contribution 0.98 0.84 0.17 asserted 0.99
A single Transformer backbone computes policy scores and a correction budget from trusted policy and candidates, then bounds evidence corrections by that budget. method 1.00 0.80 0.07 asserted 1.00
Paired supervision encourages stable predictions under instruction insertions that preserve task facts and changed preferences when edits change the correct answer. method 0.98 0.76 0.04 asserted 0.98
Authority supervision accommodates evidence when the policy alone cannot determine the decision. method 0.95 0.37 0.06 asserted 0.97
We evaluate a 0.6B implementation against 14 baselines on three datasets covering authentication, instruction hierarchy, and executable authorization rules. method 0.77 0.96 0.04 asserted 0.99
TrustJev achieves the highest mean Balance, the harmonic mean of accuracy on non-attack and attack inputs, exceeding the strongest baseline on each by 0.57-1.05 percentage points. result 1.00 0.97 0.57 asserted 1.00
In the authorization benchmark, it correctly answers all 32 attacks per source in a fixed pool for 95.77% of sources, compared with 80.98% for the strongest comparator. result 1.00 0.96 0.18 asserted 1.00
TrustJev maintains comparable inference memory and median latency, with additional overhead primarily during training. result 1.00 0.94 0.06 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.95

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 7,447 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (5)
Figure 1 of the paper
Figure 1. A compensation workflow in which customer emails supply decision facts but have no authority to change the administrator’s policy. The selected
Figure 2 of the paper
Figure 2. Overview of TrustJev. (a) A single Transformer backbone supports both scoring branches. Attention constraints prevent external evidence from altering
Figure 3 of the paper
Figure 3. AEB paired diagnostics for Table IV. Dots and intervals show means
Figure 4 of the paper
Figure 4. Robustness under attack variants, composition, and multiple attempts.
Figure 5 of the paper
Figure 5. Request latency and input scaling. Panel (a) shows test inputs from three datasets, with P50 labeled. Panel (b) varies evidence length with

Figures are the authors', reproduced from the paper.

References (51)
  1. [1] D. Almeida, “Introducing system one models & Jev,” 2026, typeSafe AI, September 15, 2026. Accessed October 21, 2026. [Online]. Available: https://typesafe.ai/blog/introducing-system-one-models-and-jev on Pith
  2. [2] Z. Chen, F. Yu, H. Huang, Y . Li, A. Chen, and L. Chen, “SecJev: Bringing security expertise to system one decision models,” arXiv:2610.03073, 2026. [Online]. Available: https://arxiv.org/abs/2610. 03073 on PithPith's reviewarXiv
  3. [3] J. Palmer, “Kev-0.8B: a calibrated decision model on Qwen3.5- 0.8B,” 2026, model card. Accessed October 21, 2026. [Online]. Available: https://github.com/jaredpalmer/kev/blob/main/docs/ model-cards/kev-0.8b.md on Pith
  4. [4] Y . Liu, Y . Jia, R. Geng, J. Jia, and N. Z. Gong, “Formalizing and benchmarking prompt injection attacks and defenses,” in33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, 2024, pp. 1831–1847. [Online]. Available: https://www.usenix.org/ conference/usen... on Pith
  5. [5] T. Wu and W. Y . B. Lim, “Decision hijacking: Prompt injection attacks on Jev’s typed probabilistic decisions,” arXiv:2609.28613, 2026. [Online]. Available: https://arxiv.org/abs/2609.28613 on PithPith's reviewarXiv
  6. [6] S. Chen, J. Piet, C. Sitawarin, and D. Wagner, “StruQ: Defending against prompt injection with structured queries,” in34th USENIX Security Symposium (USENIX Security 25). USENIX Association, 2025, pp. 2383–2400. [Online]. Available: https://www.usenix.org/ conference/usenixsec... on Pith
  7. [7] S. Chen, A. Zharmagambetov, S. Mahloujifar, K. Chaudhuri, D. Wagner, and C. Guo, “SecAlign: Defending against prompt injection with preference optimization,” inThe ACM Conference on Computer and Communications Security (CCS), 2025. [Online]. Available: https://sizhe-chen.githu... on Pith
  8. [8] R. Liu, Y . Lin, Z. Huang, and J. S. Dong, “DRIP: Defending prompt injection via token-wise representation editing and residual instruction fusion,” in33rd ACM Conference on Computer and Communications Security (CCS), 2026, accepted for publication. [Online]. Available: https:... on PitharXiv
  9. [9] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems, vol. 30, 2017. [Online]. Available: https://papers.nips.cc/paper files/paper/2017/hash/ 3f5e... on Pith
  10. [10] S. Toyer, O. Watkins, E. Mendes, J. Svegliato, L. Bailey, T. Wang, I. Ong, K. Elmaaroufi, P. Abbeel, T. Darrell, A. Ritter, and S. Russell, “Tensor Trust: Interpretable prompt injection attacks from an online game,” inInternational Conference on Learning Representations, 2024.... on Pith
  11. [11] Z. Zhang, S. Li, Z. Zhang, X. Liu, H. Jiang, X. Tang, Y . Gao, Z. Li, H. Wang, Z. Tan, Y . Li, Q. Yin, B. Yin, and M. Jiang, “IHEval: Evaluating language models on following the instruction hierarchy,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapte... on Pith
  12. [12] Z. Wang, J. Li, M. Ma, Z. Li, Y . Kang, C. Zhang, C. Bansal, M. Chintalapati, S. Rajmohan, Q. Lin, D. Zhang, C. Pei, and G. Xie, “Large language models can provide accurate and interpretable incident triage,” in2024 IEEE 35th International Symposium on Software Reliability Eng... on Pith
  13. [13] M. Du, F. Li, G. Zheng, and V . Srikumar, “DeepLog: Anomaly detection and diagnosis from system logs through deep learning,” inProceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017. [Online]. Available: https: //dl.acm.org/doi/10.1145/3133... on Pithdoi:10.1145/3133956.3134015
  14. [14] X. Zhang, Y . Xu, Q. Lin, B. Qiao, H. Zhang, Y . Dang, C. Xie, X. Yang, Q. Cheng, Z. Li, J. Chen, X. He, R. Yao, J.-G. Lou, M. Chintalapati, F. Shen, and D. Zhang, “Robust log-based anomaly detection on unstable log data,” inProceedings of the 27th ACM Joint European Software ... on Pithdoi:10.1145/3338906.3338931
  15. [15] Z. Xu, Y . Wu, S. Wang, J. Gao, T. Qiu, Z. Wang, H. Wan, and X. Zhao, “Deep learning-based intrusion detection systems: A survey,” arXiv:2504.07839, 2025. [Online]. Available: https: //arxiv.org/abs/2504.07839 on PitharXiv
  16. [16] X. Liu, Y . Han, Z. Zhang, K. Chang, Z. Xu, T. Qiu, W. Deng, J. Gao, X. Peng, H. Wan, and X. Zhao, “DiagChain: A diagnostic benchmark for evaluating LLM agents on evidence-grounded attack chain reconstruction,” arXiv:2608.03591, 2026. [Online]. Available: https://arxiv.org/abs... on PithPith's reviewarXiv
  17. [17] Z. Xu, T. Qiu, H. Gao, W. Jiao, Q. Ma, J. Wang, J. Zhao, J. Gao, H. Wan, and X. Zhao, “Cerberus: Robust endpoint threat detection in practice,” inProceedings of the 34th ACM International Conference on the Foundations of Software Engineering, 2026. [Online]. Available: https:/... on Pithdoi:10.1145/3803437.3805249
  18. [18] D. Hendrycks and K. Gimpel, “Gaussian error linear units (GELUs),” arXiv:1606.08415, 2016. [Online]. Available: https://arxiv.org/abs/1606. 08415 on PithPith's reviewarXiv
  19. [19] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: A simple way to prevent neural networks from overfitting,”Journal of Machine Learning Research, vol. 15, no. 56, pp. 1929–1958, 2014. [Online]. Available: https://www.jmlr.org/papers/v15/sri... on Pith
  20. [20] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, “Towards deep learning models resistant to adversarial attacks,” inInternational Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=rJzIBfZAb on Pith
show 31 more
  1. [21] T. Hasan, A. Bhattacharjee, M. S. Islam, K. Mubasshir, Y .-F. Li, Y .-B. Kang, M. S. Rahman, and R. Shahriyar, “XL-Sum: Large-scale multilingual abstractive summarization for 44 languages,” inFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Associati... on Pith
  2. [22] H. Li, X. Liu, N. Zhang, and C. Xiao, “PIGuard: Prompt injection guardrail via mitigating overdefense for free,” inProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 2025, ... on Pith
  3. [23] X. Liang, L. Wu, J. Li, Y . Wang, Q. Meng, T. Qin, W. Chen, M. Zhang, and T.-Y . Liu, “R-Drop: Regularized dropout for neural networks,” inAdvances in Neural Information Processing Systems, vol. 34, on Pith
  4. [24] S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang, “Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization,” inInternational Conference on Learning Representations, 2020. [Online]. Available: https://iclr.c... on Pith
  5. [25] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” inAdvances in Neural Information Processing Systems, vol. 36, 2023. [Online]. Available: https://papers.neurips.cc/paper... on Pith
  6. [26] E. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” in International Conference on Learning Representations, 2022. [Online]. Available: https://iclr.cc/virtual/2022/poster/6319 on Pith
  7. [27] I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” in International Conference on Learning Representations, 2019. [Online]. Available: https://openreview.net/forum?id=Bkg6RiCqY7 on Pith
  8. [28] J. M. Zhang, M. Harman, L. Ma, and Y . Liu, “Machine learning testing: Survey, landscapes and horizons,”IEEE Transactions on Software Engineering, 2022. [Online]. Available: https://ieeexplore.ieee. org/document/9000651/ on Pith
  9. [29] S. Li, J. Guo, J.-G. Lou, M. Fan, T. Liu, and D. Zhang, “Testing machine learning systems in industry: An empirical study,” inProceedings of the 44th International Conference on Software Engineering, 2022. [On- line]. Available: https://www.microsoft.com/en-us/research/publica... on Pith
  10. [30] A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” inProceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning 11 Research, vol. 80, 2018... on Pith
  11. [31] F. Tramer, N. Carlini, W. Brendel, and A. Madry, “On adaptive attacks to adversarial example defenses,” inAdvances in Neural Information Processing Systems, vol. 33, 2020. [Online]. Available: https://papers.neurips.cc/paper files/paper/2020/ hash/11f38f8ecd71867b42433548d1078... on Pith
  12. [32] K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world LLM- integrated applications with indirect prompt injection,” inProceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023. [... on Pithdoi:10.1145/3605764.3623985
  13. [33] E. Debenedetti, I. Shumailov, T. Fan, J. Hayes, N. Carlini, D. Fabian, C. Kern, C. Shi, A. Terzis, and F. Tram `er, “Defeating prompt injections by design,” arXiv:2503.18813, 2025. [Online]. Available: https://arxiv.org/abs/2503.18813 on PithPith's reviewarXiv
  14. [34] E. Wallace, K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel, “The instruction hierarchy: Training LLMs to prioritize privileged instructions,” arXiv:2404.13208, 2024. [Online]. Available: https://arxiv.org/abs/2404.13208 on PithPith's reviewarXiv
  15. [35] K. Hines, G. Lopez, M. Hall, F. Zarfati, Y . Zunger, and E. Kıcıman, “Defending against indirect prompt injection attacks with spotlighting,” inProceedings of the Conference on Applied Machine Learning in Information Security (CAMLIS 2024), 2024. [Online]. Available: https://c... on Pith
  16. [36] T. Wu, S. Zhang, K. Song, S. Xu, S. Zhao, R. Agrawal, S. R. Indurthi, C. Xiang, P. Mittal, and W. Zhou, “Instructional segment embedding: Improving LLM safety with instruction hierarchy,” inInternational Conference on Learning Representations, 2025. [Online]. Available: https:... on Pith
  17. [37] E. Zverev, E. Kortukov, A. Panfilov, A. V olkova, S. Tabesh, S. Lapuschkin, W. Samek, and C. H. Lampert, “ASIDE: Architectural separation of instructions and data in language models,” inInternational Conference on Learning Representations, 2026. [Online]. Available: https://pr... on Pith
  18. [38] T. Y . Chen, F.-C. Kuo, H. Liu, P.-L. Poon, D. Towey, T. H. Tse, and Z. Q. Zhou, “Metamorphic testing: A review of challenges and opportunities,”ACM Computing Surveys, vol. 51, no. 1, 2018. [Online]. Available: https://dl.acm.org/doi/10.1145/3143561 on Pithdoi:10.1145/3143561
  19. [39] E. T. Barr, M. Harman, P. McMinn, M. Shahbaz, and S. Yoo, “The oracle problem in software testing: A survey,”IEEE Transactions on Software Engineering, vol. 41, no. 5, pp. 507–525, 2015. [Online]. Available: https://discovery.ucl.ac.uk/id/eprint/1471263/ on Pith
  20. [40] Y . Tian, K. Pei, S. Jana, and B. Ray, “DeepTest: Automated testing of deep-neural-network-driven autonomous cars,” inProceedings of the 40th International Conference on Software Engineering, 2018. [Online]. Available: https://dl.acm.org/doi/10.1145/3180155.3180220 on Pithdoi:10.1145/3180155.3180220
  21. [41] M. Zhang, Y . Zhang, L. Zhang, C. Liu, and S. Khurshid, “DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems,” inProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering, 2018. [Online]. Avail... on Pithdoi:10.1145/3238147.3238187
  22. [42] P. He, C. Meister, and Z. Su, “Structure-invariant testing for machine translation,” inProceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020. [Online]. Available: https://dl.acm.org/doi/10.1145/3377811.3380339 on Pithdoi:10.1145/3377811.3380339
  23. [43] Z. Sun, J. M. Zhang, M. Harman, M. Papadakis, and L. Zhang, “Automatic testing and improvement of machine translation,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 974–985. [Online]. Available: https://dl.acm.org/doi/10.1145/... on Pithdoi:10.1145/3377811.3380420
  24. [44] P. He, C. Meister, and Z. Su, “Testing machine translation via referential transparency,” in2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021. [Online]. Available: https://pinjiahe.github.io/files/pdf/research/ICSE21.pdf on Pith
  25. [45] M. T. Ribeiro, T. Wu, C. Guestrin, and S. Singh, “Beyond accuracy: Behavioral testing of NLP models with CheckList,” inProceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2020, pp. 4902–4912. [Onli... on Pith
  26. [46] D. Kaushik, E. Hovy, and Z. C. Lipton, “Learning the difference that makes a difference with counterfactually-augmented data,” in International Conference on Learning Representations, 2020. [Online]. Available: https://openreview.net/forum?id=Sklgs0NFvr on Pith
  27. [47] J. Yi, Y . Xie, B. Zhu, E. Kiciman, G. Sun, X. Xie, and F. Wu, “Benchmarking and defending against indirect prompt injection attacks on large language models,” inProceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025. [Online]. Available: ht... on Pithdoi:10.1145/3690624.3709179
  28. [48] Q. Zhan, Z. Liang, Z. Ying, and D. Kang, “InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents,” inFindings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 10 471–10 506. [Online]. Available: https: //aclantho... on Pith
  29. [49] E. Debenedetti, J. Zhang, M. Balunovic, L. Beurer-Kellner, M. Fischer, and F. Tram `er, “AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents,” inAdvances in Neural Information Processing Systems, vol. 37, on Pith
  30. [2021] [Online]. Available: https://proceedings.neurips.cc/paper files/ paper/2021/hash/5a66b9200f29ac3fa0ae244cc2a51b39-Abstract.html on Pith
  31. [2024] [Online]. Available: https://proceedings.nips.cc/paper files/paper/ 2024/hash/97091a5177d8dc64b1da8bf3e1f6fb54-Abstract-Datasets and Benchmarks Track.html 12 on Pith

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment